2024-09-19T03:55:04.688 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T03:55:04.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T03:55:04.814 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911624' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxSf4IuCEVcD+LR7xkJwxdOh1pmTEY7ORtaYIKHDyKQSqpHz4Q7AWUUFauWO1ton1i3tDS3YfVcEVSQJ6be5E0= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBq5CI8lApddq8TTMNZnSvzWnjVwsi5/2td73khbsWszbdqWN3dzqTruePuTVZ0F9LX/o4YBfLjxUtBQXFMso9Q= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3bIfsLhSMrw7qj0Lgf9/KON0KPd27nhwOfDWpnZBxr22VGQJuUHAok9RaPYoRDguG4E+K9YkAhknCkNTU4dq8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T03:55:04.814 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T03:55:04.815 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T03:55:04.815 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T03:55:04.816 INFO:teuthology.task.internal:Checking packages... 2024-09-19T03:55:04.839 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T03:55:04.839 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T03:55:04.839 INFO:teuthology.packaging:ref: None 2024-09-19T03:55:04.839 INFO:teuthology.packaging:tag: None 2024-09-19T03:55:04.839 INFO:teuthology.packaging:branch: squid 2024-09-19T03:55:04.840 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:55:04.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T03:55:05.069 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T03:55:05.071 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T03:55:05.077 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T03:55:05.077 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T03:55:05.109 INFO:teuthology.task.internal:Saving configuration 2024-09-19T03:55:05.130 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T03:55:05.136 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T03:55:05.156 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:49:58.511946', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxSf4IuCEVcD+LR7xkJwxdOh1pmTEY7ORtaYIKHDyKQSqpHz4Q7AWUUFauWO1ton1i3tDS3YfVcEVSQJ6be5E0='} 2024-09-19T03:55:05.173 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:49:58.510241', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBq5CI8lApddq8TTMNZnSvzWnjVwsi5/2td73khbsWszbdqWN3dzqTruePuTVZ0F9LX/o4YBfLjxUtBQXFMso9Q='} 2024-09-19T03:55:05.190 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:49:58.513236', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3bIfsLhSMrw7qj0Lgf9/KON0KPd27nhwOfDWpnZBxr22VGQJuUHAok9RaPYoRDguG4E+K9YkAhknCkNTU4dq8='} 2024-09-19T03:55:05.190 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T03:55:05.195 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T03:55:05.196 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T03:55:05.196 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T03:55:05.196 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T03:55:05.287 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f69738975b0>, signals=[15]) 2024-09-19T03:55:05.287 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T03:55:05.293 INFO:teuthology.task.internal:Opening connections... 2024-09-19T03:55:05.293 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T03:55:05.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:55:05.376 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-09-19T03:55:05.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:55:05.448 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-09-19T03:55:05.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:55:05.525 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T03:55:05.531 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-09-19T03:55:05.549 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-09-19T03:55:05.550 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-09-19T03:55:05.607 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-09-19T03:55:05.607 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-09-19T03:55:05.607 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-09-19T03:55:05.607 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-09-19T03:55:05.607 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-09-19T03:55:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-09-19T03:55:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T03:55:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-09-19T03:55:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-09-19T03:55:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T03:55:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-09-19T03:55:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T03:55:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T03:55:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T03:55:05.609 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-09-19T03:55:05.628 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-09-19T03:55:05.647 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-09-19T03:55:05.647 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-09-19T03:55:05.705 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-09-19T03:55:05.705 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-09-19T03:55:05.705 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-09-19T03:55:05.705 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-09-19T03:55:05.705 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-09-19T03:55:05.705 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-09-19T03:55:05.705 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T03:55:05.705 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-09-19T03:55:05.706 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-09-19T03:55:05.706 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T03:55:05.706 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-09-19T03:55:05.706 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T03:55:05.706 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T03:55:05.706 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T03:55:05.706 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-09-19T03:55:05.720 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-09-19T03:55:05.738 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-09-19T03:55:05.738 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-09-19T03:55:05.797 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T03:55:05.798 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-09-19T03:55:05.798 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T03:55:05.798 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T03:55:05.798 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T03:55:05.798 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-09-19T03:55:05.820 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T03:55:05.829 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T03:55:05.858 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T03:55:05.858 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:55:05.861 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:55:05.863 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:55:05.880 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T03:55:05.885 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T03:55:05.886 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:55:05.919 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:55:05.921 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:55:06.010 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T03:55:06.135 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T03:55:06.391 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-19T03:55:06.392 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-19T03:55:06.392 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T03:55:06.392 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T03:55:06.393 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T03:55:06.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:55:06.410 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-09-19T03:55:06.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:55:06.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:55:06.412 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-19T03:55:06.412 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-19T03:55:06.470 INFO:teuthology.orchestra.run.smithi104.stdout:5.14.0-503.el9.x86_64 2024-09-19T03:55:06.470 INFO:teuthology.task.kernel:Running kernel on smithi104: 5.14.0-503.el9.x86_64 2024-09-19T03:55:06.470 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum install -y kernel 2024-09-19T03:55:06.472 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-503.el9.x86_64 2024-09-19T03:55:06.472 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.14.0-503.el9.x86_64 2024-09-19T03:55:06.472 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum install -y kernel 2024-09-19T03:55:06.473 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-503.el9.x86_64 2024-09-19T03:55:06.473 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-503.el9.x86_64 2024-09-19T03:55:06.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-09-19T03:55:08.700 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-19T03:55:08.919 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-19T03:55:10.371 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.3 MB 00:02 2024-09-19T03:55:10.517 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-19T03:55:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-19T03:55:14.901 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-19T03:55:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-19T03:55:16.619 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 19 kB 00:00 2024-09-19T03:55:17.793 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-19T03:55:18.782 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-19T03:55:21.006 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 11 MB/s | 23 MB 00:02 2024-09-19T03:55:22.143 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-19T03:55:23.544 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 19 kB 00:00 2024-09-19T03:55:24.983 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-19T03:55:26.367 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-19T03:55:28.274 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T03:55:28.274 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-19T03:55:28.366 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-19T03:55:28.366 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-09-19T03:55:28.366 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-19T03:55:28.366 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-19T03:55:28.366 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T03:55:28.366 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-19T03:55:28.366 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T03:55:28.366 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T03:55:28.366 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T03:55:28.367 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T03:55:28.367 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-19T03:55:28.367 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-19T03:55:28.367 INFO:teuthology.orchestra.run.smithi094.stdout:Install 4 Packages 2024-09-19T03:55:28.367 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T03:55:28.367 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 79 M 2024-09-19T03:55:28.367 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 126 M 2024-09-19T03:55:28.368 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-19T03:55:28.494 INFO:teuthology.orchestra.run.smithi094.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 341 kB/s | 36 kB 00:00 2024-09-19T03:55:29.469 INFO:teuthology.orchestra.run.smithi094.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-09-19T03:55:29.961 INFO:teuthology.orchestra.run.smithi094.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-19T03:55:30.344 INFO:teuthology.orchestra.run.smithi094.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 18 MB/s | 35 MB 00:01 2024-09-19T03:55:30.345 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-19T03:55:30.345 INFO:teuthology.orchestra.run.smithi094.stdout:Total 40 MB/s | 79 MB 00:01 2024-09-19T03:55:30.353 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-19T03:55:31.032 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-19T03:55:31.385 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-19T03:55:31.385 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-19T03:55:32.252 INFO:teuthology.orchestra.run.smithi104.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T03:55:32.252 INFO:teuthology.orchestra.run.smithi104.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T03:55:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-19T03:55:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-19T03:55:32.361 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-19T03:55:32.367 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-19T03:55:32.368 INFO:teuthology.orchestra.run.smithi104.stdout:Install 4 Packages 2024-09-19T03:55:32.368 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T03:55:32.368 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 79 M 2024-09-19T03:55:32.368 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 126 M 2024-09-19T03:55:32.368 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-19T03:55:32.730 INFO:teuthology.orchestra.run.smithi104.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 339 kB/s | 36 kB 00:00 2024-09-19T03:55:33.114 INFO:teuthology.orchestra.run.smithi104.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-19T03:55:33.539 INFO:teuthology.orchestra.run.smithi104.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-19T03:55:33.806 INFO:teuthology.orchestra.run.smithi104.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 27 MB/s | 29 MB 00:01 2024-09-19T03:55:33.806 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-19T03:55:33.806 INFO:teuthology.orchestra.run.smithi104.stdout:Total 55 MB/s | 79 MB 00:01 2024-09-19T03:55:34.492 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-19T03:55:34.801 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-19T03:55:34.844 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-19T03:55:34.844 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-19T03:55:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T03:55:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:55:35.807 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-19T03:55:35.808 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-19T03:55:38.236 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-19T03:55:38.318 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:55:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:55:38.873 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T03:55:38.955 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:55:39.635 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 23 MB 00:22 2024-09-19T03:55:41.713 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:55:41.840 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:55:43.299 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:55:43.470 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:55:45.077 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T03:55:46.676 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:55:46.782 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:55:46.984 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T03:55:46.984 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T03:55:47.086 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-19T03:55:47.095 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T03:55:47.096 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-19T03:55:47.097 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-19T03:55:47.097 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2024-09-19T03:55:47.097 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T03:55:47.097 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 79 M 2024-09-19T03:55:47.097 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 126 M 2024-09-19T03:55:47.097 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-19T03:55:47.371 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 147 kB/s | 36 kB 00:00 2024-09-19T03:55:48.330 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:55:49.387 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 6.9 MB/s | 16 MB 00:02 2024-09-19T03:55:51.642 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:55:52.096 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 6.9 MB/s | 35 MB 00:04 2024-09-19T03:55:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.7 MB/s | 29 MB 00:04 2024-09-19T03:55:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-19T03:55:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:Total 15 MB/s | 79 MB 00:05 2024-09-19T03:55:53.042 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-19T03:55:53.386 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-19T03:55:53.387 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-19T03:55:54.348 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-19T03:55:54.348 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-19T03:55:56.780 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-19T03:55:57.506 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T03:55:57.588 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:56:00.331 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:56:00.476 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:56:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:56:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:10.333 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:26.533 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:26.932 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:26.932 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T03:56:26.932 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:56:26.932 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:56:27.740 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:27.740 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T03:56:27.740 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-19T03:56:27.740 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T03:56:27.740 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T03:56:27.740 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T03:56:27.740 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T03:56:27.740 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T03:56:27.741 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-19T03:56:28.125 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T03:56:28.199 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T03:56:28.200 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T03:56:28.200 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T03:56:28.200 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T03:56:28.200 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T03:56:28.201 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-19T03:56:28.201 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T03:56:28.201 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T03:56:28.259 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T03:56:28.260 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T03:56:28.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 82.el9 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8274670 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-09-19T03:56:28.292 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T03:56:28.293 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-09-19T03:56:28.294 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T03:56:28.294 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T03:56:28.294 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T03:56:28.294 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 5.14.0-508.el9.x86_64 2024-09-19T03:56:28.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T03:56:28.938 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-09-19T03:56:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:29.695 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:30.076 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:30.076 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T03:56:30.077 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:56:30.077 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:56:30.296 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T03:56:30.324 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-09-19T03:56:30.326 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-19T03:56:30.341 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.eAc8zRj3jn 2024-09-19T03:56:30.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eAc8zRj3jn 2024-09-19T03:56:30.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.eAc8zRj3jn 2024-09-19T03:56:30.544 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.eAc8zRj3jn is 6KB 2024-09-19T03:56:30.593 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.eAc8zRj3jn 2024-09-19T03:56:30.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T03:56:30.679 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T03:56:30.679 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T03:56:30.679 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T03:56:30.679 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T03:56:30.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T03:56:30.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-09-19T03:56:30.923 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:30.923 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T03:56:30.923 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-19T03:56:30.923 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T03:56:30.923 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T03:56:30.923 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T03:56:30.923 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T03:56:30.923 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T03:56:30.924 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-19T03:56:31.289 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T03:56:31.371 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T03:56:31.371 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T03:56:31.372 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T03:56:31.372 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T03:56:31.372 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T03:56:31.372 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-19T03:56:31.372 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi104.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T03:56:31.372 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T03:56:31.430 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T03:56:31.431 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T03:56:31.431 DEBUG:teuthology.orchestra.run.smithi104:> sudo rpm -qi grub2-tools 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Name : grub2-tools 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Epoch : 1 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Version : 2.06 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Release : 82.el9 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Architecture: x86_64 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Group : Unspecified 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Size : 8274670 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:License : GPLv3+ 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T03:56:31.471 INFO:teuthology.orchestra.run.smithi104.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:Packager : builder@centos.org 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:Vendor : CentOS 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:Summary : Support tools for GRUB. 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:Description : 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout:hardware devices. 2024-09-19T03:56:31.472 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T03:56:31.473 INFO:teuthology.orchestra.run.smithi104.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T03:56:31.473 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T03:56:31.473 INFO:teuthology.task.kernel:Updating grub on smithi104 to boot 5.14.0-508.el9.x86_64 2024-09-19T03:56:31.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T03:56:32.128 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2024-09-19T03:56:33.427 INFO:teuthology.orchestra.run.smithi104.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T03:56:33.456 INFO:teuthology.orchestra.run.smithi104.stderr:done 2024-09-19T03:56:33.459 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-09-19T03:56:33.475 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.24mUWxE3bT 2024-09-19T03:56:33.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.24mUWxE3bT 2024-09-19T03:56:33.542 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.24mUWxE3bT 2024-09-19T03:56:33.678 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.24mUWxE3bT is 6KB 2024-09-19T03:56:33.727 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.24mUWxE3bT 2024-09-19T03:56:33.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T03:56:33.805 INFO:teuthology.orchestra.run.smithi104.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T03:56:33.805 INFO:teuthology.orchestra.run.smithi104.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T03:56:33.805 INFO:teuthology.orchestra.run.smithi104.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T03:56:33.805 INFO:teuthology.orchestra.run.smithi104.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T03:56:33.806 DEBUG:teuthology.orchestra.run.smithi104:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T03:56:33.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo shutdown -r now 2024-09-19T03:56:48.375 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:48.421 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:48.761 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:48.762 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T03:56:48.762 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:56:48.762 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:56:49.584 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:56:49.584 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T03:56:49.584 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-19T03:56:49.584 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T03:56:49.584 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T03:56:49.584 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T03:56:49.584 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T03:56:49.584 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T03:56:49.584 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-19T03:56:49.920 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T03:56:50.010 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T03:56:50.010 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T03:56:50.011 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T03:56:50.011 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T03:56:50.011 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T03:56:50.011 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-19T03:56:50.011 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T03:56:50.011 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T03:56:50.070 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T03:56:50.070 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T03:56:50.070 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2024-09-19T03:56:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2024-09-19T03:56:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2024-09-19T03:56:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2024-09-19T03:56:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 82.el9 2024-09-19T03:56:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2024-09-19T03:56:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T03:56:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2024-09-19T03:56:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8274670 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T03:56:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T03:56:50.110 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T03:56:50.110 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2024-09-19T03:56:50.110 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T03:56:50.110 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T03:56:50.111 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T03:56:50.111 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 5.14.0-508.el9.x86_64 2024-09-19T03:56:50.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-09-19T03:56:52.074 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T03:56:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-09-19T03:56:52.103 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-09-19T03:56:52.118 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.jgt5xuQCNT 2024-09-19T03:56:52.118 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jgt5xuQCNT 2024-09-19T03:56:52.185 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.jgt5xuQCNT 2024-09-19T03:56:52.324 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.jgt5xuQCNT is 6KB 2024-09-19T03:56:52.372 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.jgt5xuQCNT 2024-09-19T03:56:52.387 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T03:56:52.450 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T03:56:52.450 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T03:56:52.450 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T03:56:52.450 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T03:56:52.451 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T03:56:52.603 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-09-19T03:57:00.848 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-19T03:57:00.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-19T03:57:00.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:57:03.960 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-19T03:57:03.960 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-19T03:57:03.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:57:19.293 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-19T03:57:22.369 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-09-19T03:57:22.606 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-19T03:57:22.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-19T03:57:22.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:57:28.297 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-19T03:57:28.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:57:31.357 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-19T03:57:31.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-19T03:57:31.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:57:41.053 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-09-19T03:57:43.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-19T03:57:43.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:57:50.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-19T03:57:50.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:57:53.121 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-09-19T03:58:05.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-19T03:58:05.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:58:15.220 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-19T03:58:15.981 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-19T03:58:15.981 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-19T03:58:15.981 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-19T03:58:15.995 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-508.el9.x86_64 2024-09-19T03:58:15.995 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T03:58:15.995 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T03:58:15.996 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T03:58:16.997 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-19T03:58:16.997 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T03:58:17.081 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-09-19T03:58:17.102 DEBUG:teuthology.parallel:result is None 2024-09-19T03:58:23.549 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-09-19T03:58:31.373 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T03:58:38.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-19T03:58:38.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:58:38.924 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-19T03:58:39.331 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-19T03:58:39.331 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-508.el9.x86_64"... 2024-09-19T03:58:39.331 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-19T03:58:39.345 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-508.el9.x86_64 2024-09-19T03:58:39.345 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T03:58:39.345 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T03:58:39.346 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T03:58:40.346 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-19T03:58:40.346 DEBUG:teuthology.orchestra.run.smithi112:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T03:58:40.433 INFO:teuthology.orchestra.run.smithi112.stdout:ttyS1 2024-09-19T03:58:40.458 DEBUG:teuthology.parallel:result is None 2024-09-19T03:58:43.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-19T03:58:43.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:58:43.720 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-09-19T03:58:44.102 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-19T03:58:44.102 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-19T03:58:44.102 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-09-19T03:58:44.119 INFO:teuthology.orchestra.run.smithi104.stdout:5.14.0-508.el9.x86_64 2024-09-19T03:58:44.119 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T03:58:44.119 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T03:58:44.120 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T03:58:45.120 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-19T03:58:45.121 DEBUG:teuthology.orchestra.run.smithi104:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T03:58:45.188 INFO:teuthology.orchestra.run.smithi104.stdout:ttyS1 2024-09-19T03:58:45.208 DEBUG:teuthology.parallel:result is None 2024-09-19T03:58:45.208 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T03:58:45.215 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T03:58:45.215 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T03:58:45.218 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T03:58:45.220 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T03:58:45.239 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T03:58:45.246 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T03:58:45.278 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T03:58:45.278 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T03:58:45.281 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T03:58:45.283 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T03:58:45.328 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T03:58:45.334 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T03:58:45.334 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T03:58:45.363 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T03:58:45.366 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T03:58:45.396 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:58:45.401 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:58:45.408 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:58:45.410 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:58:45.412 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:58:45.422 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:58:45.423 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T03:58:45.430 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T03:58:45.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T03:58:45.433 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T03:58:45.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T03:58:45.485 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T03:58:45.494 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T03:58:45.494 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T03:58:45.501 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T03:58:45.523 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T03:58:45.541 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:58:45.594 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:58:45.677 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:58:45.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:58:45.761 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T03:58:45.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T03:58:45.827 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:58:45.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:58:45.922 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:58:46.001 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:58:46.048 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T03:58:46.048 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T03:58:46.114 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:58:46.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:58:46.217 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:58:46.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:58:46.344 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T03:58:46.344 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T03:58:46.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-09-19T03:58:46.412 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-09-19T03:58:46.415 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-09-19T03:58:46.456 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T03:58:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T03:58:46.495 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T03:58:46.890 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T03:58:46.898 INFO:teuthology.task.internal:Starting timer... 2024-09-19T03:58:46.898 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T03:58:46.939 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T03:58:46.970 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T03:58:46.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-09-19T03:58:47.014 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-09-19T03:58:47.015 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-09-19T03:58:47.063 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-09-19T03:58:47.065 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-09-19T03:58:47.128 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-09-19T03:58:47.130 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T03:58:47.131 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-09-19T03:58:47.157 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-09-19T03:58:47.157 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-09-19T03:58:47.187 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-09-19T03:58:47.187 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-09-19T03:58:47.216 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-09-19T03:58:47.216 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive'} 2024-09-19T03:58:47.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T03:58:47.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:58:47.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T03:58:47.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:58:47.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T03:58:47.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:58:47.300 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T03:58:47.300 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-09-19T03:58:47.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-09-19T03:58:47.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-09-19T03:58:47.413 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T03:58:47.422 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-19T03:58:47.426 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-19T03:58:47.535 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T03:58:47.545 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T03:58:47.546 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi104.front.sepia.ceph.com,smithi112.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T04:02:40.023 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com')] 2024-09-19T04:02:40.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-19T04:02:40.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:02:40.104 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-19T04:02:40.185 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-19T04:02:40.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-19T04:02:40.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:02:40.258 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-09-19T04:02:40.340 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-19T04:02:40.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-19T04:02:40.341 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:02:40.418 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-19T04:02:40.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-19T04:02:40.497 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T04:02:40.507 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T04:02:40.507 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:02:40.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:02:40.510 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:02:40.510 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:02:40.513 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:02:40.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:02:40.537 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:02:40.541 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:02:40.551 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:02:40.556 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:02:40.567 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:02:40.575 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-09-19T04:02:40.580 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-09-19T04:02:40.582 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:02:40.586 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-09-19T04:02:40.592 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-09-19T04:02:40.600 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:02:40.606 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-09-19T04:02:40.606 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:02:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:02:40.617 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-09-19T04:02:40.620 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:02:40.631 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:02:40.646 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:02:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:02:40.674 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:02:40.675 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-09-19T04:02:40.675 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.675 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.675 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.675 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.685 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:02:40.689 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:02:40.689 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-09-19T04:02:40.689 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.689 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.689 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.689 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.700 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:02:40.704 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:02:40.704 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-19T04:02:40.704 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.704 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.704 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.704 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:02:40.705 INFO:teuthology.run_tasks:Running task install... 2024-09-19T04:02:40.713 DEBUG:teuthology.task.install:project ceph 2024-09-19T04:02:40.713 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T04:02:40.713 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T04:02:40.713 INFO:teuthology.task.install:Using flavor: default 2024-09-19T04:02:40.719 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T04:02:40.719 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T04:02:40.720 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:02:40.721 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:02:40.723 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:02:40.723 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:02:40.724 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:02:40.724 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:02:40.873 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:02:40.873 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:02:40.877 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:02:40.878 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:02:40.878 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:02:40.879 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:02:40.986 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:02:40.987 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:02:40.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:02:40.994 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:02:40.994 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:02:40.994 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:02:40.997 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:02:40.997 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:02:40.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:02:41.014 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:02:41.014 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:02:41.022 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:02:41.022 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:02:41.023 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:02:41.023 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:02:41.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:02:41.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:02:41.088 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:02:41.154 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:02:41.162 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:02:41.164 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:02:41.218 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-09-19T04:02:41.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-09-19T04:02:41.227 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-09-19T04:02:41.227 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-09-19T04:02:41.228 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-09-19T04:02:41.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-09-19T04:02:41.445 INFO:teuthology.orchestra.run.smithi112.stdout:45 files removed 2024-09-19T04:02:41.452 INFO:teuthology.orchestra.run.smithi104.stdout:45 files removed 2024-09-19T04:02:41.467 INFO:teuthology.orchestra.run.smithi094.stdout:45 files removed 2024-09-19T04:02:41.470 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:02:41.480 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:02:41.493 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:02:42.033 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-19T04:02:42.035 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 300 kB/s | 84 kB 00:00 2024-09-19T04:02:42.050 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-19T04:02:42.317 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T04:02:42.319 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T04:02:42.335 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T04:02:42.596 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T04:02:42.600 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-19T04:02:42.610 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T04:02:43.028 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.3 MB 00:00 2024-09-19T04:02:43.043 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.3 MB 00:00 2024-09-19T04:02:43.117 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-19T04:02:44.845 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-19T04:02:45.380 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-19T04:02:48.202 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 20 MB 00:04 2024-09-19T04:02:49.822 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.5 MB 00:00 2024-09-19T04:02:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-19T04:02:51.609 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-19T04:02:52.239 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 19 kB 00:00 2024-09-19T04:02:52.594 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-19T04:02:52.723 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-19T04:02:54.819 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-19T04:02:55.363 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 23 MB 00:02 2024-09-19T04:02:58.077 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-19T04:03:00.025 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 23 MB 00:05 2024-09-19T04:03:00.263 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:03:00.264 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:03:00.275 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:03:00.276 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:03:00.277 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:03:00.278 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:03:00.279 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:03:00.282 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:03:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:03:00.346 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-19T04:03:00.365 INFO:teuthology.orchestra.run.smithi112.stdout:====================================================================================== 2024-09-19T04:03:00.365 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-19T04:03:00.365 INFO:teuthology.orchestra.run.smithi112.stdout:====================================================================================== 2024-09-19T04:03:00.365 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-19T04:03:00.365 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:03:00.366 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:03:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:03:00.368 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:03:00.369 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:03:00.369 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:03:00.369 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:03:00.369 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:03:00.369 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-09-19T04:03:00.369 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:03:00.369 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:03:00.370 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:03:00.371 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:03:00.372 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:03:00.373 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:03:00.374 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:03:00.375 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:03:00.376 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:03:00.376 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:03:00.376 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:03:00.376 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:03:00.376 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:03:00.376 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:03:00.376 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:03:00.376 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:03:00.376 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:03:00.377 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:03:00.377 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:03:00.377 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:03:00.377 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:03:00.377 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:03:00.377 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:03:00.377 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:03:00.377 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:03:00.377 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:03:00.378 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:03:00.379 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:00.380 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-19T04:03:00.381 INFO:teuthology.orchestra.run.smithi112.stdout:====================================================================================== 2024-09-19T04:03:00.381 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-09-19T04:03:00.381 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 8 Packages 2024-09-19T04:03:00.381 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:00.381 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 226 M 2024-09-19T04:03:00.381 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-19T04:03:00.776 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T04:03:00.804 INFO:teuthology.orchestra.run.smithi112.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 56 kB/s | 6.6 kB 00:00 2024-09-19T04:03:00.987 INFO:teuthology.orchestra.run.smithi112.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.8 MB/s | 896 kB 00:00 2024-09-19T04:03:01.062 INFO:teuthology.orchestra.run.smithi112.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.9 MB/s | 145 kB 00:00 2024-09-19T04:03:01.138 INFO:teuthology.orchestra.run.smithi112.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-19T04:03:01.246 INFO:teuthology.orchestra.run.smithi112.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 14 MB/s | 1.5 MB 00:00 2024-09-19T04:03:01.296 INFO:teuthology.orchestra.run.smithi112.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 10 MB/s | 2.4 MB 00:00 2024-09-19T04:03:01.507 INFO:teuthology.orchestra.run.smithi112.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-19T04:03:01.930 INFO:teuthology.orchestra.run.smithi112.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 17 MB/s | 21 MB 00:01 2024-09-19T04:03:01.980 INFO:teuthology.orchestra.run.smithi112.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 506 kB/s | 25 kB 00:00 2024-09-19T04:03:02.181 INFO:teuthology.orchestra.run.smithi112.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 16 MB/s | 11 MB 00:00 2024-09-19T04:03:02.239 INFO:teuthology.orchestra.run.smithi112.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.8 MB/s | 228 kB 00:00 2024-09-19T04:03:02.406 INFO:teuthology.orchestra.run.smithi112.stdout:(12/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 15 MB/s | 17 MB 00:01 2024-09-19T04:03:02.431 INFO:teuthology.orchestra.run.smithi112.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 141 kB/s | 27 kB 00:00 2024-09-19T04:03:02.457 INFO:teuthology.orchestra.run.smithi112.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 634 kB/s | 32 kB 00:00 2024-09-19T04:03:02.499 INFO:teuthology.orchestra.run.smithi112.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 751 kB 00:00 2024-09-19T04:03:02.533 INFO:teuthology.orchestra.run.smithi112.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T04:03:02.558 INFO:teuthology.orchestra.run.smithi112.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T04:03:02.591 INFO:teuthology.orchestra.run.smithi112.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 8.5 MB/s | 506 kB 00:00 2024-09-19T04:03:02.633 INFO:teuthology.orchestra.run.smithi112.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 1.1 MB/s | 45 kB 00:00 2024-09-19T04:03:02.700 INFO:teuthology.orchestra.run.smithi112.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T04:03:02.767 INFO:teuthology.orchestra.run.smithi112.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.4 MB/s | 161 kB 00:00 2024-09-19T04:03:02.842 INFO:teuthology.orchestra.run.smithi112.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.2 MB/s | 320 kB 00:00 2024-09-19T04:03:02.926 INFO:teuthology.orchestra.run.smithi112.stdout:(23/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-19T04:03:02.951 INFO:teuthology.orchestra.run.smithi112.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 2.7 MB/s | 300 kB 00:00 2024-09-19T04:03:02.977 INFO:teuthology.orchestra.run.smithi112.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T04:03:02.981 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:03:02.981 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:03:02.995 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:03:02.996 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:03:02.997 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:03:02.998 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:03:03.000 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:03:03.002 INFO:teuthology.orchestra.run.smithi112.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T04:03:03.002 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:03:03.003 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:03:03.052 INFO:teuthology.orchestra.run.smithi112.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.4 MB/s | 171 kB 00:00 2024-09-19T04:03:03.066 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-19T04:03:03.076 INFO:teuthology.orchestra.run.smithi094.stdout:====================================================================================== 2024-09-19T04:03:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-19T04:03:03.076 INFO:teuthology.orchestra.run.smithi094.stdout:====================================================================================== 2024-09-19T04:03:03.076 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-19T04:03:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:03:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:03:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:03:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:03:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:03:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:03:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:03:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:03:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:03:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:03:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:03:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:03:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:03:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:03:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:03:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:03:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:03:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:03:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:03:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:03:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:03:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout:====================================================================================== 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 8 Packages 2024-09-19T04:03:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:03:03.090 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 226 M 2024-09-19T04:03:03.090 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-19T04:03:03.119 INFO:teuthology.orchestra.run.smithi112.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 479 kB/s | 32 kB 00:00 2024-09-19T04:03:03.169 INFO:teuthology.orchestra.run.smithi112.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 16 MB/s | 3.1 MB 00:00 2024-09-19T04:03:03.245 INFO:teuthology.orchestra.run.smithi112.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.1 MB/s | 146 kB 00:00 2024-09-19T04:03:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 13 MB/s | 3.6 MB 00:00 2024-09-19T04:03:03.512 INFO:teuthology.orchestra.run.smithi112.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.3 MB/s | 249 kB 00:00 2024-09-19T04:03:03.578 INFO:teuthology.orchestra.run.smithi112.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 745 kB/s | 49 kB 00:00 2024-09-19T04:03:03.654 INFO:teuthology.orchestra.run.smithi112.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 219 kB/s | 16 kB 00:00 2024-09-19T04:03:03.754 INFO:teuthology.orchestra.run.smithi112.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-19T04:03:03.779 INFO:teuthology.orchestra.run.smithi112.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 2.2 MB/s | 285 kB 00:00 2024-09-19T04:03:03.813 INFO:teuthology.orchestra.run.smithi112.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 13 MB/s | 765 kB 00:00 2024-09-19T04:03:03.838 INFO:teuthology.orchestra.run.smithi112.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 313 kB/s | 18 kB 00:00 2024-09-19T04:03:04.247 INFO:teuthology.orchestra.run.smithi112.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 21 MB/s | 49 MB 00:02 2024-09-19T04:03:04.294 INFO:teuthology.orchestra.run.smithi094.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 28 kB/s | 6.6 kB 00:00 2024-09-19T04:03:04.520 INFO:teuthology.orchestra.run.smithi094.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-19T04:03:04.628 INFO:teuthology.orchestra.run.smithi094.stdout:(3/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 9.5 MB/s | 5.4 MB 00:00 2024-09-19T04:03:04.654 INFO:teuthology.orchestra.run.smithi094.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 1.1 MB/s | 145 kB 00:00 2024-09-19T04:03:04.787 INFO:teuthology.orchestra.run.smithi094.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 15 MB/s | 2.4 MB 00:00 2024-09-19T04:03:04.805 INFO:teuthology.orchestra.run.smithi112.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 80 kB/s | 80 kB 00:00 2024-09-19T04:03:04.829 INFO:teuthology.orchestra.run.smithi094.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-19T04:03:04.972 INFO:teuthology.orchestra.run.smithi112.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 264 kB/s | 191 kB 00:00 2024-09-19T04:03:05.055 INFO:teuthology.orchestra.run.smithi094.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-19T04:03:05.139 INFO:teuthology.orchestra.run.smithi112.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 616 kB/s | 801 kB 00:01 2024-09-19T04:03:05.455 INFO:teuthology.orchestra.run.smithi094.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 15 MB/s | 21 MB 00:01 2024-09-19T04:03:05.505 INFO:teuthology.orchestra.run.smithi094.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 506 kB/s | 25 kB 00:00 2024-09-19T04:03:05.580 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-19T04:03:05.656 INFO:teuthology.orchestra.run.smithi112.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 298 kB/s | 253 kB 00:00 2024-09-19T04:03:05.797 INFO:teuthology.orchestra.run.smithi094.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 14 MB/s | 11 MB 00:00 2024-09-19T04:03:05.815 INFO:teuthology.orchestra.run.smithi112.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-19T04:03:05.856 INFO:teuthology.orchestra.run.smithi094.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.8 MB/s | 228 kB 00:00 2024-09-19T04:03:05.948 INFO:teuthology.orchestra.run.smithi112.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-19T04:03:06.048 INFO:teuthology.orchestra.run.smithi094.stdout:(12/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 14 MB/s | 17 MB 00:01 2024-09-19T04:03:06.073 INFO:teuthology.orchestra.run.smithi094.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 125 kB/s | 27 kB 00:00 2024-09-19T04:03:06.098 INFO:teuthology.orchestra.run.smithi094.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 635 kB/s | 32 kB 00:00 2024-09-19T04:03:06.141 INFO:teuthology.orchestra.run.smithi094.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 751 kB 00:00 2024-09-19T04:03:06.174 INFO:teuthology.orchestra.run.smithi094.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T04:03:06.199 INFO:teuthology.orchestra.run.smithi094.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T04:03:06.241 INFO:teuthology.orchestra.run.smithi094.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T04:03:06.308 INFO:teuthology.orchestra.run.smithi094.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 676 kB/s | 45 kB 00:00 2024-09-19T04:03:06.375 INFO:teuthology.orchestra.run.smithi094.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T04:03:06.382 INFO:teuthology.orchestra.run.smithi112.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 186 kB/s | 135 kB 00:00 2024-09-19T04:03:06.443 INFO:teuthology.orchestra.run.smithi094.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.4 MB/s | 161 kB 00:00 2024-09-19T04:03:06.507 INFO:teuthology.orchestra.run.smithi112.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 180 kB/s | 125 kB 00:00 2024-09-19T04:03:06.518 INFO:teuthology.orchestra.run.smithi094.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 5.3 MB 00:00 2024-09-19T04:03:06.551 INFO:teuthology.orchestra.run.smithi094.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 2.9 MB/s | 320 kB 00:00 2024-09-19T04:03:06.566 INFO:teuthology.orchestra.run.smithi112.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-19T04:03:06.576 INFO:teuthology.orchestra.run.smithi094.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.0 MB/s | 300 kB 00:00 2024-09-19T04:03:06.602 INFO:teuthology.orchestra.run.smithi094.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T04:03:06.608 INFO:teuthology.orchestra.run.smithi112.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-19T04:03:06.627 INFO:teuthology.orchestra.run.smithi094.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T04:03:06.650 INFO:teuthology.orchestra.run.smithi112.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-09-19T04:03:06.686 INFO:teuthology.orchestra.run.smithi094.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-19T04:03:06.692 INFO:teuthology.orchestra.run.smithi112.stdout:(51/156): python3-urllib3-1.26.5-6.el9.noarch.r 289 kB/s | 215 kB 00:00 2024-09-19T04:03:06.725 INFO:teuthology.orchestra.run.smithi112.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-19T04:03:06.753 INFO:teuthology.orchestra.run.smithi094.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 478 kB/s | 32 kB 00:00 2024-09-19T04:03:06.759 INFO:teuthology.orchestra.run.smithi112.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 943 kB 00:00 2024-09-19T04:03:06.792 INFO:teuthology.orchestra.run.smithi112.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-09-19T04:03:06.811 INFO:teuthology.orchestra.run.smithi094.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-19T04:03:06.818 INFO:teuthology.orchestra.run.smithi112.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-19T04:03:06.837 INFO:teuthology.orchestra.run.smithi094.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.7 MB/s | 146 kB 00:00 2024-09-19T04:03:06.843 INFO:teuthology.orchestra.run.smithi112.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-19T04:03:06.868 INFO:teuthology.orchestra.run.smithi112.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T04:03:06.927 INFO:teuthology.orchestra.run.smithi112.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-19T04:03:06.952 INFO:teuthology.orchestra.run.smithi112.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-19T04:03:06.986 INFO:teuthology.orchestra.run.smithi112.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-19T04:03:07.011 INFO:teuthology.orchestra.run.smithi112.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-19T04:03:07.036 INFO:teuthology.orchestra.run.smithi112.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-19T04:03:07.061 INFO:teuthology.orchestra.run.smithi112.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-19T04:03:07.087 INFO:teuthology.orchestra.run.smithi112.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-19T04:03:07.112 INFO:teuthology.orchestra.run.smithi112.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-19T04:03:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 12 MB/s | 3.6 MB 00:00 2024-09-19T04:03:07.137 INFO:teuthology.orchestra.run.smithi112.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-19T04:03:07.188 INFO:teuthology.orchestra.run.smithi094.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.3 MB/s | 249 kB 00:00 2024-09-19T04:03:07.229 INFO:teuthology.orchestra.run.smithi112.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-19T04:03:07.271 INFO:teuthology.orchestra.run.smithi112.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-19T04:03:07.279 INFO:teuthology.orchestra.run.smithi094.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-19T04:03:07.297 INFO:teuthology.orchestra.run.smithi094.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 455 kB/s | 49 kB 00:00 2024-09-19T04:03:07.322 INFO:teuthology.orchestra.run.smithi094.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 392 kB/s | 16 kB 00:00 2024-09-19T04:03:07.346 INFO:teuthology.orchestra.run.smithi112.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 19 MB/s | 4.9 MB 00:00 2024-09-19T04:03:07.356 INFO:teuthology.orchestra.run.smithi094.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.8 MB/s | 285 kB 00:00 2024-09-19T04:03:07.372 INFO:teuthology.orchestra.run.smithi112.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-19T04:03:07.389 INFO:teuthology.orchestra.run.smithi094.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 11 MB/s | 765 kB 00:00 2024-09-19T04:03:07.405 INFO:teuthology.orchestra.run.smithi112.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-19T04:03:07.406 INFO:teuthology.orchestra.run.smithi094.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 363 kB/s | 18 kB 00:00 2024-09-19T04:03:07.447 INFO:teuthology.orchestra.run.smithi112.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-19T04:03:07.472 INFO:teuthology.orchestra.run.smithi112.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T04:03:07.506 INFO:teuthology.orchestra.run.smithi112.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-19T04:03:07.581 INFO:teuthology.orchestra.run.smithi112.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-19T04:03:07.606 INFO:teuthology.orchestra.run.smithi112.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-19T04:03:07.632 INFO:teuthology.orchestra.run.smithi112.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-19T04:03:07.649 INFO:teuthology.orchestra.run.smithi112.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 607 kB/s | 26 kB 00:00 2024-09-19T04:03:07.666 INFO:teuthology.orchestra.run.smithi112.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 568 kB/s | 19 kB 00:00 2024-09-19T04:03:07.691 INFO:teuthology.orchestra.run.smithi112.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-19T04:03:07.708 INFO:teuthology.orchestra.run.smithi112.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 830 kB/s | 35 kB 00:00 2024-09-19T04:03:07.748 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:03:07.749 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:03:07.760 INFO:teuthology.orchestra.run.smithi104.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:03:07.761 INFO:teuthology.orchestra.run.smithi104.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:03:07.762 INFO:teuthology.orchestra.run.smithi104.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:03:07.763 INFO:teuthology.orchestra.run.smithi104.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:03:07.764 INFO:teuthology.orchestra.run.smithi104.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:03:07.766 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:03:07.767 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:03:07.783 INFO:teuthology.orchestra.run.smithi112.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-19T04:03:07.809 INFO:teuthology.orchestra.run.smithi112.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-19T04:03:07.829 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-19T04:03:07.834 INFO:teuthology.orchestra.run.smithi112.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout:====================================================================================== 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout:====================================================================================== 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:03:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:03:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:03:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:03:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:03:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:03:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:03:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:03:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:03:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:03:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:03:07.850 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-09-19T04:03:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:03:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:03:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:03:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:03:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:03:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:03:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:03:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:03:07.851 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-19T04:03:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:03:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:03:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:03:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:03:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:03:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:03:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:03:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:03:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:03:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:03:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:03:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:03:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:03:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:03:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:03:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:03:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:03:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:03:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:03:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:03:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:03:07.861 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-09-19T04:03:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:03:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:03:07.861 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-19T04:03:07.861 INFO:teuthology.orchestra.run.smithi104.stdout:====================================================================================== 2024-09-19T04:03:07.861 INFO:teuthology.orchestra.run.smithi104.stdout:Install 148 Packages 2024-09-19T04:03:07.861 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 8 Packages 2024-09-19T04:03:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:03:07.862 INFO:teuthology.orchestra.run.smithi112.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T04:03:07.862 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 226 M 2024-09-19T04:03:07.862 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-19T04:03:07.882 INFO:teuthology.orchestra.run.smithi094.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 20 MB/s | 49 MB 00:02 2024-09-19T04:03:07.884 INFO:teuthology.orchestra.run.smithi112.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-19T04:03:07.918 INFO:teuthology.orchestra.run.smithi112.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T04:03:07.943 INFO:teuthology.orchestra.run.smithi112.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-19T04:03:07.977 INFO:teuthology.orchestra.run.smithi112.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-19T04:03:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-09-19T04:03:08.027 INFO:teuthology.orchestra.run.smithi112.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T04:03:08.094 INFO:teuthology.orchestra.run.smithi112.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-19T04:03:08.119 INFO:teuthology.orchestra.run.smithi112.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 454 kB/s | 42 kB 00:00 2024-09-19T04:03:08.161 INFO:teuthology.orchestra.run.smithi112.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 663 kB/s | 44 kB 00:00 2024-09-19T04:03:08.186 INFO:teuthology.orchestra.run.smithi112.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-19T04:03:08.212 INFO:teuthology.orchestra.run.smithi112.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 712 kB/s | 36 kB 00:00 2024-09-19T04:03:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 96 kB/s | 80 kB 00:00 2024-09-19T04:03:08.237 INFO:teuthology.orchestra.run.smithi112.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-19T04:03:08.271 INFO:teuthology.orchestra.run.smithi112.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-19T04:03:08.296 INFO:teuthology.orchestra.run.smithi112.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 989 kB/s | 58 kB 00:00 2024-09-19T04:03:08.340 INFO:teuthology.orchestra.run.smithi094.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 417 kB/s | 191 kB 00:00 2024-09-19T04:03:08.358 INFO:teuthology.orchestra.run.smithi104.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-19T04:03:08.371 INFO:teuthology.orchestra.run.smithi112.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 956 kB/s | 96 kB 00:00 2024-09-19T04:03:08.396 INFO:teuthology.orchestra.run.smithi112.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 662 kB/s | 64 kB 00:00 2024-09-19T04:03:08.422 INFO:teuthology.orchestra.run.smithi112.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-09-19T04:03:08.508 INFO:teuthology.orchestra.run.smithi094.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 728 kB/s | 801 kB 00:01 2024-09-19T04:03:08.522 INFO:teuthology.orchestra.run.smithi112.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-09-19T04:03:08.717 INFO:teuthology.orchestra.run.smithi104.stdout:(2/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T04:03:08.722 INFO:teuthology.orchestra.run.smithi112.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-19T04:03:08.749 INFO:teuthology.orchestra.run.smithi094.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 482 kB/s | 253 kB 00:00 2024-09-19T04:03:08.750 INFO:teuthology.orchestra.run.smithi104.stdout:(3/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.2 MB/s | 896 kB 00:00 2024-09-19T04:03:08.764 INFO:teuthology.orchestra.run.smithi112.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-09-19T04:03:08.776 INFO:teuthology.orchestra.run.smithi104.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 2.4 MB/s | 145 kB 00:00 2024-09-19T04:03:08.789 INFO:teuthology.orchestra.run.smithi112.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 839 kB/s | 19 kB 00:00 2024-09-19T04:03:08.831 INFO:teuthology.orchestra.run.smithi112.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-19T04:03:08.857 INFO:teuthology.orchestra.run.smithi112.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-19T04:03:08.859 INFO:teuthology.orchestra.run.smithi104.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 18 MB/s | 1.5 MB 00:00 2024-09-19T04:03:08.882 INFO:teuthology.orchestra.run.smithi112.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-09-19T04:03:08.924 INFO:teuthology.orchestra.run.smithi112.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2024-09-19T04:03:08.941 INFO:teuthology.orchestra.run.smithi094.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 246 kB/s | 106 kB 00:00 2024-09-19T04:03:08.958 INFO:teuthology.orchestra.run.smithi112.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.7 MB/s | 151 kB 00:00 2024-09-19T04:03:08.974 INFO:teuthology.orchestra.run.smithi112.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 413 kB/s | 49 kB 00:00 2024-09-19T04:03:09.033 INFO:teuthology.orchestra.run.smithi112.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-19T04:03:09.033 INFO:teuthology.orchestra.run.smithi094.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.8 MB/s | 1.2 MB 00:00 2024-09-19T04:03:09.100 INFO:teuthology.orchestra.run.smithi112.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-19T04:03:09.117 INFO:teuthology.orchestra.run.smithi112.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 519 kB/s | 43 kB 00:00 2024-09-19T04:03:09.134 INFO:teuthology.orchestra.run.smithi112.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 961 kB/s | 32 kB 00:00 2024-09-19T04:03:09.134 INFO:teuthology.orchestra.run.smithi104.stdout:(6/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-19T04:03:09.151 INFO:teuthology.orchestra.run.smithi112.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 423 kB/s | 14 kB 00:00 2024-09-19T04:03:09.176 INFO:teuthology.orchestra.run.smithi112.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-09-19T04:03:09.183 INFO:teuthology.orchestra.run.smithi094.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 311 kB/s | 135 kB 00:00 2024-09-19T04:03:09.201 INFO:teuthology.orchestra.run.smithi112.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-19T04:03:09.226 INFO:teuthology.orchestra.run.smithi112.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-19T04:03:09.252 INFO:teuthology.orchestra.run.smithi112.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T04:03:09.277 INFO:teuthology.orchestra.run.smithi112.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-09-19T04:03:09.302 INFO:teuthology.orchestra.run.smithi112.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-19T04:03:09.319 INFO:teuthology.orchestra.run.smithi112.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-09-19T04:03:09.336 INFO:teuthology.orchestra.run.smithi112.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 526 kB/s | 18 kB 00:00 2024-09-19T04:03:09.343 INFO:teuthology.orchestra.run.smithi104.stdout:(7/156): ceph-common-19.1.1-361.g2f599a46.el9.x 19 MB/s | 21 MB 00:01 2024-09-19T04:03:09.353 INFO:teuthology.orchestra.run.smithi112.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 688 kB/s | 23 kB 00:00 2024-09-19T04:03:09.370 INFO:teuthology.orchestra.run.smithi112.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 580 kB/s | 19 kB 00:00 2024-09-19T04:03:09.384 INFO:teuthology.orchestra.run.smithi094.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 282 kB/s | 125 kB 00:00 2024-09-19T04:03:09.387 INFO:teuthology.orchestra.run.smithi112.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 589 kB/s | 20 kB 00:00 2024-09-19T04:03:09.410 INFO:teuthology.orchestra.run.smithi104.stdout:(8/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.6 MB/s | 2.4 MB 00:00 2024-09-19T04:03:09.412 INFO:teuthology.orchestra.run.smithi112.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:03:09.438 INFO:teuthology.orchestra.run.smithi112.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T04:03:09.454 INFO:teuthology.orchestra.run.smithi112.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-19T04:03:09.488 INFO:teuthology.orchestra.run.smithi112.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-19T04:03:09.493 INFO:teuthology.orchestra.run.smithi104.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 302 kB/s | 25 kB 00:00 2024-09-19T04:03:09.505 INFO:teuthology.orchestra.run.smithi112.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 987 kB/s | 16 kB 00:00 2024-09-19T04:03:09.530 INFO:teuthology.orchestra.run.smithi112.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T04:03:09.551 INFO:teuthology.orchestra.run.smithi094.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 415 kB/s | 215 kB 00:00 2024-09-19T04:03:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T04:03:09.572 INFO:teuthology.orchestra.run.smithi112.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-19T04:03:09.598 INFO:teuthology.orchestra.run.smithi112.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 735 kB/s | 31 kB 00:00 2024-09-19T04:03:09.623 INFO:teuthology.orchestra.run.smithi112.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T04:03:09.648 INFO:teuthology.orchestra.run.smithi112.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T04:03:09.674 INFO:teuthology.orchestra.run.smithi112.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-19T04:03:09.724 INFO:teuthology.orchestra.run.smithi112.stdout:(141/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-09-19T04:03:09.749 INFO:teuthology.orchestra.run.smithi112.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 738 kB/s | 74 kB 00:00 2024-09-19T04:03:09.775 INFO:teuthology.orchestra.run.smithi112.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T04:03:09.800 INFO:teuthology.orchestra.run.smithi112.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-19T04:03:09.825 INFO:teuthology.orchestra.run.smithi112.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-19T04:03:09.850 INFO:teuthology.orchestra.run.smithi112.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:03:09.876 INFO:teuthology.orchestra.run.smithi112.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T04:03:09.918 INFO:teuthology.orchestra.run.smithi094.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 939 kB/s | 689 kB 00:00 2024-09-19T04:03:09.926 INFO:teuthology.orchestra.run.smithi112.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T04:03:09.952 INFO:teuthology.orchestra.run.smithi104.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 21 MB/s | 17 MB 00:00 2024-09-19T04:03:10.010 INFO:teuthology.orchestra.run.smithi094.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-19T04:03:10.035 INFO:teuthology.orchestra.run.smithi094.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 185 kB/s | 22 kB 00:00 2024-09-19T04:03:10.069 INFO:teuthology.orchestra.run.smithi104.stdout:(11/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 15 MB/s | 11 MB 00:00 2024-09-19T04:03:10.094 INFO:teuthology.orchestra.run.smithi104.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.6 MB/s | 228 kB 00:00 2024-09-19T04:03:10.128 INFO:teuthology.orchestra.run.smithi104.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 462 kB/s | 27 kB 00:00 2024-09-19T04:03:10.143 INFO:teuthology.orchestra.run.smithi112.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T04:03:10.143 INFO:teuthology.orchestra.run.smithi094.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-19T04:03:10.153 INFO:teuthology.orchestra.run.smithi104.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 542 kB/s | 32 kB 00:00 2024-09-19T04:03:10.202 INFO:teuthology.orchestra.run.smithi112.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-19T04:03:10.203 INFO:teuthology.orchestra.run.smithi104.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 3.2 MB/s | 163 kB 00:00 2024-09-19T04:03:10.237 INFO:teuthology.orchestra.run.smithi104.stdout:(16/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 6.8 MB/s | 751 kB 00:00 2024-09-19T04:03:10.262 INFO:teuthology.orchestra.run.smithi104.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T04:03:10.313 INFO:teuthology.orchestra.run.smithi104.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 6.6 MB/s | 506 kB 00:00 2024-09-19T04:03:10.319 INFO:teuthology.orchestra.run.smithi094.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-19T04:03:10.344 INFO:teuthology.orchestra.run.smithi094.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 939 kB/s | 313 kB 00:00 2024-09-19T04:03:10.371 INFO:teuthology.orchestra.run.smithi104.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 775 kB/s | 45 kB 00:00 2024-09-19T04:03:10.436 INFO:teuthology.orchestra.run.smithi094.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-19T04:03:10.438 INFO:teuthology.orchestra.run.smithi104.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T04:03:10.461 INFO:teuthology.orchestra.run.smithi094.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-19T04:03:10.513 INFO:teuthology.orchestra.run.smithi104.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 21 MB/s | 5.3 MB 00:00 2024-09-19T04:03:10.539 INFO:teuthology.orchestra.run.smithi104.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.6 MB/s | 161 kB 00:00 2024-09-19T04:03:10.687 INFO:teuthology.orchestra.run.smithi094.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-19T04:03:10.687 INFO:teuthology.orchestra.run.smithi094.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 360 kB/s | 45 kB 00:00 2024-09-19T04:03:10.687 INFO:teuthology.orchestra.run.smithi094.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-09-19T04:03:10.687 INFO:teuthology.orchestra.run.smithi104.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.2 MB/s | 320 kB 00:00 2024-09-19T04:03:10.687 INFO:teuthology.orchestra.run.smithi104.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.0 MB/s | 300 kB 00:00 2024-09-19T04:03:10.687 INFO:teuthology.orchestra.run.smithi104.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 99 kB 00:00 2024-09-19T04:03:10.698 INFO:teuthology.orchestra.run.smithi104.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 851 kB/s | 85 kB 00:00 2024-09-19T04:03:10.748 INFO:teuthology.orchestra.run.smithi104.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.4 MB/s | 171 kB 00:00 2024-09-19T04:03:10.754 INFO:teuthology.orchestra.run.smithi094.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 226 kB/s | 24 kB 00:00 2024-09-19T04:03:10.768 INFO:teuthology.orchestra.run.smithi112.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 72 kB/s | 41 kB 00:00 2024-09-19T04:03:10.805 INFO:teuthology.orchestra.run.smithi094.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-19T04:03:10.807 INFO:teuthology.orchestra.run.smithi104.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-19T04:03:10.824 INFO:teuthology.orchestra.run.smithi104.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 423 kB/s | 32 kB 00:00 2024-09-19T04:03:10.858 INFO:teuthology.orchestra.run.smithi104.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T04:03:10.872 INFO:teuthology.orchestra.run.smithi094.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-09-19T04:03:10.897 INFO:teuthology.orchestra.run.smithi094.stdout:(63/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-09-19T04:03:10.927 INFO:teuthology.orchestra.run.smithi112.stdout:(152/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:01 2024-09-19T04:03:10.931 INFO:teuthology.orchestra.run.smithi094.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-19T04:03:11.014 INFO:teuthology.orchestra.run.smithi094.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-19T04:03:11.108 INFO:teuthology.orchestra.run.smithi104.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 13 MB/s | 3.6 MB 00:00 2024-09-19T04:03:11.165 INFO:teuthology.orchestra.run.smithi094.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 644 kB/s | 188 kB 00:00 2024-09-19T04:03:11.200 INFO:teuthology.orchestra.run.smithi104.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 22 MB/s | 7.4 MB 00:00 2024-09-19T04:03:11.210 INFO:teuthology.orchestra.run.smithi112.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-09-19T04:03:11.233 INFO:teuthology.orchestra.run.smithi104.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.0 MB/s | 249 kB 00:00 2024-09-19T04:03:11.250 INFO:teuthology.orchestra.run.smithi104.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 986 kB/s | 49 kB 00:00 2024-09-19T04:03:11.267 INFO:teuthology.orchestra.run.smithi104.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 492 kB/s | 16 kB 00:00 2024-09-19T04:03:11.301 INFO:teuthology.orchestra.run.smithi104.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T04:03:11.343 INFO:teuthology.orchestra.run.smithi104.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 439 kB/s | 18 kB 00:00 2024-09-19T04:03:11.377 INFO:teuthology.orchestra.run.smithi104.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 6.9 MB/s | 765 kB 00:00 2024-09-19T04:03:11.502 INFO:teuthology.orchestra.run.smithi104.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-19T04:03:11.544 INFO:teuthology.orchestra.run.smithi112.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 321 kB/s | 249 kB 00:00 2024-09-19T04:03:11.585 INFO:teuthology.orchestra.run.smithi104.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 801 kB 00:00 2024-09-19T04:03:11.611 INFO:teuthology.orchestra.run.smithi104.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-19T04:03:11.645 INFO:teuthology.orchestra.run.smithi104.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-19T04:03:11.695 INFO:teuthology.orchestra.run.smithi104.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T04:03:11.753 INFO:teuthology.orchestra.run.smithi104.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-09-19T04:03:11.779 INFO:teuthology.orchestra.run.smithi104.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-19T04:03:11.812 INFO:teuthology.orchestra.run.smithi104.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-19T04:03:11.869 INFO:teuthology.orchestra.run.smithi112.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-19T04:03:11.871 INFO:teuthology.orchestra.run.smithi104.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-19T04:03:11.898 INFO:teuthology.orchestra.run.smithi094.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-09-19T04:03:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-19T04:03:12.130 INFO:teuthology.orchestra.run.smithi104.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-19T04:03:12.180 INFO:teuthology.orchestra.run.smithi104.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-19T04:03:12.213 INFO:teuthology.orchestra.run.smithi104.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-19T04:03:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-19T04:03:12.240 INFO:teuthology.orchestra.run.smithi094.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.8 MB/s | 4.9 MB 00:01 2024-09-19T04:03:12.244 INFO:teuthology.orchestra.run.smithi112.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-19T04:03:12.251 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:03:12.251 INFO:teuthology.orchestra.run.smithi112.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-19T04:03:12.264 INFO:teuthology.orchestra.run.smithi104.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-09-19T04:03:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-19T04:03:12.365 INFO:teuthology.orchestra.run.smithi104.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-19T04:03:12.423 INFO:teuthology.orchestra.run.smithi104.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T04:03:12.473 INFO:teuthology.orchestra.run.smithi104.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-09-19T04:03:12.532 INFO:teuthology.orchestra.run.smithi094.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 471 kB/s | 137 kB 00:00 2024-09-19T04:03:12.590 INFO:teuthology.orchestra.run.smithi104.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-19T04:03:12.908 INFO:teuthology.orchestra.run.smithi104.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 1.8 MB/s | 581 kB 00:00 2024-09-19T04:03:12.966 INFO:teuthology.orchestra.run.smithi104.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 420 kB/s | 24 kB 00:00 2024-09-19T04:03:13.017 INFO:teuthology.orchestra.run.smithi104.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-19T04:03:13.192 INFO:teuthology.orchestra.run.smithi104.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-19T04:03:13.282 INFO:teuthology.orchestra.run.smithi094.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 657 kB/s | 910 kB 00:01 2024-09-19T04:03:13.309 INFO:teuthology.orchestra.run.smithi104.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-19T04:03:13.351 INFO:teuthology.orchestra.run.smithi104.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 928 kB/s | 38 kB 00:00 2024-09-19T04:03:13.474 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-19T04:03:13.476 INFO:teuthology.orchestra.run.smithi104.stdout:(65/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.4 MB/s | 3.0 MB 00:01 2024-09-19T04:03:13.524 INFO:teuthology.orchestra.run.smithi094.stdout:(71/156): postgresql-13.16-1.el9.x86_64.rpm 688 kB/s | 1.6 MB 00:02 2024-09-19T04:03:13.626 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-19T04:03:13.626 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-19T04:03:13.724 INFO:teuthology.orchestra.run.smithi094.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 438 kB/s | 87 kB 00:00 2024-09-19T04:03:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 674 kB/s | 1.0 MB 00:01 2024-09-19T04:03:14.885 INFO:teuthology.orchestra.run.smithi104.stdout:(66/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 9.0 MB/s | 49 MB 00:05 2024-09-19T04:03:14.934 INFO:teuthology.orchestra.run.smithi094.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 524 kB/s | 61 kB 00:00 2024-09-19T04:03:15.243 INFO:teuthology.orchestra.run.smithi104.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-09-19T04:03:15.309 INFO:teuthology.orchestra.run.smithi094.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 653 kB/s | 245 kB 00:00 2024-09-19T04:03:15.310 INFO:teuthology.orchestra.run.smithi104.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-19T04:03:15.393 INFO:teuthology.orchestra.run.smithi104.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-19T04:03:15.569 INFO:teuthology.orchestra.run.smithi104.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.2 MB/s | 4.9 MB 00:02 2024-09-19T04:03:15.692 INFO:teuthology.orchestra.run.smithi094.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 648 kB/s | 248 kB 00:00 2024-09-19T04:03:15.793 INFO:teuthology.orchestra.run.smithi094.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-19T04:03:15.819 INFO:teuthology.orchestra.run.smithi104.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 1.5 MB/s | 910 kB 00:00 2024-09-19T04:03:15.870 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-19T04:03:15.871 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-19T04:03:15.886 INFO:teuthology.orchestra.run.smithi104.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-09-19T04:03:15.893 INFO:teuthology.orchestra.run.smithi094.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-09-19T04:03:15.936 INFO:teuthology.orchestra.run.smithi104.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 749 kB/s | 87 kB 00:00 2024-09-19T04:03:15.978 INFO:teuthology.orchestra.run.smithi104.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T04:03:16.128 INFO:teuthology.orchestra.run.smithi104.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-19T04:03:16.162 INFO:teuthology.orchestra.run.smithi104.stdout:(76/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.7 MB/s | 1.0 MB 00:00 2024-09-19T04:03:16.193 INFO:teuthology.orchestra.run.smithi094.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 574 kB/s | 172 kB 00:00 2024-09-19T04:03:16.212 INFO:teuthology.orchestra.run.smithi104.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-19T04:03:16.263 INFO:teuthology.orchestra.run.smithi104.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-19T04:03:16.288 INFO:teuthology.orchestra.run.smithi104.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-19T04:03:16.302 INFO:teuthology.orchestra.run.smithi094.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-09-19T04:03:16.363 INFO:teuthology.orchestra.run.smithi104.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-19T04:03:16.388 INFO:teuthology.orchestra.run.smithi104.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 346 kB/s | 35 kB 00:00 2024-09-19T04:03:16.413 INFO:teuthology.orchestra.run.smithi104.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-19T04:03:16.539 INFO:teuthology.orchestra.run.smithi104.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-09-19T04:03:16.564 INFO:teuthology.orchestra.run.smithi104.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-19T04:03:16.614 INFO:teuthology.orchestra.run.smithi104.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-19T04:03:16.639 INFO:teuthology.orchestra.run.smithi104.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 770 kB/s | 77 kB 00:00 2024-09-19T04:03:16.835 INFO:teuthology.orchestra.run.smithi104.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-19T04:03:16.865 INFO:teuthology.orchestra.run.smithi104.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 704 kB/s | 159 kB 00:00 2024-09-19T04:03:16.899 INFO:teuthology.orchestra.run.smithi104.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 947 kB/s | 269 kB 00:00 2024-09-19T04:03:16.924 INFO:teuthology.orchestra.run.smithi104.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 865 kB/s | 51 kB 00:00 2024-09-19T04:03:16.949 INFO:teuthology.orchestra.run.smithi104.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.4 MB/s | 279 kB 00:00 2024-09-19T04:03:16.975 INFO:teuthology.orchestra.run.smithi104.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-09-19T04:03:17.000 INFO:teuthology.orchestra.run.smithi104.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-19T04:03:17.025 INFO:teuthology.orchestra.run.smithi104.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-09-19T04:03:17.060 INFO:teuthology.orchestra.run.smithi104.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T04:03:17.075 INFO:teuthology.orchestra.run.smithi104.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-19T04:03:17.110 INFO:teuthology.orchestra.run.smithi104.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T04:03:17.160 INFO:teuthology.orchestra.run.smithi104.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T04:03:17.210 INFO:teuthology.orchestra.run.smithi104.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.9 MB/s | 96 kB 00:00 2024-09-19T04:03:17.269 INFO:teuthology.orchestra.run.smithi104.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-19T04:03:17.286 INFO:teuthology.orchestra.run.smithi104.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-09-19T04:03:17.311 INFO:teuthology.orchestra.run.smithi104.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-19T04:03:17.386 INFO:teuthology.orchestra.run.smithi104.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-19T04:03:17.445 INFO:teuthology.orchestra.run.smithi104.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T04:03:17.478 INFO:teuthology.orchestra.run.smithi104.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-19T04:03:17.495 INFO:teuthology.orchestra.run.smithi104.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-19T04:03:17.579 INFO:teuthology.orchestra.run.smithi104.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-19T04:03:17.654 INFO:teuthology.orchestra.run.smithi104.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-19T04:03:17.671 INFO:teuthology.orchestra.run.smithi104.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2024-09-19T04:03:17.688 INFO:teuthology.orchestra.run.smithi104.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-09-19T04:03:17.713 INFO:teuthology.orchestra.run.smithi104.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-19T04:03:17.738 INFO:teuthology.orchestra.run.smithi104.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-19T04:03:17.772 INFO:teuthology.orchestra.run.smithi104.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-19T04:03:17.789 INFO:teuthology.orchestra.run.smithi104.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-19T04:03:17.806 INFO:teuthology.orchestra.run.smithi104.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-19T04:03:17.831 INFO:teuthology.orchestra.run.smithi104.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-19T04:03:17.864 INFO:teuthology.orchestra.run.smithi104.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-19T04:03:17.898 INFO:teuthology.orchestra.run.smithi104.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-19T04:03:17.923 INFO:teuthology.orchestra.run.smithi104.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T04:03:17.957 INFO:teuthology.orchestra.run.smithi104.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-19T04:03:17.982 INFO:teuthology.orchestra.run.smithi104.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-19T04:03:18.025 INFO:teuthology.orchestra.run.smithi104.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-19T04:03:18.050 INFO:teuthology.orchestra.run.smithi104.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-19T04:03:18.075 INFO:teuthology.orchestra.run.smithi104.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-19T04:03:18.100 INFO:teuthology.orchestra.run.smithi104.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 934 kB/s | 23 kB 00:00 2024-09-19T04:03:18.126 INFO:teuthology.orchestra.run.smithi104.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-19T04:03:18.151 INFO:teuthology.orchestra.run.smithi104.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-19T04:03:18.201 INFO:teuthology.orchestra.run.smithi104.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-19T04:03:18.235 INFO:teuthology.orchestra.run.smithi104.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T04:03:18.260 INFO:teuthology.orchestra.run.smithi104.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T04:03:18.285 INFO:teuthology.orchestra.run.smithi104.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T04:03:18.319 INFO:teuthology.orchestra.run.smithi104.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-19T04:03:18.352 INFO:teuthology.orchestra.run.smithi104.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-19T04:03:18.378 INFO:teuthology.orchestra.run.smithi104.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-19T04:03:18.403 INFO:teuthology.orchestra.run.smithi104.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-19T04:03:18.428 INFO:teuthology.orchestra.run.smithi104.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-19T04:03:18.454 INFO:teuthology.orchestra.run.smithi104.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-19T04:03:18.479 INFO:teuthology.orchestra.run.smithi104.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-19T04:03:18.504 INFO:teuthology.orchestra.run.smithi104.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T04:03:18.529 INFO:teuthology.orchestra.run.smithi104.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-19T04:03:18.555 INFO:teuthology.orchestra.run.smithi104.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T04:03:18.580 INFO:teuthology.orchestra.run.smithi104.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T04:03:18.605 INFO:teuthology.orchestra.run.smithi104.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T04:03:18.631 INFO:teuthology.orchestra.run.smithi104.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 799 kB/s | 20 kB 00:00 2024-09-19T04:03:18.656 INFO:teuthology.orchestra.run.smithi104.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-19T04:03:18.681 INFO:teuthology.orchestra.run.smithi104.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:03:18.731 INFO:teuthology.orchestra.run.smithi104.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-19T04:03:18.982 INFO:teuthology.orchestra.run.smithi104.stdout:(148/156): librbd1-19.1.1-361.g2f599a46.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-19T04:03:19.096 INFO:teuthology.orchestra.run.smithi104.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 8.2 MB/s | 3.4 MB 00:00 2024-09-19T04:03:19.165 INFO:teuthology.orchestra.run.smithi104.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-19T04:03:19.198 INFO:teuthology.orchestra.run.smithi104.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-09-19T04:03:19.240 INFO:teuthology.orchestra.run.smithi104.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 5.8 MB/s | 249 kB 00:00 2024-09-19T04:03:19.299 INFO:teuthology.orchestra.run.smithi104.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-19T04:03:19.358 INFO:teuthology.orchestra.run.smithi104.stdout:(154/156): NetworkManager-libnm-1.51.0-1.el9.x8 6.9 MB/s | 1.8 MB 00:00 2024-09-19T04:03:19.408 INFO:teuthology.orchestra.run.smithi104.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-19T04:03:20.994 INFO:teuthology.orchestra.run.smithi094.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 699 kB/s | 5.8 MB 00:08 2024-09-19T04:03:21.369 INFO:teuthology.orchestra.run.smithi094.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 668 kB/s | 250 kB 00:00 2024-09-19T04:03:21.761 INFO:teuthology.orchestra.run.smithi094.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 567 kB/s | 222 kB 00:00 2024-09-19T04:03:21.994 INFO:teuthology.orchestra.run.smithi094.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 330 kB/s | 77 kB 00:00 2024-09-19T04:03:22.271 INFO:teuthology.orchestra.run.smithi094.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 153 kB/s | 42 kB 00:00 2024-09-19T04:03:22.298 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-19T04:03:22.406 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:03:22.448 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:03:22.453 INFO:teuthology.orchestra.run.smithi094.stdout:(86/156): python3-babel-2.9.1-2.el9.noarch.rpm 700 kB/s | 6.0 MB 00:08 2024-09-19T04:03:22.498 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:03:22.531 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:03:22.557 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:03:22.633 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:03:22.746 INFO:teuthology.orchestra.run.smithi094.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 544 kB/s | 159 kB 00:00 2024-09-19T04:03:22.779 INFO:teuthology.orchestra.run.smithi094.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 530 kB/s | 269 kB 00:00 2024-09-19T04:03:22.888 INFO:teuthology.orchestra.run.smithi094.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-09-19T04:03:22.996 INFO:teuthology.orchestra.run.smithi094.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 496 kB/s | 54 kB 00:00 2024-09-19T04:03:23.069 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:03:23.174 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:03:23.205 INFO:teuthology.orchestra.run.smithi094.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 607 kB/s | 279 kB 00:00 2024-09-19T04:03:23.279 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:03:23.308 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:03:23.322 INFO:teuthology.orchestra.run.smithi094.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-09-19T04:03:23.408 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:03:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:03:23.447 INFO:teuthology.orchestra.run.smithi094.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 353 kB/s | 44 kB 00:00 2024-09-19T04:03:23.625 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:03:23.682 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:03:23.703 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:03:23.808 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:03:23.829 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:03:23.830 INFO:teuthology.orchestra.run.smithi094.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 576 kB/s | 221 kB 00:00 2024-09-19T04:03:23.893 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:03:23.906 INFO:teuthology.orchestra.run.smithi094.stdout:(95/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 686 kB/s | 5.1 MB 00:07 2024-09-19T04:03:23.922 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:03:23.940 INFO:teuthology.orchestra.run.smithi094.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 330 kB/s | 36 kB 00:00 2024-09-19T04:03:23.946 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:03:24.015 INFO:teuthology.orchestra.run.smithi094.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-19T04:03:24.124 INFO:teuthology.orchestra.run.smithi094.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-19T04:03:24.141 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:03:24.309 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:03:24.324 INFO:teuthology.orchestra.run.smithi094.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 479 kB/s | 96 kB 00:00 2024-09-19T04:03:24.424 INFO:teuthology.orchestra.run.smithi094.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 628 kB/s | 304 kB 00:00 2024-09-19T04:03:24.453 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:03:24.533 INFO:teuthology.orchestra.run.smithi094.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 413 kB/s | 45 kB 00:00 2024-09-19T04:03:24.550 INFO:teuthology.orchestra.run.smithi094.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 282 kB/s | 64 kB 00:00 2024-09-19T04:03:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:03:24.617 INFO:teuthology.orchestra.run.smithi094.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-19T04:03:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:03:24.675 INFO:teuthology.orchestra.run.smithi094.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T04:03:24.709 INFO:teuthology.orchestra.run.smithi094.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-19T04:03:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-09-19T04:03:24.751 INFO:teuthology.orchestra.run.smithi094.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 464 kB/s | 19 kB 00:00 2024-09-19T04:03:24.777 INFO:teuthology.orchestra.run.smithi094.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.4 MB/s | 26 kB 00:00 2024-09-19T04:03:24.802 INFO:teuthology.orchestra.run.smithi094.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-19T04:03:24.844 INFO:teuthology.orchestra.run.smithi094.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-19T04:03:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:03:24.886 INFO:teuthology.orchestra.run.smithi094.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-19T04:03:24.953 INFO:teuthology.orchestra.run.smithi094.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-19T04:03:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:03:25.004 INFO:teuthology.orchestra.run.smithi094.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-19T04:03:25.020 INFO:teuthology.orchestra.run.smithi094.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-19T04:03:25.037 INFO:teuthology.orchestra.run.smithi094.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-19T04:03:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:03:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:03:25.096 INFO:teuthology.orchestra.run.smithi094.stdout:(116/156): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-19T04:03:25.121 INFO:teuthology.orchestra.run.smithi094.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 168 kB/s | 14 kB 00:00 2024-09-19T04:03:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:03:25.146 INFO:teuthology.orchestra.run.smithi094.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-19T04:03:25.180 INFO:teuthology.orchestra.run.smithi094.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T04:03:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:03:25.205 INFO:teuthology.orchestra.run.smithi094.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-19T04:03:25.240 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:03:25.241 INFO:teuthology.orchestra.run.smithi094.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-19T04:03:25.273 INFO:teuthology.orchestra.run.smithi094.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-19T04:03:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:03:25.298 INFO:teuthology.orchestra.run.smithi094.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-19T04:03:25.323 INFO:teuthology.orchestra.run.smithi094.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-19T04:03:25.330 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:03:25.349 INFO:teuthology.orchestra.run.smithi094.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-19T04:03:25.362 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:03:25.374 INFO:teuthology.orchestra.run.smithi094.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-19T04:03:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:03:25.399 INFO:teuthology.orchestra.run.smithi104.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.9 MB/s | 16 MB 00:08 2024-09-19T04:03:25.399 INFO:teuthology.orchestra.run.smithi094.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-19T04:03:25.404 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:03:25.404 INFO:teuthology.orchestra.run.smithi104.stdout:Total 13 MB/s | 226 MB 00:17 2024-09-19T04:03:25.445 INFO:teuthology.orchestra.run.smithi094.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-19T04:03:25.458 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:03:25.467 INFO:teuthology.orchestra.run.smithi094.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:03:25.492 INFO:teuthology.orchestra.run.smithi094.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T04:03:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:03:25.517 INFO:teuthology.orchestra.run.smithi094.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T04:03:25.543 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:03:25.551 INFO:teuthology.orchestra.run.smithi094.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-19T04:03:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-19T04:03:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:03:25.601 INFO:teuthology.orchestra.run.smithi094.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-19T04:03:25.627 INFO:teuthology.orchestra.run.smithi094.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-19T04:03:25.644 INFO:teuthology.orchestra.run.smithi094.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-09-19T04:03:25.661 INFO:teuthology.orchestra.run.smithi094.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 911 kB/s | 31 kB 00:00 2024-09-19T04:03:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:03:25.686 INFO:teuthology.orchestra.run.smithi094.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-09-19T04:03:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:03:25.703 INFO:teuthology.orchestra.run.smithi094.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-19T04:03:25.720 INFO:teuthology.orchestra.run.smithi094.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-19T04:03:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:03:25.745 INFO:teuthology.orchestra.run.smithi094.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-19T04:03:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:03:25.770 INFO:teuthology.orchestra.run.smithi094.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T04:03:25.796 INFO:teuthology.orchestra.run.smithi094.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T04:03:25.813 INFO:teuthology.orchestra.run.smithi094.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-19T04:03:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:03:25.847 INFO:teuthology.orchestra.run.smithi094.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-19T04:03:25.872 INFO:teuthology.orchestra.run.smithi094.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-19T04:03:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:03:25.926 INFO:teuthology.orchestra.run.smithi094.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 20 MB/s | 1.6 MB 00:00 2024-09-19T04:03:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:03:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:03:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:03:26.176 INFO:teuthology.orchestra.run.smithi094.stdout:(148/156): librados2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-19T04:03:26.235 INFO:teuthology.orchestra.run.smithi094.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-19T04:03:26.642 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-19T04:03:26.663 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:03:26.739 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:03:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:03:26.792 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-19T04:03:26.792 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-19T04:03:26.820 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:03:26.857 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:03:26.901 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:03:27.010 INFO:teuthology.orchestra.run.smithi094.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.8 MB/s | 2.3 MB 00:00 2024-09-19T04:03:27.064 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:03:27.119 INFO:teuthology.orchestra.run.smithi094.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 2.0 MB/s | 1.8 MB 00:00 2024-09-19T04:03:27.127 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:03:27.182 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:03:27.256 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:03:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:03:27.302 INFO:teuthology.orchestra.run.smithi094.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 140 kB/s | 41 kB 00:00 2024-09-19T04:03:27.322 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:03:27.369 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:03:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:03:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:03:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:03:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:03:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 489 kB/s | 249 kB 00:00 2024-09-19T04:03:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:03:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:03:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:03:27.987 INFO:teuthology.orchestra.run.smithi094.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 1.4 MB 00:00 2024-09-19T04:03:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:03:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:03:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:03:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:03:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:03:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:03:28.486 INFO:teuthology.orchestra.run.smithi094.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.8 MB/s | 2.4 MB 00:00 2024-09-19T04:03:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:03:28.690 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:03:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:03:28.765 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:03:28.797 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:03:28.827 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:03:28.861 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:03:28.895 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:03:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:03:28.965 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:03:29.051 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:03:29.089 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-19T04:03:29.090 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-19T04:03:29.104 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:03:29.135 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:03:29.238 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:03:29.285 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:03:30.004 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:03:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:03:30.083 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:03:30.182 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:03:30.237 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:03:30.282 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:03:30.360 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:03:30.402 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:03:30.585 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:03:30.616 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:03:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:03:31.339 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:03:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:03:32.688 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:03:32.785 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:03:32.813 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:03:32.837 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:03:33.146 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:03:33.184 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:03:33.257 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:03:33.289 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:03:33.330 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:03:33.803 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:03:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:03:33.916 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:03:33.947 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:03:35.330 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-19T04:03:35.469 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:03:35.507 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:03:35.560 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:03:35.594 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:03:35.619 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:03:35.684 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:03:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:03:36.194 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:03:36.227 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:03:36.246 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:03:36.368 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:03:36.405 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:03:36.488 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:03:36.518 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:03:36.699 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:03:36.754 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:03:36.780 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:03:36.783 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:03:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:03:36.888 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:03:36.911 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:03:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:03:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:03:36.911 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:36.918 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:03:36.947 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:03:36.981 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:03:37.011 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:03:37.035 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:03:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:03:37.397 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:03:37.548 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:03:37.666 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:03:37.735 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:03:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:03:38.068 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:03:38.162 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:03:38.207 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:03:38.241 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:03:38.306 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:03:38.351 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:03:38.400 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:03:38.443 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:03:38.476 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:03:38.503 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:03:38.572 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:03:38.625 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:03:38.657 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:03:38.700 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:03:38.775 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:03:38.819 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:03:38.871 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:03:38.907 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:03:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:03:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:03:39.104 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:03:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:03:39.291 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:03:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:03:39.896 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:03:39.924 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:03:39.967 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:03:40.005 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:03:40.049 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:03:40.204 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:03:40.266 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:03:40.321 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:03:40.394 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:03:40.426 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:03:40.462 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:03:40.509 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:03:40.544 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:03:40.587 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:03:40.630 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:03:40.774 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:03:40.815 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:03:40.856 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:03:40.958 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:03:41.000 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:03:41.039 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:03:41.075 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:03:41.100 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:03:41.190 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:03:41.340 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:03:41.381 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:03:41.419 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:03:41.453 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:03:41.491 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:03:41.531 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:03:41.570 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:03:41.605 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:03:41.638 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:03:41.679 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:03:41.713 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:03:41.800 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:03:41.843 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:03:41.875 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:03:41.978 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:03:42.034 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:03:42.757 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:03:42.814 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:03:42.847 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:03:42.948 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:03:43.002 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:03:43.048 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:03:43.135 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:03:43.166 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:03:43.350 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:03:43.381 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:03:43.461 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:03:44.113 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:03:44.221 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:03:45.481 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:03:45.567 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:03:45.595 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:03:45.628 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:03:45.936 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:03:45.983 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:03:46.074 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:03:46.104 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:03:46.141 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:03:46.619 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:03:46.644 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:03:46.700 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:03:46.732 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:03:46.778 INFO:teuthology.orchestra.run.smithi094.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 700 kB/s | 16 MB 00:23 2024-09-19T04:03:46.783 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:03:46.783 INFO:teuthology.orchestra.run.smithi094.stdout:Total 5.2 MB/s | 226 MB 00:43 2024-09-19T04:03:48.017 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-19T04:03:48.167 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-19T04:03:48.167 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-19T04:03:48.617 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:03:48.617 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-09-19T04:03:48.617 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-09-19T04:03:48.617 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-09-19T04:03:48.617 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-09-19T04:03:48.618 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-09-19T04:03:48.618 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-09-19T04:03:48.618 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-09-19T04:03:48.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:03:48.808 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:03:48.808 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:03:48.808 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:03:48.808 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:49.042 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:03:49.075 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:03:49.629 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:03:49.662 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:03:49.755 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:03:49.756 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:03:49.756 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:03:49.756 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:03:49.796 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:03:50.279 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:03:50.306 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:03:50.306 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:03:50.306 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:03:50.306 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:03:50.307 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:03:50.307 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-19T04:03:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-19T04:03:50.530 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:03:50.553 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:03:50.553 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:03:50.553 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:03:50.553 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:03:50.553 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:03:50.553 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:50.989 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:03:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:03:51.012 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:03:51.012 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:03:51.012 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:03:51.012 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:03:51.013 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:51.119 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:03:51.175 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:03:51.233 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:03:51.416 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:03:51.464 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:03:51.541 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:03:51.631 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:03:51.769 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:03:51.821 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:03:51.854 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:03:51.882 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:03:52.326 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:03:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:03:52.386 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:03:52.448 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:03:52.478 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:03:53.519 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:03:53.555 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:03:54.515 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:03:54.547 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:03:54.649 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:03:54.810 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:03:54.840 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:03:54.865 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:03:54.865 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:03:54.865 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:03:54.865 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:03:54.865 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:03:54.865 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:54.916 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:03:54.928 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:03:55.028 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:03:55.082 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:03:56.007 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:03:56.035 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:03:56.059 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:03:56.059 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:03:56.059 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:03:56.060 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:03:56.060 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:03:56.060 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:56.103 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:03:56.126 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:03:56.126 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:03:56.126 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:03:56.126 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:56.170 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:03:56.193 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:03:56.194 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:03:56.194 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:03:56.194 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:03:56.194 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:03:56.194 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:56.504 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:03:56.527 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:03:56.527 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:03:56.527 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:03:56.527 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:03:56.528 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:03:56.528 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:03:56.613 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-19T04:03:56.749 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:03:56.787 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:03:56.841 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:03:56.873 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:03:56.899 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:03:56.949 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:03:57.316 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:03:57.384 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:03:57.505 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:03:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:03:57.634 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:03:57.674 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:03:57.849 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:03:57.903 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:03:57.931 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:03:58.018 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:03:58.041 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:03:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:03:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:03:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:03:58.313 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:03:58.484 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:03:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:03:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:03:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:03:59.062 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:03:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:03:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:03:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:03:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:03:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:03:59.449 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:03:59.482 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:03:59.525 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:03:59.558 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:03:59.585 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:03:59.654 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:03:59.708 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:03:59.740 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:03:59.783 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:03:59.859 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:03:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:03:59.937 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:03:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:04:00.026 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:04:00.072 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:04:00.146 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:04:00.280 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:04:00.332 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:04:00.859 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:04:00.945 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:04:00.974 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:04:01.018 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:04:01.055 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:04:01.066 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:04:01.101 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:04:01.107 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:04:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:04:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:04:01.199 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:04:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:04:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:04:01.381 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:04:01.404 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:04:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:04:01.460 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:04:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:04:01.478 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:04:01.479 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-09-19T04:04:01.479 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-09-19T04:04:01.479 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-09-19T04:04:01.479 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-09-19T04:04:01.479 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-09-19T04:04:01.479 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-09-19T04:04:01.479 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-09-19T04:04:01.479 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:01.497 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:04:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:04:01.543 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:04:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:04:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:04:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:04:01.657 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:04:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:04:01.674 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:04:01.674 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:01.674 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:04:01.674 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:04:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:04:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:04:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:04:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:04:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:04:02.106 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:04:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:04:02.140 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:04:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:04:02.161 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:04:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:04:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:04:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:04:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:04:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:04:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:04:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:04:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:04:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:04:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:04:02.682 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:04:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:04:02.758 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:04:02.845 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:04:02.897 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:04:02.928 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:04:02.985 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:04:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:04:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:04:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:04:03.126 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:04:03.152 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:04:03.179 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:04:03.179 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:03.179 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:04:03.179 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:04:03.179 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:04:03.179 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:03.218 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:04:03.247 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:04:03.273 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:04:03.307 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:04:03.307 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:04:03.391 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:04:03.391 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:04:03.402 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:04:03.426 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:04:03.426 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:03.426 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:04:03.426 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:04:03.426 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:04:03.426 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:03.467 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:04:03.492 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:04:03.497 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:04:03.497 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:04:03.677 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:04:03.677 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:04:03.761 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:04:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:04:03.825 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:04:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:04:03.861 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:04:03.874 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:04:03.885 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:04:03.885 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:03.885 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:04:03.885 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:04:03.885 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:04:03.885 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:03.976 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:04:03.991 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:04:04.038 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:04:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:04:04.100 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:04:04.112 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:04:04.186 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:04:04.227 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:04:04.298 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:04:04.354 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:04:04.411 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:04:04.432 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:04:04.458 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:04:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:04:04.536 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:04:04.669 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:04:04.732 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:04:04.768 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:04:04.796 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:04:05.188 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:04:05.268 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:04:05.295 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:04:05.307 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:04:05.326 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:04:05.389 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:04:05.418 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:04:06.469 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:04:06.503 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:04:06.569 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:04:06.655 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:04:06.683 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:04:06.716 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:04:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:04:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:04:07.153 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:04:07.184 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:04:07.251 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:04:07.466 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:04:07.495 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:04:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:04:07.734 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:04:07.762 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:04:07.766 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:04:07.796 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:04:07.820 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:04:07.821 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:07.821 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:04:07.821 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:04:07.821 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:04:07.821 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:07.829 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:04:07.859 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:04:07.872 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:04:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:04:07.983 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:04:08.047 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:04:08.966 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:04:09.000 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:04:09.024 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:04:09.024 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:09.024 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:04:09.024 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:04:09.024 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:04:09.024 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:09.067 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:04:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:04:09.090 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:09.090 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:04:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:09.143 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:04:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:04:09.166 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:09.166 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:04:09.166 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:04:09.166 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:04:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:09.391 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:04:09.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:04:09.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:04:09.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:04:09.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:04:09.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:04:09.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:04:09.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:04:09.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:04:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:04:09.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:04:09.394 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:04:09.394 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:04:09.407 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:04:09.408 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:04:09.409 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:04:09.409 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:04:09.409 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:04:09.409 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:04:09.409 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:04:09.409 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:04:09.409 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:04:09.409 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:04:09.409 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:04:09.410 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:04:09.411 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:04:09.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:04:09.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:04:09.414 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:04:09.415 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:04:09.416 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:04:09.416 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:04:09.416 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:04:09.416 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:04:09.416 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:04:09.416 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:04:09.416 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:04:09.416 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:04:09.417 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:04:09.417 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:04:09.417 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:04:09.417 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:04:09.417 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:04:09.418 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:04:09.418 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:04:09.418 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:04:09.418 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:04:09.418 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:04:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:04:09.468 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:04:09.492 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:04:09.492 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:09.492 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:04:09.492 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:04:09.492 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:04:09.492 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:10.125 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:04:10.164 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:04:10.723 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:04:10.768 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:04:10.858 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:04:10.858 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:04:10.858 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:04:10.858 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:04:12.425 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:04:12.425 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:04:12.425 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-09-19T04:04:12.425 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:12.425 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:12.425 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:12.425 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:12.425 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.426 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:04:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:04:12.429 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:04:12.430 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:04:12.431 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:04:12.432 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:04:12.433 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:04:12.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:04:12.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:04:12.436 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:04:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:04:12.438 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:04:12.438 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:04:12.438 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-19T04:04:12.830 DEBUG:teuthology.parallel:result is None 2024-09-19T04:04:14.038 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:04:14.097 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:04:14.137 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:04:14.169 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:04:14.196 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:04:14.411 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:04:14.459 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:04:14.489 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:04:14.520 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:04:14.557 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:04:15.189 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:04:15.221 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:04:15.242 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:04:16.088 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:04:16.175 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:04:16.225 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:04:16.341 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:04:16.370 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:04:16.398 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:04:16.438 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:04:16.438 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:04:16.539 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:04:16.539 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:04:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:04:16.639 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:04:16.645 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:04:16.645 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:04:16.825 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:04:16.825 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:04:16.909 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:04:16.981 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:04:22.540 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:04:22.540 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:04:22.540 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:04:22.540 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:04:22.540 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:04:22.540 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:04:22.540 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:04:22.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:04:22.542 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:04:22.553 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:04:22.554 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:04:22.555 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:04:22.556 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:04:22.557 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:04:22.558 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:04:22.559 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:04:22.560 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:04:22.561 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:04:22.561 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:04:22.561 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:04:22.561 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:04:22.561 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:04:22.561 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:04:22.561 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:04:22.561 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:04:22.561 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:04:22.562 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:04:22.563 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:04:22.564 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:04:22.565 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:04:22.565 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:04:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:04:22.995 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-09-19T04:04:22.995 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-09-19T04:04:22.995 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-09-19T04:04:22.996 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-09-19T04:04:22.996 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-09-19T04:04:22.996 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-09-19T04:04:22.996 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-09-19T04:04:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:23.167 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:04:23.183 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:04:23.183 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:23.183 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:04:23.183 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:24.610 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:04:24.637 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:04:24.637 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:24.637 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:04:24.638 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:04:24.638 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:04:24.638 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:04:24.884 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:04:24.884 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:24.884 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:04:24.884 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:04:24.884 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:04:24.884 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:25.261 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:04:25.261 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:04:25.262 INFO:teuthology.orchestra.run.smithi104.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.263 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.264 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:04:25.265 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:04:25.266 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:04:25.267 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.268 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:04:25.269 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:04:25.270 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:04:25.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:04:25.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:04:25.273 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:04:25.274 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:04:25.275 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:04:25.275 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:04:25.275 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-19T04:04:25.319 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:04:25.343 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:04:25.343 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:25.343 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:04:25.343 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:04:25.344 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:04:25.344 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:25.449 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:04:25.506 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:04:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:04:25.695 DEBUG:teuthology.parallel:result is None 2024-09-19T04:04:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:04:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:04:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:04:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:04:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:04:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:04:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:04:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:04:26.612 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:04:26.638 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:04:26.670 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:04:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:04:26.761 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:04:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:04:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:04:28.797 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:04:28.830 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:04:28.933 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:04:29.101 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:04:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:04:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:04:29.156 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:29.157 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:04:29.157 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:04:29.157 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:04:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:29.208 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:04:29.220 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:04:29.321 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:04:29.384 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:04:30.303 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:04:30.344 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:04:30.368 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:04:30.368 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:30.368 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:04:30.368 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:04:30.368 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:04:30.368 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:30.411 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:04:30.434 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:04:30.434 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:30.434 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:04:30.434 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:30.478 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:04:30.502 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:04:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:04:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:04:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:04:30.502 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:30.804 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:04:30.828 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:04:30.828 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:04:30.828 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:04:30.828 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:04:30.828 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:04:30.828 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:04:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:04:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:04:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:04:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:04:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:04:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:04:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:04:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:04:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:04:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:04:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:04:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:04:37.374 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:04:37.465 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:04:37.514 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:04:37.601 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:04:37.635 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:04:37.661 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:04:37.696 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:04:37.696 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:04:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:04:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:04:37.872 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:04:37.897 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:04:37.902 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:04:37.902 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:04:38.099 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:04:38.099 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:04:38.183 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:04:38.247 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:04:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:04:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:04:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:04:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:04:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:04:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:04:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:04:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:04:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:04:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:04:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:04:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:04:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:04:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:04:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:04:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:04:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:04:46.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-19T04:04:46.855 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.856 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:04:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:04:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.861 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:04:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:04:46.863 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:04:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:04:46.865 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:04:46.866 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:04:46.867 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:04:46.867 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:04:46.867 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:04:46.867 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:04:46.867 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:04:46.867 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:04:46.867 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-19T04:04:47.291 DEBUG:teuthology.parallel:result is None 2024-09-19T04:04:47.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:04:47.432 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:04:47.455 INFO:teuthology.orchestra.run.smithi094.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:04:47.455 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:04:47.455 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:04:47.457 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:04:47.594 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:04:47.620 INFO:teuthology.orchestra.run.smithi104.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:04:47.620 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:04:47.620 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:04:47.622 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:04:47.757 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:04:47.784 INFO:teuthology.orchestra.run.smithi112.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:04:47.785 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:04:47.785 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:04:47.786 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T04:04:47.787 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:04:47.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:04:47.814 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:04:47.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:04:47.846 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:04:47.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:04:47.879 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T04:04:47.879 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:04:47.879 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:04:47.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:04:47.968 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:04:47.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:04:47.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:04:48.066 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:04:48.066 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:04:48.096 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:04:48.165 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T04:04:48.165 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:04:48.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:04:48.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:04:48.254 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:04:48.255 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:04:48.281 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:04:48.346 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:04:48.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:04:48.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:04:48.437 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T04:04:48.437 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:04:48.437 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:04:48.462 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:04:48.526 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:04:48.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:04:48.552 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:04:48.617 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:04:48.617 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:04:48.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:04:48.712 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T04:04:48.800 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-19T04:04:48.801 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:04:48.801 INFO:tasks.cephadm:Cluster fsid is 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:04:48.801 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T04:04:48.802 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.94', 'mon.b': '172.21.15.104', 'mon.c': '172.21.15.112'} 2024-09-19T04:04:48.802 INFO:tasks.cephadm:First mon is mon.a on smithi094 2024-09-19T04:04:48.802 INFO:tasks.cephadm:First mgr is x 2024-09-19T04:04:48.802 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T04:04:48.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-09-19T04:04:48.828 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-09-19T04:04:48.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-09-19T04:04:48.878 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T04:04:48.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:04:49.015 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T04:04:49.153 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:04:49.155 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:04:49.155 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:04:49.155 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:04:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:04 /home/ubuntu/cephtest/cephadm 2024-09-19T04:04:49.536 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:04:49.917 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:04 /home/ubuntu/cephtest/cephadm 2024-09-19T04:04:49.917 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:04:50.309 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:04 /home/ubuntu/cephtest/cephadm 2024-09-19T04:04:50.309 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:04:50.329 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:04:50.349 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:04:50.381 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T04:04:50.381 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:04:50.384 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:04:50.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:04:51.881 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:04:51.919 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:04:52.079 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:05:19.551 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:05:19.552 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:05:19.552 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:05:19.552 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-09-19T04:05:19.552 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:05:19.552 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:05:19.552 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:05:23.601 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-19T04:05:23.601 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:05:23.601 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:05:23.601 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-09-19T04:05:23.601 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:05:23.601 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-19T04:05:23.601 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-19T04:05:24.806 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-19T04:05:24.806 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:05:24.806 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:05:24.806 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-09-19T04:05:24.806 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:05:24.806 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-19T04:05:24.806 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-19T04:05:24.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-09-19T04:05:24.865 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-09-19T04:05:24.892 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-09-19T04:05:24.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-09-19T04:05:24.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-09-19T04:05:24.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-09-19T04:05:25.007 INFO:tasks.cephadm:Writing seed config... 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T04:05:25.009 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T04:05:25.010 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-19T04:05:25.011 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T04:05:25.012 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T04:05:25.012 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T04:05:25.012 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T04:05:25.012 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T04:05:25.012 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T04:05:25.012 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T04:05:25.012 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-19T04:05:25.013 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:05:25.013 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T04:05:25.031 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 503c4526-763c-11ef-bceb-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T04:05:25.031 DEBUG:teuthology.orchestra.run.smithi094:mon.a> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T04:05:25.074 DEBUG:teuthology.orchestra.run.smithi094:mgr.x> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T04:05:25.116 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T04:05:25.116 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 503c4526-763c-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.94 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:05:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:05:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '503c4526-763c-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.94', '--skip-admin-label'] 2024-09-19T04:05:25.341 INFO:teuthology.orchestra.run.smithi094.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T04:05:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying podman|docker is present... 2024-09-19T04:05:25.365 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:05:25.365 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying lvm2 is present... 2024-09-19T04:05:25.365 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying time synchronization is in place... 2024-09-19T04:05:25.373 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:05:25.373 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:05:25.380 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:05:25.380 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-09-19T04:05:25.386 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-09-19T04:05:25.393 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-09-19T04:05:25.393 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:05:25.393 INFO:teuthology.orchestra.run.smithi094.stdout:Repeating the final host check... 2024-09-19T04:05:25.416 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:05:25.416 INFO:teuthology.orchestra.run.smithi094.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T04:05:25.416 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl is present 2024-09-19T04:05:25.416 INFO:teuthology.orchestra.run.smithi094.stdout:lvcreate is present 2024-09-19T04:05:25.422 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:05:25.422 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:05:25.429 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:05:25.429 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-09-19T04:05:25.435 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-09-19T04:05:25.442 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-09-19T04:05:25.442 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:05:25.442 INFO:teuthology.orchestra.run.smithi094.stdout:Host looks OK 2024-09-19T04:05:25.442 INFO:teuthology.orchestra.run.smithi094.stdout:Cluster fsid: 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:25.442 INFO:teuthology.orchestra.run.smithi094.stdout:Acquiring lock 140670706119536 on /run/cephadm/503c4526-763c-11ef-bceb-c7b262605968.lock 2024-09-19T04:05:25.442 INFO:teuthology.orchestra.run.smithi094.stdout:Lock 140670706119536 acquired on /run/cephadm/503c4526-763c-11ef-bceb-c7b262605968.lock 2024-09-19T04:05:25.443 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 3300 ... 2024-09-19T04:05:25.443 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 6789 ... 2024-09-19T04:05:25.443 INFO:teuthology.orchestra.run.smithi094.stdout:Base mon IP(s) is [172.21.15.94:3300, 172.21.15.94:6789], mon addrv is [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-09-19T04:05:25.446 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.94 metric 100 2024-09-19T04:05:25.446 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.94 metric 100 2024-09-19T04:05:25.448 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T04:05:25.448 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-19T04:05:25.449 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-19T04:05:25.451 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T04:05:25.451 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T04:05:25.451 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:05:25.451 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-19T04:05:25.451 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d763/64 scope link noprefixroute 2024-09-19T04:05:25.452 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:05:25.452 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-09-19T04:05:25.452 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-09-19T04:05:25.452 INFO:teuthology.orchestra.run.smithi094.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T04:05:25.453 INFO:teuthology.orchestra.run.smithi094.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T04:05:25.453 INFO:teuthology.orchestra.run.smithi094.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:05:25.921 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:05:25.921 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:05:25.921 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T04:05:25.921 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T04:05:25.921 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T04:05:25.921 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:05:25.922 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T04:05:26.792 INFO:teuthology.orchestra.run.smithi094.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:05:26.792 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:05:26.792 INFO:teuthology.orchestra.run.smithi094.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T04:05:27.798 INFO:teuthology.orchestra.run.smithi094.stdout:stat: stdout 167 167 2024-09-19T04:05:27.798 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial keys... 2024-09-19T04:05:28.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQAIo+tmSi68BxAAFsVLGFV9XuAvIcOxX3vkZg== 2024-09-19T04:05:29.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQAJo+tmlxSOCRAAGozCxy46w0htXH4Hb4PL0w== 2024-09-19T04:05:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQAKo+tmub9vCRAA4pFt6sm1pWqzatIi1mPtIg== 2024-09-19T04:05:30.723 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial monmap... 2024-09-19T04:05:31.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:05:31.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T04:05:31.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:31.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:05:31.637 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool for a [v2:172.21.15.94:3300,v1:172.21.15.94:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:05:31.637 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-09-19T04:05:31.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: set fsid to 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:31.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:05:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:05:31.637 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mon... 2024-09-19T04:05:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:create mon.a on 2024-09-19T04:05:32.847 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T04:05:32.989 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T04:05:33.163 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-503c4526-763c-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-503c4526-763c-11ef-bceb-c7b262605968.target. 2024-09-19T04:05:33.164 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-503c4526-763c-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-503c4526-763c-11ef-bceb-c7b262605968.target. 2024-09-19T04:05:33.399 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.a 2024-09-19T04:05:33.400 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T04:05:33.563 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-503c4526-763c-11ef-bceb-c7b262605968.target.wants/ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-503c4526-763c-11ef-bceb-c7b262605968@.service. 2024-09-19T04:05:34.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:34 smithi094 podman[30581]: 2024-09-19 04:05:34.020768123 +0000 UTC m=+0.334068828 container start a37e4c93be29cfba769d101f6270a105c2f54fadef0bc0f03c79c4035b7a1215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T04:05:34.115 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:05:34.115 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-09-19T04:05:34.122 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:05:34.122 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-09-19T04:05:34.122 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-09-19T04:05:34.122 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:05:34.122 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon to start... 2024-09-19T04:05:34.123 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon... 2024-09-19T04:05:34.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:34 smithi094 ceph-mon[30601]: mkfs 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:34.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:34 smithi094 ceph-mon[30601]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:05:34.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:34 smithi094 bash[30581]: a37e4c93be29cfba769d101f6270a105c2f54fadef0bc0f03c79c4035b7a1215 2024-09-19T04:05:34.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:34 smithi094 systemd[1]: Started Ceph mon.a for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:05:35.158 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:05:35.158 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: monmap epoch 1 2024-09-19T04:05:35.158 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:35.158 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: last_changed 2024-09-19T04:05:31.044155+0000 2024-09-19T04:05:35.158 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:05:35.158 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: min_mon_release 19 (squid) 2024-09-19T04:05:35.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: election_strategy: 1 2024-09-19T04:05:35.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:05:35.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: fsmap 2024-09-19T04:05:35.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:05:35.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: mgrmap e1: no daemons active 2024-09-19T04:05:35.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:35 smithi094 ceph-mon[30601]: from='client.? 172.21.15.94:0/939813594' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T04:05:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T04:05:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout id: 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T04:05:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:05:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout services: 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.559611s) 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout data: 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.548+0000 7fab1bb91640 1 Processor -- start 2024-09-19T04:05:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.548+0000 7fab1bb91640 1 -- start start 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.548+0000 7fab1bb91640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab14107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.548+0000 7fab1bb91640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab141080b0 con 0x7fab141076e0 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.548+0000 7fab1992d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab14107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.549+0000 7fab1992d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab14107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54916/0 (socket says 172.21.15.94:54916) 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.549+0000 7fab1992d640 1 -- 172.21.15.94:0/2584377281 learned_addr learned my addr 172.21.15.94:0/2584377281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.549+0000 7fab1992d640 1 -- 172.21.15.94:0/2584377281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab14108840 con 0x7fab141076e0 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.550+0000 7fab1992d640 1 --2- 172.21.15.94:0/2584377281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab14107ae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab04009820 tx=0x7fab0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5346f878e3ab19ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1892b640 1 -- 172.21.15.94:0/2584377281 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab04037070 con 0x7fab141076e0 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1892b640 1 -- 172.21.15.94:0/2584377281 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab04032440 con 0x7fab141076e0 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1892b640 1 -- 172.21.15.94:0/2584377281 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab04032740 con 0x7fab141076e0 2024-09-19T04:05:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1bb91640 1 -- 172.21.15.94:0/2584377281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 msgr2=0x7fab14107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1bb91640 1 --2- 172.21.15.94:0/2584377281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab14107ae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab04009820 tx=0x7fab0402a270 comp rx=0 tx=0).stop 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1bb91640 1 -- 172.21.15.94:0/2584377281 shutdown_connections 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1bb91640 1 --2- 172.21.15.94:0/2584377281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab14107ae0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1bb91640 1 -- 172.21.15.94:0/2584377281 >> 172.21.15.94:0/2584377281 conn(0x7fab14102ed0 msgr2=0x7fab141052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1bb91640 1 -- 172.21.15.94:0/2584377281 shutdown_connections 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.551+0000 7fab1bb91640 1 -- 172.21.15.94:0/2584377281 wait complete. 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.552+0000 7fab1bb91640 1 Processor -- start 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.552+0000 7fab1bb91640 1 -- start start 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.552+0000 7fab1bb91640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab1407db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.552+0000 7fab1bb91640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab14108de0 con 0x7fab141076e0 2024-09-19T04:05:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.552+0000 7fab1992d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab1407db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.552+0000 7fab1992d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab1407db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54932/0 (socket says 172.21.15.94:54932) 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.552+0000 7fab1992d640 1 -- 172.21.15.94:0/939813594 learned_addr learned my addr 172.21.15.94:0/939813594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.552+0000 7fab1992d640 1 -- 172.21.15.94:0/939813594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1407e090 con 0x7fab141076e0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.553+0000 7fab1992d640 1 --2- 172.21.15.94:0/939813594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab1407db50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab040090b0 tx=0x7fab040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.553+0000 7fab0affd640 1 -- 172.21.15.94:0/939813594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab04041070 con 0x7fab141076e0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.553+0000 7fab0affd640 1 -- 172.21.15.94:0/939813594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab0403c7a0 con 0x7fab141076e0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.553+0000 7fab1bb91640 1 -- 172.21.15.94:0/939813594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1407c280 con 0x7fab141076e0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.553+0000 7fab0affd640 1 -- 172.21.15.94:0/939813594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab04037040 con 0x7fab141076e0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.553+0000 7fab1bb91640 1 -- 172.21.15.94:0/939813594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1407c7c0 con 0x7fab141076e0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.553+0000 7fab0affd640 1 -- 172.21.15.94:0/939813594 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab04044070 con 0x7fab141076e0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.553+0000 7fab0affd640 1 -- 172.21.15.94:0/939813594 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fab0403d3f0 con 0x7fab141076e0 2024-09-19T04:05:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.554+0000 7fab1bb91640 1 -- 172.21.15.94:0/939813594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab14109430 con 0x7fab141076e0 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.556+0000 7fab0affd640 1 -- 172.21.15.94:0/939813594 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fab04046450 con 0x7fab141076e0 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.615+0000 7fab1bb91640 1 -- 172.21.15.94:0/939813594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fab1407cec0 con 0x7fab141076e0 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.615+0000 7fab0affd640 1 -- 172.21.15.94:0/939813594 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fab04046630 con 0x7fab141076e0 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.616+0000 7fab1bb91640 1 -- 172.21.15.94:0/939813594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 msgr2=0x7fab1407db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.616+0000 7fab1bb91640 1 --2- 172.21.15.94:0/939813594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab1407db50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab040090b0 tx=0x7fab040046c0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.616+0000 7fab1bb91640 1 -- 172.21.15.94:0/939813594 shutdown_connections 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.616+0000 7fab1bb91640 1 --2- 172.21.15.94:0/939813594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab141076e0 0x7fab1407db50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.616+0000 7fab1bb91640 1 -- 172.21.15.94:0/939813594 >> 172.21.15.94:0/939813594 conn(0x7fab14102ed0 msgr2=0x7fab1418f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.617+0000 7fab1bb91640 1 -- 172.21.15.94:0/939813594 shutdown_connections 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:34.617+0000 7fab1bb91640 1 -- 172.21.15.94:0/939813594 wait complete. 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:mon is available 2024-09-19T04:05:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T04:05:36.230 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:36 smithi094 ceph-mon[30601]: from='client.? 172.21.15.94:0/1729245795' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:05:36.230 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:36 smithi094 ceph-mon[30601]: from='client.? 172.21.15.94:0/1729245795' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T04:05:36.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:05:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.545+0000 7fba35432640 1 Processor -- start 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.545+0000 7fba35432640 1 -- start start 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.545+0000 7fba35432640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba30107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.545+0000 7fba35432640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba301080b0 con 0x7fba301076e0 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.545+0000 7fba2effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba30107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.545+0000 7fba2effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba30107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54936/0 (socket says 172.21.15.94:54936) 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.545+0000 7fba2effd640 1 -- 172.21.15.94:0/4237968045 learned_addr learned my addr 172.21.15.94:0/4237968045 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.545+0000 7fba2effd640 1 -- 172.21.15.94:0/4237968045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba30108840 con 0x7fba301076e0 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.546+0000 7fba2effd640 1 --2- 172.21.15.94:0/4237968045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba30107ae0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba18009820 tx=0x7fba1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe4ce73ee1b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.546+0000 7fba2dffb640 1 -- 172.21.15.94:0/4237968045 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba18037070 con 0x7fba301076e0 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.546+0000 7fba2dffb640 1 -- 172.21.15.94:0/4237968045 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba18032440 con 0x7fba301076e0 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.546+0000 7fba2dffb640 1 -- 172.21.15.94:0/4237968045 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba18032740 con 0x7fba301076e0 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.546+0000 7fba35432640 1 -- 172.21.15.94:0/4237968045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 msgr2=0x7fba30107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.546+0000 7fba35432640 1 --2- 172.21.15.94:0/4237968045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba30107ae0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba18009820 tx=0x7fba1802a270 comp rx=0 tx=0).stop 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.546+0000 7fba35432640 1 -- 172.21.15.94:0/4237968045 shutdown_connections 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.546+0000 7fba35432640 1 --2- 172.21.15.94:0/4237968045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba30107ae0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.546+0000 7fba35432640 1 -- 172.21.15.94:0/4237968045 >> 172.21.15.94:0/4237968045 conn(0x7fba30102ed0 msgr2=0x7fba301052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.547+0000 7fba35432640 1 -- 172.21.15.94:0/4237968045 shutdown_connections 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.547+0000 7fba35432640 1 -- 172.21.15.94:0/4237968045 wait complete. 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.547+0000 7fba35432640 1 Processor -- start 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.547+0000 7fba35432640 1 -- start start 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.547+0000 7fba35432640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba3019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.547+0000 7fba35432640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba30108de0 con 0x7fba301076e0 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba2effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba3019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba2effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba3019ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54948/0 (socket says 172.21.15.94:54948) 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba2effd640 1 -- 172.21.15.94:0/1729245795 learned_addr learned my addr 172.21.15.94:0/1729245795 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba2effd640 1 -- 172.21.15.94:0/1729245795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3019b340 con 0x7fba301076e0 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba2effd640 1 --2- 172.21.15.94:0/1729245795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba3019ae00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba180090b0 tx=0x7fba180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba27fff640 1 -- 172.21.15.94:0/1729245795 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba18041070 con 0x7fba301076e0 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba27fff640 1 -- 172.21.15.94:0/1729245795 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1803c7a0 con 0x7fba301076e0 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba35432640 1 -- 172.21.15.94:0/1729245795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3019b5d0 con 0x7fba301076e0 2024-09-19T04:05:36.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba27fff640 1 -- 172.21.15.94:0/1729245795 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba18037040 con 0x7fba301076e0 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba35432640 1 -- 172.21.15.94:0/1729245795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3019e2c0 con 0x7fba301076e0 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba27fff640 1 -- 172.21.15.94:0/1729245795 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fba18044070 con 0x7fba301076e0 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.548+0000 7fba27fff640 1 -- 172.21.15.94:0/1729245795 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fba1803d3f0 con 0x7fba301076e0 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.549+0000 7fba35432640 1 -- 172.21.15.94:0/1729245795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba30109430 con 0x7fba301076e0 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.551+0000 7fba27fff640 1 -- 172.21.15.94:0/1729245795 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fba18046450 con 0x7fba301076e0 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.608+0000 7fba35432640 1 -- 172.21.15.94:0/1729245795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fba3019f9a0 con 0x7fba301076e0 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.612+0000 7fba27fff640 1 -- 172.21.15.94:0/1729245795 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fba18046630 con 0x7fba301076e0 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.612+0000 7fba27fff640 1 -- 172.21.15.94:0/1729245795 <== mon.0 v2:172.21.15.94:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fba18056070 con 0x7fba301076e0 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.613+0000 7fba35432640 1 -- 172.21.15.94:0/1729245795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 msgr2=0x7fba3019ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.613+0000 7fba35432640 1 --2- 172.21.15.94:0/1729245795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba3019ae00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba180090b0 tx=0x7fba180046c0 comp rx=0 tx=0).stop 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.613+0000 7fba35432640 1 -- 172.21.15.94:0/1729245795 shutdown_connections 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.613+0000 7fba35432640 1 --2- 172.21.15.94:0/1729245795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba301076e0 0x7fba3019ae00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.613+0000 7fba35432640 1 -- 172.21.15.94:0/1729245795 >> 172.21.15.94:0/1729245795 conn(0x7fba30102ed0 msgr2=0x7fba3018f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:36.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.613+0000 7fba35432640 1 -- 172.21.15.94:0/1729245795 shutdown_connections 2024-09-19T04:05:36.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:35.613+0000 7fba35432640 1 -- 172.21.15.94:0/1729245795 wait complete. 2024-09-19T04:05:36.243 INFO:teuthology.orchestra.run.smithi094.stdout:Generating new minimal ceph.conf... 2024-09-19T04:05:37.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:37 smithi094 ceph-mon[30601]: from='client.? 172.21.15.94:0/2582560608' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:05:42.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.607+0000 7f3ee70cc640 1 Processor -- start 2024-09-19T04:05:42.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.607+0000 7f3ee70cc640 1 -- start start 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.607+0000 7f3ee70cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee0107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.607+0000 7f3ee70cc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee01080b0 con 0x7f3ee01076e0 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.607+0000 7f3ee4e68640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee0107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.607+0000 7f3ee4e68640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee0107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54950/0 (socket says 172.21.15.94:54950) 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.607+0000 7f3ee4e68640 1 -- 172.21.15.94:0/1399760669 learned_addr learned my addr 172.21.15.94:0/1399760669 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.607+0000 7f3ee4e68640 1 -- 172.21.15.94:0/1399760669 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee0108840 con 0x7f3ee01076e0 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.608+0000 7f3ee4e68640 1 --2- 172.21.15.94:0/1399760669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee0107ae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009820 tx=0x7f3ed402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac791a79d39d228 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.608+0000 7f3edf7fe640 1 -- 172.21.15.94:0/1399760669 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4037070 con 0x7f3ee01076e0 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.608+0000 7f3edf7fe640 1 -- 172.21.15.94:0/1399760669 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3ed40328d0 con 0x7f3ee01076e0 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.608+0000 7f3edf7fe640 1 -- 172.21.15.94:0/1399760669 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed40307d0 con 0x7f3ee01076e0 2024-09-19T04:05:42.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.609+0000 7f3ee70cc640 1 -- 172.21.15.94:0/1399760669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 msgr2=0x7f3ee0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.609+0000 7f3ee70cc640 1 --2- 172.21.15.94:0/1399760669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee0107ae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009820 tx=0x7f3ed402a270 comp rx=0 tx=0).stop 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.609+0000 7f3ee70cc640 1 -- 172.21.15.94:0/1399760669 shutdown_connections 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.609+0000 7f3ee70cc640 1 --2- 172.21.15.94:0/1399760669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee0107ae0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.609+0000 7f3ee70cc640 1 -- 172.21.15.94:0/1399760669 >> 172.21.15.94:0/1399760669 conn(0x7f3ee0102ed0 msgr2=0x7f3ee01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.609+0000 7f3ee70cc640 1 -- 172.21.15.94:0/1399760669 shutdown_connections 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.609+0000 7f3ee70cc640 1 -- 172.21.15.94:0/1399760669 wait complete. 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.609+0000 7f3ee70cc640 1 Processor -- start 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.609+0000 7f3ee70cc640 1 -- start start 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3ee70cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3ee70cc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee010bbf0 con 0x7f3ee01076e0 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3ee4e68640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3ee4e68640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54962/0 (socket says 172.21.15.94:54962) 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3ee4e68640 1 -- 172.21.15.94:0/2582560608 learned_addr learned my addr 172.21.15.94:0/2582560608 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:42.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3ee4e68640 1 -- 172.21.15.94:0/2582560608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee007e090 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3ee4e68640 1 --2- 172.21.15.94:0/2582560608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee007db50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40097f0 tx=0x7f3ed4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3eddffb640 1 -- 172.21.15.94:0/2582560608 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4037040 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3eddffb640 1 -- 172.21.15.94:0/2582560608 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3ed4032bb0 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3eddffb640 1 -- 172.21.15.94:0/2582560608 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed403a390 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.610+0000 7f3ee70cc640 1 -- 172.21.15.94:0/2582560608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee007c280 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.611+0000 7f3ee70cc640 1 -- 172.21.15.94:0/2582560608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee007c7c0 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.611+0000 7f3eddffb640 1 -- 172.21.15.94:0/2582560608 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3ed403a580 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.611+0000 7f3eddffb640 1 -- 172.21.15.94:0/2582560608 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3ed4044b50 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.612+0000 7f3ee70cc640 1 -- 172.21.15.94:0/2582560608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee010bd70 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.614+0000 7f3eddffb640 1 -- 172.21.15.94:0/2582560608 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3ed4032d60 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.670+0000 7f3ee70cc640 1 -- 172.21.15.94:0/2582560608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3ee0107b60 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.671+0000 7f3eddffb640 1 -- 172.21.15.94:0/2582560608 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f3ed403aa00 con 0x7f3ee01076e0 2024-09-19T04:05:42.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.672+0000 7f3ee70cc640 1 -- 172.21.15.94:0/2582560608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 msgr2=0x7f3ee007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:42.927 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.672+0000 7f3ee70cc640 1 --2- 172.21.15.94:0/2582560608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee007db50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40097f0 tx=0x7f3ed4030d00 comp rx=0 tx=0).stop 2024-09-19T04:05:42.927 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.672+0000 7f3ee70cc640 1 -- 172.21.15.94:0/2582560608 shutdown_connections 2024-09-19T04:05:42.927 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.672+0000 7f3ee70cc640 1 --2- 172.21.15.94:0/2582560608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee01076e0 0x7f3ee007db50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.927 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.672+0000 7f3ee70cc640 1 -- 172.21.15.94:0/2582560608 >> 172.21.15.94:0/2582560608 conn(0x7f3ee0102ed0 msgr2=0x7f3ee0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:42.927 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.672+0000 7f3ee70cc640 1 -- 172.21.15.94:0/2582560608 shutdown_connections 2024-09-19T04:05:42.927 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:36.672+0000 7f3ee70cc640 1 -- 172.21.15.94:0/2582560608 wait complete. 2024-09-19T04:05:42.927 INFO:teuthology.orchestra.run.smithi094.stdout:Restarting the monitor... 2024-09-19T04:05:43.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:42 smithi094 systemd[1]: Stopping Ceph mon.a for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:05:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a[30597]: 2024-09-19T04:05:44.104+0000 7f33d3870640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:05:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a[30597]: 2024-09-19T04:05:44.104+0000 7f33d3870640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T04:05:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:44 smithi094 podman[30844]: 2024-09-19 04:05:44.156301525 +0000 UTC m=+1.039752534 container died a37e4c93be29cfba769d101f6270a105c2f54fadef0bc0f03c79c4035b7a1215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:05:48.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:47 smithi094 podman[30844]: 2024-09-19 04:05:47.701488984 +0000 UTC m=+4.584939990 container cleanup a37e4c93be29cfba769d101f6270a105c2f54fadef0bc0f03c79c4035b7a1215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:05:48.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:47 smithi094 bash[30844]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a 2024-09-19T04:05:49.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:49 smithi094 podman[30857]: 2024-09-19 04:05:49.585472137 +0000 UTC m=+5.420221156 container remove a37e4c93be29cfba769d101f6270a105c2f54fadef0bc0f03c79c4035b7a1215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:05:49.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:49 smithi094 systemd[1]: ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T04:05:49.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:49 smithi094 systemd[1]: Stopped Ceph mon.a for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:05:49.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:49 smithi094 systemd[1]: Starting Ceph mon.a for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:05:50.110 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 podman[30952]: 2024-09-19 04:05:50.00288111 +0000 UTC m=+0.096961043 container create cf9c5201f28527b683791423b5d1b0c262f45c959cba0fee8faf1de2ddb7ed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T04:05:50.111 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 podman[30952]: 2024-09-19 04:05:49.922336607 +0000 UTC m=+0.016416557 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:05:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T04:05:50.407 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 podman[30952]: 2024-09-19 04:05:50.213482337 +0000 UTC m=+0.307562279 container init cf9c5201f28527b683791423b5d1b0c262f45c959cba0fee8faf1de2ddb7ed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T04:05:50.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 podman[30952]: 2024-09-19 04:05:50.216792511 +0000 UTC m=+0.310872448 container start cf9c5201f28527b683791423b5d1b0c262f45c959cba0fee8faf1de2ddb7ed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:05:50.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:05:50.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:05:50.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: pidfile_write: ignore empty --pid-file 2024-09-19T04:05:50.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: load: jerasure load: lrc 2024-09-19T04:05:50.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:05:50.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Git sha 0 2024-09-19T04:05:50.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:05:50.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: DB SUMMARY 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: DB Session ID: RQ0XUAZ16JYUHDX2VE7R 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106064 ; 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:05:50.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:05:50.410 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:05:50.410 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.env: 0x55fe8043a940 2024-09-19T04:05:50.410 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:05:50.410 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.info_log: 0x55fe814a56e0 2024-09-19T04:05:50.410 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:05:50.410 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.statistics: (nil) 2024-09-19T04:05:50.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.use_fsync: 0 2024-09-19T04:05:50.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:05:50.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:05:50.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:05:50.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:05:50.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:05:50.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:05:50.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.db_log_dir: 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.wal_dir: 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:05:50.413 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.write_buffer_manager: 0x55fe814b4dc0 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.unordered_write: 0 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:05:50.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.row_cache: None 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.wal_filter: None 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.wal_compression: 0 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:05:50.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:05:50.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_open_files: -1 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:05:50.417 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Compression algorithms supported: 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: kZSTD supported: 0 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:05:50.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.merge_operator: 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_filter: None 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fe814a5320) 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:05:50.419 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:05:50.420 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:05:50.420 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_type: 0 2024-09-19T04:05:50.420 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_index_type: 0 2024-09-19T04:05:50.420 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_shortening: 1 2024-09-19T04:05:50.420 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:05:50.420 INFO:journalctl@ceph.mon.a.smithi094.stdout: checksum: 4 2024-09-19T04:05:50.420 INFO:journalctl@ceph.mon.a.smithi094.stdout: no_block_cache: 0 2024-09-19T04:05:50.420 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache: 0x55fe814991f0 2024-09-19T04:05:50.420 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_options: 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: capacity : 536870912 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_shard_bits : 4 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: strict_capacity_limit : 0 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_compressed: (nil) 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: persistent_cache: (nil) 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size: 4096 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size_deviation: 10 2024-09-19T04:05:50.421 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_restart_interval: 16 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_block_restart_interval: 1 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: metadata_block_size: 4096 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: partition_filters: 0 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: use_delta_encoding: 1 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: filter_policy: bloomfilter 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: whole_key_filtering: 1 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: verify_compression: 0 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: format_version: 5 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_index_compression: 1 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_align: 0 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: max_auto_readahead_size: 262144 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: prepopulate_block_cache: 0 2024-09-19T04:05:50.422 INFO:journalctl@ceph.mon.a.smithi094.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression: NoCompression 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.num_levels: 7 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:05:50.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:05:50.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:05:50.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:05:50.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:05:50.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.ttl: 2592000 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:05:50.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d67f6e58-e7e3-41c7-a804-08ef94f5e8f1 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726718750244042, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T04:05:50.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726718750245054, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 98919, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14506, "raw_average_key_size": 49, "raw_value_size": 91093, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726718750, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d67f6e58-e7e3-41c7-a804-08ef94f5e8f1", "db_session_id": "RQ0XUAZ16JYUHDX2VE7R", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726718750245157, "job": 1, "event": "recovery_finished"} 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fe814c6e00 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: DB pointer 0x55fe815c2000 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** DB Stats ** 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:05:50.430 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: L0 2/0 100.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: Sum 2/0 100.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 151.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:05:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative compaction: 0.00 GB write, 29.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval compaction: 0.00 GB write, 29.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache BinnedLRUCache@0x55fe814991f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-19T04:05:50.432 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: starting mon.a rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mon.a@-1(???) e1 preinit fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mon.a@-1(???).mds e1 new map 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mon.a@-1(???).mds e1 print_map 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout: e1 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout: btime 2024-09-19T04:05:34:057432+0000 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout: legacy client fscid: -1 2024-09-19T04:05:50.433 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout: No filesystems configured 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 bash[30952]: cf9c5201f28527b683791423b5d1b0c262f45c959cba0fee8faf1de2ddb7ed88 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 systemd[1]: Started Ceph mon.a for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: monmap epoch 1 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: last_changed 2024-09-19T04:05:31.044155+0000 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:05:50.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-19T04:05:50.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: election_strategy: 1 2024-09-19T04:05:50.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:05:50.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: fsmap 2024-09-19T04:05:50.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:05:50.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:50 smithi094 ceph-mon[30972]: mgrmap e1: no daemons active 2024-09-19T04:05:51.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.681+0000 7f9f7b61d640 1 Processor -- start 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.681+0000 7f9f7b61d640 1 -- start start 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.681+0000 7f9f7b61d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.681+0000 7f9f7b61d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74106f40 con 0x7f9f74106570 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.682+0000 7f9f793b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.682+0000 7f9f793b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45456/0 (socket says 172.21.15.94:45456) 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.682+0000 7f9f793b9640 1 -- 172.21.15.94:0/1639604323 learned_addr learned my addr 172.21.15.94:0/1639604323 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.682+0000 7f9f793b9640 1 -- 172.21.15.94:0/1639604323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f74107720 con 0x7f9f74106570 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.684+0000 7f9f793b9640 1 --2- 172.21.15.94:0/1639604323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74106970 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000bac0 tx=0x7f9f7002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2509816a92c6fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.684+0000 7f9f6bfff640 1 -- 172.21.15.94:0/1639604323 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f700326d0 con 0x7f9f74106570 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.684+0000 7f9f6bfff640 1 -- 172.21.15.94:0/1639604323 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9f70038070 con 0x7f9f74106570 2024-09-19T04:05:51.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.684+0000 7f9f6bfff640 1 -- 172.21.15.94:0/1639604323 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70033c90 con 0x7f9f74106570 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.685+0000 7f9f7b61d640 1 -- 172.21.15.94:0/1639604323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 msgr2=0x7f9f74106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.685+0000 7f9f7b61d640 1 --2- 172.21.15.94:0/1639604323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74106970 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000bac0 tx=0x7f9f7002c920 comp rx=0 tx=0).stop 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.685+0000 7f9f7b61d640 1 -- 172.21.15.94:0/1639604323 shutdown_connections 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.685+0000 7f9f7b61d640 1 --2- 172.21.15.94:0/1639604323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74106970 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.685+0000 7f9f7b61d640 1 -- 172.21.15.94:0/1639604323 >> 172.21.15.94:0/1639604323 conn(0x7f9f74101d80 msgr2=0x7f9f741041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.685+0000 7f9f7b61d640 1 -- 172.21.15.94:0/1639604323 shutdown_connections 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.685+0000 7f9f7b61d640 1 -- 172.21.15.94:0/1639604323 wait complete. 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.686+0000 7f9f7b61d640 1 Processor -- start 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.686+0000 7f9f7b61d640 1 -- start start 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.686+0000 7f9f7b61d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.686+0000 7f9f7b61d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7407afb0 con 0x7f9f74106570 2024-09-19T04:05:51.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.686+0000 7f9f793b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.686+0000 7f9f793b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45458/0 (socket says 172.21.15.94:45458) 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.686+0000 7f9f793b9640 1 -- 172.21.15.94:0/518775537 learned_addr learned my addr 172.21.15.94:0/518775537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.686+0000 7f9f793b9640 1 -- 172.21.15.94:0/518775537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f741999c0 con 0x7f9f74106570 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.687+0000 7f9f793b9640 1 --2- 172.21.15.94:0/518775537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74199480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000b2c0 tx=0x7f9f700327d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.687+0000 7f9f6a7fc640 1 -- 172.21.15.94:0/518775537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70042070 con 0x7f9f74106570 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.687+0000 7f9f6a7fc640 1 -- 172.21.15.94:0/518775537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9f70032bf0 con 0x7f9f74106570 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.687+0000 7f9f7b61d640 1 -- 172.21.15.94:0/518775537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f74199c50 con 0x7f9f74106570 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.687+0000 7f9f6a7fc640 1 -- 172.21.15.94:0/518775537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f7003c580 con 0x7f9f74106570 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.687+0000 7f9f7b61d640 1 -- 172.21.15.94:0/518775537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7419c940 con 0x7f9f74106570 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.687+0000 7f9f6a7fc640 1 -- 172.21.15.94:0/518775537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f70005990 con 0x7f9f74106570 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.687+0000 7f9f6a7fc640 1 -- 172.21.15.94:0/518775537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f70030050 con 0x7f9f74106570 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.688+0000 7f9f7b61d640 1 -- 172.21.15.94:0/518775537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f741069f0 con 0x7f9f74106570 2024-09-19T04:05:51.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.690+0000 7f9f6a7fc640 1 -- 172.21.15.94:0/518775537 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9f70054020 con 0x7f9f74106570 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.747+0000 7f9f7b61d640 1 -- 172.21.15.94:0/518775537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f9f741928f0 con 0x7f9f74106570 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.748+0000 7f9f6a7fc640 1 -- 172.21.15.94:0/518775537 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9f70032da0 con 0x7f9f74106570 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.749+0000 7f9f7b61d640 1 -- 172.21.15.94:0/518775537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 msgr2=0x7f9f74199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.749+0000 7f9f7b61d640 1 --2- 172.21.15.94:0/518775537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74199480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000b2c0 tx=0x7f9f700327d0 comp rx=0 tx=0).stop 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.749+0000 7f9f7b61d640 1 -- 172.21.15.94:0/518775537 shutdown_connections 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.749+0000 7f9f7b61d640 1 --2- 172.21.15.94:0/518775537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f74106570 0x7f9f74199480 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.749+0000 7f9f7b61d640 1 -- 172.21.15.94:0/518775537 >> 172.21.15.94:0/518775537 conn(0x7f9f74101d80 msgr2=0x7f9f7418fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.749+0000 7f9f7b61d640 1 -- 172.21.15.94:0/518775537 shutdown_connections 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:50.749+0000 7f9f7b61d640 1 -- 172.21.15.94:0/518775537 wait complete. 2024-09-19T04:05:51.330 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T04:05:51.345 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:05:51.345 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mgr... 2024-09-19T04:05:51.345 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T04:05:51.346 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T04:05:51.603 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.x 2024-09-19T04:05:51.603 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T04:05:51.751 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-503c4526-763c-11ef-bceb-c7b262605968.target.wants/ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-503c4526-763c-11ef-bceb-c7b262605968@.service. 2024-09-19T04:05:52.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:51 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/518775537' entity='client.admin' 2024-09-19T04:05:52.255 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:05:52.255 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-09-19T04:05:52.263 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:05:52.263 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-09-19T04:05:52.263 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-09-19T04:05:52.263 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:05:52.271 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:05:52.271 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-09-19T04:05:52.279 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:05:52.279 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-09-19T04:05:52.279 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-09-19T04:05:52.279 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T04:05:52.279 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr to start... 2024-09-19T04:05:52.279 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr... 2024-09-19T04:05:53.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:52 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3601525045' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:05:53.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:52.862+0000 7fc28867f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:05:53.390 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:53 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:53.185+0000 7fc28867f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:05:53.390 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:53 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:53.247+0000 7fc28867f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "503c4526-763c-11ef-bceb-c7b262605968", 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:05:53.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:05:53.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:05:53.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:05:53.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:05:34:057432+0000", 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:05:53.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:05:34.058042+0000", 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:05:53.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.729+0000 7fb6cb6ce640 1 Processor -- start 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.729+0000 7fb6cb6ce640 1 -- start start 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.730+0000 7fb6cb6ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c407bcd0 0x7fb6c410a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.730+0000 7fb6cb6ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c410a7a0 con 0x7fb6c407bcd0 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.730+0000 7fb6c946a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c407bcd0 0x7fb6c410a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.730+0000 7fb6c946a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c407bcd0 0x7fb6c410a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52094/0 (socket says 172.21.15.94:52094) 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.730+0000 7fb6c946a640 1 -- 172.21.15.94:0/3977261596 learned_addr learned my addr 172.21.15.94:0/3977261596 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.730+0000 7fb6c946a640 1 -- 172.21.15.94:0/3977261596 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c410a920 con 0x7fb6c407bcd0 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.731+0000 7fb6c946a640 1 --2- 172.21.15.94:0/3977261596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c407bcd0 0x7fb6c410a1d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009820 tx=0x7fb6b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6a49adee6e199e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.731+0000 7fb6b7fff640 1 -- 172.21.15.94:0/3977261596 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b8037070 con 0x7fb6c407bcd0 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.731+0000 7fb6b7fff640 1 -- 172.21.15.94:0/3977261596 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb6b80328d0 con 0x7fb6c407bcd0 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.731+0000 7fb6b7fff640 1 -- 172.21.15.94:0/3977261596 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b80307d0 con 0x7fb6c407bcd0 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.732+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3977261596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c407bcd0 msgr2=0x7fb6c410a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:53.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.732+0000 7fb6cb6ce640 1 --2- 172.21.15.94:0/3977261596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c407bcd0 0x7fb6c410a1d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009820 tx=0x7fb6b802a270 comp rx=0 tx=0).stop 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.732+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3977261596 shutdown_connections 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.732+0000 7fb6cb6ce640 1 --2- 172.21.15.94:0/3977261596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c407bcd0 0x7fb6c410a1d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.732+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3977261596 >> 172.21.15.94:0/3977261596 conn(0x7fb6c4101d80 msgr2=0x7fb6c41041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.732+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3977261596 shutdown_connections 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.732+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3977261596 wait complete. 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.733+0000 7fb6cb6ce640 1 Processor -- start 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.733+0000 7fb6cb6ce640 1 -- start start 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.733+0000 7fb6cb6ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c419b5f0 0x7fb6c419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.733+0000 7fb6cb6ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c410b9c0 con 0x7fb6c419b5f0 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.733+0000 7fb6c946a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c419b5f0 0x7fb6c419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.733+0000 7fb6c946a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c419b5f0 0x7fb6c419ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52096/0 (socket says 172.21.15.94:52096) 2024-09-19T04:05:53.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.733+0000 7fb6c946a640 1 -- 172.21.15.94:0/3601525045 learned_addr learned my addr 172.21.15.94:0/3601525045 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.734+0000 7fb6c946a640 1 -- 172.21.15.94:0/3601525045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c419bf50 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.734+0000 7fb6c946a640 1 --2- 172.21.15.94:0/3601525045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c419b5f0 0x7fb6c419ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8030520 tx=0x7fb6b8032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.734+0000 7fb6b67fc640 1 -- 172.21.15.94:0/3601525045 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b8042070 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.734+0000 7fb6b67fc640 1 -- 172.21.15.94:0/3601525045 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb6b8030cc0 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.734+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3601525045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c419c1e0 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.734+0000 7fb6b67fc640 1 -- 172.21.15.94:0/3601525045 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b8037040 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.734+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3601525045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c419ecc0 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.734+0000 7fb6b67fc640 1 -- 172.21.15.94:0/3601525045 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb6b8045070 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.734+0000 7fb6b67fc640 1 -- 172.21.15.94:0/3601525045 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb6b8047a30 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.735+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3601525045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c4108240 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.737+0000 7fb6b67fc640 1 -- 172.21.15.94:0/3601525045 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb6b803d430 con 0x7fb6c419b5f0 2024-09-19T04:05:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.796+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3601525045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb6c407bdc0 con 0x7fb6c419b5f0 2024-09-19T04:05:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.797+0000 7fb6b67fc640 1 -- 172.21.15.94:0/3601525045 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fb6b803d610 con 0x7fb6c419b5f0 2024-09-19T04:05:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.798+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3601525045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c419b5f0 msgr2=0x7fb6c419ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.798+0000 7fb6cb6ce640 1 --2- 172.21.15.94:0/3601525045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c419b5f0 0x7fb6c419ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8030520 tx=0x7fb6b8032dd0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.798+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3601525045 shutdown_connections 2024-09-19T04:05:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.798+0000 7fb6cb6ce640 1 --2- 172.21.15.94:0/3601525045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c419b5f0 0x7fb6c419ba10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.798+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3601525045 >> 172.21.15.94:0/3601525045 conn(0x7fb6c4101d80 msgr2=0x7fb6c41029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.798+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3601525045 shutdown_connections 2024-09-19T04:05:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:52.798+0000 7fb6cb6ce640 1 -- 172.21.15.94:0/3601525045 wait complete. 2024-09-19T04:05:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (1/15)... 2024-09-19T04:05:53.840 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:53 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:53.437+0000 7fc28867f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:05:54.590 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:54 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:54.309+0000 7fc28867f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:05:54.590 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:54 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:54.376+0000 7fc28867f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:05:55.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:55.093+0000 7fc28867f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:05:55.350 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:55.207+0000 7fc28867f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:05:55.350 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:55.348+0000 7fc28867f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:05:55.634 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:55.409+0000 7fc28867f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:05:55.634 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:55.565+0000 7fc28867f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:05:55.634 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:55.632+0000 7fc28867f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:05:55.928 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:55.695+0000 7fc28867f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:05:55.928 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:55.858+0000 7fc28867f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:05:55.928 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:55.926+0000 7fc28867f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:05:56.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:55 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3118461818' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:05:56.224 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:56.012+0000 7fc28867f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:05:56.224 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:56.111+0000 7fc28867f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:05:56.224 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:56.171+0000 7fc28867f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:05:56.524 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:56.231+0000 7fc28867f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:05:56.536 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:05:56.536 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:05:56.536 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "503c4526-763c-11ef-bceb-c7b262605968", 2024-09-19T04:05:56.536 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:05:56.536 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:05:56.537 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:56.538 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:05:34:057432+0000", 2024-09-19T04:05:56.539 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:05:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:05:34.058042+0000", 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.862+0000 7f9b52768640 1 Processor -- start 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.862+0000 7f9b52768640 1 -- start start 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.862+0000 7f9b52768640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c10a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.862+0000 7f9b52768640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c07be40 con 0x7f9b4c1080e0 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.862+0000 7f9b4bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c10a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.862+0000 7f9b4bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c10a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52110/0 (socket says 172.21.15.94:52110) 2024-09-19T04:05:56.541 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.862+0000 7f9b4bfff640 1 -- 172.21.15.94:0/1136166890 learned_addr learned my addr 172.21.15.94:0/1136166890 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.863+0000 7f9b4bfff640 1 -- 172.21.15.94:0/1136166890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c10aa10 con 0x7f9b4c1080e0 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.863+0000 7f9b4bfff640 1 --2- 172.21.15.94:0/1136166890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c10a4d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009820 tx=0x7f9b3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f281857d60c399aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.863+0000 7f9b4affd640 1 -- 172.21.15.94:0/1136166890 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c037070 con 0x7f9b4c1080e0 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.864+0000 7f9b4affd640 1 -- 172.21.15.94:0/1136166890 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b3c0328d0 con 0x7f9b4c1080e0 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.864+0000 7f9b4affd640 1 -- 172.21.15.94:0/1136166890 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c0307d0 con 0x7f9b4c1080e0 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.864+0000 7f9b52768640 1 -- 172.21.15.94:0/1136166890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 msgr2=0x7f9b4c10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.864+0000 7f9b52768640 1 --2- 172.21.15.94:0/1136166890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c10a4d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009820 tx=0x7f9b3c02a270 comp rx=0 tx=0).stop 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.864+0000 7f9b52768640 1 -- 172.21.15.94:0/1136166890 shutdown_connections 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.864+0000 7f9b52768640 1 --2- 172.21.15.94:0/1136166890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c10a4d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.864+0000 7f9b52768640 1 -- 172.21.15.94:0/1136166890 >> 172.21.15.94:0/1136166890 conn(0x7f9b4c101d80 msgr2=0x7f9b4c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.864+0000 7f9b52768640 1 -- 172.21.15.94:0/1136166890 shutdown_connections 2024-09-19T04:05:56.542 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.864+0000 7f9b52768640 1 -- 172.21.15.94:0/1136166890 wait complete. 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.865+0000 7f9b52768640 1 Processor -- start 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.865+0000 7f9b52768640 1 -- start start 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.865+0000 7f9b52768640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c107960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.865+0000 7f9b52768640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c10b9f0 con 0x7f9b4c1080e0 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.865+0000 7f9b4bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.865+0000 7f9b4bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52120/0 (socket says 172.21.15.94:52120) 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.865+0000 7f9b4bfff640 1 -- 172.21.15.94:0/3118461818 learned_addr learned my addr 172.21.15.94:0/3118461818 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.866+0000 7f9b4bfff640 1 -- 172.21.15.94:0/3118461818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c107ea0 con 0x7f9b4c1080e0 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.866+0000 7f9b4bfff640 1 --2- 172.21.15.94:0/3118461818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c107960 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c032e10 tx=0x7f9b3c032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.866+0000 7f9b497fa640 1 -- 172.21.15.94:0/3118461818 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c042070 con 0x7f9b4c1080e0 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.866+0000 7f9b497fa640 1 -- 172.21.15.94:0/3118461818 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b3c030c80 con 0x7f9b4c1080e0 2024-09-19T04:05:56.543 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.866+0000 7f9b52768640 1 -- 172.21.15.94:0/3118461818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4c106090 con 0x7f9b4c1080e0 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.866+0000 7f9b497fa640 1 -- 172.21.15.94:0/3118461818 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c037040 con 0x7f9b4c1080e0 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.866+0000 7f9b52768640 1 -- 172.21.15.94:0/3118461818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4c1065d0 con 0x7f9b4c1080e0 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.866+0000 7f9b497fa640 1 -- 172.21.15.94:0/3118461818 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b3c045070 con 0x7f9b4c1080e0 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.866+0000 7f9b497fa640 1 -- 172.21.15.94:0/3118461818 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b3c0479e0 con 0x7f9b4c1080e0 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.867+0000 7f9b52768640 1 -- 172.21.15.94:0/3118461818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b4c108540 con 0x7f9b4c1080e0 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.869+0000 7f9b497fa640 1 -- 172.21.15.94:0/3118461818 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9b3c047be0 con 0x7f9b4c1080e0 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.931+0000 7f9b52768640 1 -- 172.21.15.94:0/3118461818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9b4c106910 con 0x7f9b4c1080e0 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.931+0000 7f9b497fa640 1 -- 172.21.15.94:0/3118461818 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f9b3c03c5f0 con 0x7f9b4c1080e0 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.933+0000 7f9b52768640 1 -- 172.21.15.94:0/3118461818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 msgr2=0x7f9b4c107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.933+0000 7f9b52768640 1 --2- 172.21.15.94:0/3118461818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c107960 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c032e10 tx=0x7f9b3c032e40 comp rx=0 tx=0).stop 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.933+0000 7f9b52768640 1 -- 172.21.15.94:0/3118461818 shutdown_connections 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.933+0000 7f9b52768640 1 --2- 172.21.15.94:0/3118461818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b4c1080e0 0x7f9b4c107960 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:56.544 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.933+0000 7f9b52768640 1 -- 172.21.15.94:0/3118461818 >> 172.21.15.94:0/3118461818 conn(0x7f9b4c101d80 msgr2=0x7f9b4c102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:56.545 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.933+0000 7f9b52768640 1 -- 172.21.15.94:0/3118461818 shutdown_connections 2024-09-19T04:05:56.545 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:55.933+0000 7f9b52768640 1 -- 172.21.15.94:0/3118461818 wait complete. 2024-09-19T04:05:56.545 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (2/15)... 2024-09-19T04:05:57.340 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:05:57.019+0000 7fc28867f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: Activating manager daemon x 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: mgrmap e2: x(active, starting, since 0.00322404s) 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: Manager daemon x is now available 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:05:57.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' 2024-09-19T04:05:57.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' 2024-09-19T04:05:57.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:57 smithi094 ceph-mon[30972]: from='mgr.14100 172.21.15.94:0/3009908638' entity='mgr.x' 2024-09-19T04:05:59.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:05:59 smithi094 ceph-mon[30972]: mgrmap e3: x(active, since 1.00622s) 2024-09-19T04:05:59.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:05:59.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:05:59.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "503c4526-763c-11ef-bceb-c7b262605968", 2024-09-19T04:05:59.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:05:59.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:59.820 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:05:59.820 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:05:59.820 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:59.820 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:05:59.820 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:59.820 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:05:59.820 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:05:59.820 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:05:59.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:05:59.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:05:59.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:05:59.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:59.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:05:59.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:05:34:057432+0000", 2024-09-19T04:05:59.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:05:59.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:05:34.058042+0000", 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.965+0000 7f3f8f6be640 1 Processor -- start 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.965+0000 7f3f8f6be640 1 -- start start 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.965+0000 7f3f8f6be640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f88107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.965+0000 7f3f8f6be640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f881080b0 con 0x7f3f881076e0 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.965+0000 7f3f8d45a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f88107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.965+0000 7f3f8d45a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f88107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52174/0 (socket says 172.21.15.94:52174) 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.965+0000 7f3f8d45a640 1 -- 172.21.15.94:0/2707351446 learned_addr learned my addr 172.21.15.94:0/2707351446 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:59.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.965+0000 7f3f8d45a640 1 -- 172.21.15.94:0/2707351446 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f88108840 con 0x7f3f881076e0 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.966+0000 7f3f8d45a640 1 --2- 172.21.15.94:0/2707351446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f88107ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009820 tx=0x7f3f8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1ead441d89896b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.966+0000 7f3f7ffff640 1 -- 172.21.15.94:0/2707351446 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f84037070 con 0x7f3f881076e0 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.966+0000 7f3f7ffff640 1 -- 172.21.15.94:0/2707351446 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3f840328d0 con 0x7f3f881076e0 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.966+0000 7f3f7ffff640 1 -- 172.21.15.94:0/2707351446 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f840307d0 con 0x7f3f881076e0 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.967+0000 7f3f8f6be640 1 -- 172.21.15.94:0/2707351446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 msgr2=0x7f3f88107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.967+0000 7f3f8f6be640 1 --2- 172.21.15.94:0/2707351446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f88107ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009820 tx=0x7f3f8402a270 comp rx=0 tx=0).stop 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.967+0000 7f3f8f6be640 1 -- 172.21.15.94:0/2707351446 shutdown_connections 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.967+0000 7f3f8f6be640 1 --2- 172.21.15.94:0/2707351446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f88107ae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.967+0000 7f3f8f6be640 1 -- 172.21.15.94:0/2707351446 >> 172.21.15.94:0/2707351446 conn(0x7f3f88102ed0 msgr2=0x7f3f881052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.967+0000 7f3f8f6be640 1 -- 172.21.15.94:0/2707351446 shutdown_connections 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.967+0000 7f3f8f6be640 1 -- 172.21.15.94:0/2707351446 wait complete. 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.968+0000 7f3f8f6be640 1 Processor -- start 2024-09-19T04:05:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.968+0000 7f3f8f6be640 1 -- start start 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.968+0000 7f3f8f6be640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f8819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.968+0000 7f3f8f6be640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8810bbf0 con 0x7f3f881076e0 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.968+0000 7f3f8d45a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f8819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.968+0000 7f3f8d45a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f8819b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52190/0 (socket says 172.21.15.94:52190) 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.968+0000 7f3f8d45a640 1 -- 172.21.15.94:0/3018270555 learned_addr learned my addr 172.21.15.94:0/3018270555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.968+0000 7f3f8d45a640 1 -- 172.21.15.94:0/3018270555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8819bb50 con 0x7f3f881076e0 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.968+0000 7f3f8d45a640 1 --2- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f8819b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009940 tx=0x7f3f84032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.969+0000 7f3f7e7fc640 1 -- 172.21.15.94:0/3018270555 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f84042070 con 0x7f3f881076e0 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.969+0000 7f3f7e7fc640 1 -- 172.21.15.94:0/3018270555 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3f84030c80 con 0x7f3f881076e0 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.969+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8819bde0 con 0x7f3f881076e0 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.969+0000 7f3f7e7fc640 1 -- 172.21.15.94:0/3018270555 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f84037040 con 0x7f3f881076e0 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.969+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8819ead0 con 0x7f3f881076e0 2024-09-19T04:05:59.833 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.970+0000 7f3f7e7fc640 1 -- 172.21.15.94:0/3018270555 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7f3f84045070 con 0x7f3f881076e0 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.970+0000 7f3f7e7fc640 1 --2- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f3f740390a0 0x7f3f7403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.970+0000 7f3f7e7fc640 1 -- 172.21.15.94:0/3018270555 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3f8406bf20 con 0x7f3f881076e0 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.970+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8810bd70 con 0x7f3f881076e0 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.970+0000 7f3f8cc59640 1 --2- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f3f740390a0 0x7f3f7403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.970+0000 7f3f8cc59640 1 --2- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f3f740390a0 0x7f3f7403b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f780098c0 tx=0x7f3f78006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:58.975+0000 7f3f7e7fc640 1 -- 172.21.15.94:0/3018270555 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f8409b210 con 0x7f3f881076e0 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.027+0000 7f3f7e7fc640 1 -- 172.21.15.94:0/3018270555 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f3f8403cac0 con 0x7f3f881076e0 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.208+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3f8819eca0 con 0x7f3f881076e0 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.209+0000 7f3f7e7fc640 1 -- 172.21.15.94:0/3018270555 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f3f84045360 con 0x7f3f881076e0 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f3f740390a0 msgr2=0x7f3f7403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 --2- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f3f740390a0 0x7f3f7403b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f780098c0 tx=0x7f3f78006db0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.834 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 msgr2=0x7f3f8819b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 --2- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f8819b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009940 tx=0x7f3f84032e10 comp rx=0 tx=0).stop 2024-09-19T04:05:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 shutdown_connections 2024-09-19T04:05:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 --2- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f3f740390a0 0x7f3f7403b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 --2- 172.21.15.94:0/3018270555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f881076e0 0x7f3f8819b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 >> 172.21.15.94:0/3018270555 conn(0x7f3f88102ed0 msgr2=0x7f3f88103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 shutdown_connections 2024-09-19T04:05:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:05:59.211+0000 7f3f8f6be640 1 -- 172.21.15.94:0/3018270555 wait complete. 2024-09-19T04:05:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:mgr is available 2024-09-19T04:06:00.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:00 smithi094 ceph-mon[30972]: mgrmap e4: x(active, since 2s) 2024-09-19T04:06:00.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:00 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3018270555' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:06:00.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:06:00.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:06:00.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:06:00.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-09-19T04:06:00.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:06:00.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:06:00.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.199+0000 7f5ebc0bf640 1 Processor -- start 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.199+0000 7f5ebc0bf640 1 -- start start 2024-09-19T04:06:00.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.199+0000 7f5ebc0bf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb4107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.199+0000 7f5ebc0bf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41080b0 con 0x7f5eb41076e0 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.199+0000 7f5eb9e5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb4107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.199+0000 7f5eb9e5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb4107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52196/0 (socket says 172.21.15.94:52196) 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.199+0000 7f5eb9e5b640 1 -- 172.21.15.94:0/341082024 learned_addr learned my addr 172.21.15.94:0/341082024 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.199+0000 7f5eb9e5b640 1 -- 172.21.15.94:0/341082024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb4108840 con 0x7f5eb41076e0 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.200+0000 7f5eb9e5b640 1 --2- 172.21.15.94:0/341082024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb4107ae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009820 tx=0x7f5ea802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e88aebdc503ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.200+0000 7f5eb8e59640 1 -- 172.21.15.94:0/341082024 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ea8037070 con 0x7f5eb41076e0 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.200+0000 7f5eb8e59640 1 -- 172.21.15.94:0/341082024 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5ea80328d0 con 0x7f5eb41076e0 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.200+0000 7f5eb8e59640 1 -- 172.21.15.94:0/341082024 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ea80307d0 con 0x7f5eb41076e0 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.201+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/341082024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 msgr2=0x7f5eb4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.201+0000 7f5ebc0bf640 1 --2- 172.21.15.94:0/341082024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb4107ae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009820 tx=0x7f5ea802a270 comp rx=0 tx=0).stop 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.201+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/341082024 shutdown_connections 2024-09-19T04:06:00.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.201+0000 7f5ebc0bf640 1 --2- 172.21.15.94:0/341082024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb4107ae0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.201+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/341082024 >> 172.21.15.94:0/341082024 conn(0x7f5eb4102ed0 msgr2=0x7f5eb41052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.201+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/341082024 shutdown_connections 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.201+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/341082024 wait complete. 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5ebc0bf640 1 Processor -- start 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5ebc0bf640 1 -- start start 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5ebc0bf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5ebc0bf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb410bbf0 con 0x7f5eb41076e0 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5eb9e5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5eb9e5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb419b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52202/0 (socket says 172.21.15.94:52202) 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5eb9e5b640 1 -- 172.21.15.94:0/2374734940 learned_addr learned my addr 172.21.15.94:0/2374734940 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5eb9e5b640 1 -- 172.21.15.94:0/2374734940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb419bb50 con 0x7f5eb41076e0 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5eb9e5b640 1 --2- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb419b610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009940 tx=0x7f5ea8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5ea6ffd640 1 -- 172.21.15.94:0/2374734940 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ea8042070 con 0x7f5eb41076e0 2024-09-19T04:06:00.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5ea6ffd640 1 -- 172.21.15.94:0/2374734940 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5ea8030c80 con 0x7f5eb41076e0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb419bde0 con 0x7f5eb41076e0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.202+0000 7f5ea6ffd640 1 -- 172.21.15.94:0/2374734940 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ea8037040 con 0x7f5eb41076e0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.203+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb419ead0 con 0x7f5eb41076e0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.203+0000 7f5ea6ffd640 1 -- 172.21.15.94:0/2374734940 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f5ea8045070 con 0x7f5eb41076e0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.203+0000 7f5ea6ffd640 1 --2- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f5ea0039160 0x7f5ea003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.203+0000 7f5ea6ffd640 1 -- 172.21.15.94:0/2374734940 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ea806c0a0 con 0x7f5eb41076e0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.203+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb410bd70 con 0x7f5eb41076e0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.203+0000 7f5eb965a640 1 --2- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f5ea0039160 0x7f5ea003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.204+0000 7f5eb965a640 1 --2- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f5ea0039160 0x7f5ea003b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00098c0 tx=0x7f5eb0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.208+0000 7f5ea6ffd640 1 -- 172.21.15.94:0/2374734940 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5ea8045360 con 0x7f5eb41076e0 2024-09-19T04:06:00.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.372+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5eb4107b60 con 0x7f5eb41076e0 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.373+0000 7f5ea6ffd640 1 -- 172.21.15.94:0/2374734940 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f5ea802f2a0 con 0x7f5eb41076e0 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.375+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f5ea0039160 msgr2=0x7f5ea003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.375+0000 7f5ebc0bf640 1 --2- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f5ea0039160 0x7f5ea003b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00098c0 tx=0x7f5eb0006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.375+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 msgr2=0x7f5eb419b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.375+0000 7f5ebc0bf640 1 --2- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb419b610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009940 tx=0x7f5ea8032e10 comp rx=0 tx=0).stop 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.375+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 shutdown_connections 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.375+0000 7f5ebc0bf640 1 --2- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f5ea0039160 0x7f5ea003b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.375+0000 7f5ebc0bf640 1 --2- 172.21.15.94:0/2374734940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb41076e0 0x7f5eb419b610 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.375+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 >> 172.21.15.94:0/2374734940 conn(0x7f5eb4102ed0 msgr2=0x7f5eb4103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.376+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 shutdown_connections 2024-09-19T04:06:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:00.376+0000 7f5ebc0bf640 1 -- 172.21.15.94:0/2374734940 wait complete. 2024-09-19T04:06:00.978 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling cephadm module... 2024-09-19T04:06:01.230 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:01 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2374734940' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:06:02.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:02 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/892268772' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T04:06:02.341 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: ignoring --setuser ceph since I am not root 2024-09-19T04:06:02.341 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: ignoring --setgroup ceph since I am not root 2024-09-19T04:06:02.341 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:02.173+0000 7ff4b3e86040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:06:02.341 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:02.229+0000 7ff4b3e86040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:06:02.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:02.377+0000 7ff4b3e86040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:06:02.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:02.443+0000 7ff4b3e86040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:06:02.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:02.589+0000 7ff4b3e86040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:06:02.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.399+0000 7f86bacb9640 1 Processor -- start 2024-09-19T04:06:02.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.399+0000 7f86bacb9640 1 -- start start 2024-09-19T04:06:02.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.399+0000 7f86bacb9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b4105d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:02.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.399+0000 7f86bacb9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86b4106310 con 0x7f86b4105940 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.400+0000 7f86b8a55640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b4105d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.400+0000 7f86b8a55640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b4105d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52214/0 (socket says 172.21.15.94:52214) 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.400+0000 7f86b8a55640 1 -- 172.21.15.94:0/1836457235 learned_addr learned my addr 172.21.15.94:0/1836457235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.400+0000 7f86b8a55640 1 -- 172.21.15.94:0/1836457235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b4108920 con 0x7f86b4105940 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.400+0000 7f86b8a55640 1 --2- 172.21.15.94:0/1836457235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b4105d40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009820 tx=0x7f86a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf3d6e7fb758772 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86b37fe640 1 -- 172.21.15.94:0/1836457235 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86a4037070 con 0x7f86b4105940 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86b37fe640 1 -- 172.21.15.94:0/1836457235 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f86a40328d0 con 0x7f86b4105940 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86b37fe640 1 -- 172.21.15.94:0/1836457235 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86a40307d0 con 0x7f86b4105940 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86bacb9640 1 -- 172.21.15.94:0/1836457235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 msgr2=0x7f86b4105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86bacb9640 1 --2- 172.21.15.94:0/1836457235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b4105d40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009820 tx=0x7f86a402a270 comp rx=0 tx=0).stop 2024-09-19T04:06:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86bacb9640 1 -- 172.21.15.94:0/1836457235 shutdown_connections 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86bacb9640 1 --2- 172.21.15.94:0/1836457235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b4105d40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86bacb9640 1 -- 172.21.15.94:0/1836457235 >> 172.21.15.94:0/1836457235 conn(0x7f86b4101490 msgr2=0x7f86b41038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86bacb9640 1 -- 172.21.15.94:0/1836457235 shutdown_connections 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.401+0000 7f86bacb9640 1 -- 172.21.15.94:0/1836457235 wait complete. 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.402+0000 7f86bacb9640 1 Processor -- start 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.402+0000 7f86bacb9640 1 -- start start 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.402+0000 7f86bacb9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.402+0000 7f86bacb9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86b410bb50 con 0x7f86b4105940 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.402+0000 7f86b8a55640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.402+0000 7f86b8a55640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b419b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52226/0 (socket says 172.21.15.94:52226) 2024-09-19T04:06:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.402+0000 7f86b8a55640 1 -- 172.21.15.94:0/892268772 learned_addr learned my addr 172.21.15.94:0/892268772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.402+0000 7f86b8a55640 1 -- 172.21.15.94:0/892268772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b419bbe0 con 0x7f86b4105940 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.403+0000 7f86b8a55640 1 --2- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b419b6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f86a40097f0 tx=0x7f86a4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.403+0000 7f86b1ffb640 1 -- 172.21.15.94:0/892268772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86a4037040 con 0x7f86b4105940 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.403+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b419be70 con 0x7f86b4105940 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.403+0000 7f86b1ffb640 1 -- 172.21.15.94:0/892268772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f86a4032bb0 con 0x7f86b4105940 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.403+0000 7f86b1ffb640 1 -- 172.21.15.94:0/892268772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86a403a390 con 0x7f86b4105940 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.403+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b419eb60 con 0x7f86b4105940 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.404+0000 7f86b1ffb640 1 -- 172.21.15.94:0/892268772 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f86a403b3f0 con 0x7f86b4105940 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.404+0000 7f86b1ffb640 1 --2- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f869c039160 0x7f869c03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.404+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b4105dc0 con 0x7f86b4105940 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.404+0000 7f86b1ffb640 1 -- 172.21.15.94:0/892268772 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f86a406c420 con 0x7f86b4105940 2024-09-19T04:06:02.722 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.404+0000 7f86b3fff640 1 --2- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f869c039160 0x7f869c03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.404+0000 7f86b3fff640 1 --2- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f869c039160 0x7f869c03b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86a80098c0 tx=0x7f86a8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.409+0000 7f86b1ffb640 1 -- 172.21.15.94:0/892268772 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86a4032d60 con 0x7f86b4105940 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:01.621+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f86b419ed30 con 0x7f86b4105940 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.030+0000 7f86b1ffb640 1 -- 172.21.15.94:0/892268772 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f86b419ed30 con 0x7f86b4105940 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.032+0000 7f86b1ffb640 1 -- 172.21.15.94:0/892268772 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f86a403a5d0 con 0x7f86b4105940 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f869c039160 msgr2=0x7f869c03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 --2- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f869c039160 0x7f869c03b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86a80098c0 tx=0x7f86a8006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 msgr2=0x7f86b419b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 --2- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b419b6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f86a40097f0 tx=0x7f86a4030d00 comp rx=0 tx=0).stop 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 shutdown_connections 2024-09-19T04:06:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 --2- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f869c039160 0x7f869c03b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.724 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 --2- 172.21.15.94:0/892268772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b4105940 0x7f86b419b6a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.724 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 >> 172.21.15.94:0/892268772 conn(0x7f86b4101490 msgr2=0x7f86b4102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:02.724 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 shutdown_connections 2024-09-19T04:06:02.724 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:02.034+0000 7f86bacb9640 1 -- 172.21.15.94:0/892268772 wait complete. 2024-09-19T04:06:03.016 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:02.731+0000 7ff4b3e86040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:06:03.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:03 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/892268772' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T04:06:03.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:03 smithi094 ceph-mon[30972]: mgrmap e5: x(active, since 5s) 2024-09-19T04:06:03.301 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:03.051+0000 7ff4b3e86040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:06:03.301 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:03.116+0000 7ff4b3e86040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:06:03.590 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:03.304+0000 7ff4b3e86040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:06:03.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.092+0000 7f51c0fc5640 1 Processor -- start 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.092+0000 7f51c0fc5640 1 -- start start 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.092+0000 7f51c0fc5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc07bcd0 0x7f51bc10a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.092+0000 7f51c0fc5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc10a7a0 con 0x7f51bc07bcd0 2024-09-19T04:06:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.092+0000 7f51ba59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc07bcd0 0x7f51bc10a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.092+0000 7f51ba59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc07bcd0 0x7f51bc10a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46536/0 (socket says 172.21.15.94:46536) 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.093+0000 7f51ba59c640 1 -- 172.21.15.94:0/1994238120 learned_addr learned my addr 172.21.15.94:0/1994238120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.093+0000 7f51ba59c640 1 -- 172.21.15.94:0/1994238120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc10a920 con 0x7f51bc07bcd0 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.093+0000 7f51ba59c640 1 --2- 172.21.15.94:0/1994238120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc07bcd0 0x7f51bc10a1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009820 tx=0x7f51ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e2181019ab7ec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.094+0000 7f51b9d9b640 1 -- 172.21.15.94:0/1994238120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac037070 con 0x7f51bc07bcd0 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.094+0000 7f51b9d9b640 1 -- 172.21.15.94:0/1994238120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f51ac0328d0 con 0x7f51bc07bcd0 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.094+0000 7f51b9d9b640 1 -- 172.21.15.94:0/1994238120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac0307d0 con 0x7f51bc07bcd0 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.094+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1994238120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc07bcd0 msgr2=0x7f51bc10a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.094+0000 7f51c0fc5640 1 --2- 172.21.15.94:0/1994238120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc07bcd0 0x7f51bc10a1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009820 tx=0x7f51ac02a270 comp rx=0 tx=0).stop 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.095+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1994238120 shutdown_connections 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.095+0000 7f51c0fc5640 1 --2- 172.21.15.94:0/1994238120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc07bcd0 0x7f51bc10a1d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.095+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1994238120 >> 172.21.15.94:0/1994238120 conn(0x7f51bc101d80 msgr2=0x7f51bc1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.095+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1994238120 shutdown_connections 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.095+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1994238120 wait complete. 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.095+0000 7f51c0fc5640 1 Processor -- start 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.096+0000 7f51c0fc5640 1 -- start start 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.096+0000 7f51c0fc5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc19b5f0 0x7f51bc19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.096+0000 7f51c0fc5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc10b9c0 con 0x7f51bc19b5f0 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.096+0000 7f51ba59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc19b5f0 0x7f51bc19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.096+0000 7f51ba59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc19b5f0 0x7f51bc19ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46552/0 (socket says 172.21.15.94:46552) 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.096+0000 7f51ba59c640 1 -- 172.21.15.94:0/1851426195 learned_addr learned my addr 172.21.15.94:0/1851426195 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.096+0000 7f51ba59c640 1 -- 172.21.15.94:0/1851426195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc19bf50 con 0x7f51bc19b5f0 2024-09-19T04:06:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.096+0000 7f51ba59c640 1 --2- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc19b5f0 0x7f51bc19ba10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0097f0 tx=0x7f51ac030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.097+0000 7f51b37fe640 1 -- 172.21.15.94:0/1851426195 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac037040 con 0x7f51bc19b5f0 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.097+0000 7f51b37fe640 1 -- 172.21.15.94:0/1851426195 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f51ac03b3f0 con 0x7f51bc19b5f0 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.097+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51bc19c1e0 con 0x7f51bc19b5f0 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.097+0000 7f51b37fe640 1 -- 172.21.15.94:0/1851426195 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac03a520 con 0x7f51bc19b5f0 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.097+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51bc19ecc0 con 0x7f51bc19b5f0 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.097+0000 7f51b37fe640 1 -- 172.21.15.94:0/1851426195 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f51ac03a710 con 0x7f51bc19b5f0 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.098+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51bc108240 con 0x7f51bc19b5f0 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.098+0000 7f51b37fe640 1 --2- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f51a80391b0 0x7f51a803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.098+0000 7f51b37fe640 1 -- 172.21.15.94:0/1851426195 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f51ac06c630 con 0x7f51bc19b5f0 2024-09-19T04:06:03.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.098+0000 7f51b3fff640 1 -- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f51a80391b0 msgr2=0x7f51a803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2229490078 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.098+0000 7f51b3fff640 1 --2- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f51a80391b0 0x7f51a803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.102+0000 7f51b37fe640 1 -- 172.21.15.94:0/1851426195 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f51ac032820 con 0x7f51bc19b5f0 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.299+0000 7f51b3fff640 1 -- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f51a80391b0 msgr2=0x7f51a803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2229490078 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.299+0000 7f51b3fff640 1 --2- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f51a80391b0 0x7f51a803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.317+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f51bc19f1d0 con 0x7f51bc19b5f0 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.318+0000 7f51b37fe640 1 -- 172.21.15.94:0/1851426195 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f51ac06c050 con 0x7f51bc19b5f0 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f51a80391b0 msgr2=0x7f51a803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 --2- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f51a80391b0 0x7f51a803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc19b5f0 msgr2=0x7f51bc19ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 --2- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc19b5f0 0x7f51bc19ba10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0097f0 tx=0x7f51ac030d00 comp rx=0 tx=0).stop 2024-09-19T04:06:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 shutdown_connections 2024-09-19T04:06:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 --2- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f51a80391b0 0x7f51a803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 --2- 172.21.15.94:0/1851426195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51bc19b5f0 0x7f51bc19ba10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 >> 172.21.15.94:0/1851426195 conn(0x7f51bc101d80 msgr2=0x7f51bc1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 shutdown_connections 2024-09-19T04:06:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:03.320+0000 7f51c0fc5640 1 -- 172.21.15.94:0/1851426195 wait complete. 2024-09-19T04:06:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-09-19T04:06:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 5... 2024-09-19T04:06:04.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:04 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1851426195' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:06:04.341 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:04.198+0000 7ff4b3e86040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:06:04.341 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:04.264+0000 7ff4b3e86040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:06:05.226 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:04.971+0000 7ff4b3e86040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:06:05.227 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.085+0000 7ff4b3e86040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:06:05.500 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.225+0000 7ff4b3e86040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:06:05.500 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.285+0000 7ff4b3e86040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:06:05.500 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.432+0000 7ff4b3e86040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:06:05.500 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.498+0000 7ff4b3e86040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:06:05.759 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.558+0000 7ff4b3e86040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:06:05.759 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.713+0000 7ff4b3e86040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:06:06.041 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.777+0000 7ff4b3e86040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:06:06.041 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.842+0000 7ff4b3e86040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:06:06.041 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.923+0000 7ff4b3e86040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:06:06.042 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:05.982+0000 7ff4b3e86040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:06:06.042 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:06.039+0000 7ff4b3e86040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:06:07.090 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:06.812+0000 7ff4b3e86040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:06:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: Active manager daemon x restarted 2024-09-19T04:06:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: Activating manager daemon x 2024-09-19T04:06:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T04:06:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: mgrmap e6: x(active, starting, since 0.00336424s) 2024-09-19T04:06:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:06:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:06:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:06:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:06:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:06:07.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: Manager daemon x is now available 2024-09-19T04:06:07.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:07.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:07.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:07.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:07.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:06:07.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:06 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:06:08.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:07 smithi094 ceph-mon[30972]: Found migration_current of "None". Setting to last migration. 2024-09-19T04:06:08.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:07 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:08.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:07 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:08.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:07 smithi094 ceph-mon[30972]: mgrmap e7: x(active, since 1.00541s) 2024-09-19T04:06:08.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.350+0000 7f33943e3640 1 Processor -- start 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.350+0000 7f33943e3640 1 -- start start 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.351+0000 7f33943e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c10a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.351+0000 7f33943e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f338c07be00 con 0x7f338c108120 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.351+0000 7f339217f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c10a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.351+0000 7f339217f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c10a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46568/0 (socket says 172.21.15.94:46568) 2024-09-19T04:06:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.351+0000 7f339217f640 1 -- 172.21.15.94:0/3280072246 learned_addr learned my addr 172.21.15.94:0/3280072246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.351+0000 7f339217f640 1 -- 172.21.15.94:0/3280072246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c10aa50 con 0x7f338c108120 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.352+0000 7f339217f640 1 --2- 172.21.15.94:0/3280072246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c10a510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3388009820 tx=0x7f338802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0194a991890b15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.352+0000 7f339117d640 1 -- 172.21.15.94:0/3280072246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3388037070 con 0x7f338c108120 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.352+0000 7f339117d640 1 -- 172.21.15.94:0/3280072246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f33880328d0 con 0x7f338c108120 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.352+0000 7f339117d640 1 -- 172.21.15.94:0/3280072246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33880307d0 con 0x7f338c108120 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.353+0000 7f33943e3640 1 -- 172.21.15.94:0/3280072246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 msgr2=0x7f338c10a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.353+0000 7f33943e3640 1 --2- 172.21.15.94:0/3280072246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c10a510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3388009820 tx=0x7f338802a270 comp rx=0 tx=0).stop 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.353+0000 7f33943e3640 1 -- 172.21.15.94:0/3280072246 shutdown_connections 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.353+0000 7f33943e3640 1 --2- 172.21.15.94:0/3280072246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c10a510 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.353+0000 7f33943e3640 1 -- 172.21.15.94:0/3280072246 >> 172.21.15.94:0/3280072246 conn(0x7f338c101d80 msgr2=0x7f338c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.353+0000 7f33943e3640 1 -- 172.21.15.94:0/3280072246 shutdown_connections 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.353+0000 7f33943e3640 1 -- 172.21.15.94:0/3280072246 wait complete. 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.353+0000 7f33943e3640 1 Processor -- start 2024-09-19T04:06:08.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.353+0000 7f33943e3640 1 -- start start 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f33943e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c1079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f33943e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f338c10b9d0 con 0x7f338c108120 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f339217f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c1079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f339217f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c1079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46578/0 (socket says 172.21.15.94:46578) 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f339217f640 1 -- 172.21.15.94:0/2565281794 learned_addr learned my addr 172.21.15.94:0/2565281794 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f339217f640 1 -- 172.21.15.94:0/2565281794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c107ee0 con 0x7f338c108120 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f339217f640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c1079a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3388030520 tx=0x7f3388032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3388042070 con 0x7f338c108120 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f33943e3640 1 -- 172.21.15.94:0/2565281794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338c1060d0 con 0x7f338c108120 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3388030cc0 con 0x7f338c108120 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3388037040 con 0x7f338c108120 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.354+0000 7f33943e3640 1 -- 172.21.15.94:0/2565281794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338c106610 con 0x7f338c108120 2024-09-19T04:06:08.464 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.355+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f3388045070 con 0x7f338c108120 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.355+0000 7f33837fe640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 0x7f337803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.355+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 --> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f337803bd80 con 0x7f33780391b0 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.355+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f338806c420 con 0x7f338c108120 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.355+0000 7f339197e640 1 -- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 msgr2=0x7f337803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2229490078 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.355+0000 7f339197e640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 0x7f337803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.556+0000 7f339197e640 1 -- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 msgr2=0x7f337803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2229490078 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.556+0000 7f339197e640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 0x7f337803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.957+0000 7f339197e640 1 -- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 msgr2=0x7f337803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2229490078 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:04.957+0000 7f339197e640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 0x7f337803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:05.758+0000 7f339197e640 1 -- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 msgr2=0x7f337803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2229490078 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:05.758+0000 7f339197e640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 0x7f337803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:06.816+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f338806aee0 con 0x7f338c108120 2024-09-19T04:06:08.465 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:06.816+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 msgr2=0x7f337803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:06.816+0000 7f33837fe640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 0x7f337803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.817+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f338803cb20 con 0x7f338c108120 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.817+0000 7f33837fe640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f337803cae0 0x7f337803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.817+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3388045e70 con 0x7f337803cae0 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.817+0000 7f339197e640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f337803cae0 0x7f337803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.818+0000 7f339197e640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f337803cae0 0x7f337803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f337c003980 tx=0x7f337c006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.819+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 <== mgr.14118 v2:172.21.15.94:6800/3136401482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3388045e70 con 0x7f337803cae0 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.825+0000 7f33943e3640 1 -- 172.21.15.94:0/2565281794 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f338c108580 con 0x7f337803cae0 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33837fe640 1 -- 172.21.15.94:0/2565281794 <== mgr.14118 v2:172.21.15.94:6800/3136401482 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f338c108580 con 0x7f337803cae0 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 -- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f337803cae0 msgr2=0x7f337803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f337803cae0 0x7f337803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f337c003980 tx=0x7f337c006e10 comp rx=0 tx=0).stop 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 -- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 msgr2=0x7f338c1079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c1079a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3388030520 tx=0x7f3388032dd0 comp rx=0 tx=0).stop 2024-09-19T04:06:08.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 -- 172.21.15.94:0/2565281794 shutdown_connections 2024-09-19T04:06:08.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f337803cae0 0x7f337803eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:08.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:6800/2229490078,v1:172.21.15.94:6801/2229490078] conn(0x7f33780391b0 0x7f337803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:08.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 --2- 172.21.15.94:0/2565281794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f338c108120 0x7f338c1079a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:08.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 -- 172.21.15.94:0/2565281794 >> 172.21.15.94:0/2565281794 conn(0x7f338c101d80 msgr2=0x7f338c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:08.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 -- 172.21.15.94:0/2565281794 shutdown_connections 2024-09-19T04:06:08.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:07.826+0000 7f33943e3640 1 -- 172.21.15.94:0/2565281794 wait complete. 2024-09-19T04:06:08.467 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 5 is available 2024-09-19T04:06:08.467 INFO:teuthology.orchestra.run.smithi094.stdout:Setting orchestrator backend to cephadm... 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:08.441+0000 7ff41b1e0640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:06:08] ENGINE Error in HTTPServer.serve 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: Traceback (most recent call last): 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self._connections.run(self.expiration_interval) 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self._run(expiration_interval) 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:06:08.756 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: s = self.context.wrap_socket( 2024-09-19T04:06:08.757 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:06:08.757 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: return self.sslsocket_class._create( 2024-09-19T04:06:08.757 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:06:08.757 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self.do_handshake() 2024-09-19T04:06:08.757 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:06:08.757 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self._sslobj.do_handshake() 2024-09-19T04:06:08.757 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:06:08.757 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:09.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-mon[30972]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T04:06:09.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-mon[30972]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T04:06:09.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:08 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:09.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.831+0000 7f1fa820a640 1 Processor -- start 2024-09-19T04:06:09.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.831+0000 7f1fa820a640 1 -- start start 2024-09-19T04:06:09.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.832+0000 7f1fa820a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.832+0000 7f1fa820a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa007c820 con 0x7f1fa007dde0 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.832+0000 7f1fa5fa6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.832+0000 7f1fa5fa6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46638/0 (socket says 172.21.15.94:46638) 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.832+0000 7f1fa5fa6640 1 -- 172.21.15.94:0/3529389938 learned_addr learned my addr 172.21.15.94:0/3529389938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.832+0000 7f1fa5fa6640 1 -- 172.21.15.94:0/3529389938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa007c9a0 con 0x7f1fa007dde0 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.833+0000 7f1fa5fa6640 1 --2- 172.21.15.94:0/3529389938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa007c2e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009820 tx=0x7f1f9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc02ff6a2194c674 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.833+0000 7f1fa4fa4640 1 -- 172.21.15.94:0/3529389938 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f9c037070 con 0x7f1fa007dde0 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.833+0000 7f1fa4fa4640 1 -- 172.21.15.94:0/3529389938 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1f9c0328d0 con 0x7f1fa007dde0 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.833+0000 7f1fa4fa4640 1 -- 172.21.15.94:0/3529389938 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f9c0307d0 con 0x7f1fa007dde0 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.833+0000 7f1fa820a640 1 -- 172.21.15.94:0/3529389938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 msgr2=0x7f1fa007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.833+0000 7f1fa820a640 1 --2- 172.21.15.94:0/3529389938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa007c2e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009820 tx=0x7f1f9c02a270 comp rx=0 tx=0).stop 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.833+0000 7f1fa820a640 1 -- 172.21.15.94:0/3529389938 shutdown_connections 2024-09-19T04:06:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.833+0000 7f1fa820a640 1 --2- 172.21.15.94:0/3529389938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa007c2e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.833+0000 7f1fa820a640 1 -- 172.21.15.94:0/3529389938 >> 172.21.15.94:0/3529389938 conn(0x7f1fa0102ed0 msgr2=0x7f1fa01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.834+0000 7f1fa820a640 1 -- 172.21.15.94:0/3529389938 shutdown_connections 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.834+0000 7f1fa820a640 1 -- 172.21.15.94:0/3529389938 wait complete. 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.834+0000 7f1fa820a640 1 Processor -- start 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.834+0000 7f1fa820a640 1 -- start start 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.834+0000 7f1fa820a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.834+0000 7f1fa820a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa010dc30 con 0x7f1fa007dde0 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.834+0000 7f1fa5fa6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.834+0000 7f1fa5fa6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa019d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46648/0 (socket says 172.21.15.94:46648) 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.834+0000 7f1fa5fa6640 1 -- 172.21.15.94:0/498166946 learned_addr learned my addr 172.21.15.94:0/498166946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.835+0000 7f1fa5fa6640 1 -- 172.21.15.94:0/498166946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa019dd90 con 0x7f1fa007dde0 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.835+0000 7f1fa5fa6640 1 --2- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa019d850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0097f0 tx=0x7f1f9c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.835+0000 7f1f92ffd640 1 -- 172.21.15.94:0/498166946 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f9c037040 con 0x7f1fa007dde0 2024-09-19T04:06:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.835+0000 7f1f92ffd640 1 -- 172.21.15.94:0/498166946 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1f9c032bb0 con 0x7f1fa007dde0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.835+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa019e020 con 0x7f1fa007dde0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.835+0000 7f1f92ffd640 1 -- 172.21.15.94:0/498166946 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f9c03a390 con 0x7f1fa007dde0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.835+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa01a0d10 con 0x7f1fa007dde0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.836+0000 7f1f92ffd640 1 -- 172.21.15.94:0/498166946 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f1f9c03a600 con 0x7f1fa007dde0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.836+0000 7f1f92ffd640 1 --2- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f1f8c0390f0 0x7f1f8c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.836+0000 7f1f92ffd640 1 -- 172.21.15.94:0/498166946 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1f9c06c300 con 0x7f1fa007dde0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.836+0000 7f1fa57a5640 1 --2- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f1f8c0390f0 0x7f1f8c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.836+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa007c360 con 0x7f1fa007dde0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.837+0000 7f1fa57a5640 1 --2- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f1f8c0390f0 0x7f1f8c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f940098c0 tx=0x7f1f94006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.841+0000 7f1f92ffd640 1 -- 172.21.15.94:0/498166946 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1f9c032d60 con 0x7f1fa007dde0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:08.869+0000 7f1f92ffd640 1 -- 172.21.15.94:0/498166946 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f1f9c03a940 con 0x7f1fa007dde0 2024-09-19T04:06:09.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.021+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f1fa0104180 con 0x7f1f8c0390f0 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.026+0000 7f1f92ffd640 1 -- 172.21.15.94:0/498166946 <== mgr.14118 v2:172.21.15.94:6800/3136401482 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1fa0104180 con 0x7f1f8c0390f0 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f1f8c0390f0 msgr2=0x7f1f8c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 --2- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f1f8c0390f0 0x7f1f8c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f940098c0 tx=0x7f1f94006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 msgr2=0x7f1fa019d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 --2- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa019d850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0097f0 tx=0x7f1f9c030d00 comp rx=0 tx=0).stop 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 shutdown_connections 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 --2- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f1f8c0390f0 0x7f1f8c03b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 --2- 172.21.15.94:0/498166946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa007dde0 0x7f1fa019d850 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 >> 172.21.15.94:0/498166946 conn(0x7f1fa0102ed0 msgr2=0x7f1fa0103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 shutdown_connections 2024-09-19T04:06:09.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:09.028+0000 7f1fa820a640 1 -- 172.21.15.94:0/498166946 wait complete. 2024-09-19T04:06:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:09 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:08] ENGINE Bus STARTING 2024-09-19T04:06:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:09 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:08] ENGINE Serving on http://172.21.15.94:8765 2024-09-19T04:06:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:09 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:08] ENGINE Serving on https://172.21.15.94:7150 2024-09-19T04:06:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:09 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:08] ENGINE Bus STARTED 2024-09-19T04:06:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:09 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:08] ENGINE Error in HTTPServer.serve 2024-09-19T04:06:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: Traceback (most recent call last): 2024-09-19T04:06:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:06:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:06:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:06:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: self._run(expiration_interval) 2024-09-19T04:06:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:06:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:06:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:06:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:06:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:06:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: s = self.context.wrap_socket( 2024-09-19T04:06:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:06:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: return self.sslsocket_class._create( 2024-09-19T04:06:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:06:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout: self.do_handshake() 2024-09-19T04:06:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:06:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout: self._sslobj.do_handshake() 2024-09-19T04:06:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:06:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:09 smithi094 ceph-mon[30972]: mgrmap e8: x(active, since 2s) 2024-09-19T04:06:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:09 smithi094 ceph-mon[30972]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:09 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:09 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:10.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T04:06:10.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.040+0000 7f61af91a640 1 Processor -- start 2024-09-19T04:06:10.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.040+0000 7f61af91a640 1 -- start start 2024-09-19T04:06:10.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.040+0000 7f61af91a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a8106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.040+0000 7f61af91a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61a8106f10 con 0x7f61a8106540 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.040+0000 7f61ad6b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a8106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.040+0000 7f61ad6b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a8106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46650/0 (socket says 172.21.15.94:46650) 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.040+0000 7f61ad6b6640 1 -- 172.21.15.94:0/3408893789 learned_addr learned my addr 172.21.15.94:0/3408893789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.040+0000 7f61ad6b6640 1 -- 172.21.15.94:0/3408893789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a8107740 con 0x7f61a8106540 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.041+0000 7f61ad6b6640 1 --2- 172.21.15.94:0/3408893789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a8106940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f619c009820 tx=0x7f619c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c284f7b248c685 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.041+0000 7f619bfff640 1 -- 172.21.15.94:0/3408893789 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f619c037070 con 0x7f61a8106540 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.041+0000 7f619bfff640 1 -- 172.21.15.94:0/3408893789 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f619c0328d0 con 0x7f61a8106540 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.041+0000 7f619bfff640 1 -- 172.21.15.94:0/3408893789 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f619c0307d0 con 0x7f61a8106540 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.042+0000 7f61af91a640 1 -- 172.21.15.94:0/3408893789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 msgr2=0x7f61a8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.042+0000 7f61af91a640 1 --2- 172.21.15.94:0/3408893789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a8106940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f619c009820 tx=0x7f619c02a270 comp rx=0 tx=0).stop 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.042+0000 7f61af91a640 1 -- 172.21.15.94:0/3408893789 shutdown_connections 2024-09-19T04:06:10.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.042+0000 7f61af91a640 1 --2- 172.21.15.94:0/3408893789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a8106940 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.042+0000 7f61af91a640 1 -- 172.21.15.94:0/3408893789 >> 172.21.15.94:0/3408893789 conn(0x7f61a8101d90 msgr2=0x7f61a81041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.042+0000 7f61af91a640 1 -- 172.21.15.94:0/3408893789 shutdown_connections 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.042+0000 7f61af91a640 1 -- 172.21.15.94:0/3408893789 wait complete. 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.042+0000 7f61af91a640 1 Processor -- start 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61af91a640 1 -- start start 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61af91a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a807ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61af91a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810aaf0 con 0x7f61a8106540 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61ad6b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a807ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61ad6b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a807ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46654/0 (socket says 172.21.15.94:46654) 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61ad6b6640 1 -- 172.21.15.94:0/2646474421 learned_addr learned my addr 172.21.15.94:0/2646474421 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61ad6b6640 1 -- 172.21.15.94:0/2646474421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a819e350 con 0x7f61a8106540 2024-09-19T04:06:10.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61ad6b6640 1 --2- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a807ba30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f619c0097f0 tx=0x7f619c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f619a7fc640 1 -- 172.21.15.94:0/2646474421 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f619c037040 con 0x7f61a8106540 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a819e520 con 0x7f61a8106540 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f619a7fc640 1 -- 172.21.15.94:0/2646474421 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f619c032bb0 con 0x7f61a8106540 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f619a7fc640 1 -- 172.21.15.94:0/2646474421 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f619c03a390 con 0x7f61a8106540 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.043+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a819e8c0 con 0x7f61a8106540 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.044+0000 7f619a7fc640 1 -- 172.21.15.94:0/2646474421 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f619c03b3f0 con 0x7f61a8106540 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.044+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a81069c0 con 0x7f61a8106540 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.044+0000 7f619a7fc640 1 --2- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f61940391b0 0x7f619403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.045+0000 7f619a7fc640 1 -- 172.21.15.94:0/2646474421 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f619c06c4d0 con 0x7f61a8106540 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.045+0000 7f61aceb5640 1 --2- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f61940391b0 0x7f619403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.045+0000 7f61aceb5640 1 --2- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f61940391b0 0x7f619403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61a40098c0 tx=0x7f61a4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.049+0000 7f619a7fc640 1 -- 172.21.15.94:0/2646474421 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f619c049c80 con 0x7f61a8106540 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.217+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f61a8103010 con 0x7f61940391b0 2024-09-19T04:06:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.218+0000 7f619a7fc640 1 -- 172.21.15.94:0/2646474421 <== mgr.14118 v2:172.21.15.94:6800/3136401482 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f61a8103010 con 0x7f61940391b0 2024-09-19T04:06:10.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.220+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f61940391b0 msgr2=0x7f619403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.220+0000 7f61af91a640 1 --2- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f61940391b0 0x7f619403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61a40098c0 tx=0x7f61a4006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.220+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 msgr2=0x7f61a807ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.220+0000 7f61af91a640 1 --2- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a807ba30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f619c0097f0 tx=0x7f619c030d00 comp rx=0 tx=0).stop 2024-09-19T04:06:10.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.220+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 shutdown_connections 2024-09-19T04:06:10.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.220+0000 7f61af91a640 1 --2- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f61940391b0 0x7f619403b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.220+0000 7f61af91a640 1 --2- 172.21.15.94:0/2646474421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61a8106540 0x7f61a807ba30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.221+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 >> 172.21.15.94:0/2646474421 conn(0x7f61a8101d90 msgr2=0x7f61a8102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:10.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.221+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 shutdown_connections 2024-09-19T04:06:10.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:10.221+0000 7f61af91a640 1 -- 172.21.15.94:0/2646474421 wait complete. 2024-09-19T04:06:10.805 INFO:teuthology.orchestra.run.smithi094.stdout:Generating ssh key... 2024-09-19T04:06:11.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-mon[30972]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:11.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: Generating public/private rsa key pair. 2024-09-19T04:06:11.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: Your identification has been saved in /tmp/tmpu25ury26/key 2024-09-19T04:06:11.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: Your public key has been saved in /tmp/tmpu25ury26/key.pub 2024-09-19T04:06:11.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: The key fingerprint is: 2024-09-19T04:06:11.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: SHA256:RzahkYB+HwnmBmDbdIXg3vrGT75zdi7Me70KPKxcT5c ceph-503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:06:11.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: The key's randomart image is: 2024-09-19T04:06:11.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: +---[RSA 3072]----+ 2024-09-19T04:06:11.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: | o.ooo+o.. | 2024-09-19T04:06:11.842 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: | . =o.+ .o . | 2024-09-19T04:06:11.842 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: | ..o+ ...+ | 2024-09-19T04:06:11.842 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: | ...+ oo . | 2024-09-19T04:06:11.842 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: | .o..S.. | 2024-09-19T04:06:11.842 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: | . .+ . | 2024-09-19T04:06:11.842 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: | .. .o* ..E | 2024-09-19T04:06:11.842 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: | .o+.o*=o.. | 2024-09-19T04:06:11.842 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: | ...*=.*+... | 2024-09-19T04:06:11.842 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:11 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: +----[SHA256]-----+ 2024-09-19T04:06:12.108 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.196+0000 7ffb8173d640 1 Processor -- start 2024-09-19T04:06:12.108 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.196+0000 7ffb8173d640 1 -- start start 2024-09-19T04:06:12.108 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.196+0000 7ffb8173d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:12.108 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.196+0000 7ffb8173d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1080b0 con 0x7ffb7c1076e0 2024-09-19T04:06:12.108 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.196+0000 7ffb7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:12.108 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.196+0000 7ffb7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46658/0 (socket says 172.21.15.94:46658) 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.197+0000 7ffb7affd640 1 -- 172.21.15.94:0/2825984418 learned_addr learned my addr 172.21.15.94:0/2825984418 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.197+0000 7ffb7affd640 1 -- 172.21.15.94:0/2825984418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c108840 con 0x7ffb7c1076e0 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.197+0000 7ffb7affd640 1 --2- 172.21.15.94:0/2825984418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c107ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009820 tx=0x7ffb6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6e6800412627bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.197+0000 7ffb79ffb640 1 -- 172.21.15.94:0/2825984418 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb6c037070 con 0x7ffb7c1076e0 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.198+0000 7ffb79ffb640 1 -- 172.21.15.94:0/2825984418 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ffb6c0328d0 con 0x7ffb7c1076e0 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.198+0000 7ffb79ffb640 1 -- 172.21.15.94:0/2825984418 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb6c0307d0 con 0x7ffb7c1076e0 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.198+0000 7ffb8173d640 1 -- 172.21.15.94:0/2825984418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 msgr2=0x7ffb7c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.198+0000 7ffb8173d640 1 --2- 172.21.15.94:0/2825984418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c107ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009820 tx=0x7ffb6c02a270 comp rx=0 tx=0).stop 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.198+0000 7ffb8173d640 1 -- 172.21.15.94:0/2825984418 shutdown_connections 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.198+0000 7ffb8173d640 1 --2- 172.21.15.94:0/2825984418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c107ae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.198+0000 7ffb8173d640 1 -- 172.21.15.94:0/2825984418 >> 172.21.15.94:0/2825984418 conn(0x7ffb7c102ed0 msgr2=0x7ffb7c1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.198+0000 7ffb8173d640 1 -- 172.21.15.94:0/2825984418 shutdown_connections 2024-09-19T04:06:12.109 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.198+0000 7ffb8173d640 1 -- 172.21.15.94:0/2825984418 wait complete. 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.199+0000 7ffb8173d640 1 Processor -- start 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.199+0000 7ffb8173d640 1 -- start start 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.199+0000 7ffb8173d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c19b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.199+0000 7ffb8173d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10bbf0 con 0x7ffb7c1076e0 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.199+0000 7ffb7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c19b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.199+0000 7ffb7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c19b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46672/0 (socket says 172.21.15.94:46672) 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.199+0000 7ffb7affd640 1 -- 172.21.15.94:0/316542363 learned_addr learned my addr 172.21.15.94:0/316542363 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.199+0000 7ffb7affd640 1 -- 172.21.15.94:0/316542363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c19bb50 con 0x7ffb7c1076e0 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.200+0000 7ffb7affd640 1 --2- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c19b610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009940 tx=0x7ffb6c032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.200+0000 7ffb63fff640 1 -- 172.21.15.94:0/316542363 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb6c042070 con 0x7ffb7c1076e0 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.200+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c19bde0 con 0x7ffb7c1076e0 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.200+0000 7ffb63fff640 1 -- 172.21.15.94:0/316542363 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ffb6c030c80 con 0x7ffb7c1076e0 2024-09-19T04:06:12.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.200+0000 7ffb63fff640 1 -- 172.21.15.94:0/316542363 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb6c037040 con 0x7ffb7c1076e0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.200+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c19ead0 con 0x7ffb7c1076e0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.201+0000 7ffb63fff640 1 -- 172.21.15.94:0/316542363 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ffb6c045070 con 0x7ffb7c1076e0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.201+0000 7ffb63fff640 1 --2- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7ffb640391b0 0x7ffb6403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.201+0000 7ffb63fff640 1 -- 172.21.15.94:0/316542363 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffb6c06c350 con 0x7ffb7c1076e0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.201+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7c10bd70 con 0x7ffb7c1076e0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.201+0000 7ffb7a7fc640 1 --2- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7ffb640391b0 0x7ffb6403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.201+0000 7ffb7a7fc640 1 --2- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7ffb640391b0 0x7ffb6403b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffb700098c0 tx=0x7ffb70006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.206+0000 7ffb63fff640 1 -- 172.21.15.94:0/316542363 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffb6c045360 con 0x7ffb7c1076e0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.373+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ffb7c104130 con 0x7ffb640391b0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.562+0000 7ffb63fff640 1 -- 172.21.15.94:0/316542363 <== mgr.14118 v2:172.21.15.94:6800/3136401482 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffb7c104130 con 0x7ffb640391b0 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.564+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7ffb640391b0 msgr2=0x7ffb6403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.564+0000 7ffb8173d640 1 --2- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7ffb640391b0 0x7ffb6403b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffb700098c0 tx=0x7ffb70006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:12.111 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.564+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 msgr2=0x7ffb7c19b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:12.112 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.564+0000 7ffb8173d640 1 --2- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c19b610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009940 tx=0x7ffb6c032e10 comp rx=0 tx=0).stop 2024-09-19T04:06:12.112 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.565+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 shutdown_connections 2024-09-19T04:06:12.112 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.565+0000 7ffb8173d640 1 --2- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7ffb640391b0 0x7ffb6403b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:12.112 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.565+0000 7ffb8173d640 1 --2- 172.21.15.94:0/316542363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c1076e0 0x7ffb7c19b610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:12.112 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.565+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 >> 172.21.15.94:0/316542363 conn(0x7ffb7c102ed0 msgr2=0x7ffb7c103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:12.112 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.565+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 shutdown_connections 2024-09-19T04:06:12.112 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:11.565+0000 7ffb8173d640 1 -- 172.21.15.94:0/316542363 wait complete. 2024-09-19T04:06:12.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:12 smithi094 ceph-mon[30972]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:12.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:12 smithi094 ceph-mon[30972]: Generating ssh key... 2024-09-19T04:06:12.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:12 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:12.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:12 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:13.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQO1N3pjspPUsujM8AnLcWTKmJDQUxhNso1yA3ObI9+OgF+9zDWCGNidtBioDDcaZ3IsOyhsGzuo3xTNOg18g5c894eedX++/3aU7gtPYgLdf+GbtH6/a5AG7geATCz/TO3HHoQBggnknWodAC3p4ROryECmPmtPPbW4H0cYo1h80vAhxCBWJeuX6Ro7q/ldt45Rl60gMk2G754QExCgLpgfSJ2M0nQ3gPMgdiv4eTTSRLDjci2dNZH8cE0njGOA/LjU8ewUAKwKyx4FN0LvdL2/O/8VqZ/lDFtr+YIknoVtyzT1jadjD+UH3sOQKSXIE6BK7RS4HJzTx/TBrBGMRbrjqtn1oHIGplnMATmg469gejv4JGbk4q5h0qq3bMeBD18plQ3rexYsH7ek4ZcWErkTxzKksmMFNBHJk1hmkBnQSUGpVLVZXC4SOJ91yFgH+zXZrLdVYs335O8EPkMLVJ1xFxLRfuGD7vPI4XClGsUwQNcEYpVoJ+vChgzlBdQiE= ceph-503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.485+0000 7f1134e2a640 1 Processor -- start 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.485+0000 7f1134e2a640 1 -- start start 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.485+0000 7f1134e2a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f1130107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.485+0000 7f1134e2a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11301080b0 con 0x7f11301076e0 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.485+0000 7f112e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f1130107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.485+0000 7f112e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f1130107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57362/0 (socket says 172.21.15.94:57362) 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.485+0000 7f112e59c640 1 -- 172.21.15.94:0/1575824424 learned_addr learned my addr 172.21.15.94:0/1575824424 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.486+0000 7f112e59c640 1 -- 172.21.15.94:0/1575824424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1130108840 con 0x7f11301076e0 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.487+0000 7f112e59c640 1 --2- 172.21.15.94:0/1575824424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f1130107ae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1120009820 tx=0x7f112002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9325345a128d927e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:13.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.487+0000 7f112dd9b640 1 -- 172.21.15.94:0/1575824424 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1120037070 con 0x7f11301076e0 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.487+0000 7f112dd9b640 1 -- 172.21.15.94:0/1575824424 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f11200328d0 con 0x7f11301076e0 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.487+0000 7f112dd9b640 1 -- 172.21.15.94:0/1575824424 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11200307d0 con 0x7f11301076e0 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.488+0000 7f1134e2a640 1 -- 172.21.15.94:0/1575824424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 msgr2=0x7f1130107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.488+0000 7f1134e2a640 1 --2- 172.21.15.94:0/1575824424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f1130107ae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1120009820 tx=0x7f112002a270 comp rx=0 tx=0).stop 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.488+0000 7f1134e2a640 1 -- 172.21.15.94:0/1575824424 shutdown_connections 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.488+0000 7f1134e2a640 1 --2- 172.21.15.94:0/1575824424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f1130107ae0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.488+0000 7f1134e2a640 1 -- 172.21.15.94:0/1575824424 >> 172.21.15.94:0/1575824424 conn(0x7f1130102ed0 msgr2=0x7f11301052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.488+0000 7f1134e2a640 1 -- 172.21.15.94:0/1575824424 shutdown_connections 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.488+0000 7f1134e2a640 1 -- 172.21.15.94:0/1575824424 wait complete. 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.489+0000 7f1134e2a640 1 Processor -- start 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.489+0000 7f1134e2a640 1 -- start start 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.489+0000 7f1134e2a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f113019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:13.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.489+0000 7f1134e2a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f113010bbf0 con 0x7f11301076e0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.489+0000 7f112e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f113019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.489+0000 7f112e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f113019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57374/0 (socket says 172.21.15.94:57374) 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.489+0000 7f112e59c640 1 -- 172.21.15.94:0/2235204367 learned_addr learned my addr 172.21.15.94:0/2235204367 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.490+0000 7f112e59c640 1 -- 172.21.15.94:0/2235204367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113019dda0 con 0x7f11301076e0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.490+0000 7f112e59c640 1 --2- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f113019d860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f11200097f0 tx=0x7f1120030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.490+0000 7f11277fe640 1 -- 172.21.15.94:0/2235204367 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1120037040 con 0x7f11301076e0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.490+0000 7f11277fe640 1 -- 172.21.15.94:0/2235204367 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1120032bb0 con 0x7f11301076e0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.490+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113019e030 con 0x7f11301076e0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.490+0000 7f11277fe640 1 -- 172.21.15.94:0/2235204367 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f112003a390 con 0x7f11301076e0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.490+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11301a0d20 con 0x7f11301076e0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.491+0000 7f11277fe640 1 -- 172.21.15.94:0/2235204367 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f112003b3f0 con 0x7f11301076e0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.491+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113010bd70 con 0x7f11301076e0 2024-09-19T04:06:13.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.492+0000 7f11277fe640 1 --2- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f111c0391b0 0x7f111c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.492+0000 7f11277fe640 1 -- 172.21.15.94:0/2235204367 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f112006c4d0 con 0x7f11301076e0 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.492+0000 7f1127fff640 1 --2- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f111c0391b0 0x7f111c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.492+0000 7f1127fff640 1 --2- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f111c0391b0 0x7f111c03b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f11180098c0 tx=0x7f1118006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.497+0000 7f11277fe640 1 -- 172.21.15.94:0/2235204367 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1120049c80 con 0x7f11301076e0 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.670+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f11301040b0 con 0x7f111c0391b0 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.671+0000 7f11277fe640 1 -- 172.21.15.94:0/2235204367 <== mgr.14118 v2:172.21.15.94:6800/3136401482 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f11301040b0 con 0x7f111c0391b0 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f111c0391b0 msgr2=0x7f111c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 --2- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f111c0391b0 0x7f111c03b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f11180098c0 tx=0x7f1118006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 msgr2=0x7f113019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 --2- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f113019d860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f11200097f0 tx=0x7f1120030d00 comp rx=0 tx=0).stop 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 shutdown_connections 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 --2- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f111c0391b0 0x7f111c03b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:13.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 --2- 172.21.15.94:0/2235204367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11301076e0 0x7f113019d860 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:13.253 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 >> 172.21.15.94:0/2235204367 conn(0x7f1130102ed0 msgr2=0x7f11301039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:13.253 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 shutdown_connections 2024-09-19T04:06:13.253 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:12.673+0000 7f1134e2a640 1 -- 172.21.15.94:0/2235204367 wait complete. 2024-09-19T04:06:13.253 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:06:13.253 INFO:teuthology.orchestra.run.smithi094.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T04:06:13.255 INFO:teuthology.orchestra.run.smithi094.stdout:Adding host smithi094... 2024-09-19T04:06:13.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:13 smithi094 ceph-mon[30972]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:14.684 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:14 smithi094 ceph-mon[30972]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:15.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:15 smithi094 ceph-mon[30972]: Deploying cephadm binary to smithi094 2024-09-19T04:06:17.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Added host 'smithi094' with addr '172.21.15.94' 2024-09-19T04:06:17.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.689+0000 7feb454d2640 1 Processor -- start 2024-09-19T04:06:17.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.689+0000 7feb454d2640 1 -- start start 2024-09-19T04:06:17.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.689+0000 7feb454d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb40105d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:17.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.689+0000 7feb454d2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb40106310 con 0x7feb40105940 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.689+0000 7feb3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb40105d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.689+0000 7feb3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb40105d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57390/0 (socket says 172.21.15.94:57390) 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.689+0000 7feb3effd640 1 -- 172.21.15.94:0/1374583416 learned_addr learned my addr 172.21.15.94:0/1374583416 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.689+0000 7feb3effd640 1 -- 172.21.15.94:0/1374583416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb40108920 con 0x7feb40105940 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.690+0000 7feb3effd640 1 --2- 172.21.15.94:0/1374583416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb40105d40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feb30009820 tx=0x7feb3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef552515409a3eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.690+0000 7feb3e7fc640 1 -- 172.21.15.94:0/1374583416 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb30037070 con 0x7feb40105940 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.690+0000 7feb3e7fc640 1 -- 172.21.15.94:0/1374583416 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7feb300328d0 con 0x7feb40105940 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.690+0000 7feb3e7fc640 1 -- 172.21.15.94:0/1374583416 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb300307d0 con 0x7feb40105940 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 -- 172.21.15.94:0/1374583416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 msgr2=0x7feb40105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 --2- 172.21.15.94:0/1374583416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb40105d40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feb30009820 tx=0x7feb3002a270 comp rx=0 tx=0).stop 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 -- 172.21.15.94:0/1374583416 shutdown_connections 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 --2- 172.21.15.94:0/1374583416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb40105d40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 -- 172.21.15.94:0/1374583416 >> 172.21.15.94:0/1374583416 conn(0x7feb40101490 msgr2=0x7feb401038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 -- 172.21.15.94:0/1374583416 shutdown_connections 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 -- 172.21.15.94:0/1374583416 wait complete. 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 Processor -- start 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 -- start start 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.691+0000 7feb454d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb4019d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb454d2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb4010bb50 con 0x7feb40105940 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb4019d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb4019d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57406/0 (socket says 172.21.15.94:57406) 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb3effd640 1 -- 172.21.15.94:0/3048178563 learned_addr learned my addr 172.21.15.94:0/3048178563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb3effd640 1 -- 172.21.15.94:0/3048178563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4019ddf0 con 0x7feb40105940 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb3effd640 1 --2- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb4019d8b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feb30030520 tx=0x7feb30032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb3cff9640 1 -- 172.21.15.94:0/3048178563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb30042070 con 0x7feb40105940 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb3cff9640 1 -- 172.21.15.94:0/3048178563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7feb30030cc0 con 0x7feb40105940 2024-09-19T04:06:17.635 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4019e080 con 0x7feb40105940 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb3cff9640 1 -- 172.21.15.94:0/3048178563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb30037040 con 0x7feb40105940 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.692+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb401a0d70 con 0x7feb40105940 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.693+0000 7feb3cff9640 1 -- 172.21.15.94:0/3048178563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7feb30045070 con 0x7feb40105940 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.693+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb40105dc0 con 0x7feb40105940 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.693+0000 7feb3cff9640 1 --2- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7feb2c039200 0x7feb2c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.693+0000 7feb3cff9640 1 -- 172.21.15.94:0/3048178563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feb3006c1f0 con 0x7feb40105940 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.693+0000 7feb37fff640 1 --2- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7feb2c039200 0x7feb2c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.694+0000 7feb37fff640 1 --2- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7feb2c039200 0x7feb2c03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb280098c0 tx=0x7feb28006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.698+0000 7feb3cff9640 1 -- 172.21.15.94:0/3048178563 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb30045360 con 0x7feb40105940 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:13.868+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}) -- 0x7feb40102750 con 0x7feb2c039200 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.073+0000 7feb3cff9640 1 -- 172.21.15.94:0/3048178563 <== mgr.14118 v2:172.21.15.94:6800/3136401482 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7feb40102750 con 0x7feb2c039200 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.075+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7feb2c039200 msgr2=0x7feb2c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.075+0000 7feb454d2640 1 --2- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7feb2c039200 0x7feb2c03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb280098c0 tx=0x7feb28006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.075+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 msgr2=0x7feb4019d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:17.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.075+0000 7feb454d2640 1 --2- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb4019d8b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feb30030520 tx=0x7feb30032dd0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.075+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 shutdown_connections 2024-09-19T04:06:17.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.075+0000 7feb454d2640 1 --2- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7feb2c039200 0x7feb2c03b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.075+0000 7feb454d2640 1 --2- 172.21.15.94:0/3048178563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb40105940 0x7feb4019d8b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.075+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 >> 172.21.15.94:0/3048178563 conn(0x7feb40101490 msgr2=0x7feb40102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:17.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.075+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 shutdown_connections 2024-09-19T04:06:17.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:17.076+0000 7feb454d2640 1 -- 172.21.15.94:0/3048178563 wait complete. 2024-09-19T04:06:17.637 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mon service... 2024-09-19T04:06:18.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:18 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:18.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:18 smithi094 ceph-mon[30972]: Added host smithi094 2024-09-19T04:06:18.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:18 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:18.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T04:06:18.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.066+0000 7fa875718640 1 Processor -- start 2024-09-19T04:06:18.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.066+0000 7fa875718640 1 -- start start 2024-09-19T04:06:18.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.067+0000 7fa875718640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87007bcd0 0x7fa87010a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:18.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.067+0000 7fa875718640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa87010a7a0 con 0x7fa87007bcd0 2024-09-19T04:06:18.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.067+0000 7fa86effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87007bcd0 0x7fa87010a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:18.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.067+0000 7fa86effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87007bcd0 0x7fa87010a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57420/0 (socket says 172.21.15.94:57420) 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.067+0000 7fa86effd640 1 -- 172.21.15.94:0/4269055152 learned_addr learned my addr 172.21.15.94:0/4269055152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.067+0000 7fa86effd640 1 -- 172.21.15.94:0/4269055152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87010a920 con 0x7fa87007bcd0 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.068+0000 7fa86effd640 1 --2- 172.21.15.94:0/4269055152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87007bcd0 0x7fa87010a1d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa860009820 tx=0x7fa86002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a64c26f90a19c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.068+0000 7fa86dffb640 1 -- 172.21.15.94:0/4269055152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa860037070 con 0x7fa87007bcd0 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.068+0000 7fa86dffb640 1 -- 172.21.15.94:0/4269055152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa8600328d0 con 0x7fa87007bcd0 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.068+0000 7fa86dffb640 1 -- 172.21.15.94:0/4269055152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8600307d0 con 0x7fa87007bcd0 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.069+0000 7fa875718640 1 -- 172.21.15.94:0/4269055152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87007bcd0 msgr2=0x7fa87010a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.069+0000 7fa875718640 1 --2- 172.21.15.94:0/4269055152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87007bcd0 0x7fa87010a1d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa860009820 tx=0x7fa86002a270 comp rx=0 tx=0).stop 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.069+0000 7fa875718640 1 -- 172.21.15.94:0/4269055152 shutdown_connections 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.069+0000 7fa875718640 1 --2- 172.21.15.94:0/4269055152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87007bcd0 0x7fa87010a1d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.069+0000 7fa875718640 1 -- 172.21.15.94:0/4269055152 >> 172.21.15.94:0/4269055152 conn(0x7fa870101d80 msgr2=0x7fa8701041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.069+0000 7fa875718640 1 -- 172.21.15.94:0/4269055152 shutdown_connections 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.069+0000 7fa875718640 1 -- 172.21.15.94:0/4269055152 wait complete. 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.070+0000 7fa875718640 1 Processor -- start 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.070+0000 7fa875718640 1 -- start start 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.070+0000 7fa875718640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87019b5f0 0x7fa87019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.070+0000 7fa875718640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa87010b9c0 con 0x7fa87019b5f0 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.070+0000 7fa86effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87019b5f0 0x7fa87019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.070+0000 7fa86effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87019b5f0 0x7fa87019ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57434/0 (socket says 172.21.15.94:57434) 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.070+0000 7fa86effd640 1 -- 172.21.15.94:0/2756656051 learned_addr learned my addr 172.21.15.94:0/2756656051 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.070+0000 7fa86effd640 1 -- 172.21.15.94:0/2756656051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87019bf50 con 0x7fa87019b5f0 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.070+0000 7fa86effd640 1 --2- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87019b5f0 0x7fa87019ba10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa860030520 tx=0x7fa860032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:18.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.071+0000 7fa857fff640 1 -- 172.21.15.94:0/2756656051 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa860042070 con 0x7fa87019b5f0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.071+0000 7fa857fff640 1 -- 172.21.15.94:0/2756656051 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa860030cc0 con 0x7fa87019b5f0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.071+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87019c1e0 con 0x7fa87019b5f0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.071+0000 7fa857fff640 1 -- 172.21.15.94:0/2756656051 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa860037040 con 0x7fa87019b5f0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.071+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87019ecc0 con 0x7fa87019b5f0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.072+0000 7fa857fff640 1 -- 172.21.15.94:0/2756656051 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa860045070 con 0x7fa87019b5f0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.072+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa870108240 con 0x7fa87019b5f0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.072+0000 7fa857fff640 1 --2- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fa8580391b0 0x7fa85803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.072+0000 7fa86e7fc640 1 --2- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fa8580391b0 0x7fa85803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.072+0000 7fa86e7fc640 1 --2- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fa8580391b0 0x7fa85803b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa8640098c0 tx=0x7fa864006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.074+0000 7fa857fff640 1 -- 172.21.15.94:0/2756656051 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa86003d7b0 con 0x7fa87019b5f0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.079+0000 7fa857fff640 1 -- 172.21.15.94:0/2756656051 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa86004b170 con 0x7fa87019b5f0 2024-09-19T04:06:18.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.259+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa8701030f0 con 0x7fa8580391b0 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.265+0000 7fa857fff640 1 -- 172.21.15.94:0/2756656051 <== mgr.14118 v2:172.21.15.94:6800/3136401482 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa8701030f0 con 0x7fa8580391b0 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fa8580391b0 msgr2=0x7fa85803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 --2- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fa8580391b0 0x7fa85803b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa8640098c0 tx=0x7fa864006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87019b5f0 msgr2=0x7fa87019ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 --2- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87019b5f0 0x7fa87019ba10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa860030520 tx=0x7fa860032dd0 comp rx=0 tx=0).stop 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 shutdown_connections 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 --2- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fa8580391b0 0x7fa85803b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 --2- 172.21.15.94:0/2756656051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa87019b5f0 0x7fa87019ba10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 >> 172.21.15.94:0/2756656051 conn(0x7fa870101d80 msgr2=0x7fa8701029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 shutdown_connections 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:18.267+0000 7fa875718640 1 -- 172.21.15.94:0/2756656051 wait complete. 2024-09-19T04:06:18.883 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mgr service... 2024-09-19T04:06:19.445 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:19 smithi094 ceph-mon[30972]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:19.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:19 smithi094 ceph-mon[30972]: Saving service mon spec with placement count:5 2024-09-19T04:06:19.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:19 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:20.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T04:06:20.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.252+0000 7f601be5b640 1 Processor -- start 2024-09-19T04:06:20.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.253+0000 7f601be5b640 1 -- start start 2024-09-19T04:06:20.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.253+0000 7f601be5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f6014107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:20.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.253+0000 7f601be5b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6014108090 con 0x7f60141076c0 2024-09-19T04:06:20.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.253+0000 7f6019bf7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f6014107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:20.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.253+0000 7f6019bf7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f6014107ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57440/0 (socket says 172.21.15.94:57440) 2024-09-19T04:06:20.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.253+0000 7f6019bf7640 1 -- 172.21.15.94:0/1869023620 learned_addr learned my addr 172.21.15.94:0/1869023620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.253+0000 7f6019bf7640 1 -- 172.21.15.94:0/1869023620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6014108820 con 0x7f60141076c0 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.254+0000 7f6019bf7640 1 --2- 172.21.15.94:0/1869023620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f6014107ac0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6008009820 tx=0x7f600802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe564823abf4e99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.254+0000 7f6018bf5640 1 -- 172.21.15.94:0/1869023620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6008037070 con 0x7f60141076c0 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.254+0000 7f6018bf5640 1 -- 172.21.15.94:0/1869023620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f60080328d0 con 0x7f60141076c0 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.254+0000 7f6018bf5640 1 -- 172.21.15.94:0/1869023620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60080307d0 con 0x7f60141076c0 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.255+0000 7f601be5b640 1 -- 172.21.15.94:0/1869023620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 msgr2=0x7f6014107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.255+0000 7f601be5b640 1 --2- 172.21.15.94:0/1869023620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f6014107ac0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6008009820 tx=0x7f600802a270 comp rx=0 tx=0).stop 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.255+0000 7f601be5b640 1 -- 172.21.15.94:0/1869023620 shutdown_connections 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.255+0000 7f601be5b640 1 --2- 172.21.15.94:0/1869023620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f6014107ac0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.255+0000 7f601be5b640 1 -- 172.21.15.94:0/1869023620 >> 172.21.15.94:0/1869023620 conn(0x7f6014102ed0 msgr2=0x7f60141052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.255+0000 7f601be5b640 1 -- 172.21.15.94:0/1869023620 shutdown_connections 2024-09-19T04:06:20.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.255+0000 7f601be5b640 1 -- 172.21.15.94:0/1869023620 wait complete. 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.255+0000 7f601be5b640 1 Processor -- start 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f601be5b640 1 -- start start 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f601be5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f601419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f601be5b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f601410bbd0 con 0x7f60141076c0 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f6019bf7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f601419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f6019bf7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f601419d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57454/0 (socket says 172.21.15.94:57454) 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f6019bf7640 1 -- 172.21.15.94:0/3197363761 learned_addr learned my addr 172.21.15.94:0/3197363761 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f6019bf7640 1 -- 172.21.15.94:0/3197363761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601419dd50 con 0x7f60141076c0 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f6019bf7640 1 --2- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f601419d810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6008009940 tx=0x7f6008032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f6006ffd640 1 -- 172.21.15.94:0/3197363761 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6008042070 con 0x7f60141076c0 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601419dfe0 con 0x7f60141076c0 2024-09-19T04:06:20.250 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f6006ffd640 1 -- 172.21.15.94:0/3197363761 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6008030c80 con 0x7f60141076c0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f6006ffd640 1 -- 172.21.15.94:0/3197363761 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6008037040 con 0x7f60141076c0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.256+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60141a0cd0 con 0x7f60141076c0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.257+0000 7f6006ffd640 1 -- 172.21.15.94:0/3197363761 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6008045070 con 0x7f60141076c0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.257+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601410bd50 con 0x7f60141076c0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.257+0000 7f6006ffd640 1 --2- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f60000391b0 0x7f600003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.257+0000 7f6006ffd640 1 -- 172.21.15.94:0/3197363761 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f600806c180 con 0x7f60141076c0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.258+0000 7f60193f6640 1 --2- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f60000391b0 0x7f600003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.258+0000 7f60193f6640 1 --2- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f60000391b0 0x7f600003b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f60100098c0 tx=0x7f6010006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.262+0000 7f6006ffd640 1 -- 172.21.15.94:0/3197363761 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6008045360 con 0x7f60141076c0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.443+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6014104090 con 0x7f60000391b0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.447+0000 7f6006ffd640 1 -- 172.21.15.94:0/3197363761 <== mgr.14118 v2:172.21.15.94:6800/3136401482 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6014104090 con 0x7f60000391b0 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f60000391b0 msgr2=0x7f600003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 --2- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f60000391b0 0x7f600003b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f60100098c0 tx=0x7f6010006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:20.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 msgr2=0x7f601419d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:20.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 --2- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f601419d810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6008009940 tx=0x7f6008032e10 comp rx=0 tx=0).stop 2024-09-19T04:06:20.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 shutdown_connections 2024-09-19T04:06:20.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 --2- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f60000391b0 0x7f600003b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:20.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 --2- 172.21.15.94:0/3197363761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60141076c0 0x7f601419d810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:20.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 >> 172.21.15.94:0/3197363761 conn(0x7f6014102ed0 msgr2=0x7f6014103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:20.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 shutdown_connections 2024-09-19T04:06:20.252 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:19.449+0000 7f601be5b640 1 -- 172.21.15.94:0/3197363761 wait complete. 2024-09-19T04:06:20.836 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:20 smithi094 ceph-mon[30972]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:20.836 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:20 smithi094 ceph-mon[30972]: Saving service mgr spec with placement count:2 2024-09-19T04:06:20.837 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:20 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:22.287 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:21 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:22.288 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:21 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:22.288 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:21 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/837353369' entity='client.admin' 2024-09-19T04:06:22.295 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.347+0000 7f4c540d2640 1 Processor -- start 2024-09-19T04:06:22.295 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.347+0000 7f4c540d2640 1 -- start start 2024-09-19T04:06:22.295 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.347+0000 7f4c540d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c10a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:22.295 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.347+0000 7f4c540d2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c07bde0 con 0x7f4c4c108140 2024-09-19T04:06:22.295 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.347+0000 7f4c51e6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c10a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:22.295 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.348+0000 7f4c51e6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c10a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57458/0 (socket says 172.21.15.94:57458) 2024-09-19T04:06:22.295 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.348+0000 7f4c51e6e640 1 -- 172.21.15.94:0/1165890148 learned_addr learned my addr 172.21.15.94:0/1165890148 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:22.295 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.348+0000 7f4c51e6e640 1 -- 172.21.15.94:0/1165890148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c10aa70 con 0x7f4c4c108140 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.348+0000 7f4c51e6e640 1 --2- 172.21.15.94:0/1165890148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c10a530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009820 tx=0x7f4c4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e184239bce6e96e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.349+0000 7f4c50e6c640 1 -- 172.21.15.94:0/1165890148 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c48037070 con 0x7f4c4c108140 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.349+0000 7f4c50e6c640 1 -- 172.21.15.94:0/1165890148 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c480328f0 con 0x7f4c4c108140 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.349+0000 7f4c50e6c640 1 -- 172.21.15.94:0/1165890148 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c48030740 con 0x7f4c4c108140 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.349+0000 7f4c540d2640 1 -- 172.21.15.94:0/1165890148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 msgr2=0x7f4c4c10a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.349+0000 7f4c540d2640 1 --2- 172.21.15.94:0/1165890148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c10a530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009820 tx=0x7f4c4802a270 comp rx=0 tx=0).stop 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.349+0000 7f4c540d2640 1 -- 172.21.15.94:0/1165890148 shutdown_connections 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.349+0000 7f4c540d2640 1 --2- 172.21.15.94:0/1165890148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c10a530 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.349+0000 7f4c540d2640 1 -- 172.21.15.94:0/1165890148 >> 172.21.15.94:0/1165890148 conn(0x7f4c4c101d80 msgr2=0x7f4c4c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.350+0000 7f4c540d2640 1 -- 172.21.15.94:0/1165890148 shutdown_connections 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.350+0000 7f4c540d2640 1 -- 172.21.15.94:0/1165890148 wait complete. 2024-09-19T04:06:22.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.350+0000 7f4c540d2640 1 Processor -- start 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.350+0000 7f4c540d2640 1 -- start start 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.350+0000 7f4c540d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.350+0000 7f4c540d2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c10ba30 con 0x7f4c4c108140 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.350+0000 7f4c51e6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.350+0000 7f4c51e6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c19d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57468/0 (socket says 172.21.15.94:57468) 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.350+0000 7f4c51e6e640 1 -- 172.21.15.94:0/837353369 learned_addr learned my addr 172.21.15.94:0/837353369 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.351+0000 7f4c51e6e640 1 -- 172.21.15.94:0/837353369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c19dd50 con 0x7f4c4c108140 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.351+0000 7f4c51e6e640 1 --2- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c19d810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c480327b0 tx=0x7f4c480327e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.351+0000 7f4c42ffd640 1 -- 172.21.15.94:0/837353369 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c48042070 con 0x7f4c4c108140 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.351+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c19dfe0 con 0x7f4c4c108140 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.351+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c1a0cd0 con 0x7f4c4c108140 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.351+0000 7f4c42ffd640 1 -- 172.21.15.94:0/837353369 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c480328d0 con 0x7f4c4c108140 2024-09-19T04:06:22.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.351+0000 7f4c42ffd640 1 -- 172.21.15.94:0/837353369 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c48037040 con 0x7f4c4c108140 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.352+0000 7f4c42ffd640 1 -- 172.21.15.94:0/837353369 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4c48045070 con 0x7f4c4c108140 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.352+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c300052c0 con 0x7f4c4c108140 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.352+0000 7f4c42ffd640 1 --2- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4c380391b0 0x7f4c3803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.353+0000 7f4c42ffd640 1 -- 172.21.15.94:0/837353369 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4c48030d50 con 0x7f4c4c108140 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.353+0000 7f4c5166d640 1 --2- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4c380391b0 0x7f4c3803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.353+0000 7f4c5166d640 1 --2- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4c380391b0 0x7f4c3803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0098c0 tx=0x7f4c3c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.357+0000 7f4c42ffd640 1 -- 172.21.15.94:0/837353369 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4c4806c8b0 con 0x7f4c4c108140 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.541+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4c300055b0 con 0x7f4c4c108140 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.543+0000 7f4c42ffd640 1 -- 172.21.15.94:0/837353369 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4c48042210 con 0x7f4c4c108140 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4c380391b0 msgr2=0x7f4c3803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 --2- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4c380391b0 0x7f4c3803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0098c0 tx=0x7f4c3c006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:22.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 msgr2=0x7f4c4c19d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:22.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 --2- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c19d810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c480327b0 tx=0x7f4c480327e0 comp rx=0 tx=0).stop 2024-09-19T04:06:22.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 shutdown_connections 2024-09-19T04:06:22.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 --2- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4c380391b0 0x7f4c3803b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:22.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 --2- 172.21.15.94:0/837353369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c4c108140 0x7f4c4c19d810 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:22.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 >> 172.21.15.94:0/837353369 conn(0x7f4c4c101d80 msgr2=0x7f4c4c102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:22.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 shutdown_connections 2024-09-19T04:06:22.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:21.546+0000 7f4c540d2640 1 -- 172.21.15.94:0/837353369 wait complete. 2024-09-19T04:06:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.659+0000 7f4da584e640 1 Processor -- start 2024-09-19T04:06:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.659+0000 7f4da584e640 1 -- start start 2024-09-19T04:06:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.659+0000 7f4da584e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da0107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.659+0000 7f4da584e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01080b0 con 0x7f4da01076e0 2024-09-19T04:06:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.659+0000 7f4d9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da0107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.659+0000 7f4d9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da0107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33788/0 (socket says 172.21.15.94:33788) 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.659+0000 7f4d9effd640 1 -- 172.21.15.94:0/3843811410 learned_addr learned my addr 172.21.15.94:0/3843811410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.659+0000 7f4d9effd640 1 -- 172.21.15.94:0/3843811410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da0108840 con 0x7f4da01076e0 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.660+0000 7f4d9effd640 1 --2- 172.21.15.94:0/3843811410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da0107ae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009820 tx=0x7f4d9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f852817603ed66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.660+0000 7f4d9e7fc640 1 -- 172.21.15.94:0/3843811410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d90037070 con 0x7f4da01076e0 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.660+0000 7f4d9e7fc640 1 -- 172.21.15.94:0/3843811410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d900328f0 con 0x7f4da01076e0 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.660+0000 7f4d9e7fc640 1 -- 172.21.15.94:0/3843811410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d90030740 con 0x7f4da01076e0 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.661+0000 7f4da584e640 1 -- 172.21.15.94:0/3843811410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 msgr2=0x7f4da0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.661+0000 7f4da584e640 1 --2- 172.21.15.94:0/3843811410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da0107ae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009820 tx=0x7f4d9002a270 comp rx=0 tx=0).stop 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.661+0000 7f4da584e640 1 -- 172.21.15.94:0/3843811410 shutdown_connections 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.661+0000 7f4da584e640 1 --2- 172.21.15.94:0/3843811410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da0107ae0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.661+0000 7f4da584e640 1 -- 172.21.15.94:0/3843811410 >> 172.21.15.94:0/3843811410 conn(0x7f4da0102ed0 msgr2=0x7f4da01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.661+0000 7f4da584e640 1 -- 172.21.15.94:0/3843811410 shutdown_connections 2024-09-19T04:06:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.661+0000 7f4da584e640 1 -- 172.21.15.94:0/3843811410 wait complete. 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.661+0000 7f4da584e640 1 Processor -- start 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.661+0000 7f4da584e640 1 -- start start 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4da584e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4da584e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4da010bbf0 con 0x7f4da01076e0 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4d9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4d9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33792/0 (socket says 172.21.15.94:33792) 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4d9effd640 1 -- 172.21.15.94:0/665084837 learned_addr learned my addr 172.21.15.94:0/665084837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4d9effd640 1 -- 172.21.15.94:0/665084837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da007e090 con 0x7f4da01076e0 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4d9effd640 1 --2- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da007db50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009940 tx=0x7f4d90030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:23.707 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4da484c640 1 -- 172.21.15.94:0/665084837 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d90042070 con 0x7f4da01076e0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4da484c640 1 -- 172.21.15.94:0/665084837 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d90030e30 con 0x7f4da01076e0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da007c280 con 0x7f4da01076e0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4da484c640 1 -- 172.21.15.94:0/665084837 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d90037040 con 0x7f4da01076e0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.662+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da007c7c0 con 0x7f4da01076e0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.663+0000 7f4da484c640 1 -- 172.21.15.94:0/665084837 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4d90045070 con 0x7f4da01076e0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.664+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da0107b60 con 0x7f4da01076e0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.664+0000 7f4da484c640 1 --2- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4d8c039200 0x7f4d8c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.664+0000 7f4da484c640 1 -- 172.21.15.94:0/665084837 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d9006c440 con 0x7f4da01076e0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.664+0000 7f4d967fc640 1 --2- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4d8c039200 0x7f4d8c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.664+0000 7f4d967fc640 1 --2- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4d8c039200 0x7f4d8c03b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4d880098c0 tx=0x7f4d88006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.668+0000 7f4da484c640 1 -- 172.21.15.94:0/665084837 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4d90045360 con 0x7f4da01076e0 2024-09-19T04:06:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.836+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4da007ca40 con 0x7f4da01076e0 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.839+0000 7f4da484c640 1 -- 172.21.15.94:0/665084837 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4d9003b330 con 0x7f4da01076e0 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4d8c039200 msgr2=0x7f4d8c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 --2- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4d8c039200 0x7f4d8c03b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4d880098c0 tx=0x7f4d88006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 msgr2=0x7f4da007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 --2- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da007db50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009940 tx=0x7f4d90030cc0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 shutdown_connections 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 --2- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f4d8c039200 0x7f4d8c03b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 --2- 172.21.15.94:0/665084837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da01076e0 0x7f4da007db50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 >> 172.21.15.94:0/665084837 conn(0x7f4da0102ed0 msgr2=0x7f4da0103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 shutdown_connections 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:22.841+0000 7f4da584e640 1 -- 172.21.15.94:0/665084837 wait complete. 2024-09-19T04:06:23.709 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling the dashboard module... 2024-09-19T04:06:24.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:23 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/665084837' entity='client.admin' 2024-09-19T04:06:24.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:23 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:25.534 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-mon[30972]: from='mgr.14118 172.21.15.94:0/69125653' entity='mgr.x' 2024-09-19T04:06:25.534 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2588603070' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T04:06:25.534 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: ignoring --setuser ceph since I am not root 2024-09-19T04:06:25.535 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: ignoring --setgroup ceph since I am not root 2024-09-19T04:06:25.535 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:25.375+0000 7f7555e32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:06:25.535 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:25.432+0000 7f7555e32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:06:25.796 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:25.582+0000 7f7555e32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:06:25.796 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:25.648+0000 7f7555e32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:06:25.796 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:25.794+0000 7f7555e32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:06:26.090 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:25.937+0000 7f7555e32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:06:26.254 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.156+0000 7faf32941640 1 Processor -- start 2024-09-19T04:06:26.254 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.156+0000 7faf32941640 1 -- start start 2024-09-19T04:06:26.254 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.157+0000 7faf32941640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c10a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.157+0000 7faf32941640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c07bdc0 con 0x7faf2c108160 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.157+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c10a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.157+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c10a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33806/0 (socket says 172.21.15.94:33806) 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.157+0000 7faf2bfff640 1 -- 172.21.15.94:0/1738388660 learned_addr learned my addr 172.21.15.94:0/1738388660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.157+0000 7faf2bfff640 1 -- 172.21.15.94:0/1738388660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c10aa90 con 0x7faf2c108160 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.158+0000 7faf2bfff640 1 --2- 172.21.15.94:0/1738388660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c10a550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009820 tx=0x7faf1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3138821a24e99f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.158+0000 7faf2b7fe640 1 -- 172.21.15.94:0/1738388660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf1c037070 con 0x7faf2c108160 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.158+0000 7faf2b7fe640 1 -- 172.21.15.94:0/1738388660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf1c0328f0 con 0x7faf2c108160 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.158+0000 7faf2b7fe640 1 -- 172.21.15.94:0/1738388660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf1c030740 con 0x7faf2c108160 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 -- 172.21.15.94:0/1738388660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 msgr2=0x7faf2c10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 --2- 172.21.15.94:0/1738388660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c10a550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009820 tx=0x7faf1c02a270 comp rx=0 tx=0).stop 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 -- 172.21.15.94:0/1738388660 shutdown_connections 2024-09-19T04:06:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 --2- 172.21.15.94:0/1738388660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c10a550 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 -- 172.21.15.94:0/1738388660 >> 172.21.15.94:0/1738388660 conn(0x7faf2c101d80 msgr2=0x7faf2c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 -- 172.21.15.94:0/1738388660 shutdown_connections 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 -- 172.21.15.94:0/1738388660 wait complete. 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 Processor -- start 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 -- start start 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.159+0000 7faf32941640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c1079e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf32941640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c10b9f0 con 0x7faf2c108160 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c1079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c1079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33818/0 (socket says 172.21.15.94:33818) 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf2bfff640 1 -- 172.21.15.94:0/2588603070 learned_addr learned my addr 172.21.15.94:0/2588603070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf2bfff640 1 -- 172.21.15.94:0/2588603070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c107f20 con 0x7faf2c108160 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf2bfff640 1 --2- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c1079e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0097f0 tx=0x7faf1c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:26.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf29ffb640 1 -- 172.21.15.94:0/2588603070 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf1c042070 con 0x7faf2c108160 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf29ffb640 1 -- 172.21.15.94:0/2588603070 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf1c030e80 con 0x7faf2c108160 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2c106110 con 0x7faf2c108160 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf29ffb640 1 -- 172.21.15.94:0/2588603070 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf1c037040 con 0x7faf2c108160 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.160+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2c106650 con 0x7faf2c108160 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.161+0000 7faf29ffb640 1 -- 172.21.15.94:0/2588603070 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7faf1c045070 con 0x7faf2c108160 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.161+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf2c1085c0 con 0x7faf2c108160 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.161+0000 7faf29ffb640 1 --2- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7faf180391b0 0x7faf1803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.161+0000 7faf29ffb640 1 -- 172.21.15.94:0/2588603070 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faf1c06c350 con 0x7faf2c108160 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.161+0000 7faf23fff640 1 --2- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7faf180391b0 0x7faf1803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.162+0000 7faf23fff640 1 --2- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7faf180391b0 0x7faf1803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf140098c0 tx=0x7faf14006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.166+0000 7faf29ffb640 1 -- 172.21.15.94:0/2588603070 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf1c04b170 con 0x7faf2c108160 2024-09-19T04:06:26.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:24.399+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7faf2c106f30 con 0x7faf2c108160 2024-09-19T04:06:26.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.230+0000 7faf29ffb640 1 -- 172.21.15.94:0/2588603070 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7faf1c03d7b0 con 0x7faf2c108160 2024-09-19T04:06:26.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.231+0000 7faf29ffb640 1 -- 172.21.15.94:0/2588603070 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7faf1c03db70 con 0x7faf2c108160 2024-09-19T04:06:26.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7faf180391b0 msgr2=0x7faf1803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:26.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 --2- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7faf180391b0 0x7faf1803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf140098c0 tx=0x7faf14006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 msgr2=0x7faf2c1079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:26.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 --2- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c1079e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0097f0 tx=0x7faf1c030c70 comp rx=0 tx=0).stop 2024-09-19T04:06:26.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 shutdown_connections 2024-09-19T04:06:26.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 --2- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7faf180391b0 0x7faf1803b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.260 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 --2- 172.21.15.94:0/2588603070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2c108160 0x7faf2c1079e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.260 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 >> 172.21.15.94:0/2588603070 conn(0x7faf2c101d80 msgr2=0x7faf2c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:26.260 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 shutdown_connections 2024-09-19T04:06:26.260 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:25.233+0000 7faf32941640 1 -- 172.21.15.94:0/2588603070 wait complete. 2024-09-19T04:06:26.506 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:26 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2588603070' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T04:06:26.506 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:26 smithi094 ceph-mon[30972]: mgrmap e9: x(active, since 18s) 2024-09-19T04:06:26.506 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:26 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:26.248+0000 7f7555e32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:06:26.506 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:26 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:26.313+0000 7f7555e32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:06:26.506 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:26 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:26.504+0000 7f7555e32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:06:27.471 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:27 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/981118319' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:06:27.472 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:27 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:27.421+0000 7f7555e32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:06:27.474 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:06:27.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T04:06:27.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:06:27.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:06:27.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.644+0000 7f9f1f997640 1 Processor -- start 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.644+0000 7f9f1f997640 1 -- start start 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.644+0000 7f9f1f997640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1810a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.644+0000 7f9f1f997640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1807be90 con 0x7f9f181080b0 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.645+0000 7f9f1d733640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1810a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.645+0000 7f9f1d733640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1810a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33844/0 (socket says 172.21.15.94:33844) 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.645+0000 7f9f1d733640 1 -- 172.21.15.94:0/4037191575 learned_addr learned my addr 172.21.15.94:0/4037191575 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.645+0000 7f9f1d733640 1 -- 172.21.15.94:0/4037191575 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1810a9e0 con 0x7f9f181080b0 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.646+0000 7f9f1d733640 1 --2- 172.21.15.94:0/4037191575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1810a4a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009820 tx=0x7f9f0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1863937c0179999 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.646+0000 7f9f0ffff640 1 -- 172.21.15.94:0/4037191575 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f08037070 con 0x7f9f181080b0 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.646+0000 7f9f0ffff640 1 -- 172.21.15.94:0/4037191575 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f080328f0 con 0x7f9f181080b0 2024-09-19T04:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.646+0000 7f9f0ffff640 1 -- 172.21.15.94:0/4037191575 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f08030740 con 0x7f9f181080b0 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.647+0000 7f9f1f997640 1 -- 172.21.15.94:0/4037191575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 msgr2=0x7f9f1810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.647+0000 7f9f1f997640 1 --2- 172.21.15.94:0/4037191575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1810a4a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009820 tx=0x7f9f0802a270 comp rx=0 tx=0).stop 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.647+0000 7f9f1f997640 1 -- 172.21.15.94:0/4037191575 shutdown_connections 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.647+0000 7f9f1f997640 1 --2- 172.21.15.94:0/4037191575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1810a4a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.647+0000 7f9f1f997640 1 -- 172.21.15.94:0/4037191575 >> 172.21.15.94:0/4037191575 conn(0x7f9f18101d90 msgr2=0x7f9f181041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.647+0000 7f9f1f997640 1 -- 172.21.15.94:0/4037191575 shutdown_connections 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.647+0000 7f9f1f997640 1 -- 172.21.15.94:0/4037191575 wait complete. 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.647+0000 7f9f1f997640 1 Processor -- start 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.647+0000 7f9f1f997640 1 -- start start 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f1f997640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f1f997640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810ba60 con 0x7f9f181080b0 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f1d733640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f1d733640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1819b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33856/0 (socket says 172.21.15.94:33856) 2024-09-19T04:06:27.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f1d733640 1 -- 172.21.15.94:0/981118319 learned_addr learned my addr 172.21.15.94:0/981118319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f1d733640 1 -- 172.21.15.94:0/981118319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1819bb00 con 0x7f9f181080b0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f1d733640 1 --2- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1819b5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f080097f0 tx=0x7f9f08030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f0e7fc640 1 -- 172.21.15.94:0/981118319 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f08042070 con 0x7f9f181080b0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f0e7fc640 1 -- 172.21.15.94:0/981118319 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f08030d80 con 0x7f9f181080b0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1819bd90 con 0x7f9f181080b0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f0e7fc640 1 -- 172.21.15.94:0/981118319 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f08037040 con 0x7f9f181080b0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.648+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1819ea80 con 0x7f9f181080b0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.649+0000 7f9f0e7fc640 1 -- 172.21.15.94:0/981118319 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f9f08045070 con 0x7f9f181080b0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.650+0000 7f9f0e7fc640 1 --2- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f9f04039260 0x7f9f0403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.650+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f18108510 con 0x7f9f181080b0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.650+0000 7f9f1cf32640 1 -- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f9f04039260 msgr2=0x7f9f0403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3136401482 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.650+0000 7f9f0e7fc640 1 -- 172.21.15.94:0/981118319 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9f0806c480 con 0x7f9f181080b0 2024-09-19T04:06:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.650+0000 7f9f1cf32640 1 --2- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f9f04039260 0x7f9f0403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.655+0000 7f9f0e7fc640 1 -- 172.21.15.94:0/981118319 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f0803add0 con 0x7f9f181080b0 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.851+0000 7f9f1cf32640 1 -- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f9f04039260 msgr2=0x7f9f0403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3136401482 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.851+0000 7f9f1cf32640 1 --2- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f9f04039260 0x7f9f0403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.888+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9f1819ec50 con 0x7f9f181080b0 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.889+0000 7f9f0e7fc640 1 -- 172.21.15.94:0/981118319 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f9f1819ec50 con 0x7f9f181080b0 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f9f04039260 msgr2=0x7f9f0403b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 --2- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f9f04039260 0x7f9f0403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 msgr2=0x7f9f1819b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 --2- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1819b5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f080097f0 tx=0x7f9f08030c70 comp rx=0 tx=0).stop 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 shutdown_connections 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 --2- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7f9f04039260 0x7f9f0403b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:27.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 --2- 172.21.15.94:0/981118319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f181080b0 0x7f9f1819b5c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:27.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 >> 172.21.15.94:0/981118319 conn(0x7f9f18101d90 msgr2=0x7f9f181028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:27.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 shutdown_connections 2024-09-19T04:06:27.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:26.891+0000 7f9f1f997640 1 -- 172.21.15.94:0/981118319 wait complete. 2024-09-19T04:06:27.485 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-09-19T04:06:27.485 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 9... 2024-09-19T04:06:27.724 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:27 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:27.491+0000 7f7555e32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:06:28.466 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:28.212+0000 7f7555e32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:06:28.466 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:28.325+0000 7f7555e32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:06:28.740 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:28.465+0000 7f7555e32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:06:28.740 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:28.526+0000 7f7555e32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:06:28.740 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:28.672+0000 7f7555e32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:06:28.740 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:28.738+0000 7f7555e32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:06:29.018 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:28.797+0000 7f7555e32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:06:29.018 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:28.952+0000 7f7555e32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:06:29.018 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:29 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:29.016+0000 7f7555e32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:06:29.280 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:29 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:29.081+0000 7f7555e32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:06:29.281 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:29 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:29.162+0000 7f7555e32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:06:29.281 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:29 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:29.220+0000 7f7555e32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:06:29.281 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:29 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:29.278+0000 7f7555e32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:06:30.340 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:30.051+0000 7f7555e32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:06:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: Active manager daemon x restarted 2024-09-19T04:06:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: Activating manager daemon x 2024-09-19T04:06:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T04:06:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: mgrmap e10: x(active, starting, since 0.00332982s) 2024-09-19T04:06:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:06:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:06:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:06:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:06:30.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:06:30.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: Manager daemon x is now available 2024-09-19T04:06:30.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:30.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:06:30.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:06:31.498 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:31.108+0000 7f74bb8fb640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:06:31] ENGINE Error in HTTPServer.serve 2024-09-19T04:06:31.498 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: Traceback (most recent call last): 2024-09-19T04:06:31.498 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:06:31.498 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self._connections.run(self.expiration_interval) 2024-09-19T04:06:31.498 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:06:31.498 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self._run(expiration_interval) 2024-09-19T04:06:31.498 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:06:31.498 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: s = self.context.wrap_socket( 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: return self.sslsocket_class._create( 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self.do_handshake() 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self._sslobj.do_handshake() 2024-09-19T04:06:31.499 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:06:31.500 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:06:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.910+0000 7fdd8338c640 1 Processor -- start 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.910+0000 7fdd8338c640 1 -- start start 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.910+0000 7fdd8338c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c10a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.910+0000 7fdd8338c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c07be40 con 0x7fdd7c1080e0 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.911+0000 7fdd81128640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c10a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.911+0000 7fdd81128640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c10a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33870/0 (socket says 172.21.15.94:33870) 2024-09-19T04:06:31.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.911+0000 7fdd81128640 1 -- 172.21.15.94:0/1529665159 learned_addr learned my addr 172.21.15.94:0/1529665159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:31.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.911+0000 7fdd81128640 1 -- 172.21.15.94:0/1529665159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c10aa10 con 0x7fdd7c1080e0 2024-09-19T04:06:31.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.911+0000 7fdd81128640 1 --2- 172.21.15.94:0/1529665159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c10a4d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009820 tx=0x7fdd7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3a70ef8eb1f4a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:31.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd73fff640 1 -- 172.21.15.94:0/1529665159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd78037070 con 0x7fdd7c1080e0 2024-09-19T04:06:31.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd73fff640 1 -- 172.21.15.94:0/1529665159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd780328f0 con 0x7fdd7c1080e0 2024-09-19T04:06:31.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd73fff640 1 -- 172.21.15.94:0/1529665159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd78030740 con 0x7fdd7c1080e0 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd8338c640 1 -- 172.21.15.94:0/1529665159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 msgr2=0x7fdd7c10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd8338c640 1 --2- 172.21.15.94:0/1529665159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c10a4d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009820 tx=0x7fdd7802a270 comp rx=0 tx=0).stop 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd8338c640 1 -- 172.21.15.94:0/1529665159 shutdown_connections 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd8338c640 1 --2- 172.21.15.94:0/1529665159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c10a4d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd8338c640 1 -- 172.21.15.94:0/1529665159 >> 172.21.15.94:0/1529665159 conn(0x7fdd7c101d80 msgr2=0x7fdd7c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd8338c640 1 -- 172.21.15.94:0/1529665159 shutdown_connections 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.912+0000 7fdd8338c640 1 -- 172.21.15.94:0/1529665159 wait complete. 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.913+0000 7fdd8338c640 1 Processor -- start 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.913+0000 7fdd8338c640 1 -- start start 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.913+0000 7fdd8338c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c19b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.913+0000 7fdd8338c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c10ba30 con 0x7fdd7c1080e0 2024-09-19T04:06:31.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.913+0000 7fdd81128640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c19b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.913+0000 7fdd81128640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c19b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33882/0 (socket says 172.21.15.94:33882) 2024-09-19T04:06:31.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.913+0000 7fdd81128640 1 -- 172.21.15.94:0/1585352368 learned_addr learned my addr 172.21.15.94:0/1585352368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:31.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.913+0000 7fdd81128640 1 -- 172.21.15.94:0/1585352368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c19bb50 con 0x7fdd7c1080e0 2024-09-19T04:06:31.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.914+0000 7fdd81128640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c19b610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdd780327b0 tx=0x7fdd780327e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:31.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.914+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd78042070 con 0x7fdd7c1080e0 2024-09-19T04:06:31.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.914+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd78030d50 con 0x7fdd7c1080e0 2024-09-19T04:06:31.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.914+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7c19bde0 con 0x7fdd7c1080e0 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.914+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd78037040 con 0x7fdd7c1080e0 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.914+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd7c19ead0 con 0x7fdd7c1080e0 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.915+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fdd78045070 con 0x7fdd7c1080e0 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.915+0000 7fdd727fc640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 0x7fdd6803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.915+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 --> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd7c108540 con 0x7fdd68039200 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.915+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdd7806c350 con 0x7fdd7c1080e0 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.915+0000 7fdd80927640 1 -- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 msgr2=0x7fdd6803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3136401482 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:27.915+0000 7fdd80927640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 0x7fdd6803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:28.116+0000 7fdd80927640 1 -- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 msgr2=0x7fdd6803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3136401482 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:28.116+0000 7fdd80927640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 0x7fdd6803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:28.517+0000 7fdd80927640 1 -- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 msgr2=0x7fdd6803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3136401482 2024-09-19T04:06:31.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:28.517+0000 7fdd80927640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 0x7fdd6803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:29.318+0000 7fdd80927640 1 -- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 msgr2=0x7fdd6803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3136401482 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:29.318+0000 7fdd80927640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 0x7fdd6803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:30.054+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7fdd7806ae00 con 0x7fdd7c1080e0 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:30.055+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 msgr2=0x7fdd6803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:30.055+0000 7fdd727fc640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 0x7fdd6803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.056+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fdd7803b330 con 0x7fdd7c1080e0 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.056+0000 7fdd727fc640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdd6803cad0 0x7fdd6803eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.056+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd7c108540 con 0x7fdd6803cad0 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.057+0000 7fdd80927640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdd6803cad0 0x7fdd6803eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.057+0000 7fdd80927640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdd6803cad0 0x7fdd6803eec0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c003980 tx=0x7fdd6c0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:31.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.058+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fdd7c108540 con 0x7fdd6803cad0 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fdd7c108540 con 0x7fdd6803cad0 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd727fc640 1 -- 172.21.15.94:0/1585352368 <== mgr.14150 v2:172.21.15.94:6800/2308569937 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fdd7c108540 con 0x7fdd6803cad0 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdd6803cad0 msgr2=0x7fdd6803eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdd6803cad0 0x7fdd6803eec0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c003980 tx=0x7fdd6c0092b0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 msgr2=0x7fdd7c19b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c19b610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdd780327b0 tx=0x7fdd780327e0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 shutdown_connections 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdd6803cad0 0x7fdd6803eec0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:6800/3136401482,v1:172.21.15.94:6801/3136401482] conn(0x7fdd68039200 0x7fdd6803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 --2- 172.21.15.94:0/1585352368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd7c1080e0 0x7fdd7c19b610 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 >> 172.21.15.94:0/1585352368 conn(0x7fdd7c101d80 msgr2=0x7fdd7c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 shutdown_connections 2024-09-19T04:06:31.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:31.065+0000 7fdd8338c640 1 -- 172.21.15.94:0/1585352368 wait complete. 2024-09-19T04:06:31.867 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 9 is available 2024-09-19T04:06:31.867 INFO:teuthology.orchestra.run.smithi094.stdout:Generating a dashboard self-signed certificate... 2024-09-19T04:06:32.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:32 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:30] ENGINE Bus STARTING 2024-09-19T04:06:32.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:32 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:30] ENGINE Serving on http://172.21.15.94:8765 2024-09-19T04:06:32.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:32 smithi094 ceph-mon[30972]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T04:06:32.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:32 smithi094 ceph-mon[30972]: mgrmap e11: x(active, since 1.00576s) 2024-09-19T04:06:32.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:32 smithi094 ceph-mon[30972]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T04:06:32.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:32 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:31] ENGINE Serving on https://172.21.15.94:7150 2024-09-19T04:06:32.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:32 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:31] ENGINE Bus STARTED 2024-09-19T04:06:32.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:32 smithi094 ceph-mon[30972]: [19/Sep/2024:04:06:31] ENGINE Error in HTTPServer.serve 2024-09-19T04:06:32.102 INFO:journalctl@ceph.mon.a.smithi094.stdout: Traceback (most recent call last): 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: self._run(expiration_interval) 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: s = self.context.wrap_socket( 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: return self.sslsocket_class._create( 2024-09-19T04:06:32.103 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:06:32.104 INFO:journalctl@ceph.mon.a.smithi094.stdout: self.do_handshake() 2024-09-19T04:06:32.104 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:06:32.104 INFO:journalctl@ceph.mon.a.smithi094.stdout: self._sslobj.do_handshake() 2024-09-19T04:06:32.104 INFO:journalctl@ceph.mon.a.smithi094.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:06:33.158 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T04:06:33.158 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.295+0000 7fd7f0dcf640 1 Processor -- start 2024-09-19T04:06:33.158 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.295+0000 7fd7f0dcf640 1 -- start start 2024-09-19T04:06:33.158 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.296+0000 7fd7f0dcf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:33.158 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.296+0000 7fd7f0dcf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec1080b0 con 0x7fd7ec1076e0 2024-09-19T04:06:33.158 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.296+0000 7fd7ea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:33.158 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.296+0000 7fd7ea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60828/0 (socket says 172.21.15.94:60828) 2024-09-19T04:06:33.158 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.296+0000 7fd7ea59c640 1 -- 172.21.15.94:0/246047994 learned_addr learned my addr 172.21.15.94:0/246047994 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:33.158 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.296+0000 7fd7ea59c640 1 -- 172.21.15.94:0/246047994 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec108840 con 0x7fd7ec1076e0 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.297+0000 7fd7ea59c640 1 --2- 172.21.15.94:0/246047994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec107ae0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009820 tx=0x7fd7e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba93fc1228c7216c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.297+0000 7fd7e959a640 1 -- 172.21.15.94:0/246047994 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e0037070 con 0x7fd7ec1076e0 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.297+0000 7fd7e959a640 1 -- 172.21.15.94:0/246047994 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7e00328f0 con 0x7fd7ec1076e0 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.297+0000 7fd7e959a640 1 -- 172.21.15.94:0/246047994 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e0030740 con 0x7fd7ec1076e0 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.298+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/246047994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 msgr2=0x7fd7ec107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.298+0000 7fd7f0dcf640 1 --2- 172.21.15.94:0/246047994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec107ae0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009820 tx=0x7fd7e002a270 comp rx=0 tx=0).stop 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.298+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/246047994 shutdown_connections 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.298+0000 7fd7f0dcf640 1 --2- 172.21.15.94:0/246047994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec107ae0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.298+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/246047994 >> 172.21.15.94:0/246047994 conn(0x7fd7ec102ed0 msgr2=0x7fd7ec1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.298+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/246047994 shutdown_connections 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.298+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/246047994 wait complete. 2024-09-19T04:06:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.298+0000 7fd7f0dcf640 1 Processor -- start 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7f0dcf640 1 -- start start 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7f0dcf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7f0dcf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec10bbf0 con 0x7fd7ec1076e0 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7ea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7ea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec19b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60830/0 (socket says 172.21.15.94:60830) 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7ea59c640 1 -- 172.21.15.94:0/4206796803 learned_addr learned my addr 172.21.15.94:0/4206796803 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7ea59c640 1 -- 172.21.15.94:0/4206796803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec19bba0 con 0x7fd7ec1076e0 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7ea59c640 1 --2- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec19b660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009940 tx=0x7fd7e0030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7db7fe640 1 -- 172.21.15.94:0/4206796803 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e0042070 con 0x7fd7ec1076e0 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ec19be30 con 0x7fd7ec1076e0 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7db7fe640 1 -- 172.21.15.94:0/4206796803 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7e0030e30 con 0x7fd7ec1076e0 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7db7fe640 1 -- 172.21.15.94:0/4206796803 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e0037040 con 0x7fd7ec1076e0 2024-09-19T04:06:33.160 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.299+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ec19eb20 con 0x7fd7ec1076e0 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.300+0000 7fd7db7fe640 1 -- 172.21.15.94:0/4206796803 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fd7e0045070 con 0x7fd7ec1076e0 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.300+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ec10bd70 con 0x7fd7ec1076e0 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.300+0000 7fd7db7fe640 1 --2- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd7d4039200 0x7fd7d403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.300+0000 7fd7db7fe640 1 -- 172.21.15.94:0/4206796803 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd7e006c450 con 0x7fd7ec1076e0 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.301+0000 7fd7e9d9b640 1 --2- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd7d4039200 0x7fd7d403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.301+0000 7fd7e9d9b640 1 --2- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd7d4039200 0x7fd7d403b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0098c0 tx=0x7fd7dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.305+0000 7fd7db7fe640 1 -- 172.21.15.94:0/4206796803 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd7e004b170 con 0x7fd7ec1076e0 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.504+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fd7ec104180 con 0x7fd7d4039200 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.560+0000 7fd7db7fe640 1 -- 172.21.15.94:0/4206796803 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fd7ec104180 con 0x7fd7d4039200 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.562+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd7d4039200 msgr2=0x7fd7d403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.562+0000 7fd7f0dcf640 1 --2- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd7d4039200 0x7fd7d403b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0098c0 tx=0x7fd7dc006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:33.161 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.562+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 msgr2=0x7fd7ec19b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:33.162 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.562+0000 7fd7f0dcf640 1 --2- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec19b660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009940 tx=0x7fd7e0030cc0 comp rx=0 tx=0).stop 2024-09-19T04:06:33.162 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.562+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 shutdown_connections 2024-09-19T04:06:33.162 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.562+0000 7fd7f0dcf640 1 --2- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd7d4039200 0x7fd7d403b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:33.162 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.562+0000 7fd7f0dcf640 1 --2- 172.21.15.94:0/4206796803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7ec1076e0 0x7fd7ec19b660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:33.162 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.562+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 >> 172.21.15.94:0/4206796803 conn(0x7fd7ec102ed0 msgr2=0x7fd7ec103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:33.162 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.563+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 shutdown_connections 2024-09-19T04:06:33.162 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:32.563+0000 7fd7f0dcf640 1 -- 172.21.15.94:0/4206796803 wait complete. 2024-09-19T04:06:33.162 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial admin user... 2024-09-19T04:06:33.276 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:33 smithi094 ceph-mon[30972]: mgrmap e12: x(active, since 2s) 2024-09-19T04:06:33.276 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:33.276 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:33.276 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:34.291 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:34 smithi094 ceph-mon[30972]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:34.291 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:34.291 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$h.lS1Paa.FAtgjCtXhXOI.Y3AqNPDrtIKjRvLExYajA4t//Rwa7H.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726718793, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.574+0000 7fd07f45a640 1 Processor -- start 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.574+0000 7fd07f45a640 1 -- start start 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.575+0000 7fd07f45a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd078107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.575+0000 7fd07f45a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781080b0 con 0x7fd0781076e0 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.575+0000 7fd07d1f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd078107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.575+0000 7fd07d1f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd078107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60832/0 (socket says 172.21.15.94:60832) 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.575+0000 7fd07d1f6640 1 -- 172.21.15.94:0/1025616473 learned_addr learned my addr 172.21.15.94:0/1025616473 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.575+0000 7fd07d1f6640 1 -- 172.21.15.94:0/1025616473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd078108840 con 0x7fd0781076e0 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.576+0000 7fd07d1f6640 1 --2- 172.21.15.94:0/1025616473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd078107ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd068009e80 tx=0x7fd06802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a164e8ef4eb3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:34.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.576+0000 7fd077fff640 1 -- 172.21.15.94:0/1025616473 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd068036070 con 0x7fd0781076e0 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.576+0000 7fd077fff640 1 -- 172.21.15.94:0/1025616473 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0680318b0 con 0x7fd0781076e0 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.576+0000 7fd077fff640 1 -- 172.21.15.94:0/1025616473 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd068007930 con 0x7fd0781076e0 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 -- 172.21.15.94:0/1025616473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 msgr2=0x7fd078107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 --2- 172.21.15.94:0/1025616473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd078107ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd068009e80 tx=0x7fd06802a7a0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 -- 172.21.15.94:0/1025616473 shutdown_connections 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 --2- 172.21.15.94:0/1025616473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd078107ae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 -- 172.21.15.94:0/1025616473 >> 172.21.15.94:0/1025616473 conn(0x7fd078102ed0 msgr2=0x7fd0781052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 -- 172.21.15.94:0/1025616473 shutdown_connections 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 -- 172.21.15.94:0/1025616473 wait complete. 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 Processor -- start 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 -- start start 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd07819b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.577+0000 7fd07f45a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd07810bbf0 con 0x7fd0781076e0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd07d1f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd07819b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd07d1f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd07819b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60846/0 (socket says 172.21.15.94:60846) 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd07d1f6640 1 -- 172.21.15.94:0/4286520522 learned_addr learned my addr 172.21.15.94:0/4286520522 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd07d1f6640 1 -- 172.21.15.94:0/4286520522 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07819bba0 con 0x7fd0781076e0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd07d1f6640 1 --2- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd07819b660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd06802acc0 tx=0x7fd068031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd0767fc640 1 -- 172.21.15.94:0/4286520522 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd068036050 con 0x7fd0781076e0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd0767fc640 1 -- 172.21.15.94:0/4286520522 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd068037040 con 0x7fd0781076e0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd0767fc640 1 -- 172.21.15.94:0/4286520522 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd06803d9c0 con 0x7fd0781076e0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07819be30 con 0x7fd0781076e0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.578+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07819eb20 con 0x7fd0781076e0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.579+0000 7fd0767fc640 1 -- 172.21.15.94:0/4286520522 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fd068044020 con 0x7fd0781076e0 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.579+0000 7fd0767fc640 1 --2- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd060039260 0x7fd06003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.579+0000 7fd0767fc640 1 -- 172.21.15.94:0/4286520522 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd068042020 con 0x7fd0781076e0 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.579+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07810bd70 con 0x7fd0781076e0 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.579+0000 7fd07c9f5640 1 --2- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd060039260 0x7fd06003b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.580+0000 7fd07c9f5640 1 --2- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd060039260 0x7fd06003b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0098c0 tx=0x7fd06c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.584+0000 7fd0767fc640 1 -- 172.21.15.94:0/4286520522 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd06804b020 con 0x7fd0781076e0 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.769+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fd078107b60 con 0x7fd060039260 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:33.999+0000 7fd0767fc640 1 -- 172.21.15.94:0/4286520522 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fd078107b60 con 0x7fd060039260 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd060039260 msgr2=0x7fd06003b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 --2- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd060039260 0x7fd06003b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0098c0 tx=0x7fd06c006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 msgr2=0x7fd07819b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 --2- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd07819b660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd06802acc0 tx=0x7fd068031c90 comp rx=0 tx=0).stop 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 shutdown_connections 2024-09-19T04:06:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 --2- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd060039260 0x7fd06003b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 --2- 172.21.15.94:0/4286520522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0781076e0 0x7fd07819b660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 >> 172.21.15.94:0/4286520522 conn(0x7fd078102ed0 msgr2=0x7fd078103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:34.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 shutdown_connections 2024-09-19T04:06:34.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.001+0000 7fd07f45a640 1 -- 172.21.15.94:0/4286520522 wait complete. 2024-09-19T04:06:34.571 INFO:teuthology.orchestra.run.smithi094.stdout:Fetching dashboard port number... 2024-09-19T04:06:35.138 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:35 smithi094 ceph-mon[30972]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:35.844 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T04:06:35.844 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.947+0000 7f05f0798640 1 Processor -- start 2024-09-19T04:06:35.844 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.947+0000 7f05f0798640 1 -- start start 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.947+0000 7f05f0798640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e8107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.947+0000 7f05f0798640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05e81080b0 con 0x7f05e81076e0 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.948+0000 7f05ee534640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e8107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.948+0000 7f05ee534640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e8107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60854/0 (socket says 172.21.15.94:60854) 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.948+0000 7f05ee534640 1 -- 172.21.15.94:0/952320608 learned_addr learned my addr 172.21.15.94:0/952320608 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.948+0000 7f05ee534640 1 -- 172.21.15.94:0/952320608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e8108840 con 0x7f05e81076e0 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.948+0000 7f05ee534640 1 --2- 172.21.15.94:0/952320608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e8107ae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009820 tx=0x7f05e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46136fe733e2dc40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05ed532640 1 -- 172.21.15.94:0/952320608 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4037070 con 0x7f05e81076e0 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05ed532640 1 -- 172.21.15.94:0/952320608 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05e40328f0 con 0x7f05e81076e0 2024-09-19T04:06:35.845 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05ed532640 1 -- 172.21.15.94:0/952320608 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4030740 con 0x7f05e81076e0 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05f0798640 1 -- 172.21.15.94:0/952320608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 msgr2=0x7f05e8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05f0798640 1 --2- 172.21.15.94:0/952320608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e8107ae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009820 tx=0x7f05e402a270 comp rx=0 tx=0).stop 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05f0798640 1 -- 172.21.15.94:0/952320608 shutdown_connections 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05f0798640 1 --2- 172.21.15.94:0/952320608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e8107ae0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05f0798640 1 -- 172.21.15.94:0/952320608 >> 172.21.15.94:0/952320608 conn(0x7f05e8102ed0 msgr2=0x7f05e81052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05f0798640 1 -- 172.21.15.94:0/952320608 shutdown_connections 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.949+0000 7f05f0798640 1 -- 172.21.15.94:0/952320608 wait complete. 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.950+0000 7f05f0798640 1 Processor -- start 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.950+0000 7f05f0798640 1 -- start start 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.950+0000 7f05f0798640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.950+0000 7f05f0798640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05e810bbf0 con 0x7f05e81076e0 2024-09-19T04:06:35.846 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.950+0000 7f05ee534640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.950+0000 7f05ee534640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60862/0 (socket says 172.21.15.94:60862) 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.950+0000 7f05ee534640 1 -- 172.21.15.94:0/38527475 learned_addr learned my addr 172.21.15.94:0/38527475 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.950+0000 7f05ee534640 1 -- 172.21.15.94:0/38527475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e819bb70 con 0x7f05e81076e0 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.951+0000 7f05ee534640 1 --2- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e819b630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009940 tx=0x7f05e4030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.951+0000 7f05df7fe640 1 -- 172.21.15.94:0/38527475 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4042070 con 0x7f05e81076e0 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.951+0000 7f05df7fe640 1 -- 172.21.15.94:0/38527475 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05e4030e30 con 0x7f05e81076e0 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.951+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e819be00 con 0x7f05e81076e0 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.951+0000 7f05df7fe640 1 -- 172.21.15.94:0/38527475 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4037040 con 0x7f05e81076e0 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.951+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e819eaf0 con 0x7f05e81076e0 2024-09-19T04:06:35.847 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.952+0000 7f05df7fe640 1 -- 172.21.15.94:0/38527475 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f05e4045070 con 0x7f05e81076e0 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.952+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e810bd70 con 0x7f05e81076e0 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.952+0000 7f05df7fe640 1 --2- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f05d4039200 0x7f05d403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.952+0000 7f05df7fe640 1 -- 172.21.15.94:0/38527475 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f05e406c570 con 0x7f05e81076e0 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.952+0000 7f05edd33640 1 --2- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f05d4039200 0x7f05d403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.952+0000 7f05edd33640 1 --2- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f05d4039200 0x7f05d403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098c0 tx=0x7f05d8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:34.957+0000 7f05df7fe640 1 -- 172.21.15.94:0/38527475 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05e403c650 con 0x7f05e81076e0 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.144+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f05e819ecc0 con 0x7f05e81076e0 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.144+0000 7f05df7fe640 1 -- 172.21.15.94:0/38527475 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f05e4046bd0 con 0x7f05e81076e0 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.146+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f05d4039200 msgr2=0x7f05d403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.146+0000 7f05f0798640 1 --2- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f05d4039200 0x7f05d403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098c0 tx=0x7f05d8006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.146+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 msgr2=0x7f05e819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:35.849 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.146+0000 7f05f0798640 1 --2- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e819b630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009940 tx=0x7f05e4030cc0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.849 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.146+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 shutdown_connections 2024-09-19T04:06:35.849 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.146+0000 7f05f0798640 1 --2- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f05d4039200 0x7f05d403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.849 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.146+0000 7f05f0798640 1 --2- 172.21.15.94:0/38527475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05e81076e0 0x7f05e819b630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.849 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.146+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 >> 172.21.15.94:0/38527475 conn(0x7f05e8102ed0 msgr2=0x7f05e8103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:35.849 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.147+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 shutdown_connections 2024-09-19T04:06:35.849 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:35.147+0000 7f05f0798640 1 -- 172.21.15.94:0/38527475 wait complete. 2024-09-19T04:06:35.851 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:06:35.851 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-09-19T04:06:35.859 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:06:35.859 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-09-19T04:06:35.859 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-09-19T04:06:35.859 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T04:06:35.860 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph Dashboard is now available at: 2024-09-19T04:06:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: URL: https://smithi094.front.sepia.ceph.com:8443/ 2024-09-19T04:06:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: User: admin 2024-09-19T04:06:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: Password: 6skbxh8ppj 2024-09-19T04:06:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:35.861 INFO:teuthology.orchestra.run.smithi094.stdout:Saving cluster configuration to /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config directory 2024-09-19T04:06:36.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:36 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/38527475' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T04:06:37.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.393+0000 7f29e80f8640 1 Processor -- start 2024-09-19T04:06:37.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.393+0000 7f29e80f8640 1 -- start start 2024-09-19T04:06:37.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.393+0000 7f29e80f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e010a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:37.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.393+0000 7f29e80f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29e007bde0 con 0x7f29e0108140 2024-09-19T04:06:37.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.394+0000 7f29e5e94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e010a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:37.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.394+0000 7f29e5e94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e010a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60866/0 (socket says 172.21.15.94:60866) 2024-09-19T04:06:37.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.394+0000 7f29e5e94640 1 -- 172.21.15.94:0/3159852164 learned_addr learned my addr 172.21.15.94:0/3159852164 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.394+0000 7f29e5e94640 1 -- 172.21.15.94:0/3159852164 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e010aa70 con 0x7f29e0108140 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.395+0000 7f29e5e94640 1 --2- 172.21.15.94:0/3159852164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e010a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009e80 tx=0x7f29dc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1854e98a7c0dbc25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.395+0000 7f29e4e92640 1 -- 172.21.15.94:0/3159852164 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29dc036070 con 0x7f29e0108140 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.395+0000 7f29e4e92640 1 -- 172.21.15.94:0/3159852164 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29dc0318b0 con 0x7f29e0108140 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.395+0000 7f29e4e92640 1 -- 172.21.15.94:0/3159852164 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29dc007930 con 0x7f29e0108140 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.395+0000 7f29e80f8640 1 -- 172.21.15.94:0/3159852164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 msgr2=0x7f29e010a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.395+0000 7f29e80f8640 1 --2- 172.21.15.94:0/3159852164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e010a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009e80 tx=0x7f29dc02a7a0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.395+0000 7f29e80f8640 1 -- 172.21.15.94:0/3159852164 shutdown_connections 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.395+0000 7f29e80f8640 1 --2- 172.21.15.94:0/3159852164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e010a530 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.395+0000 7f29e80f8640 1 -- 172.21.15.94:0/3159852164 >> 172.21.15.94:0/3159852164 conn(0x7f29e0101d80 msgr2=0x7f29e01041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.396+0000 7f29e80f8640 1 -- 172.21.15.94:0/3159852164 shutdown_connections 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.396+0000 7f29e80f8640 1 -- 172.21.15.94:0/3159852164 wait complete. 2024-09-19T04:06:37.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.396+0000 7f29e80f8640 1 Processor -- start 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.396+0000 7f29e80f8640 1 -- start start 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.396+0000 7f29e80f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.396+0000 7f29e80f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29e010ba30 con 0x7f29e0108140 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.396+0000 7f29e5e94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.396+0000 7f29e5e94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e019b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60872/0 (socket says 172.21.15.94:60872) 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.396+0000 7f29e5e94640 1 -- 172.21.15.94:0/383660247 learned_addr learned my addr 172.21.15.94:0/383660247 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.397+0000 7f29e5e94640 1 -- 172.21.15.94:0/383660247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e019bb50 con 0x7f29e0108140 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.397+0000 7f29e5e94640 1 --2- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e019b610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f29dc031cc0 tx=0x7f29dc031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.397+0000 7f29d6ffd640 1 -- 172.21.15.94:0/383660247 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29dc036050 con 0x7f29e0108140 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.397+0000 7f29d6ffd640 1 -- 172.21.15.94:0/383660247 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29dc042070 con 0x7f29e0108140 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.397+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e019bde0 con 0x7f29e0108140 2024-09-19T04:06:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.397+0000 7f29d6ffd640 1 -- 172.21.15.94:0/383660247 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29dc03db10 con 0x7f29e0108140 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.397+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e019ead0 con 0x7f29e0108140 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.398+0000 7f29d6ffd640 1 -- 172.21.15.94:0/383660247 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f29dc03dcb0 con 0x7f29e0108140 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.398+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e01085a0 con 0x7f29e0108140 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.398+0000 7f29d6ffd640 1 --2- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29cc039300 0x7f29cc03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.398+0000 7f29d6ffd640 1 -- 172.21.15.94:0/383660247 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f29dc06dcd0 con 0x7f29e0108140 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.398+0000 7f29e5693640 1 --2- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29cc039300 0x7f29cc03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.399+0000 7f29e5693640 1 --2- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29cc039300 0x7f29cc03b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29d00098c0 tx=0x7f29d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.403+0000 7f29d6ffd640 1 -- 172.21.15.94:0/383660247 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29dc073020 con 0x7f29e0108140 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.660+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f29e019eca0 con 0x7f29e0108140 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.662+0000 7f29d6ffd640 1 -- 172.21.15.94:0/383660247 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f29dc044310 con 0x7f29e0108140 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29cc039300 msgr2=0x7f29cc03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:37.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 --2- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29cc039300 0x7f29cc03b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29d00098c0 tx=0x7f29d0006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 msgr2=0x7f29e019b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 --2- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e019b610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f29dc031cc0 tx=0x7f29dc031cf0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 shutdown_connections 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 --2- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29cc039300 0x7f29cc03b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 --2- 172.21.15.94:0/383660247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e0108140 0x7f29e019b610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 >> 172.21.15.94:0/383660247 conn(0x7f29e0101d80 msgr2=0x7f29e0102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 shutdown_connections 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-09-19T04:06:36.665+0000 7f29e80f8640 1 -- 172.21.15.94:0/383660247 wait complete. 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:37.309 INFO:teuthology.orchestra.run.smithi094.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout: ceph telemetry on 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout:For more information see: 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:37.310 INFO:teuthology.orchestra.run.smithi094.stdout:Bootstrap complete. 2024-09-19T04:06:37.329 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:37 smithi094 ceph-mon[30972]: mgrmap e13: x(active, since 6s) 2024-09-19T04:06:37.329 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:37 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/383660247' entity='client.admin' 2024-09-19T04:06:37.343 INFO:tasks.cephadm:Fetching config... 2024-09-19T04:06:37.343 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:06:37.343 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T04:06:37.360 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T04:06:37.360 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:06:37.360 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T04:06:37.415 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T04:06:37.416 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:06:37.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T04:06:37.481 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T04:06:37.481 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:06:37.481 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T04:06:37.537 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T04:06:37.537 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQO1N3pjspPUsujM8AnLcWTKmJDQUxhNso1yA3ObI9+OgF+9zDWCGNidtBioDDcaZ3IsOyhsGzuo3xTNOg18g5c894eedX++/3aU7gtPYgLdf+GbtH6/a5AG7geATCz/TO3HHoQBggnknWodAC3p4ROryECmPmtPPbW4H0cYo1h80vAhxCBWJeuX6Ro7q/ldt45Rl60gMk2G754QExCgLpgfSJ2M0nQ3gPMgdiv4eTTSRLDjci2dNZH8cE0njGOA/LjU8ewUAKwKyx4FN0LvdL2/O/8VqZ/lDFtr+YIknoVtyzT1jadjD+UH3sOQKSXIE6BK7RS4HJzTx/TBrBGMRbrjqtn1oHIGplnMATmg469gejv4JGbk4q5h0qq3bMeBD18plQ3rexYsH7ek4ZcWErkTxzKksmMFNBHJk1hmkBnQSUGpVLVZXC4SOJ91yFgH+zXZrLdVYs335O8EPkMLVJ1xFxLRfuGD7vPI4XClGsUwQNcEYpVoJ+vChgzlBdQiE= ceph-503c4526-763c-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:06:37.610 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQO1N3pjspPUsujM8AnLcWTKmJDQUxhNso1yA3ObI9+OgF+9zDWCGNidtBioDDcaZ3IsOyhsGzuo3xTNOg18g5c894eedX++/3aU7gtPYgLdf+GbtH6/a5AG7geATCz/TO3HHoQBggnknWodAC3p4ROryECmPmtPPbW4H0cYo1h80vAhxCBWJeuX6Ro7q/ldt45Rl60gMk2G754QExCgLpgfSJ2M0nQ3gPMgdiv4eTTSRLDjci2dNZH8cE0njGOA/LjU8ewUAKwKyx4FN0LvdL2/O/8VqZ/lDFtr+YIknoVtyzT1jadjD+UH3sOQKSXIE6BK7RS4HJzTx/TBrBGMRbrjqtn1oHIGplnMATmg469gejv4JGbk4q5h0qq3bMeBD18plQ3rexYsH7ek4ZcWErkTxzKksmMFNBHJk1hmkBnQSUGpVLVZXC4SOJ91yFgH+zXZrLdVYs335O8EPkMLVJ1xFxLRfuGD7vPI4XClGsUwQNcEYpVoJ+vChgzlBdQiE= ceph-503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:06:37.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQO1N3pjspPUsujM8AnLcWTKmJDQUxhNso1yA3ObI9+OgF+9zDWCGNidtBioDDcaZ3IsOyhsGzuo3xTNOg18g5c894eedX++/3aU7gtPYgLdf+GbtH6/a5AG7geATCz/TO3HHoQBggnknWodAC3p4ROryECmPmtPPbW4H0cYo1h80vAhxCBWJeuX6Ro7q/ldt45Rl60gMk2G754QExCgLpgfSJ2M0nQ3gPMgdiv4eTTSRLDjci2dNZH8cE0njGOA/LjU8ewUAKwKyx4FN0LvdL2/O/8VqZ/lDFtr+YIknoVtyzT1jadjD+UH3sOQKSXIE6BK7RS4HJzTx/TBrBGMRbrjqtn1oHIGplnMATmg469gejv4JGbk4q5h0qq3bMeBD18plQ3rexYsH7ek4ZcWErkTxzKksmMFNBHJk1hmkBnQSUGpVLVZXC4SOJ91yFgH+zXZrLdVYs335O8EPkMLVJ1xFxLRfuGD7vPI4XClGsUwQNcEYpVoJ+vChgzlBdQiE= ceph-503c4526-763c-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:06:37.891 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQO1N3pjspPUsujM8AnLcWTKmJDQUxhNso1yA3ObI9+OgF+9zDWCGNidtBioDDcaZ3IsOyhsGzuo3xTNOg18g5c894eedX++/3aU7gtPYgLdf+GbtH6/a5AG7geATCz/TO3HHoQBggnknWodAC3p4ROryECmPmtPPbW4H0cYo1h80vAhxCBWJeuX6Ro7q/ldt45Rl60gMk2G754QExCgLpgfSJ2M0nQ3gPMgdiv4eTTSRLDjci2dNZH8cE0njGOA/LjU8ewUAKwKyx4FN0LvdL2/O/8VqZ/lDFtr+YIknoVtyzT1jadjD+UH3sOQKSXIE6BK7RS4HJzTx/TBrBGMRbrjqtn1oHIGplnMATmg469gejv4JGbk4q5h0qq3bMeBD18plQ3rexYsH7ek4ZcWErkTxzKksmMFNBHJk1hmkBnQSUGpVLVZXC4SOJ91yFgH+zXZrLdVYs335O8EPkMLVJ1xFxLRfuGD7vPI4XClGsUwQNcEYpVoJ+vChgzlBdQiE= ceph-503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:06:37.905 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQO1N3pjspPUsujM8AnLcWTKmJDQUxhNso1yA3ObI9+OgF+9zDWCGNidtBioDDcaZ3IsOyhsGzuo3xTNOg18g5c894eedX++/3aU7gtPYgLdf+GbtH6/a5AG7geATCz/TO3HHoQBggnknWodAC3p4ROryECmPmtPPbW4H0cYo1h80vAhxCBWJeuX6Ro7q/ldt45Rl60gMk2G754QExCgLpgfSJ2M0nQ3gPMgdiv4eTTSRLDjci2dNZH8cE0njGOA/LjU8ewUAKwKyx4FN0LvdL2/O/8VqZ/lDFtr+YIknoVtyzT1jadjD+UH3sOQKSXIE6BK7RS4HJzTx/TBrBGMRbrjqtn1oHIGplnMATmg469gejv4JGbk4q5h0qq3bMeBD18plQ3rexYsH7ek4ZcWErkTxzKksmMFNBHJk1hmkBnQSUGpVLVZXC4SOJ91yFgH+zXZrLdVYs335O8EPkMLVJ1xFxLRfuGD7vPI4XClGsUwQNcEYpVoJ+vChgzlBdQiE= ceph-503c4526-763c-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:06:38.023 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQO1N3pjspPUsujM8AnLcWTKmJDQUxhNso1yA3ObI9+OgF+9zDWCGNidtBioDDcaZ3IsOyhsGzuo3xTNOg18g5c894eedX++/3aU7gtPYgLdf+GbtH6/a5AG7geATCz/TO3HHoQBggnknWodAC3p4ROryECmPmtPPbW4H0cYo1h80vAhxCBWJeuX6Ro7q/ldt45Rl60gMk2G754QExCgLpgfSJ2M0nQ3gPMgdiv4eTTSRLDjci2dNZH8cE0njGOA/LjU8ewUAKwKyx4FN0LvdL2/O/8VqZ/lDFtr+YIknoVtyzT1jadjD+UH3sOQKSXIE6BK7RS4HJzTx/TBrBGMRbrjqtn1oHIGplnMATmg469gejv4JGbk4q5h0qq3bMeBD18plQ3rexYsH7ek4ZcWErkTxzKksmMFNBHJk1hmkBnQSUGpVLVZXC4SOJ91yFgH+zXZrLdVYs335O8EPkMLVJ1xFxLRfuGD7vPI4XClGsUwQNcEYpVoJ+vChgzlBdQiE= ceph-503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:06:38.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T04:06:38.292 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.836+0000 7f710079c640 1 -- 172.21.15.94:0/3786916732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 msgr2=0x7f70f81008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.836+0000 7f710079c640 1 --2- 172.21.15.94:0/3786916732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 0x7f70f81008e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009880 tx=0x7f70f402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:06:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.836+0000 7f710079c640 1 -- 172.21.15.94:0/3786916732 shutdown_connections 2024-09-19T04:06:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.836+0000 7f710079c640 1 --2- 172.21.15.94:0/3786916732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 0x7f70f81008e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.836+0000 7f710079c640 1 -- 172.21.15.94:0/3786916732 >> 172.21.15.94:0/3786916732 conn(0x7f70f80fbc70 msgr2=0x7f70f80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.836+0000 7f710079c640 1 -- 172.21.15.94:0/3786916732 shutdown_connections 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.836+0000 7f710079c640 1 -- 172.21.15.94:0/3786916732 wait complete. 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.837+0000 7f710079c640 1 Processor -- start 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.837+0000 7f710079c640 1 -- start start 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.837+0000 7f710079c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 0x7f70f806a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.837+0000 7f710079c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f70f81064a0 con 0x7f70f81004e0 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.837+0000 7f70fe538640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 0x7f70f806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.837+0000 7f70fe538640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 0x7f70f806a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60888/0 (socket says 172.21.15.94:60888) 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.837+0000 7f70fe538640 1 -- 172.21.15.94:0/4074660503 learned_addr learned my addr 172.21.15.94:0/4074660503 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.838+0000 7f70fe538640 1 -- 172.21.15.94:0/4074660503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f81999c0 con 0x7f70f81004e0 2024-09-19T04:06:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.838+0000 7f70fe538640 1 --2- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 0x7f70f806a990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009230 tx=0x7f70f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.838+0000 7f70ef7fe640 1 -- 172.21.15.94:0/4074660503 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f403f070 con 0x7f70f81004e0 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.838+0000 7f70ef7fe640 1 -- 172.21.15.94:0/4074660503 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70f4003aa0 con 0x7f70f81004e0 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.838+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f8199b90 con 0x7f70f81004e0 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.838+0000 7f70ef7fe640 1 -- 172.21.15.94:0/4074660503 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f4035030 con 0x7f70f81004e0 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.838+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f8199fb0 con 0x7f70f81004e0 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.839+0000 7f70ef7fe640 1 -- 172.21.15.94:0/4074660503 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f70f4042070 con 0x7f70f81004e0 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.839+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f8106c20 con 0x7f70f81004e0 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.839+0000 7f70ef7fe640 1 --2- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f70e40392a0 0x7f70e403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.840+0000 7f70ef7fe640 1 -- 172.21.15.94:0/4074660503 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f70f406d1b0 con 0x7f70f81004e0 2024-09-19T04:06:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.840+0000 7f70fdd37640 1 --2- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f70e40392a0 0x7f70e403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.840+0000 7f70fdd37640 1 --2- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f70e40392a0 0x7f70e403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70e80098c0 tx=0x7f70e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:38.844+0000 7f70ef7fe640 1 -- 172.21.15.94:0/4074660503 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f70f4039350 con 0x7f70f81004e0 2024-09-19T04:06:39.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.019+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f70f818f260 con 0x7f70f81004e0 2024-09-19T04:06:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.021+0000 7f70ef7fe640 1 -- 172.21.15.94:0/4074660503 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f70f402f300 con 0x7f70f81004e0 2024-09-19T04:06:39.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f70e40392a0 msgr2=0x7f70e403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:39.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 --2- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f70e40392a0 0x7f70e403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70e80098c0 tx=0x7f70e8006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 msgr2=0x7f70f806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:39.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 --2- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 0x7f70f806a990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009230 tx=0x7f70f40048f0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 shutdown_connections 2024-09-19T04:06:39.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 --2- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f70e40392a0 0x7f70e403b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 --2- 172.21.15.94:0/4074660503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70f81004e0 0x7f70f806a990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 >> 172.21.15.94:0/4074660503 conn(0x7f70f80fbc70 msgr2=0x7f70f80fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:39.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 shutdown_connections 2024-09-19T04:06:39.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:39.024+0000 7f710079c640 1 -- 172.21.15.94:0/4074660503 wait complete. 2024-09-19T04:06:39.700 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T04:06:39.700 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T04:06:40.160 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:40.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:40 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4074660503' entity='client.admin' 2024-09-19T04:06:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.665+0000 7f9df5947640 1 -- 172.21.15.94:0/4217612139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 msgr2=0x7f9df0100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.665+0000 7f9df5947640 1 --2- 172.21.15.94:0/4217612139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 0x7f9df0100940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009880 tx=0x7f9de002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:06:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.665+0000 7f9df5947640 1 -- 172.21.15.94:0/4217612139 shutdown_connections 2024-09-19T04:06:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.665+0000 7f9df5947640 1 --2- 172.21.15.94:0/4217612139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 0x7f9df0100940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.665+0000 7f9df5947640 1 -- 172.21.15.94:0/4217612139 >> 172.21.15.94:0/4217612139 conn(0x7f9df00fbcd0 msgr2=0x7f9df00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.665+0000 7f9df5947640 1 -- 172.21.15.94:0/4217612139 shutdown_connections 2024-09-19T04:06:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.665+0000 7f9df5947640 1 -- 172.21.15.94:0/4217612139 wait complete. 2024-09-19T04:06:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.665+0000 7f9df5947640 1 Processor -- start 2024-09-19T04:06:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.665+0000 7f9df5947640 1 -- start start 2024-09-19T04:06:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9df5947640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 0x7f9df006a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9df5947640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9df01064a0 con 0x7f9df0100540 2024-09-19T04:06:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9deeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 0x7f9df006a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9deeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 0x7f9df006a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60904/0 (socket says 172.21.15.94:60904) 2024-09-19T04:06:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9deeffd640 1 -- 172.21.15.94:0/2491847638 learned_addr learned my addr 172.21.15.94:0/2491847638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9deeffd640 1 -- 172.21.15.94:0/2491847638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df0199a90 con 0x7f9df0100540 2024-09-19T04:06:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9deeffd640 1 --2- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 0x7f9df006a130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9de0005ab0 tx=0x7f9de00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9df4945640 1 -- 172.21.15.94:0/2491847638 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9de003f070 con 0x7f9df0100540 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9df4945640 1 -- 172.21.15.94:0/2491847638 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9de0003aa0 con 0x7f9df0100540 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df0199c60 con 0x7f9df0100540 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9df4945640 1 -- 172.21.15.94:0/2491847638 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9de0035030 con 0x7f9df0100540 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.666+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df019a000 con 0x7f9df0100540 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.667+0000 7f9df4945640 1 -- 172.21.15.94:0/2491847638 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9de0042070 con 0x7f9df0100540 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.667+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df0106c20 con 0x7f9df0100540 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.667+0000 7f9df4945640 1 --2- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9ddc0392a0 0x7f9ddc03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.668+0000 7f9df4945640 1 -- 172.21.15.94:0/2491847638 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9de006d000 con 0x7f9df0100540 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.668+0000 7f9de67fc640 1 --2- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9ddc0392a0 0x7f9ddc03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.668+0000 7f9de67fc640 1 --2- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9ddc0392a0 0x7f9ddc03b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80098c0 tx=0x7f9dd8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.672+0000 7f9df4945640 1 -- 172.21.15.94:0/2491847638 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9de0042360 con 0x7f9df0100540 2024-09-19T04:06:40.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.845+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f9df002d930 con 0x7f9ddc0392a0 2024-09-19T04:06:40.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.848+0000 7f9df4945640 1 -- 172.21.15.94:0/2491847638 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9df002d930 con 0x7f9ddc0392a0 2024-09-19T04:06:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9ddc0392a0 msgr2=0x7f9ddc03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 --2- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9ddc0392a0 0x7f9ddc03b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80098c0 tx=0x7f9dd8006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 msgr2=0x7f9df006a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 --2- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 0x7f9df006a130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9de0005ab0 tx=0x7f9de00048f0 comp rx=0 tx=0).stop 2024-09-19T04:06:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 shutdown_connections 2024-09-19T04:06:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 --2- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9ddc0392a0 0x7f9ddc03b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 --2- 172.21.15.94:0/2491847638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df0100540 0x7f9df006a130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 >> 172.21.15.94:0/2491847638 conn(0x7f9df00fbcd0 msgr2=0x7f9df00fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 shutdown_connections 2024-09-19T04:06:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:40.850+0000 7f9df5947640 1 -- 172.21.15.94:0/2491847638 wait complete. 2024-09-19T04:06:42.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:41 smithi094 ceph-mon[30972]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:42.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:41 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:42.300 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-09-19T04:06:42.300 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:06:42.301 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:06:42.319 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:06:42.319 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:06:42.377 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-09-19T04:06:42.377 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch host add smithi104 2024-09-19T04:06:42.641 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:43.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.167+0000 7fee12cf0640 1 -- 172.21.15.94:0/439136869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 msgr2=0x7fee0c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:43.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.167+0000 7fee12cf0640 1 --2- 172.21.15.94:0/439136869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 0x7fee0c107a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee00009880 tx=0x7fee0002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.167+0000 7fee12cf0640 1 -- 172.21.15.94:0/439136869 shutdown_connections 2024-09-19T04:06:43.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.167+0000 7fee12cf0640 1 --2- 172.21.15.94:0/439136869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 0x7fee0c107a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.167+0000 7fee12cf0640 1 -- 172.21.15.94:0/439136869 >> 172.21.15.94:0/439136869 conn(0x7fee0c0fd290 msgr2=0x7fee0c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:43.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.167+0000 7fee12cf0640 1 -- 172.21.15.94:0/439136869 shutdown_connections 2024-09-19T04:06:43.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.167+0000 7fee12cf0640 1 -- 172.21.15.94:0/439136869 wait complete. 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.168+0000 7fee12cf0640 1 Processor -- start 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.168+0000 7fee12cf0640 1 -- start start 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.168+0000 7fee12cf0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 0x7fee0c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.168+0000 7fee12cf0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c10bb40 con 0x7fee0c107660 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.168+0000 7fee10a8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 0x7fee0c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.168+0000 7fee10a8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 0x7fee0c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40164/0 (socket says 172.21.15.94:40164) 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.168+0000 7fee10a8c640 1 -- 172.21.15.94:0/1787586600 learned_addr learned my addr 172.21.15.94:0/1787586600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.168+0000 7fee10a8c640 1 -- 172.21.15.94:0/1787586600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c0763e0 con 0x7fee0c107660 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.168+0000 7fee10a8c640 1 --2- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 0x7fee0c0744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee00009230 tx=0x7fee000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.169+0000 7fee09ffb640 1 -- 172.21.15.94:0/1787586600 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee0003f070 con 0x7fee0c107660 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.169+0000 7fee09ffb640 1 -- 172.21.15.94:0/1787586600 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee00004d10 con 0x7fee0c107660 2024-09-19T04:06:43.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.169+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c074a30 con 0x7fee0c107660 2024-09-19T04:06:43.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.169+0000 7fee09ffb640 1 -- 172.21.15.94:0/1787586600 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee00035030 con 0x7fee0c107660 2024-09-19T04:06:43.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.169+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c074ef0 con 0x7fee0c107660 2024-09-19T04:06:43.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.170+0000 7fee09ffb640 1 -- 172.21.15.94:0/1787586600 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fee00042070 con 0x7fee0c107660 2024-09-19T04:06:43.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.170+0000 7fee09ffb640 1 --2- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fedfc039300 0x7fedfc03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:43.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.170+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c102cd0 con 0x7fee0c107660 2024-09-19T04:06:43.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.170+0000 7fee09ffb640 1 -- 172.21.15.94:0/1787586600 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fee0006cf60 con 0x7fee0c107660 2024-09-19T04:06:43.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.170+0000 7fee0bfff640 1 --2- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fedfc039300 0x7fedfc03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:43.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.170+0000 7fee0bfff640 1 --2- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fedfc039300 0x7fedfc03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fedf80098c0 tx=0x7fedf8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:43.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.175+0000 7fee09ffb640 1 -- 172.21.15.94:0/1787586600 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fee00042360 con 0x7fee0c107660 2024-09-19T04:06:43.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: Updating smithi094:/etc/ceph/ceph.conf 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:06:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:43.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:43.347+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}) -- 0x7fee0c075a20 con 0x7fedfc039300 2024-09-19T04:06:44.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:44 smithi094 ceph-mon[30972]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:06:44.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:44 smithi094 ceph-mon[30972]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:06:44.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:44 smithi094 ceph-mon[30972]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:06:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:45 smithi094 ceph-mon[30972]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:45 smithi094 ceph-mon[30972]: Deploying cephadm binary to smithi104 2024-09-19T04:06:46.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.513+0000 7fee09ffb640 1 -- 172.21.15.94:0/1787586600 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fee0c075a20 con 0x7fedfc039300 2024-09-19T04:06:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-09-19T04:06:46.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fedfc039300 msgr2=0x7fedfc03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:46.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 --2- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fedfc039300 0x7fedfc03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fedf80098c0 tx=0x7fedf8006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 msgr2=0x7fee0c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:46.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 --2- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 0x7fee0c0744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee00009230 tx=0x7fee000048f0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 shutdown_connections 2024-09-19T04:06:46.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 --2- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fedfc039300 0x7fedfc03b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 --2- 172.21.15.94:0/1787586600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee0c107660 0x7fee0c0744f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 >> 172.21.15.94:0/1787586600 conn(0x7fee0c0fd290 msgr2=0x7fee0c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:46.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 shutdown_connections 2024-09-19T04:06:46.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:46.515+0000 7fee12cf0640 1 -- 172.21.15.94:0/1787586600 wait complete. 2024-09-19T04:06:47.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:06:47.378 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:47.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:47.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:47 smithi094 ceph-mon[30972]: Added host smithi104 2024-09-19T04:06:47.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:47.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:47.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.819+0000 7fc04eae4640 1 -- 172.21.15.94:0/519083251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 msgr2=0x7fc048106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.819+0000 7fc04eae4640 1 --2- 172.21.15.94:0/519083251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 0x7fc048106940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc038009880 tx=0x7fc03802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:06:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.819+0000 7fc04eae4640 1 -- 172.21.15.94:0/519083251 shutdown_connections 2024-09-19T04:06:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.819+0000 7fc04eae4640 1 --2- 172.21.15.94:0/519083251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 0x7fc048106940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.819+0000 7fc04eae4640 1 -- 172.21.15.94:0/519083251 >> 172.21.15.94:0/519083251 conn(0x7fc0480fc1b0 msgr2=0x7fc0480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.819+0000 7fc04eae4640 1 -- 172.21.15.94:0/519083251 shutdown_connections 2024-09-19T04:06:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.819+0000 7fc04eae4640 1 -- 172.21.15.94:0/519083251 wait complete. 2024-09-19T04:06:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.820+0000 7fc04eae4640 1 Processor -- start 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.820+0000 7fc04eae4640 1 -- start start 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.820+0000 7fc04eae4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 0x7fc048197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.820+0000 7fc04eae4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc04810aa40 con 0x7fc048106560 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.820+0000 7fc04c880640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 0x7fc048197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.820+0000 7fc04c880640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 0x7fc048197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40182/0 (socket says 172.21.15.94:40182) 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.820+0000 7fc04c880640 1 -- 172.21.15.94:0/681723860 learned_addr learned my addr 172.21.15.94:0/681723860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.820+0000 7fc04c880640 1 -- 172.21.15.94:0/681723860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc048197550 con 0x7fc048106560 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.821+0000 7fc04c880640 1 --2- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 0x7fc048197010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc038009230 tx=0x7fc0380048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.821+0000 7fc045ffb640 1 -- 172.21.15.94:0/681723860 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03803f070 con 0x7fc048106560 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.821+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0481977e0 con 0x7fc048106560 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.821+0000 7fc045ffb640 1 -- 172.21.15.94:0/681723860 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc038004d10 con 0x7fc048106560 2024-09-19T04:06:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.821+0000 7fc045ffb640 1 -- 172.21.15.94:0/681723860 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc038035030 con 0x7fc048106560 2024-09-19T04:06:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.821+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04819b8e0 con 0x7fc048106560 2024-09-19T04:06:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.822+0000 7fc045ffb640 1 -- 172.21.15.94:0/681723860 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc038042070 con 0x7fc048106560 2024-09-19T04:06:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.822+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc048101bd0 con 0x7fc048106560 2024-09-19T04:06:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.822+0000 7fc045ffb640 1 --2- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc0300392a0 0x7fc03003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.822+0000 7fc045ffb640 1 -- 172.21.15.94:0/681723860 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc03806d160 con 0x7fc048106560 2024-09-19T04:06:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.822+0000 7fc047fff640 1 --2- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc0300392a0 0x7fc03003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.822+0000 7fc047fff640 1 --2- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc0300392a0 0x7fc03003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0098c0 tx=0x7fc03c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:47.827+0000 7fc045ffb640 1 -- 172.21.15.94:0/681723860 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc038039350 con 0x7fc048106560 2024-09-19T04:06:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.001+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc048198370 con 0x7fc0300392a0 2024-09-19T04:06:48.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.002+0000 7fc045ffb640 1 -- 172.21.15.94:0/681723860 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fc048198370 con 0x7fc0300392a0 2024-09-19T04:06:48.003 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc0300392a0 msgr2=0x7fc03003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 --2- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc0300392a0 0x7fc03003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0098c0 tx=0x7fc03c006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 msgr2=0x7fc048197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 --2- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 0x7fc048197010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc038009230 tx=0x7fc0380048f0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 shutdown_connections 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 --2- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc0300392a0 0x7fc03003b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 --2- 172.21.15.94:0/681723860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048106560 0x7fc048197010 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 >> 172.21.15.94:0/681723860 conn(0x7fc0480fc1b0 msgr2=0x7fc0480fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 shutdown_connections 2024-09-19T04:06:48.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:48.004+0000 7fc04eae4640 1 -- 172.21.15.94:0/681723860 wait complete. 2024-09-19T04:06:48.706 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi112 2024-09-19T04:06:48.706 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:06:48.706 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:06:48.725 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:06:48.725 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:06:48.782 INFO:tasks.cephadm:Adding host smithi112 to orchestrator... 2024-09-19T04:06:48.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch host add smithi112 2024-09-19T04:06:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:49.045 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:48 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:49.045 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:48 smithi094 ceph-mon[30972]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:06:49.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 -- 172.21.15.94:0/3463109907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 msgr2=0x7f0484106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:49.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 --2- 172.21.15.94:0/3463109907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 0x7f0484106940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0478009880 tx=0x7f047802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:06:49.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 -- 172.21.15.94:0/3463109907 shutdown_connections 2024-09-19T04:06:49.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 --2- 172.21.15.94:0/3463109907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 0x7f0484106940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:49.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 -- 172.21.15.94:0/3463109907 >> 172.21.15.94:0/3463109907 conn(0x7f04840fc1b0 msgr2=0x7f04840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:49.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 -- 172.21.15.94:0/3463109907 shutdown_connections 2024-09-19T04:06:49.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 -- 172.21.15.94:0/3463109907 wait complete. 2024-09-19T04:06:49.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 Processor -- start 2024-09-19T04:06:49.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 -- start start 2024-09-19T04:06:49.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 0x7f0484197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:49.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.477+0000 7f048aad5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f048410aa40 con 0x7f0484106560 2024-09-19T04:06:49.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f0488871640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 0x7f0484197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:49.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f0488871640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 0x7f0484197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40212/0 (socket says 172.21.15.94:40212) 2024-09-19T04:06:49.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f0488871640 1 -- 172.21.15.94:0/3787920661 learned_addr learned my addr 172.21.15.94:0/3787920661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:49.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f0488871640 1 -- 172.21.15.94:0/3787920661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0484198390 con 0x7f0484106560 2024-09-19T04:06:49.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f0488871640 1 --2- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 0x7f0484197e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0478009230 tx=0x7f04780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:49.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f0481ffb640 1 -- 172.21.15.94:0/3787920661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f047803f070 con 0x7f0484106560 2024-09-19T04:06:49.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f0481ffb640 1 -- 172.21.15.94:0/3787920661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0478004d10 con 0x7f0484106560 2024-09-19T04:06:49.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0484198620 con 0x7f0484106560 2024-09-19T04:06:49.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f0481ffb640 1 -- 172.21.15.94:0/3787920661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0478035030 con 0x7f0484106560 2024-09-19T04:06:49.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.478+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0484198a80 con 0x7f0484106560 2024-09-19T04:06:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.479+0000 7f0481ffb640 1 -- 172.21.15.94:0/3787920661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f0478042070 con 0x7f0484106560 2024-09-19T04:06:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.479+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0484101bd0 con 0x7f0484106560 2024-09-19T04:06:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.479+0000 7f0481ffb640 1 --2- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04740392a0 0x7f047403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.479+0000 7f0481ffb640 1 -- 172.21.15.94:0/3787920661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f047806d260 con 0x7f0484106560 2024-09-19T04:06:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.480+0000 7f0483fff640 1 --2- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04740392a0 0x7f047403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.480+0000 7f0483fff640 1 --2- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04740392a0 0x7f047403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f04700098c0 tx=0x7f0470006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.484+0000 7f0481ffb640 1 -- 172.21.15.94:0/3787920661 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0478039350 con 0x7f0484106560 2024-09-19T04:06:49.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:49.658+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "target": ["mon-mgr", ""]}) -- 0x7f04841069c0 con 0x7f04740392a0 2024-09-19T04:06:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:51 smithi094 ceph-mon[30972]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:51 smithi094 ceph-mon[30972]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:06:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:06:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:06:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:06:52.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:52 smithi094 ceph-mon[30972]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-19T04:06:52.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:52 smithi094 ceph-mon[30972]: Deploying cephadm binary to smithi112 2024-09-19T04:06:52.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:52 smithi094 ceph-mon[30972]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:06:52.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:52 smithi094 ceph-mon[30972]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:06:52.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:52.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:52.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:53.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.013+0000 7f0481ffb640 1 -- 172.21.15.94:0/3787920661 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f04841069c0 con 0x7f04740392a0 2024-09-19T04:06:53.016 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi112' with addr '172.21.15.112' 2024-09-19T04:06:53.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04740392a0 msgr2=0x7f047403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 --2- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04740392a0 0x7f047403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f04700098c0 tx=0x7f0470006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 msgr2=0x7f0484197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 --2- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 0x7f0484197e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0478009230 tx=0x7f04780048f0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 shutdown_connections 2024-09-19T04:06:53.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 --2- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04740392a0 0x7f047403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 --2- 172.21.15.94:0/3787920661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0484106560 0x7f0484197e50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 >> 172.21.15.94:0/3787920661 conn(0x7f04840fc1b0 msgr2=0x7f04840fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:53.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 shutdown_connections 2024-09-19T04:06:53.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:53.016+0000 7f048aad5640 1 -- 172.21.15.94:0/3787920661 wait complete. 2024-09-19T04:06:53.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:53 smithi094 ceph-mon[30972]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:06:53.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:53 smithi094 ceph-mon[30972]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:06:53.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:53.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:53.698 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:06:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:54.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:54 smithi094 ceph-mon[30972]: Added host smithi112 2024-09-19T04:06:54.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:54.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.421+0000 7f4ca8601640 1 -- 172.21.15.94:0/509577226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 msgr2=0x7f4ca0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.421+0000 7f4ca8601640 1 --2- 172.21.15.94:0/509577226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 0x7f4ca0106940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009880 tx=0x7f4c9c02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.421+0000 7f4ca8601640 1 -- 172.21.15.94:0/509577226 shutdown_connections 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.421+0000 7f4ca8601640 1 --2- 172.21.15.94:0/509577226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 0x7f4ca0106940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.421+0000 7f4ca8601640 1 -- 172.21.15.94:0/509577226 >> 172.21.15.94:0/509577226 conn(0x7f4ca00fc1b0 msgr2=0x7f4ca00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.421+0000 7f4ca8601640 1 -- 172.21.15.94:0/509577226 shutdown_connections 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.421+0000 7f4ca8601640 1 -- 172.21.15.94:0/509577226 wait complete. 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.422+0000 7f4ca8601640 1 Processor -- start 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.422+0000 7f4ca8601640 1 -- start start 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.422+0000 7f4ca8601640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 0x7f4ca0195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.422+0000 7f4ca8601640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca010aa40 con 0x7f4ca0106560 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.422+0000 7f4ca639d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 0x7f4ca0195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.422+0000 7f4ca639d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 0x7f4ca0195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38418/0 (socket says 172.21.15.94:38418) 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.422+0000 7f4ca639d640 1 -- 172.21.15.94:0/455507213 learned_addr learned my addr 172.21.15.94:0/455507213 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.422+0000 7f4ca639d640 1 -- 172.21.15.94:0/455507213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca0196440 con 0x7f4ca0106560 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.423+0000 7f4ca639d640 1 --2- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 0x7f4ca0195f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009230 tx=0x7f4c9c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.423+0000 7f4c977fe640 1 -- 172.21.15.94:0/455507213 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c9c03f070 con 0x7f4ca0106560 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.423+0000 7f4c977fe640 1 -- 172.21.15.94:0/455507213 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c9c004d10 con 0x7f4ca0106560 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.423+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca01966d0 con 0x7f4ca0106560 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.423+0000 7f4c977fe640 1 -- 172.21.15.94:0/455507213 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c9c035030 con 0x7f4ca0106560 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.423+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca019a7d0 con 0x7f4ca0106560 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.424+0000 7f4c977fe640 1 -- 172.21.15.94:0/455507213 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f4c9c042070 con 0x7f4ca0106560 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.424+0000 7f4c977fe640 1 --2- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4c8c0392a0 0x7f4c8c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.424+0000 7f4c977fe640 1 -- 172.21.15.94:0/455507213 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4c9c06cff0 con 0x7f4ca0106560 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.424+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca0101bd0 con 0x7f4ca0106560 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.424+0000 7f4ca5b9c640 1 --2- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4c8c0392a0 0x7f4c8c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.424+0000 7f4ca5b9c640 1 --2- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4c8c0392a0 0x7f4c8c03b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4c900098c0 tx=0x7f4c90006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.429+0000 7f4c977fe640 1 -- 172.21.15.94:0/455507213 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4c9c039350 con 0x7f4ca0106560 2024-09-19T04:06:54.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.602+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ca01069c0 con 0x7f4c8c0392a0 2024-09-19T04:06:54.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.603+0000 7f4c977fe640 1 -- 172.21.15.94:0/455507213 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f4ca01069c0 con 0x7f4c8c0392a0 2024-09-19T04:06:54.604 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:06:54.605 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}, {"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}] 2024-09-19T04:06:54.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4c8c0392a0 msgr2=0x7f4c8c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 --2- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4c8c0392a0 0x7f4c8c03b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4c900098c0 tx=0x7f4c90006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 msgr2=0x7f4ca0195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 --2- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 0x7f4ca0195f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009230 tx=0x7f4c9c0048f0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 shutdown_connections 2024-09-19T04:06:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 --2- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4c8c0392a0 0x7f4c8c03b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 --2- 172.21.15.94:0/455507213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca0106560 0x7f4ca0195f00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 >> 172.21.15.94:0/455507213 conn(0x7f4ca00fc1b0 msgr2=0x7f4ca00fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 shutdown_connections 2024-09-19T04:06:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:54.605+0000 7f4ca8601640 1 -- 172.21.15.94:0/455507213 wait complete. 2024-09-19T04:06:55.246 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T04:06:55.247 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T04:06:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:55 smithi094 ceph-mon[30972]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:06:55.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:55 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:55.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.950+0000 7f36c058c640 1 -- 172.21.15.94:0/597284322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 msgr2=0x7f36b8106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:55.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.950+0000 7f36c058c640 1 --2- 172.21.15.94:0/597284322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 0x7f36b8106920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009880 tx=0x7f36b402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:06:55.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36c058c640 1 -- 172.21.15.94:0/597284322 shutdown_connections 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36c058c640 1 --2- 172.21.15.94:0/597284322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 0x7f36b8106920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36c058c640 1 -- 172.21.15.94:0/597284322 >> 172.21.15.94:0/597284322 conn(0x7f36b80fc130 msgr2=0x7f36b80fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36c058c640 1 -- 172.21.15.94:0/597284322 shutdown_connections 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36c058c640 1 -- 172.21.15.94:0/597284322 wait complete. 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36c058c640 1 Processor -- start 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36c058c640 1 -- start start 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36c058c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 0x7f36b8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36c058c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36b810aa20 con 0x7f36b8106540 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.951+0000 7f36be328640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 0x7f36b8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.952+0000 7f36be328640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 0x7f36b8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38442/0 (socket says 172.21.15.94:38442) 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.952+0000 7f36be328640 1 -- 172.21.15.94:0/119488852 learned_addr learned my addr 172.21.15.94:0/119488852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:06:55.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.952+0000 7f36be328640 1 -- 172.21.15.94:0/119488852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b8198440 con 0x7f36b8106540 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.952+0000 7f36be328640 1 --2- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 0x7f36b8197f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009230 tx=0x7f36b40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.952+0000 7f36af7fe640 1 -- 172.21.15.94:0/119488852 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36b403f070 con 0x7f36b8106540 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.952+0000 7f36af7fe640 1 -- 172.21.15.94:0/119488852 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b4003aa0 con 0x7f36b8106540 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.952+0000 7f36af7fe640 1 -- 172.21.15.94:0/119488852 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36b4035030 con 0x7f36b8106540 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.952+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b81986d0 con 0x7f36b8106540 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.952+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b8198b30 con 0x7f36b8106540 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.953+0000 7f36af7fe640 1 -- 172.21.15.94:0/119488852 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f36b4042070 con 0x7f36b8106540 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.953+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b8101bb0 con 0x7f36b8106540 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.953+0000 7f36af7fe640 1 --2- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f36a4039300 0x7f36a403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.953+0000 7f36af7fe640 1 -- 172.21.15.94:0/119488852 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f36b406cee0 con 0x7f36b8106540 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.953+0000 7f36bdb27640 1 --2- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f36a4039300 0x7f36a403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:55.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.954+0000 7f36bdb27640 1 --2- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f36a4039300 0x7f36a403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f36a80098c0 tx=0x7f36a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:55.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:55.958+0000 7f36af7fe640 1 -- 172.21.15.94:0/119488852 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f36b404a260 con 0x7f36b8106540 2024-09-19T04:06:56.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.131+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f36b81069a0 con 0x7f36b8106540 2024-09-19T04:06:56.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.292+0000 7f36af7fe640 1 -- 172.21.15.94:0/119488852 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f36b4031400 con 0x7f36b8106540 2024-09-19T04:06:56.293 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f36a4039300 msgr2=0x7f36a403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 --2- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f36a4039300 0x7f36a403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f36a80098c0 tx=0x7f36a8006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 msgr2=0x7f36b8197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 --2- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 0x7f36b8197f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009230 tx=0x7f36b40048f0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 shutdown_connections 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 --2- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f36a4039300 0x7f36a403b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 --2- 172.21.15.94:0/119488852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36b8106540 0x7f36b8197f00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 >> 172.21.15.94:0/119488852 conn(0x7f36b80fc130 msgr2=0x7f36b80fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 shutdown_connections 2024-09-19T04:06:56.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:06:56.295+0000 7f36c058c640 1 -- 172.21.15.94:0/119488852 wait complete. 2024-09-19T04:06:56.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:56 smithi094 ceph-mon[30972]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:06:56.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:56 smithi094 ceph-mon[30972]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:06:56.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:56 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/119488852' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:06:56.899 INFO:tasks.cephadm:Adding mon.a on smithi094 2024-09-19T04:06:56.899 INFO:tasks.cephadm:Adding mon.b on smithi104 2024-09-19T04:06:56.899 INFO:tasks.cephadm:Adding mon.c on smithi112 2024-09-19T04:06:56.900 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi094:172.21.15.94=a;smithi104:172.21.15.104=b;smithi112:172.21.15.112=c' 2024-09-19T04:06:57.144 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T04:06:57.208 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T04:06:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/119488852' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-19T04:06:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:57 smithi094 ceph-mon[30972]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:06:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 -- 172.21.15.112:0/2070104478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 msgr2=0x7f23a0074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 --2- 172.21.15.112:0/2070104478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 0x7f23a0074680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f239c009880 tx=0x7f239c02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 -- 172.21.15.112:0/2070104478 shutdown_connections 2024-09-19T04:06:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 --2- 172.21.15.112:0/2070104478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 0x7f23a0074680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 -- 172.21.15.112:0/2070104478 >> 172.21.15.112:0/2070104478 conn(0x7f23a00fcdb0 msgr2=0x7f23a00ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 -- 172.21.15.112:0/2070104478 shutdown_connections 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 -- 172.21.15.112:0/2070104478 wait complete. 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 Processor -- start 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 -- start start 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 0x7f23a0198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.691+0000 7f23a6e0e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23a01097f0 con 0x7f23a0076220 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f23a4baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 0x7f23a0198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f23a4baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 0x7f23a0198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.112:45386/0 (socket says 172.21.15.112:45386) 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f23a4baa640 1 -- 172.21.15.112:0/3265287234 learned_addr learned my addr 172.21.15.112:0/3265287234 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f23a4baa640 1 -- 172.21.15.112:0/3265287234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a0199430 con 0x7f23a0076220 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f23a4baa640 1 --2- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 0x7f23a0198ef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f239c009230 tx=0x7f239c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f2395ffb640 1 -- 172.21.15.112:0/3265287234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f239c03f070 con 0x7f23a0076220 2024-09-19T04:06:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f2395ffb640 1 -- 172.21.15.112:0/3265287234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f239c003aa0 con 0x7f23a0076220 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a01996c0 con 0x7f23a0076220 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f2395ffb640 1 -- 172.21.15.112:0/3265287234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f239c035030 con 0x7f23a0076220 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.692+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a019c3b0 con 0x7f23a0076220 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.693+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a0109f70 con 0x7f23a0076220 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.693+0000 7f2395ffb640 1 -- 172.21.15.112:0/3265287234 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f239c042070 con 0x7f23a0076220 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.694+0000 7f2395ffb640 1 --2- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2388039300 0x7f238803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.694+0000 7f2395ffb640 1 -- 172.21.15.112:0/3265287234 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f239c06d120 con 0x7f23a0076220 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.694+0000 7f2397fff640 1 --2- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2388039300 0x7f238803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.694+0000 7f2397fff640 1 --2- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2388039300 0x7f238803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f23900098c0 tx=0x7f2390006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.698+0000 7f2395ffb640 1 -- 172.21.15.112:0/3265287234 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f239c039350 con 0x7f23a0076220 2024-09-19T04:06:57.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.885+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi104:172.21.15.104=b;smithi112:172.21.15.112=c", "target": ["mon-mgr", ""]}) -- 0x7f23a002d830 con 0x7f2388039300 2024-09-19T04:06:57.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.891+0000 7f2395ffb640 1 -- 172.21.15.112:0/3265287234 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f23a002d830 con 0x7f2388039300 2024-09-19T04:06:57.892 INFO:teuthology.orchestra.run.smithi112.stdout:Scheduled mon update... 2024-09-19T04:06:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2388039300 msgr2=0x7f238803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 --2- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2388039300 0x7f238803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f23900098c0 tx=0x7f2390006db0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 msgr2=0x7f23a0198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 --2- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 0x7f23a0198ef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f239c009230 tx=0x7f239c0048f0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 shutdown_connections 2024-09-19T04:06:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 --2- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2388039300 0x7f238803b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 --2- 172.21.15.112:0/3265287234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a0076220 0x7f23a0198ef0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 >> 172.21.15.112:0/3265287234 conn(0x7f23a00fcdb0 msgr2=0x7f23a00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 shutdown_connections 2024-09-19T04:06:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:06:57.893+0000 7f23a6e0e640 1 -- 172.21.15.112:0/3265287234 wait complete. 2024-09-19T04:06:58.430 DEBUG:teuthology.orchestra.run.smithi104:mon.b> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T04:06:58.433 DEBUG:teuthology.orchestra.run.smithi112:mon.c> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T04:06:58.435 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:06:58.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi104:172.21.15.104=b;smithi112:172.21.15.112=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi104:172.21.15.104=b;smithi112:172.21.15.112=c;count:3 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:06:59.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: Deploying daemon mon.c on smithi112 2024-09-19T04:06:59.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:06:58 smithi094 ceph-mon[30972]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:06:59.556 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.957+0000 7fe7548e1640 1 -- 172.21.15.112:0/2822976699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 msgr2=0x7fe750106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.957+0000 7fe7548e1640 1 --2- 172.21.15.112:0/2822976699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 0x7fe750106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe740009810 tx=0x7fe74002a540 comp rx=0 tx=0).stop 2024-09-19T04:07:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.957+0000 7fe7548e1640 1 -- 172.21.15.112:0/2822976699 shutdown_connections 2024-09-19T04:07:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.957+0000 7fe7548e1640 1 --2- 172.21.15.112:0/2822976699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 0x7fe750106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.957+0000 7fe7548e1640 1 -- 172.21.15.112:0/2822976699 >> 172.21.15.112:0/2822976699 conn(0x7fe750077310 msgr2=0x7fe7500fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.957+0000 7fe7548e1640 1 -- 172.21.15.112:0/2822976699 shutdown_connections 2024-09-19T04:07:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.957+0000 7fe7548e1640 1 -- 172.21.15.112:0/2822976699 wait complete. 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.958+0000 7fe7548e1640 1 Processor -- start 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.958+0000 7fe7548e1640 1 -- start start 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.958+0000 7fe7548e1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 0x7fe750197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.958+0000 7fe7548e1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe75010aa20 con 0x7fe750106540 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.958+0000 7fe74e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 0x7fe750197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.958+0000 7fe74e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 0x7fe750197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.112:53536/0 (socket says 172.21.15.112:53536) 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.958+0000 7fe74e59c640 1 -- 172.21.15.112:0/1424499785 learned_addr learned my addr 172.21.15.112:0/1424499785 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.959+0000 7fe74e59c640 1 -- 172.21.15.112:0/1424499785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe750197580 con 0x7fe750106540 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.959+0000 7fe74e59c640 1 --2- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 0x7fe750197040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe740030b00 tx=0x7fe740030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.959+0000 7fe73f7fe640 1 -- 172.21.15.112:0/1424499785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe740040070 con 0x7fe750106540 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.959+0000 7fe73f7fe640 1 -- 172.21.15.112:0/1424499785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7400318b0 con 0x7fe750106540 2024-09-19T04:07:00.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.959+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe750197810 con 0x7fe750106540 2024-09-19T04:07:00.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.959+0000 7fe73f7fe640 1 -- 172.21.15.112:0/1424499785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe740003de0 con 0x7fe750106540 2024-09-19T04:07:00.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.959+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe750197c70 con 0x7fe750106540 2024-09-19T04:07:00.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.960+0000 7fe73f7fe640 1 -- 172.21.15.112:0/1424499785 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fe740039040 con 0x7fe750106540 2024-09-19T04:07:00.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.960+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe750101bb0 con 0x7fe750106540 2024-09-19T04:07:00.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.960+0000 7fe73f7fe640 1 --2- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe7380392a0 0x7fe73803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:00.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.961+0000 7fe73f7fe640 1 -- 172.21.15.112:0/1424499785 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe74006d810 con 0x7fe750106540 2024-09-19T04:07:00.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.961+0000 7fe74dd9b640 1 --2- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe7380392a0 0x7fe73803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:00.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.961+0000 7fe74dd9b640 1 --2- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe7380392a0 0x7fe73803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7440098c0 tx=0x7fe744006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:00.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:00.966+0000 7fe73f7fe640 1 -- 172.21.15.112:0/1424499785 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe740031b60 con 0x7fe750106540 2024-09-19T04:07:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.215+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe750198400 con 0x7fe750106540 2024-09-19T04:07:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.216+0000 7fe73f7fe640 1 -- 172.21.15.112:0/1424499785 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fe740003730 con 0x7fe750106540 2024-09-19T04:07:01.217 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:07:01.217 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":1,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","modified":"2024-09-19T04:05:31.044155Z","created":"2024-09-19T04:05:31.044155Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T04:07:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:dumped monmap epoch 1 2024-09-19T04:07:01.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe7380392a0 msgr2=0x7fe73803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:01.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 --2- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe7380392a0 0x7fe73803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7440098c0 tx=0x7fe744006db0 comp rx=0 tx=0).stop 2024-09-19T04:07:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 msgr2=0x7fe750197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 --2- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 0x7fe750197040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe740030b00 tx=0x7fe740030b30 comp rx=0 tx=0).stop 2024-09-19T04:07:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 shutdown_connections 2024-09-19T04:07:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 --2- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe7380392a0 0x7fe73803b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 --2- 172.21.15.112:0/1424499785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe750106540 0x7fe750197040 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 >> 172.21.15.112:0/1424499785 conn(0x7fe750077310 msgr2=0x7fe7500feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 shutdown_connections 2024-09-19T04:07:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:01.218+0000 7fe7548e1640 1 -- 172.21.15.112:0/1424499785 wait complete. 2024-09-19T04:07:01.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:01 smithi094 ceph-mon[30972]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:02.262 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:02 smithi112 ceph-mon[35051]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T04:07:02.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:02 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/1424499785' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:07:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:02 smithi112 bash[35030]: 5450f6bd98af8715ce3df9d89be17a04b843f698c19716c835cd8e718ab83dac 2024-09-19T04:07:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:02 smithi112 systemd[1]: Started Ceph mon.c for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:07:02.817 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:07:02.817 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:07:03.070 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:07:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:07:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: mon.a calling monitor election 2024-09-19T04:07:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: mon.c calling monitor election 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: monmap epoch 2 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: last_changed 2024-09-19T04:07:02.233155+0000 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:07:07.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: election_strategy: 1 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: fsmap 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: mgrmap e13: x(active, since 37s) 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:07.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:07.594 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:07 smithi094 ceph-mon[30972]: Deploying daemon mon.b on smithi104 2024-09-19T04:07:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:07:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: mon.a calling monitor election 2024-09-19T04:07:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: mon.c calling monitor election 2024-09-19T04:07:07.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:07.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:07.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:07:07.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: monmap epoch 2 2024-09-19T04:07:07.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:07.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: last_changed 2024-09-19T04:07:02.233155+0000 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: min_mon_release 19 (squid) 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: election_strategy: 1 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: fsmap 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: mgrmap e13: x(active, since 37s) 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: overall HEALTH_OK 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:07.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:07.790 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:07.790 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:07 smithi112 ceph-mon[35051]: Deploying daemon mon.b on smithi104 2024-09-19T04:07:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.873+0000 7faafa176640 1 -- 172.21.15.112:0/2465272257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faadc003530 msgr2=0x7faadc0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.873+0000 7faafa176640 1 --2- 172.21.15.112:0/2465272257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faadc003530 0x7faadc0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faaf406aa60 tx=0x7faae402c490 comp rx=0 tx=0).stop 2024-09-19T04:07:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.873+0000 7faafa176640 1 -- 172.21.15.112:0/2465272257 shutdown_connections 2024-09-19T04:07:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.873+0000 7faafa176640 1 --2- 172.21.15.112:0/2465272257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faadc003530 0x7faadc0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.873+0000 7faafa176640 1 -- 172.21.15.112:0/2465272257 >> 172.21.15.112:0/2465272257 conn(0x7faaf40fc220 msgr2=0x7faaf40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.873+0000 7faafa176640 1 -- 172.21.15.112:0/2465272257 shutdown_connections 2024-09-19T04:07:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.873+0000 7faafa176640 1 -- 172.21.15.112:0/2465272257 wait complete. 2024-09-19T04:07:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.874+0000 7faafa176640 1 Processor -- start 2024-09-19T04:07:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.874+0000 7faafa176640 1 -- start start 2024-09-19T04:07:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faafa176640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 0x7faaf4100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faafa176640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf41009c0 0x7faaf4100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faafa176640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4109630 con 0x7faaf41009c0 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faafa176640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41094b0 con 0x7faadc003530 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 0x7faaf4100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 0x7faaf4100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35342/0 (socket says 172.21.15.112:35342) 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf37fe640 1 -- 172.21.15.112:0/3302295384 learned_addr learned my addr 172.21.15.112:0/3302295384 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf2ffd640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf41009c0 0x7faaf4100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf37fe640 1 -- 172.21.15.112:0/3302295384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 msgr2=0x7faaf4100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf37fe640 1 -- 172.21.15.112:0/3302295384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 msgr2=0x7faaf4100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf37fe640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 0x7faaf4100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf37fe640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 0x7faaf4100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf2ffd640 1 -- 172.21.15.112:0/3302295384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 msgr2=0x7faaf4100480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:07:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf2ffd640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 0x7faaf4100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.875+0000 7faaf2ffd640 1 -- 172.21.15.112:0/3302295384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf4102370 con 0x7faaf41009c0 2024-09-19T04:07:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.876+0000 7faaf2ffd640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf41009c0 0x7faaf4100e20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faae800b400 tx=0x7faae800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.876+0000 7faaf0ff9640 1 -- 172.21.15.112:0/3302295384 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7faae8011630 con 0x7faaf41009c0 2024-09-19T04:07:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.876+0000 7faaf0ff9640 1 -- 172.21.15.112:0/3302295384 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faae8011c80 con 0x7faaf41009c0 2024-09-19T04:07:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.876+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf41a34b0 con 0x7faaf41009c0 2024-09-19T04:07:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.876+0000 7faaf0ff9640 1 -- 172.21.15.112:0/3302295384 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7faae8010d10 con 0x7faaf41009c0 2024-09-19T04:07:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.876+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41a3a40 con 0x7faaf41009c0 2024-09-19T04:07:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.877+0000 7faaf0ff9640 1 -- 172.21.15.112:0/3302295384 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7faae8018020 con 0x7faaf41009c0 2024-09-19T04:07:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.877+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf406ad10 con 0x7faaf41009c0 2024-09-19T04:07:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.877+0000 7faaf0ff9640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faae0039090 0x7faae003b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.877+0000 7faaf0ff9640 1 -- 172.21.15.112:0/3302295384 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faae804bfa0 con 0x7faaf41009c0 2024-09-19T04:07:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.878+0000 7faaf37fe640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faae0039090 0x7faae003b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.878+0000 7faaf37fe640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faae0039090 0x7faae003b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faae4012c10 tx=0x7faae40323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:07.882+0000 7faaf0ff9640 1 -- 172.21.15.112:0/3302295384 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faae801f020 con 0x7faaf41009c0 2024-09-19T04:07:08.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.117+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7faaf41a3d50 con 0x7faaf41009c0 2024-09-19T04:07:08.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.118+0000 7faaf0ff9640 1 -- 172.21.15.112:0/3302295384 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7faae8018310 con 0x7faaf41009c0 2024-09-19T04:07:08.119 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:07:08.120 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","modified":"2024-09-19T04:07:02.233155Z","created":"2024-09-19T04:05:31.044155Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:07:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:dumped monmap epoch 2 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faae0039090 msgr2=0x7faae003b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faae0039090 0x7faae003b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faae4012c10 tx=0x7faae40323a0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf41009c0 msgr2=0x7faaf4100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf41009c0 0x7faaf4100e20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faae800b400 tx=0x7faae800b8c0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 shutdown_connections 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faae0039090 0x7faae003b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf41009c0 0x7faaf4100e20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 --2- 172.21.15.112:0/3302295384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faadc003530 0x7faaf4100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 >> 172.21.15.112:0/3302295384 conn(0x7faaf40fc220 msgr2=0x7faaf41055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 shutdown_connections 2024-09-19T04:07:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:08.120+0000 7faafa176640 1 -- 172.21.15.112:0/3302295384 wait complete. 2024-09-19T04:07:08.460 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:08 smithi112 ceph-mon[35051]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:08.460 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:08 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/3302295384' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:07:08.460 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:08.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:08 smithi094 ceph-mon[30972]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:08.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:08 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/3302295384' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:07:08.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:09.703 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:07:09.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:07:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:07:10.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.333+0000 7f236ffbb640 1 -- 172.21.15.112:0/1502756113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2368100610 msgr2=0x7f23681009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:10.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.333+0000 7f236ffbb640 1 --2- 172.21.15.112:0/1502756113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2368100610 0x7f23681009f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2364009880 tx=0x7f236402a670 comp rx=0 tx=0).stop 2024-09-19T04:07:10.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.333+0000 7f236ffbb640 1 -- 172.21.15.112:0/1502756113 shutdown_connections 2024-09-19T04:07:10.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.333+0000 7f236ffbb640 1 --2- 172.21.15.112:0/1502756113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2368100610 0x7f23681009f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.333+0000 7f236ffbb640 1 -- 172.21.15.112:0/1502756113 >> 172.21.15.112:0/1502756113 conn(0x7f23680fc220 msgr2=0x7f23680fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:10.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.334+0000 7f236ffbb640 1 -- 172.21.15.112:0/1502756113 shutdown_connections 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.334+0000 7f236ffbb640 1 -- 172.21.15.112:0/1502756113 wait complete. 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.334+0000 7f236ffbb640 1 Processor -- start 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.334+0000 7f236ffbb640 1 -- start start 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236ffbb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 0x7f23681a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236ffbb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23681a3b80 0x7f236819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236ffbb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2368112220 con 0x7f23681a3b80 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236ffbb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23681120a0 con 0x7f2368100610 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236dd57640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 0x7f23681a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236dd57640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 0x7f23681a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36292/0 (socket says 172.21.15.112:36292) 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236dd57640 1 -- 172.21.15.112:0/4082979558 learned_addr learned my addr 172.21.15.112:0/4082979558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236dd57640 1 -- 172.21.15.112:0/4082979558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 msgr2=0x7f23681a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236dd57640 1 -- 172.21.15.112:0/4082979558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 msgr2=0x7f23681a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236dd57640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 0x7f23681a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236d556640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23681a3b80 0x7f236819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236dd57640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 0x7f23681a3640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236d556640 1 -- 172.21.15.112:0/4082979558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 msgr2=0x7f23681a3640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236d556640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 0x7f23681a3640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236d556640 1 -- 172.21.15.112:0/4082979558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23681a4120 con 0x7f23681a3b80 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.335+0000 7f236d556640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23681a3b80 0x7f236819d710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f235c00b3d0 tx=0x7f235c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.336+0000 7f235affd640 1 -- 172.21.15.112:0/4082979558 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f235c011630 con 0x7f23681a3b80 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.336+0000 7f235affd640 1 -- 172.21.15.112:0/4082979558 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f235c011c80 con 0x7f23681a3b80 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.336+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236819dcb0 con 0x7f23681a3b80 2024-09-19T04:07:10.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.336+0000 7f235affd640 1 -- 172.21.15.112:0/4082979558 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f235c010d80 con 0x7f23681a3b80 2024-09-19T04:07:10.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.336+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236819e240 con 0x7f23681a3b80 2024-09-19T04:07:10.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.337+0000 7f235affd640 1 -- 172.21.15.112:0/4082979558 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f235c018020 con 0x7f23681a3b80 2024-09-19T04:07:10.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.337+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236806ad10 con 0x7f23681a3b80 2024-09-19T04:07:10.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.337+0000 7f235affd640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2354039030 0x7f235403b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:10.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.337+0000 7f235affd640 1 -- 172.21.15.112:0/4082979558 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f235c04bfb0 con 0x7f23681a3b80 2024-09-19T04:07:10.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.337+0000 7f236dd57640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2354039030 0x7f235403b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:10.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.338+0000 7f236dd57640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2354039030 0x7f235403b4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f236402ac60 tx=0x7f236402ab70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:10.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.342+0000 7f235affd640 1 -- 172.21.15.112:0/4082979558 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f235c01f020 con 0x7f23681a3b80 2024-09-19T04:07:10.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.572+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f236819e5b0 con 0x7f23681a3b80 2024-09-19T04:07:10.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.572+0000 7f235affd640 1 -- 172.21.15.112:0/4082979558 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f235c0151c0 con 0x7f23681a3b80 2024-09-19T04:07:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:07:10.574 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","modified":"2024-09-19T04:07:02.233155Z","created":"2024-09-19T04:05:31.044155Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:07:10.574 INFO:teuthology.orchestra.run.smithi112.stderr:dumped monmap epoch 2 2024-09-19T04:07:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2354039030 msgr2=0x7f235403b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2354039030 0x7f235403b4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f236402ac60 tx=0x7f236402ab70 comp rx=0 tx=0).stop 2024-09-19T04:07:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23681a3b80 msgr2=0x7f236819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23681a3b80 0x7f236819d710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f235c00b3d0 tx=0x7f235c00b890 comp rx=0 tx=0).stop 2024-09-19T04:07:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 shutdown_connections 2024-09-19T04:07:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2354039030 0x7f235403b4f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23681a3b80 0x7f236819d710 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 --2- 172.21.15.112:0/4082979558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2368100610 0x7f23681a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 >> 172.21.15.112:0/4082979558 conn(0x7f23680fc220 msgr2=0x7f23680fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:10.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.574+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 shutdown_connections 2024-09-19T04:07:10.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:10.575+0000 7f236ffbb640 1 -- 172.21.15.112:0/4082979558 wait complete. 2024-09-19T04:07:10.889 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(???) e0 preinit fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:10.889 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).mds e1 new map 2024-09-19T04:07:10.889 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-19T04:07:10.889 INFO:journalctl@ceph.mon.b.smithi104.stdout: e1 2024-09-19T04:07:10.889 INFO:journalctl@ceph.mon.b.smithi104.stdout: btime 2024-09-19T04:05:34:057432+0000 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout: legacy client fscid: -1 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout: No filesystems configured 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:07:10.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:07:11.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:07:11.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/119488852' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/119488852' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:07:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi104:172.21.15.104=b;smithi112:172.21.15.112=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi104:172.21.15.104=b;smithi112:172.21.15.112=c;count:3 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: Deploying daemon mon.c on smithi112 2024-09-19T04:07:11.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:11.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:11.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/1424499785' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:07:11.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:11.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:07:11.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.a calling monitor election 2024-09-19T04:07:11.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:11.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:11.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.c calling monitor election 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: monmap epoch 2 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: last_changed 2024-09-19T04:07:02.233155+0000 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: min_mon_release 19 (squid) 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: election_strategy: 1 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:07:11.358 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: fsmap 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mgrmap e13: x(active, since 37s) 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: overall HEALTH_OK 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: Deploying daemon mon.b on smithi104 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/3302295384' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 ceph-mon[34145]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 bash[34126]: c97384baf7fc62c69677d163de40a388d2919a20678cb75e560c713850e557b8 2024-09-19T04:07:11.359 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:10 smithi104 systemd[1]: Started Ceph mon.b for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:07:12.154 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:07:12.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:07:12.402 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:07:16.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.015+0000 7f88a42d7640 1 -- 172.21.15.112:0/3015388352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8884003530 msgr2=0x7f88840059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:16.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.015+0000 7f88a42d7640 1 --2- 172.21.15.112:0/3015388352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8884003530 0x7f88840059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f889c1016a0 tx=0x7f889402c150 comp rx=0 tx=0).stop 2024-09-19T04:07:16.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.016+0000 7f88a42d7640 1 -- 172.21.15.112:0/3015388352 shutdown_connections 2024-09-19T04:07:16.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.016+0000 7f88a42d7640 1 --2- 172.21.15.112:0/3015388352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8884003530 0x7f88840059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.016+0000 7f88a42d7640 1 -- 172.21.15.112:0/3015388352 >> 172.21.15.112:0/3015388352 conn(0x7f889c0fc220 msgr2=0x7f889c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.016+0000 7f88a42d7640 1 -- 172.21.15.112:0/3015388352 shutdown_connections 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.016+0000 7f88a42d7640 1 -- 172.21.15.112:0/3015388352 wait complete. 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.016+0000 7f88a42d7640 1 Processor -- start 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a42d7640 1 -- start start 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a42d7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8884003530 0x7f889c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a42d7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c1977d0 0x7f889c19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a42d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 0x7f889c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a42d7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f889c108520 con 0x7f8884003530 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a42d7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f889c1083a0 con 0x7f889c1977d0 2024-09-19T04:07:16.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a42d7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f889c1086a0 con 0x7f889c19c0a0 2024-09-19T04:07:16.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a2874640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 0x7f889c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:16.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a2874640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 0x7f889c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36308/0 (socket says 172.21.15.112:36308) 2024-09-19T04:07:16.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a2073640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8884003530 0x7f889c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:16.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a2874640 1 -- 172.21.15.112:0/2504878852 learned_addr learned my addr 172.21.15.112:0/2504878852 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:07:16.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a1872640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c1977d0 0x7f889c19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a2874640 1 -- 172.21.15.112:0/2504878852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 msgr2=0x7f889c19c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a2874640 1 -- 172.21.15.112:0/2504878852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 msgr2=0x7f889c19c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a2874640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 0x7f889c19c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a1872640 1 -- 172.21.15.112:0/2504878852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c1977d0 msgr2=0x7f889c19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a2874640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 0x7f889c19c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a1872640 1 -- 172.21.15.112:0/2504878852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c1977d0 msgr2=0x7f889c19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a1872640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c1977d0 0x7f889c19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.017+0000 7f88a1872640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c1977d0 0x7f889c19bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f88a2073640 1 -- 172.21.15.112:0/2504878852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 msgr2=0x7f889c19c550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f88a2073640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 0x7f889c19c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f88a2073640 1 -- 172.21.15.112:0/2504878852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c1977d0 msgr2=0x7f889c19bb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f88a2073640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c1977d0 0x7f889c19bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f88a2073640 1 -- 172.21.15.112:0/2504878852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c1a3e10 con 0x7f8884003530 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f88a2073640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8884003530 0x7f889c197290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f889c1016a0 tx=0x7f8894004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f889b7fe640 1 -- 172.21.15.112:0/2504878852 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894032040 con 0x7f8884003530 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f889b7fe640 1 -- 172.21.15.112:0/2504878852 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f889402cb60 con 0x7f8884003530 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c1a3fe0 con 0x7f8884003530 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f889b7fe640 1 -- 172.21.15.112:0/2504878852 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894036670 con 0x7f8884003530 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.018+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c1a4490 con 0x7f8884003530 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.019+0000 7f889b7fe640 1 -- 172.21.15.112:0/2504878852 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f88940368d0 con 0x7f8884003530 2024-09-19T04:07:16.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.019+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889c1095d0 con 0x7f8884003530 2024-09-19T04:07:16.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.020+0000 7f889b7fe640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8888039730 0x7f888803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:16.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.020+0000 7f889b7fe640 1 -- 172.21.15.112:0/2504878852 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f889406dff0 con 0x7f8884003530 2024-09-19T04:07:16.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.020+0000 7f88a1872640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8888039730 0x7f888803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:16.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.020+0000 7f88a1872640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8888039730 0x7f888803bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f88900070b0 tx=0x7f8890007040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:16.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.025+0000 7f889b7fe640 1 -- 172.21.15.112:0/2504878852 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f889404c200 con 0x7f8884003530 2024-09-19T04:07:16.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.267+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f889c062e50 con 0x7f8884003530 2024-09-19T04:07:16.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.268+0000 7f889b7fe640 1 -- 172.21.15.112:0/2504878852 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f889402cd10 con 0x7f8884003530 2024-09-19T04:07:16.270 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:07:16.271 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","modified":"2024-09-19T04:07:10.893365Z","created":"2024-09-19T04:05:31.044155Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:07:16.271 INFO:teuthology.orchestra.run.smithi112.stderr:dumped monmap epoch 3 2024-09-19T04:07:16.271 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:07:16.271 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: mon.c calling monitor election 2024-09-19T04:07:16.271 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: mon.a calling monitor election 2024-09-19T04:07:16.271 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: monmap epoch 3 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:16.272 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: last_changed 2024-09-19T04:07:10.893365+0000 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: min_mon_release 19 (squid) 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: election_strategy: 1 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: fsmap 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: mgrmap e13: x(active, since 45s) 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: overall HEALTH_OK 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:16.273 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:16.274 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:16.274 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:16.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8888039730 msgr2=0x7f888803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:16.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8888039730 0x7f888803bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f88900070b0 tx=0x7f8890007040 comp rx=0 tx=0).stop 2024-09-19T04:07:16.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8884003530 msgr2=0x7f889c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8884003530 0x7f889c197290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f889c1016a0 tx=0x7f8894004f80 comp rx=0 tx=0).stop 2024-09-19T04:07:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 shutdown_connections 2024-09-19T04:07:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8888039730 0x7f888803bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c19c0a0 0x7f889c19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c1977d0 0x7f889c19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 --2- 172.21.15.112:0/2504878852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8884003530 0x7f889c197290 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 >> 172.21.15.112:0/2504878852 conn(0x7f889c0fc220 msgr2=0x7f889c1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 shutdown_connections 2024-09-19T04:07:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:16.270+0000 7f88a42d7640 1 -- 172.21.15.112:0/2504878852 wait complete. 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: mon.c calling monitor election 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: mon.a calling monitor election 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: monmap epoch 3 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: last_changed 2024-09-19T04:07:10.893365+0000 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: election_strategy: 1 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-19T04:07:16.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-19T04:07:16.343 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: fsmap 2024-09-19T04:07:16.343 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:16.343 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: mgrmap e13: x(active, since 45s) 2024-09-19T04:07:16.343 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T04:07:16.343 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:16.343 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:16.343 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:16.343 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:16.842 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T04:07:16.842 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T04:07:17.096 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:17.216 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:16 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:17.216 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:16 smithi094 ceph-mon[30972]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:17.216 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:16 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/2504878852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:07:17.217 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:16 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:17.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:16 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:17.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:16 smithi112 ceph-mon[35051]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:17.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:16 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/2504878852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:07:17.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:16 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.524+0000 7f0bc6696640 1 -- 172.21.15.94:0/3061719253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 msgr2=0x7f0bc0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.524+0000 7f0bc6696640 1 --2- 172.21.15.94:0/3061719253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 0x7f0bc0106940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009880 tx=0x7f0bb002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.524+0000 7f0bc6696640 1 -- 172.21.15.94:0/3061719253 shutdown_connections 2024-09-19T04:07:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.524+0000 7f0bc6696640 1 --2- 172.21.15.94:0/3061719253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 0x7f0bc0106940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.524+0000 7f0bc6696640 1 -- 172.21.15.94:0/3061719253 >> 172.21.15.94:0/3061719253 conn(0x7f0bc00fc1b0 msgr2=0x7f0bc00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.524+0000 7f0bc6696640 1 -- 172.21.15.94:0/3061719253 shutdown_connections 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.524+0000 7f0bc6696640 1 -- 172.21.15.94:0/3061719253 wait complete. 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bc6696640 1 Processor -- start 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bc6696640 1 -- start start 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bc6696640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 0x7f0bc01994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bc6696640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0bc0199a10 0x7f0bc019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bc6696640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc019e2e0 0x7f0bc019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bc6696640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc010aa40 con 0x7f0bc0106560 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bc6696640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc010a8c0 con 0x7f0bc0199a10 2024-09-19T04:07:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bc6696640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc010abc0 con 0x7f0bc019e2e0 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 0x7f0bc01994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 0x7f0bc01994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33084/0 (socket says 172.21.15.94:33084) 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bbffff640 1 -- 172.21.15.94:0/215112407 learned_addr learned my addr 172.21.15.94:0/215112407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.525+0000 7f0bc4c33640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc019e2e0 0x7f0bc019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bbffff640 1 -- 172.21.15.94:0/215112407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc019e2e0 msgr2=0x7f0bc019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bbffff640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc019e2e0 0x7f0bc019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bbffff640 1 -- 172.21.15.94:0/215112407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0bc0199a10 msgr2=0x7f0bc019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bbffff640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0bc0199a10 0x7f0bc019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bbffff640 1 -- 172.21.15.94:0/215112407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc01a6050 con 0x7f0bc0106560 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bc4c33640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc019e2e0 0x7f0bc019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bbffff640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 0x7f0bc01994d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009230 tx=0x7f0bb0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bbd7fa640 1 -- 172.21.15.94:0/215112407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb003f070 con 0x7f0bc0106560 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bbd7fa640 1 -- 172.21.15.94:0/215112407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bb0031d60 con 0x7f0bc0106560 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bbd7fa640 1 -- 172.21.15.94:0/215112407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0035030 con 0x7f0bc0106560 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc01a6220 con 0x7f0bc0106560 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.526+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc01a6720 con 0x7f0bc0106560 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.527+0000 7f0bbd7fa640 1 -- 172.21.15.94:0/215112407 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f0bb0042070 con 0x7f0bc0106560 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.527+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc01069c0 con 0x7f0bc0106560 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.527+0000 7f0bbd7fa640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0ba8039730 0x7f0ba803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.527+0000 7f0bbd7fa640 1 -- 172.21.15.94:0/215112407 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0bb006d1c0 con 0x7f0bc0106560 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.527+0000 7f0bbf7fe640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0ba8039730 0x7f0ba803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.528+0000 7f0bbf7fe640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0ba8039730 0x7f0ba803bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4006fd0 tx=0x7f0bb4006d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.532+0000 7f0bbd7fa640 1 -- 172.21.15.94:0/215112407 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0bb0042bf0 con 0x7f0bc0106560 2024-09-19T04:07:17.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.723+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0bc002d7c0 con 0x7f0bc0106560 2024-09-19T04:07:17.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.723+0000 7f0bbd7fa640 1 -- 172.21.15.94:0/215112407 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f0bb003a3c0 con 0x7f0bc0106560 2024-09-19T04:07:17.726 INFO:teuthology.orchestra.run.smithi094.stdout:# minimal ceph.conf for 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:17.726 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-19T04:07:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: mon_host = [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0ba8039730 msgr2=0x7f0ba803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0ba8039730 0x7f0ba803bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4006fd0 tx=0x7f0bb4006d40 comp rx=0 tx=0).stop 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 msgr2=0x7f0bc01994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 0x7f0bc01994d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009230 tx=0x7f0bb0003d50 comp rx=0 tx=0).stop 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 shutdown_connections 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0ba8039730 0x7f0ba803bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc019e2e0 0x7f0bc019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0bc0199a10 0x7f0bc019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 --2- 172.21.15.94:0/215112407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc0106560 0x7f0bc01994d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 >> 172.21.15.94:0/215112407 conn(0x7f0bc00fc1b0 msgr2=0x7f0bc0102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:17.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.725+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 shutdown_connections 2024-09-19T04:07:17.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:17.726+0000 7f0bc6696640 1 -- 172.21.15.94:0/215112407 wait complete. 2024-09-19T04:07:18.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:07:18.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: mon.c calling monitor election 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: mon.a calling monitor election 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:07:18.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: monmap epoch 3 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: last_changed 2024-09-19T04:07:10.893365+0000 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: min_mon_release 19 (squid) 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: election_strategy: 1 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: fsmap 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: mgrmap e13: x(active, since 45s) 2024-09-19T04:07:18.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: overall HEALTH_OK 2024-09-19T04:07:18.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:18.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:18.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/2504878852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:07:18.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:18.436 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T04:07:18.436 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:07:18.436 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:07:18.463 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:07:18.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:07:18.529 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:07:18.529 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:07:18.556 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:07:18.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:07:18.622 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:07:18.622 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:07:18.649 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:07:18.649 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:07:18.715 INFO:tasks.cephadm:Adding mgr.x on smithi094 2024-09-19T04:07:18.715 INFO:tasks.cephadm:Adding mgr.y on smithi104 2024-09-19T04:07:18.715 INFO:tasks.cephadm:Adding mgr.z on smithi112 2024-09-19T04:07:18.715 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi094=x;smithi104=y;smithi112=z' 2024-09-19T04:07:18.799 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: mon.b calling monitor election 2024-09-19T04:07:18.799 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: Updating smithi094:/etc/ceph/ceph.conf 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: mon.b calling monitor election 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: mon.c calling monitor election 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: mon.a calling monitor election 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: monmap epoch 3 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:18.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: last_changed 2024-09-19T04:07:10.893365+0000 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: election_strategy: 1 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: fsmap 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: mgrmap e13: x(active, since 47s) 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.801 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:07:18.803 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:18.803 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:07:18.803 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:18.803 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:18 smithi094 ceph-mon[30972]: Reconfiguring daemon mon.a on smithi094 2024-09-19T04:07:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: mon.b calling monitor election 2024-09-19T04:07:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: Updating smithi094:/etc/ceph/ceph.conf 2024-09-19T04:07:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-19T04:07:18.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-19T04:07:18.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:18.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:18.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:18.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: mon.b calling monitor election 2024-09-19T04:07:18.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: mon.c calling monitor election 2024-09-19T04:07:18.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: mon.a calling monitor election 2024-09-19T04:07:18.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: monmap epoch 3 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: last_changed 2024-09-19T04:07:10.893365+0000 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: min_mon_release 19 (squid) 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: election_strategy: 1 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-19T04:07:18.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: fsmap 2024-09-19T04:07:18.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:18.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: mgrmap e13: x(active, since 47s) 2024-09-19T04:07:18.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: overall HEALTH_OK 2024-09-19T04:07:18.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:18.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:18.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:07:18.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:18.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:07:18.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:18.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:18 smithi104 ceph-mon[34145]: Reconfiguring daemon mon.a on smithi094 2024-09-19T04:07:18.989 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:07:19.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: mon.b calling monitor election 2024-09-19T04:07:19.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: Updating smithi094:/etc/ceph/ceph.conf 2024-09-19T04:07:19.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: mon.b calling monitor election 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: mon.c calling monitor election 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: mon.a calling monitor election 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: monmap epoch 3 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:07:19.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: last_changed 2024-09-19T04:07:10.893365+0000 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: created 2024-09-19T04:05:31.044155+0000 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: min_mon_release 19 (squid) 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: election_strategy: 1 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: fsmap 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: mgrmap e13: x(active, since 47s) 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: overall HEALTH_OK 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.037 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:19.038 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:07:19.039 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:19.039 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:18 smithi112 ceph-mon[35051]: Reconfiguring daemon mon.a on smithi094 2024-09-19T04:07:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.427+0000 7fb8f6626640 1 -- 172.21.15.112:0/223049962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0102180 msgr2=0x7fb8f0104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.427+0000 7fb8f6626640 1 --2- 172.21.15.112:0/223049962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0102180 0x7fb8f0104570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009880 tx=0x7fb8e002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:07:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 -- 172.21.15.112:0/223049962 shutdown_connections 2024-09-19T04:07:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 --2- 172.21.15.112:0/223049962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0102180 0x7fb8f0104570 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 -- 172.21.15.112:0/223049962 >> 172.21.15.112:0/223049962 conn(0x7fb8f00fbda0 msgr2=0x7fb8f00fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 -- 172.21.15.112:0/223049962 shutdown_connections 2024-09-19T04:07:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 -- 172.21.15.112:0/223049962 wait complete. 2024-09-19T04:07:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 Processor -- start 2024-09-19T04:07:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 -- start start 2024-09-19T04:07:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102180 0x7fb8f0196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f01973b0 0x7fb8f019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0197830 0x7fb8f019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01075f0 con 0x7fb8f0197830 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0107470 con 0x7fb8f01973b0 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.428+0000 7fb8f6626640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0107770 con 0x7fb8f0102180 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8effff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102180 0x7fb8f0196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8effff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102180 0x7fb8f0196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56852/0 (socket says 172.21.15.112:56852) 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8effff640 1 -- 172.21.15.112:0/1290397154 learned_addr learned my addr 172.21.15.112:0/1290397154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8f4bc3640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0197830 0x7fb8f019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8e7fff640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f01973b0 0x7fb8f019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8effff640 1 -- 172.21.15.112:0/1290397154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f01973b0 msgr2=0x7fb8f019c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8effff640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f01973b0 0x7fb8f019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8effff640 1 -- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0197830 msgr2=0x7fb8f019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8effff640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0197830 0x7fb8f019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8effff640 1 -- 172.21.15.112:0/1290397154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f019f270 con 0x7fb8f0102180 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8f4bc3640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0197830 0x7fb8f019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8effff640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102180 0x7fb8f0196e70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8edffb640 1 -- 172.21.15.112:0/1290397154 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e003f030 con 0x7fb8f0102180 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f019f500 con 0x7fb8f0102180 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8edffb640 1 -- 172.21.15.112:0/1290397154 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8e0003cb0 con 0x7fb8f0102180 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8edffb640 1 -- 172.21.15.112:0/1290397154 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0035070 con 0x7fb8f0102180 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.429+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f019fb10 con 0x7fb8f0102180 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.430+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f01903b0 con 0x7fb8f0102180 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.431+0000 7fb8edffb640 1 -- 172.21.15.112:0/1290397154 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb8e0042030 con 0x7fb8f0102180 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.431+0000 7fb8edffb640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8dc039730 0x7fb8dc03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:19.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.431+0000 7fb8edffb640 1 -- 172.21.15.112:0/1290397154 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb8e006d190 con 0x7fb8f0102180 2024-09-19T04:07:19.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.431+0000 7fb8e7fff640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8dc039730 0x7fb8dc03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:19.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.432+0000 7fb8e7fff640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8dc039730 0x7fb8dc03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006fd0 tx=0x7fb8d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:19.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.436+0000 7fb8edffb640 1 -- 172.21.15.112:0/1290397154 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb8e0030e10 con 0x7fb8f0102180 2024-09-19T04:07:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.619+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi104=y;smithi112=z", "target": ["mon-mgr", ""]}) -- 0x7fb8f00626c0 con 0x7fb8dc039730 2024-09-19T04:07:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.624+0000 7fb8edffb640 1 -- 172.21.15.112:0/1290397154 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb8f00626c0 con 0x7fb8dc039730 2024-09-19T04:07:19.626 INFO:teuthology.orchestra.run.smithi112.stdout:Scheduled mgr update... 2024-09-19T04:07:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8dc039730 msgr2=0x7fb8dc03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8dc039730 0x7fb8dc03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006fd0 tx=0x7fb8d8008040 comp rx=0 tx=0).stop 2024-09-19T04:07:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102180 msgr2=0x7fb8f0196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102180 0x7fb8f0196e70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e00042f0 comp rx=0 tx=0).stop 2024-09-19T04:07:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 shutdown_connections 2024-09-19T04:07:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8dc039730 0x7fb8dc03bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0197830 0x7fb8f019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f01973b0 0x7fb8f019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 --2- 172.21.15.112:0/1290397154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102180 0x7fb8f0196e70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 >> 172.21.15.112:0/1290397154 conn(0x7fb8f00fbda0 msgr2=0x7fb8f00fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 shutdown_connections 2024-09-19T04:07:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:07:19.627+0000 7fb8f6626640 1 -- 172.21.15.112:0/1290397154 wait complete. 2024-09-19T04:07:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:19 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:19.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:19 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:19 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:07:20.288 DEBUG:teuthology.orchestra.run.smithi104:mgr.y> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T04:07:20.291 DEBUG:teuthology.orchestra.run.smithi112:mgr.z> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T04:07:20.293 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T04:07:20.294 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:07:20.294 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:07:20.307 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:07:19 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:07:19.892+0000 7f74f4b3b640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T04:07:20.313 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:07:20.313 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-19 04:06:41.315561431 +0000 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-19 04:02:20.596431906 +0000 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-19 04:02:20.596431906 +0000 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-19 04:02:20.596431906 +0000 2024-09-19T04:07:20.372 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:07:20.439 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-19T04:07:20.439 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-19T04:07:20.440 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000130039 s, 3.9 MB/s 2024-09-19T04:07:20.441 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:07:20.499 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-09-19T04:07:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:07:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:20.556 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T04:07:20.556 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:20.556 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:20.556 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-19 04:06:41.348560878 +0000 2024-09-19T04:07:20.556 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-19 04:02:20.905426899 +0000 2024-09-19T04:07:20.556 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-19 04:02:20.905426899 +0000 2024-09-19T04:07:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-19 04:02:20.905426899 +0000 2024-09-19T04:07:20.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:07:20.621 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-19T04:07:20.622 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-19T04:07:20.622 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000110899 s, 4.6 MB/s 2024-09-19T04:07:20.623 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:07:20.681 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-09-19T04:07:20.737 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:07:20.737 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T04:07:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-19 04:06:41.380560341 +0000 2024-09-19T04:07:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-19 04:02:21.239421487 +0000 2024-09-19T04:07:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-19 04:02:21.239421487 +0000 2024-09-19T04:07:20.738 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-19 04:02:21.239421487 +0000 2024-09-19T04:07:20.738 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:07:20.801 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-19T04:07:20.801 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-19T04:07:20.801 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000115174 s, 4.4 MB/s 2024-09-19T04:07:20.803 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:07:20.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:20.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: Reconfiguring daemon mon.b on smithi104 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi104=y;smithi112=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: Saving service mgr spec with placement smithi094=x;smithi104=y;smithi112=z;count:3 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:20.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:20.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:20.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:07:20.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:20.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:07:20.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:20.847 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: Reconfiguring daemon mon.b on smithi104 2024-09-19T04:07:20.847 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi104=y;smithi112=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:20.847 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: Saving service mgr spec with placement smithi094=x;smithi104=y;smithi112=z;count:3 2024-09-19T04:07:20.847 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:20.847 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:20.847 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:20.857 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-09-19T04:07:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:07:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:20.913 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-19T04:07:20.913 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:20.913 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:20.913 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-19 04:06:41.411559821 +0000 2024-09-19T04:07:20.913 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-19 04:02:21.547416496 +0000 2024-09-19T04:07:20.913 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-19 04:02:21.547416496 +0000 2024-09-19T04:07:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-19 04:02:21.547416496 +0000 2024-09-19T04:07:20.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:07:20.976 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-19T04:07:20.976 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-19T04:07:20.976 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000112113 s, 4.6 MB/s 2024-09-19T04:07:20.977 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:07:21.033 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-19T04:07:21.033 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:07:21.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:21.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: Reconfiguring daemon mon.b on smithi104 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi104=y;smithi112=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: Saving service mgr spec with placement smithi094=x;smithi104=y;smithi112=z;count:3 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:21.049 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:07:21.049 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-09-19T04:07:21.105 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:07:21.105 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:21.105 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T04:07:21.106 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:21.106 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:21.106 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-19 04:07:10.356141010 +0000 2024-09-19T04:07:21.106 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-19 04:02:10.720944682 +0000 2024-09-19T04:07:21.106 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-19 04:02:10.720944682 +0000 2024-09-19T04:07:21.106 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-19 04:02:10.720944682 +0000 2024-09-19T04:07:21.106 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:07:21.169 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-19T04:07:21.169 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-19T04:07:21.169 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000216088 s, 2.4 MB/s 2024-09-19T04:07:21.171 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:07:21.226 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-09-19T04:07:21.282 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:07:21.282 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:21.282 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T04:07:21.282 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:21.282 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:21.283 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-19 04:07:10.357140999 +0000 2024-09-19T04:07:21.283 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-19 04:02:11.045940877 +0000 2024-09-19T04:07:21.283 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-19 04:02:11.045940877 +0000 2024-09-19T04:07:21.283 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-19 04:02:11.045940877 +0000 2024-09-19T04:07:21.283 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:07:21.345 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-19T04:07:21.345 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-19T04:07:21.345 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000201226 s, 2.5 MB/s 2024-09-19T04:07:21.346 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:07:21.401 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-09-19T04:07:21.457 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:07:21.457 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:21.457 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T04:07:21.457 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:21.457 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:21.457 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-19 04:07:10.357140999 +0000 2024-09-19T04:07:21.458 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-19 04:02:11.368937096 +0000 2024-09-19T04:07:21.458 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-19 04:02:11.368937096 +0000 2024-09-19T04:07:21.458 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-19 04:02:11.368937096 +0000 2024-09-19T04:07:21.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:07:21.520 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-19T04:07:21.520 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-19T04:07:21.520 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000173437 s, 3.0 MB/s 2024-09-19T04:07:21.521 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:07:21.576 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-09-19T04:07:21.632 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:07:21.632 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:21.632 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-19T04:07:21.632 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:21.632 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:21.633 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-19 04:07:10.357140999 +0000 2024-09-19T04:07:21.633 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-19 04:02:11.693933292 +0000 2024-09-19T04:07:21.633 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-19 04:02:11.693933292 +0000 2024-09-19T04:07:21.633 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-19 04:02:11.693933292 +0000 2024-09-19T04:07:21.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:07:21.695 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-19T04:07:21.695 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-19T04:07:21.695 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000211125 s, 2.4 MB/s 2024-09-19T04:07:21.696 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:07:21.752 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-19T04:07:21.752 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:07:21.769 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:07:21.769 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-09-19T04:07:21.824 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:07:21.825 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T04:07:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-19 04:07:01.361350229 +0000 2024-09-19T04:07:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-19 04:02:10.722767957 +0000 2024-09-19T04:07:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-19 04:02:10.722767957 +0000 2024-09-19T04:07:21.825 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-19 04:02:10.722767957 +0000 2024-09-19T04:07:21.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:07:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-19T04:07:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-19T04:07:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000212664 s, 2.4 MB/s 2024-09-19T04:07:21.891 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:07:21.947 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-09-19T04:07:22.002 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:07:22.002 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:22.002 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-19T04:07:22.002 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:22.002 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:22.002 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-19 04:07:01.362350202 +0000 2024-09-19T04:07:22.002 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-19 04:02:11.040759496 +0000 2024-09-19T04:07:22.003 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-19 04:02:11.040759496 +0000 2024-09-19T04:07:22.003 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-19 04:02:11.040759496 +0000 2024-09-19T04:07:22.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:07:22.065 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-19T04:07:22.066 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-19T04:07:22.066 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000174596 s, 2.9 MB/s 2024-09-19T04:07:22.067 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:07:22.125 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-09-19T04:07:22.180 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:07:22.180 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:22.180 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T04:07:22.180 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:22.180 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:22.181 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-19 04:07:01.362350202 +0000 2024-09-19T04:07:22.181 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-19 04:02:11.357751062 +0000 2024-09-19T04:07:22.181 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-19 04:02:11.357751062 +0000 2024-09-19T04:07:22.181 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-19 04:02:11.357751062 +0000 2024-09-19T04:07:22.182 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:07:22.241 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:22.241 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:22.242 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:21 smithi112 ceph-mon[35051]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:07:22.242 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:22.242 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:07:22.242 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:22.242 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:21 smithi112 ceph-mon[35051]: Reconfiguring daemon mon.c on smithi112 2024-09-19T04:07:22.244 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-19T04:07:22.244 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-19T04:07:22.244 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000179101 s, 2.9 MB/s 2024-09-19T04:07:22.245 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:07:22.301 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-09-19T04:07:22.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:22.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:22.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:21 smithi094 ceph-mon[30972]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:07:22.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:22.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:07:22.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:22.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:21 smithi094 ceph-mon[30972]: Reconfiguring daemon mon.c on smithi112 2024-09-19T04:07:22.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:22.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:22.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:21 smithi104 ceph-mon[34145]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:07:22.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:07:22.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:07:22.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:22.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:21 smithi104 ceph-mon[34145]: Reconfiguring daemon mon.c on smithi112 2024-09-19T04:07:22.358 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:07:22.358 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:07:22.358 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-19T04:07:22.358 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:22.358 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:07:22.358 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-19 04:07:01.362350202 +0000 2024-09-19T04:07:22.358 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-19 04:02:11.695742069 +0000 2024-09-19T04:07:22.358 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-19 04:02:11.695742069 +0000 2024-09-19T04:07:22.358 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-19 04:02:11.695742069 +0000 2024-09-19T04:07:22.359 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:07:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-19T04:07:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-19T04:07:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000168937 s, 3.0 MB/s 2024-09-19T04:07:22.425 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:07:22.482 INFO:tasks.cephadm:Deploying osd.0 on smithi094 with /dev/vg_nvme/lv_4... 2024-09-19T04:07:22.482 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:07:22.732 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:23.382 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:23.382 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:23.382 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:23.382 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:23.382 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:23.382 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:23.382 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:23.383 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:07:23.383 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:07:23.383 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:07:23.383 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:23.383 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:23 smithi112 ceph-mon[35051]: Deploying daemon mgr.z on smithi112 2024-09-19T04:07:23.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:23 smithi094 ceph-mon[30972]: Deploying daemon mgr.z on smithi112 2024-09-19T04:07:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:23.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:07:23.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:07:23.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:07:23.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:23.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:23 smithi104 ceph-mon[34145]: Deploying daemon mgr.z on smithi112 2024-09-19T04:07:24.026 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:23 smithi112 systemd[1]: Starting Ceph mgr.z for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:07:24.281 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 podman[36844]: 2024-09-19 04:07:23.927507557 +0000 UTC m=+0.015590174 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:07:24.281 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 podman[36844]: 2024-09-19 04:07:24.033234874 +0000 UTC m=+0.121317474 container create 6a914845d4b99b9f2221d84df623be72f4ab19e1b2b2544a994208c13da38c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T04:07:24.281 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 podman[36844]: 2024-09-19 04:07:24.252507619 +0000 UTC m=+0.340590220 container init 6a914845d4b99b9f2221d84df623be72f4ab19e1b2b2544a994208c13da38c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T04:07:24.281 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 podman[36844]: 2024-09-19 04:07:24.255393102 +0000 UTC m=+0.343475703 container start 6a914845d4b99b9f2221d84df623be72f4ab19e1b2b2544a994208c13da38c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T04:07:24.281 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 ceph-mgr[36863]: -- 172.21.15.112:0/2446663441 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55935c5d90e0 con 0x55935c5b7000 2024-09-19T04:07:24.535 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 bash[36844]: 6a914845d4b99b9f2221d84df623be72f4ab19e1b2b2544a994208c13da38c72 2024-09-19T04:07:24.535 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 systemd[1]: Started Ceph mgr.z for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:07:24.535 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:24.376+0000 7efc27271040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:07:24.535 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:24.436+0000 7efc27271040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:07:24.929 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:24.582+0000 7efc27271040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:07:24.929 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:24.646+0000 7efc27271040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:07:24.929 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:24.790+0000 7efc27271040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:07:24.930 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:24.927+0000 7efc27271040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:07:25.163 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:25 smithi094 ceph-mon[30972]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:25.164 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:25.164 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:25.164 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:25.164 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:07:25.164 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:07:25.164 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:07:25.164 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:07:25.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-09-19T04:07:25.244 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-mon[35051]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:25.244 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:25.244 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:25.244 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:25.244 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:07:25.244 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:07:25.244 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:07:25.245 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:25.534 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:25.242+0000 7efc27271040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:07:25.535 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:25.303+0000 7efc27271040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:07:25.535 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:25 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:25.484+0000 7efc27271040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:07:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:25 smithi104 ceph-mon[34145]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:07:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:07:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:07:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 -- 172.21.15.94:0/1596319179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 msgr2=0x7efdf8100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 --2- 172.21.15.94:0/1596319179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 0x7efdf8100f10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efde8009900 tx=0x7efde802a410 comp rx=0 tx=0).stop 2024-09-19T04:07:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 -- 172.21.15.94:0/1596319179 shutdown_connections 2024-09-19T04:07:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 --2- 172.21.15.94:0/1596319179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdf81026d0 0x7efdf8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 --2- 172.21.15.94:0/1596319179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf8101d10 0x7efdf8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 --2- 172.21.15.94:0/1596319179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 0x7efdf8100f10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 -- 172.21.15.94:0/1596319179 >> 172.21.15.94:0/1596319179 conn(0x7efdf80fc2a0 msgr2=0x7efdf80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 -- 172.21.15.94:0/1596319179 shutdown_connections 2024-09-19T04:07:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 -- 172.21.15.94:0/1596319179 wait complete. 2024-09-19T04:07:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.884+0000 7efdfe515640 1 Processor -- start 2024-09-19T04:07:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdfe515640 1 -- start start 2024-09-19T04:07:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdfe515640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 0x7efdf806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdfe515640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf8101d10 0x7efdf819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdfe515640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdf81026d0 0x7efdf819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdfe515640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efdf810c650 con 0x7efdf8100b10 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdfe515640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efdf810c4d0 con 0x7efdf8101d10 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdfe515640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efdf810c7d0 con 0x7efdf81026d0 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 0x7efdf806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 0x7efdf806b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41900/0 (socket says 172.21.15.94:41900) 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf7fff640 1 -- 172.21.15.94:0/2138746423 learned_addr learned my addr 172.21.15.94:0/2138746423 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdeffff640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf8101d10 0x7efdf819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdfcab2640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdf81026d0 0x7efdf819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf7fff640 1 -- 172.21.15.94:0/2138746423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdf81026d0 msgr2=0x7efdf819f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf7fff640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdf81026d0 0x7efdf819f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf7fff640 1 -- 172.21.15.94:0/2138746423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf8101d10 msgr2=0x7efdf819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf7fff640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf8101d10 0x7efdf819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf7fff640 1 -- 172.21.15.94:0/2138746423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf819faf0 con 0x7efdf8100b10 2024-09-19T04:07:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf7fff640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 0x7efdf806b2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efde80314e0 tx=0x7efde80042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf5ffb640 1 -- 172.21.15.94:0/2138746423 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde803f030 con 0x7efdf8100b10 2024-09-19T04:07:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf819fd20 con 0x7efdf8100b10 2024-09-19T04:07:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf5ffb640 1 -- 172.21.15.94:0/2138746423 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efde8003cb0 con 0x7efdf8100b10 2024-09-19T04:07:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.885+0000 7efdf5ffb640 1 -- 172.21.15.94:0/2138746423 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8035070 con 0x7efdf8100b10 2024-09-19T04:07:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.886+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf81a01a0 con 0x7efdf8100b10 2024-09-19T04:07:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.887+0000 7efdf5ffb640 1 -- 172.21.15.94:0/2138746423 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7efde8042030 con 0x7efdf8100b10 2024-09-19T04:07:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.887+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf8107f80 con 0x7efdf8100b10 2024-09-19T04:07:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.887+0000 7efdf5ffb640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7efde40394d0 0x7efde403b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.887+0000 7efdf5ffb640 1 -- 172.21.15.94:0/2138746423 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efde806cd40 con 0x7efdf8100b10 2024-09-19T04:07:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.887+0000 7efdeffff640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7efde40394d0 0x7efde403b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.887+0000 7efdeffff640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7efde40394d0 0x7efde403b990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efdf806b870 tx=0x7efde000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:25.891+0000 7efdf5ffb640 1 -- 172.21.15.94:0/2138746423 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efde8036df0 con 0x7efdf8100b10 2024-09-19T04:07:26.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:26.066+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7efdf8062630 con 0x7efde40394d0 2024-09-19T04:07:26.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:26 smithi104 ceph-mon[34145]: Deploying daemon mgr.y on smithi104 2024-09-19T04:07:26.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:26 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:07:26.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:26 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:07:26.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:26 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:26.328 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 systemd[1]: Starting Ceph mgr.y for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:07:26.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:26 smithi094 ceph-mon[30972]: Deploying daemon mgr.y on smithi104 2024-09-19T04:07:26.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:26 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:07:26.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:26 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:07:26.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:26 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:26.426 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:26 smithi112 ceph-mon[35051]: Deploying daemon mgr.y on smithi104 2024-09-19T04:07:26.426 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:26 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:07:26.427 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:26 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:07:26.427 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:26 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:26.427 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:26 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:26.358+0000 7efc27271040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:07:26.427 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:26 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:26.425+0000 7efc27271040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:07:26.604 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 podman[35499]: 2024-09-19 04:07:26.229114037 +0000 UTC m=+0.015567613 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:07:26.604 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 podman[35499]: 2024-09-19 04:07:26.341472162 +0000 UTC m=+0.127925731 container create 56905ac5daf2ee0f625d4dfadea952d2bb57137abceb85973425cea40e6aeb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T04:07:26.605 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 podman[35499]: 2024-09-19 04:07:26.530366811 +0000 UTC m=+0.316820396 container init 56905ac5daf2ee0f625d4dfadea952d2bb57137abceb85973425cea40e6aeb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T04:07:26.605 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 podman[35499]: 2024-09-19 04:07:26.533770455 +0000 UTC m=+0.320224025 container start 56905ac5daf2ee0f625d4dfadea952d2bb57137abceb85973425cea40e6aeb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T04:07:26.605 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 ceph-mgr[35518]: -- 172.21.15.104:0/3134439702 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5653b1dc30e0 con 0x5653b1da1000 2024-09-19T04:07:26.882 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 bash[35499]: 56905ac5daf2ee0f625d4dfadea952d2bb57137abceb85973425cea40e6aeb07 2024-09-19T04:07:26.882 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 systemd[1]: Started Ceph mgr.y for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:07:26.882 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:26.662+0000 7f407463b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:07:26.882 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:26.724+0000 7f407463b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:07:26.883 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:26.879+0000 7f407463b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:07:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-mon[34145]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-mon[34145]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:27.281 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:26 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:26.948+0000 7f407463b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:07:27.281 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:27.115+0000 7f407463b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:07:27.281 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:27.278+0000 7f407463b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:07:27.391 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:27.170+0000 7efc27271040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:07:27.392 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:27.287+0000 7efc27271040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:07:27.392 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-mon[35051]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:27.392 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-mon[35051]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:27.392 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.392 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.392 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.392 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.392 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:27.393 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:27 smithi094 ceph-mon[30972]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:27.393 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:27 smithi094 ceph-mon[30972]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:27.393 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:27 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.393 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:27 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.393 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:27 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.393 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:27 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:27.393 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:27 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:27.656 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:27.433+0000 7efc27271040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:07:27.656 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:27.498+0000 7efc27271040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:07:27.656 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:27.654+0000 7efc27271040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:07:27.854 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:27.603+0000 7f407463b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:07:27.854 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:27.664+0000 7f407463b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:07:27.953 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:27.723+0000 7efc27271040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:07:27.953 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:27.785+0000 7efc27271040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:07:27.953 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:27.951+0000 7efc27271040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:07:28.227 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:28.019+0000 7efc27271040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:07:28.227 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:28.085+0000 7efc27271040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:07:28.227 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:28.167+0000 7efc27271040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:07:28.354 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:27 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:27.860+0000 7f407463b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:07:28.534 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:28.225+0000 7efc27271040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:07:28.535 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:28.285+0000 7efc27271040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:07:29.104 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:28.743+0000 7f407463b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:07:29.104 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:28.809+0000 7f407463b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:07:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: Reconfiguring daemon mgr.x on smithi094 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:29.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2775678384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3"}]: dispatch 2024-09-19T04:07:29.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2775678384' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3"}]': finished 2024-09-19T04:07:29.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:07:29.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:28 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:29.284 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:07:29 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:07:29.069+0000 7efc27271040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:07:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:07:29.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:07:29.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:07:29.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:29.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: Reconfiguring daemon mgr.x on smithi094 2024-09-19T04:07:29.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:29.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2775678384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3"}]: dispatch 2024-09-19T04:07:29.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2775678384' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3"}]': finished 2024-09-19T04:07:29.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:07:29.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:28 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:07:29.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:07:29.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:07:29.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:29.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: Reconfiguring daemon mgr.x on smithi094 2024-09-19T04:07:29.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:29.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2775678384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3"}]: dispatch 2024-09-19T04:07:29.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2775678384' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3"}]': finished 2024-09-19T04:07:29.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:07:29.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:28 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:29.771 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:29.515+0000 7f407463b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:07:29.772 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:29.629+0000 7f407463b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:07:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:29.894+0000 7efdf5ffb640 1 -- 172.21.15.94:0/2138746423 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7efde803a8e0 con 0x7efdf8100b10 2024-09-19T04:07:30.049 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1681463458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:07:30.049 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-mon[34145]: Standby manager daemon z started 2024-09-19T04:07:30.049 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:07:30.049 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:07:30.049 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:07:30.050 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:07:30.050 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:29.769+0000 7f407463b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:07:30.050 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:29.830+0000 7f407463b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:07:30.050 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:29 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:29.980+0000 7f407463b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:07:30.050 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:30.046+0000 7f407463b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:07:30.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:29 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1681463458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:07:30.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:29 smithi112 ceph-mon[35051]: Standby manager daemon z started 2024-09-19T04:07:30.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:29 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:07:30.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:29 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:07:30.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:29 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:07:30.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:29 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:07:30.328 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:30.106+0000 7f407463b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:07:30.328 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:30.261+0000 7f407463b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:07:30.328 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:30.325+0000 7f407463b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:07:30.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:29 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1681463458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:07:30.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:29 smithi094 ceph-mon[30972]: Standby manager daemon z started 2024-09-19T04:07:30.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:29 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:07:30.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:29 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:07:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:29 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:07:30.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:29 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.112:0/2851761926' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:07:30.590 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:30.391+0000 7f407463b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:07:30.590 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:30.471+0000 7f407463b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:07:30.590 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:30.530+0000 7f407463b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:07:30.590 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:30.587+0000 7f407463b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:07:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:30 smithi112 ceph-mon[35051]: mgrmap e14: x(active, since 59s), standbys: z 2024-09-19T04:07:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:07:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:30 smithi112 ceph-mon[35051]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:30 smithi094 ceph-mon[30972]: mgrmap e14: x(active, since 59s), standbys: z 2024-09-19T04:07:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:07:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:30 smithi094 ceph-mon[30972]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:31.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:31.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-mon[34145]: mgrmap e14: x(active, since 59s), standbys: z 2024-09-19T04:07:31.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:07:31.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-mon[34145]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:31.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:31.854 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:07:31 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:07:31.367+0000 7f407463b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:07:31.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:31.905+0000 7efdf5ffb640 1 -- 172.21.15.94:0/2138746423 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7efde802f300 con 0x7efdf8100b10 2024-09-19T04:07:32.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:31 smithi112 ceph-mon[35051]: Standby manager daemon y started 2024-09-19T04:07:32.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:31 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:07:32.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:31 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:07:32.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:31 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:07:32.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:31 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:07:32.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:31 smithi094 ceph-mon[30972]: Standby manager daemon y started 2024-09-19T04:07:32.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:31 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:07:32.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:31 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:07:32.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:31 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:07:32.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:31 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:07:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:31 smithi104 ceph-mon[34145]: Standby manager daemon y started 2024-09-19T04:07:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:31 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:07:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:31 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:07:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:31 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:07:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:31 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.104:0/1425495290' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:07:33.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:32 smithi094 ceph-mon[30972]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-09-19T04:07:33.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:32 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:07:33.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:32 smithi094 ceph-mon[30972]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:33.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:32 smithi112 ceph-mon[35051]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-09-19T04:07:33.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:32 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:07:33.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:32 smithi112 ceph-mon[35051]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:33.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:32 smithi104 ceph-mon[34145]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-09-19T04:07:33.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:32 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:07:33.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:32 smithi104 ceph-mon[34145]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:35.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:35 smithi094 ceph-mon[30972]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:35.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:35 smithi112 ceph-mon[35051]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:35 smithi104 ceph-mon[34145]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:36.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:36.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:36 smithi094 ceph-mon[30972]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:36.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:36.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:36.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:36.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:36.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:36.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:36.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:36 smithi112 ceph-mon[35051]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:36.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:36.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:36.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:36.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:36.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:36.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:36.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:36 smithi104 ceph-mon[34145]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:38.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:07:38.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:38.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:07:38.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:07:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:39.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:38 smithi112 ceph-mon[35051]: Deploying daemon osd.0 on smithi094 2024-09-19T04:07:39.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:38 smithi112 ceph-mon[35051]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:39.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:38 smithi094 ceph-mon[30972]: Deploying daemon osd.0 on smithi094 2024-09-19T04:07:39.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:38 smithi094 ceph-mon[30972]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:39.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:38 smithi104 ceph-mon[34145]: Deploying daemon osd.0 on smithi094 2024-09-19T04:07:39.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:38 smithi104 ceph-mon[34145]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:41.367 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:41 smithi094 ceph-mon[30972]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:41.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:41 smithi112 ceph-mon[35051]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:41 smithi104 ceph-mon[34145]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:43.232 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:43 smithi094 ceph-mon[30972]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:43.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:43 smithi112 ceph-mon[35051]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:43 smithi104 ceph-mon[34145]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:44.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:44.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:44.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:44.560 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:44.560 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:44.560 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:45.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:45 smithi112 ceph-mon[35051]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:45 smithi094 ceph-mon[30972]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:45 smithi104 ceph-mon[34145]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:47 smithi112 ceph-mon[35051]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:47 smithi094 ceph-mon[30972]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:47 smithi104 ceph-mon[34145]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.398+0000 7efdf5ffb640 1 -- 172.21.15.94:0/2138746423 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efdf8062630 con 0x7efde40394d0 2024-09-19T04:07:48.399 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 0 on host 'smithi094' 2024-09-19T04:07:48.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7efde40394d0 msgr2=0x7efde403b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:48.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7efde40394d0 0x7efde403b990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efdf806b870 tx=0x7efde000b390 comp rx=0 tx=0).stop 2024-09-19T04:07:48.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 msgr2=0x7efdf806b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:48.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 0x7efdf806b2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efde80314e0 tx=0x7efde80042e0 comp rx=0 tx=0).stop 2024-09-19T04:07:48.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 shutdown_connections 2024-09-19T04:07:48.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7efde40394d0 0x7efde403b990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:48.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdf81026d0 0x7efdf819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:48.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf8101d10 0x7efdf819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:48.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 --2- 172.21.15.94:0/2138746423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdf8100b10 0x7efdf806b2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:48.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.400+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 >> 172.21.15.94:0/2138746423 conn(0x7efdf80fc2a0 msgr2=0x7efdf8105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:48.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.401+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 shutdown_connections 2024-09-19T04:07:48.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:07:48.401+0000 7efdfe515640 1 -- 172.21.15.94:0/2138746423 wait complete. 2024-09-19T04:07:49.099 DEBUG:teuthology.orchestra.run.smithi094:osd.0> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T04:07:49.102 INFO:tasks.cephadm:Deploying osd.1 on smithi104 with /dev/vg_nvme/lv_4... 2024-09-19T04:07:49.102 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:07:49.339 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:07:49.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:49 smithi094 ceph-mon[30972]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:49.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:49.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:49.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:49.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:49 smithi104 ceph-mon[34145]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:49.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:49.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:49.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:49 smithi112 ceph-mon[35051]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:50.513 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:07:50 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0[45247]: 2024-09-19T04:07:50.388+0000 7f86b58a1640 -1 osd.0 0 log_to_monitors true 2024-09-19T04:07:51.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:51 smithi104 ceph-mon[34145]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:51.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:51 smithi104 ceph-mon[34145]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:07:51.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:51 smithi112 ceph-mon[35051]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:51.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:51 smithi112 ceph-mon[35051]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:07:51.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:51 smithi094 ceph-mon[30972]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:51.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:51 smithi094 ceph-mon[30972]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:07:51.772 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:07:51.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-09-19T04:07:52.050 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:07:52.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:07:52.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:07:52.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:07:52.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:52.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.441 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.441 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:07:52.441 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.441 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:52.441 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:52.441 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:07:52.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:52 smithi104 ceph-mon[34145]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:07:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:07:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:07:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:07:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:07:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:52.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:52.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:07:52.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:52 smithi112 ceph-mon[35051]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 -- 172.21.15.104:0/899154223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 msgr2=0x7fcffc107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 --2- 172.21.15.104:0/899154223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 0x7fcffc107400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009900 tx=0x7fcff802a410 comp rx=0 tx=0).stop 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 -- 172.21.15.104:0/899154223 shutdown_connections 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 --2- 172.21.15.104:0/899154223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcffc107940 0x7fcffc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 --2- 172.21.15.104:0/899154223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 0x7fcffc107400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 --2- 172.21.15.104:0/899154223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcffc1026e0 0x7fcffc104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 -- 172.21.15.104:0/899154223 >> 172.21.15.104:0/899154223 conn(0x7fcffc0fc2a0 msgr2=0x7fcffc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 -- 172.21.15.104:0/899154223 shutdown_connections 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 -- 172.21.15.104:0/899154223 wait complete. 2024-09-19T04:07:52.539 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 Processor -- start 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 -- start start 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcffc1026e0 0x7fcffc197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.522+0000 7fd003494640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 0x7fcffc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd003494640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcffc107940 0x7fcffc19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd003494640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10d7d0 con 0x7fcffc1026e0 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd003494640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10d650 con 0x7fcffc105010 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd003494640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10d950 con 0x7fcffc107940 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd000a2f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 0x7fcffc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd000a2f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 0x7fcffc198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34340/0 (socket says 172.21.15.104:34340) 2024-09-19T04:07:52.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd000a2f640 1 -- 172.21.15.104:0/1706101133 learned_addr learned my addr 172.21.15.104:0/1706101133 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd001230640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcffc1026e0 0x7fcffc197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd000a2f640 1 -- 172.21.15.104:0/1706101133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcffc107940 msgr2=0x7fcffc19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd001a31640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcffc107940 0x7fcffc19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd000a2f640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcffc107940 0x7fcffc19f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd000a2f640 1 -- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcffc1026e0 msgr2=0x7fcffc197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd000a2f640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcffc1026e0 0x7fcffc197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd000a2f640 1 -- 172.21.15.104:0/1706101133 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc19fb40 con 0x7fcffc105010 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd000a2f640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 0x7fcffc198450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcff80314e0 tx=0x7fcff8003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fcfee7fc640 1 -- 172.21.15.104:0/1706101133 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff803f070 con 0x7fcffc105010 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fcfee7fc640 1 -- 172.21.15.104:0/1706101133 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcff8031e40 con 0x7fcffc105010 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc19fdd0 con 0x7fcffc105010 2024-09-19T04:07:52.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fcfee7fc640 1 -- 172.21.15.104:0/1706101133 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8035030 con 0x7fcffc105010 2024-09-19T04:07:52.542 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.523+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc1a02f0 con 0x7fcffc105010 2024-09-19T04:07:52.542 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.524+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc0fdd90 con 0x7fcffc105010 2024-09-19T04:07:52.542 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.525+0000 7fcfee7fc640 1 -- 172.21.15.104:0/1706101133 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fcff8042070 con 0x7fcffc105010 2024-09-19T04:07:52.542 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.526+0000 7fcfee7fc640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcfe80a3260 0x7fcfe80a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:52.542 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.526+0000 7fcfee7fc640 1 -- 172.21.15.104:0/1706101133 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fcff80eeb00 con 0x7fcffc105010 2024-09-19T04:07:52.542 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.526+0000 7fd001230640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcfe80a3260 0x7fcfe80a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:52.542 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.527+0000 7fd001230640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcfe80a3260 0x7fcfe80a5720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009990 tx=0x7fcff0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:52.542 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.529+0000 7fcfee7fc640 1 -- 172.21.15.104:0/1706101133 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcff80bd3b0 con 0x7fcffc105010 2024-09-19T04:07:52.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:07:52.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:07:52.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:07:52.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:52.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:07:52.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:52.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:52.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:07:52.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:07:52.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-mon[30972]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:07:52.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:07:52.703+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcffc062630 con 0x7fcfe80a3260 2024-09-19T04:07:53.091 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0[45247]: 2024-09-19T04:07:52.823+0000 7f86b1001640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T04:07:53.091 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:07:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0[45247]: 2024-09-19T04:07:52.826+0000 7f86a9633640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:07:53.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:53 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi094 2024-09-19T04:07:53.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:53 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi094 to 17088M 2024-09-19T04:07:53.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:53 smithi104 ceph-mon[34145]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:53.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:53.401 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:53.401 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:07:53.401 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:07:53.401 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:53.401 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:53 smithi104 ceph-mon[34145]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' 2024-09-19T04:07:53.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:53 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi094 2024-09-19T04:07:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:53 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi094 to 17088M 2024-09-19T04:07:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:53 smithi112 ceph-mon[35051]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:07:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:07:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:53 smithi112 ceph-mon[35051]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' 2024-09-19T04:07:53.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:53 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi094 2024-09-19T04:07:53.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:53 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi094 to 17088M 2024-09-19T04:07:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:53 smithi094 ceph-mon[30972]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:07:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:07:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:53 smithi094 ceph-mon[30972]: from='osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567]' entity='osd.0' 2024-09-19T04:07:54.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:54 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:07:54.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:54 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:07:54.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:54 smithi104 ceph-mon[34145]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:54.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:54 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:54.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:54 smithi104 ceph-mon[34145]: osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] boot 2024-09-19T04:07:54.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:54 smithi104 ceph-mon[34145]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:07:54.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:54 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:54.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:54 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:07:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:54 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:07:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:54 smithi112 ceph-mon[35051]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:54 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:54 smithi112 ceph-mon[35051]: osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] boot 2024-09-19T04:07:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:54 smithi112 ceph-mon[35051]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:07:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:54 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:54.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:54 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:07:54.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:54 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:07:54.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:54 smithi094 ceph-mon[30972]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:54.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:54.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:54 smithi094 ceph-mon[30972]: osd.0 [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] boot 2024-09-19T04:07:54.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:54 smithi094 ceph-mon[30972]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:07:54.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:07:55.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:55 smithi112 ceph-mon[35051]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:55.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:55 smithi112 ceph-mon[35051]: from='client.? 172.21.15.104:0/4079164699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18"}]: dispatch 2024-09-19T04:07:55.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:55 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18"}]: dispatch 2024-09-19T04:07:55.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:55 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18"}]': finished 2024-09-19T04:07:55.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:55 smithi112 ceph-mon[35051]: osdmap e9: 2 total, 1 up, 2 in 2024-09-19T04:07:55.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:55 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:07:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:55 smithi094 ceph-mon[30972]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:55.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:55 smithi094 ceph-mon[30972]: from='client.? 172.21.15.104:0/4079164699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18"}]: dispatch 2024-09-19T04:07:55.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:55 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18"}]: dispatch 2024-09-19T04:07:55.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:55 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18"}]': finished 2024-09-19T04:07:55.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:55 smithi094 ceph-mon[30972]: osdmap e9: 2 total, 1 up, 2 in 2024-09-19T04:07:55.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:55 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:07:55.598 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:55 smithi104 ceph-mon[34145]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:07:55.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:55 smithi104 ceph-mon[34145]: from='client.? 172.21.15.104:0/4079164699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18"}]: dispatch 2024-09-19T04:07:55.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:55 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18"}]: dispatch 2024-09-19T04:07:55.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:55 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18"}]': finished 2024-09-19T04:07:55.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:55 smithi104 ceph-mon[34145]: osdmap e9: 2 total, 1 up, 2 in 2024-09-19T04:07:55.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:55 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:07:56.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:56 smithi112 ceph-mon[35051]: from='client.? 172.21.15.104:0/3496086209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:07:56.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:56 smithi094 ceph-mon[30972]: from='client.? 172.21.15.104:0/3496086209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:07:56.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:56 smithi104 ceph-mon[34145]: from='client.? 172.21.15.104:0/3496086209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:07:57.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:57 smithi112 ceph-mon[35051]: pgmap v42: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:07:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:57 smithi094 ceph-mon[30972]: pgmap v42: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:07:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:57 smithi104 ceph-mon[34145]: pgmap v42: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:07:59.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:07:59 smithi112 ceph-mon[35051]: pgmap v43: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:07:59.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:07:59 smithi094 ceph-mon[30972]: pgmap v43: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:07:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:07:59 smithi104 ceph-mon[34145]: pgmap v43: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:01.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:01 smithi104 ceph-mon[34145]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:01.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:01 smithi112 ceph-mon[35051]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:01.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:01 smithi094 ceph-mon[30972]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:03.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:03 smithi104 ceph-mon[34145]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:03.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:03 smithi112 ceph-mon[35051]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:03.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:03 smithi094 ceph-mon[30972]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:04.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:04 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:08:04.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:04 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:04.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:04 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:08:04.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:04 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:04 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:08:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:04 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:05.480 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:05 smithi104 ceph-mon[34145]: Deploying daemon osd.1 on smithi104 2024-09-19T04:08:05.480 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:05 smithi104 ceph-mon[34145]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:05.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:05 smithi112 ceph-mon[35051]: Deploying daemon osd.1 on smithi104 2024-09-19T04:08:05.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:05 smithi112 ceph-mon[35051]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:05.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:05 smithi094 ceph-mon[30972]: Deploying daemon osd.1 on smithi104 2024-09-19T04:08:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:05 smithi094 ceph-mon[30972]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:07.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:07 smithi112 ceph-mon[35051]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:07.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:07 smithi094 ceph-mon[30972]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:07 smithi104 ceph-mon[34145]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:09.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:09 smithi104 ceph-mon[34145]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:09.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:09 smithi112 ceph-mon[35051]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:09.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:09 smithi094 ceph-mon[30972]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:10.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:10.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:10.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:10.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:10.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:10.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:11.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:11 smithi112 ceph-mon[35051]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:11.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:11 smithi104 ceph-mon[34145]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:11.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:11 smithi094 ceph-mon[30972]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:13.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:13 smithi094 ceph-mon[30972]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:13 smithi104 ceph-mon[34145]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:13 smithi112 ceph-mon[35051]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:14.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.038+0000 7fcfee7fc640 1 -- 172.21.15.104:0/1706101133 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcffc062630 con 0x7fcfe80a3260 2024-09-19T04:08:14.040 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 1 on host 'smithi104' 2024-09-19T04:08:14.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcfe80a3260 msgr2=0x7fcfe80a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:14.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcfe80a3260 0x7fcfe80a5720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009990 tx=0x7fcff0009110 comp rx=0 tx=0).stop 2024-09-19T04:08:14.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 msgr2=0x7fcffc198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:14.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 0x7fcffc198450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcff80314e0 tx=0x7fcff8003c80 comp rx=0 tx=0).stop 2024-09-19T04:08:14.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 shutdown_connections 2024-09-19T04:08:14.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcfe80a3260 0x7fcfe80a5720 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:14.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcffc107940 0x7fcffc19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:14.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcffc105010 0x7fcffc198450 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:14.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 --2- 172.21.15.104:0/1706101133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcffc1026e0 0x7fcffc197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:14.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 >> 172.21.15.104:0/1706101133 conn(0x7fcffc0fc2a0 msgr2=0x7fcffc102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:14.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 shutdown_connections 2024-09-19T04:08:14.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:08:14.041+0000 7fd003494640 1 -- 172.21.15.104:0/1706101133 wait complete. 2024-09-19T04:08:14.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:14 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:14.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:14 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:14.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:14 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:14.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:14 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:14.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:14 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:14.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:14 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:14.689 DEBUG:teuthology.orchestra.run.smithi104:osd.1> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T04:08:14.692 INFO:tasks.cephadm:Deploying osd.2 on smithi112 with /dev/vg_nvme/lv_4... 2024-09-19T04:08:14.692 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:08:14.720 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:14 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:14.720 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:14 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:14.720 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:14 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:14.941 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:08:15.358 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:15 smithi112 ceph-mon[35051]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:15.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:15 smithi094 ceph-mon[30972]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:15 smithi104 ceph-mon[34145]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:16.604 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:16 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1[38626]: 2024-09-19T04:08:16.308+0000 7fb5ba03a640 -1 osd.1 0 log_to_monitors true 2024-09-19T04:08:17.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:08:17.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-09-19T04:08:17.425 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:17 smithi104 ceph-mon[34145]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:17.425 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:17 smithi104 ceph-mon[34145]: from='osd.1 [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:08:17.425 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:17 smithi104 ceph-mon[34145]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:08:17.463 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:17 smithi112 ceph-mon[35051]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:17.463 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:17 smithi112 ceph-mon[35051]: from='osd.1 [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:08:17.463 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:17 smithi112 ceph-mon[35051]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:08:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:08:17.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:17 smithi094 ceph-mon[30972]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:17.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:17 smithi094 ceph-mon[30972]: from='osd.1 [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:08:17.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:17 smithi094 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.922+0000 7f9189d3c640 1 -- 172.21.15.112:0/3366233063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 msgr2=0x7f9184102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.922+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3366233063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 0x7f9184102170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9178009900 tx=0x7f917802a410 comp rx=0 tx=0).stop 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.922+0000 7f9189d3c640 1 -- 172.21.15.112:0/3366233063 shutdown_connections 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.922+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3366233063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91841026b0 0x7f9184108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.922+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3366233063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 0x7f9184102170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.922+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3366233063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9184100af0 0x7f9184100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.922+0000 7f9189d3c640 1 -- 172.21.15.112:0/3366233063 >> 172.21.15.112:0/3366233063 conn(0x7f91840fc280 msgr2=0x7f91840fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.922+0000 7f9189d3c640 1 -- 172.21.15.112:0/3366233063 shutdown_connections 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.922+0000 7f9189d3c640 1 -- 172.21.15.112:0/3366233063 wait complete. 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.923+0000 7f9189d3c640 1 Processor -- start 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.923+0000 7f9189d3c640 1 -- start start 2024-09-19T04:08:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.923+0000 7f9189d3c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184100af0 0x7f9184196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9189d3c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 0x7f9184197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9189d3c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91841026b0 0x7f918419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9189d3c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f918410c620 con 0x7f9184100af0 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9189d3c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f918410c4a0 con 0x7f91841026b0 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9189d3c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f918410c7a0 con 0x7f9184101cf0 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9182ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 0x7f9184197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9182ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 0x7f9184197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35958/0 (socket says 172.21.15.112:35958) 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9182ffd640 1 -- 172.21.15.112:0/3903285085 learned_addr learned my addr 172.21.15.112:0/3903285085 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f91837fe640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184100af0 0x7f9184196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9183fff640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91841026b0 0x7f918419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9182ffd640 1 -- 172.21.15.112:0/3903285085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91841026b0 msgr2=0x7f918419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9182ffd640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91841026b0 0x7f918419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9182ffd640 1 -- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184100af0 msgr2=0x7f9184196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9182ffd640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184100af0 0x7f9184196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9182ffd640 1 -- 172.21.15.112:0/3903285085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918419e960 con 0x7f9184101cf0 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f91837fe640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184100af0 0x7f9184196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9182ffd640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 0x7f9184197270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9178005940 tx=0x7f91780042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9180ff9640 1 -- 172.21.15.112:0/3903285085 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917803f030 con 0x7f9184101cf0 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918419ebf0 con 0x7f9184101cf0 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9180ff9640 1 -- 172.21.15.112:0/3903285085 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9178003cb0 con 0x7f9184101cf0 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.924+0000 7f9180ff9640 1 -- 172.21.15.112:0/3903285085 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178035070 con 0x7f9184101cf0 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.925+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918406a790 con 0x7f9184101cf0 2024-09-19T04:08:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.925+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918406b1b0 con 0x7f9184101cf0 2024-09-19T04:08:17.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.927+0000 7f9180ff9640 1 -- 172.21.15.112:0/3903285085 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f9178042030 con 0x7f9184101cf0 2024-09-19T04:08:17.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.927+0000 7f9180ff9640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f916c0a53f0 0x7f916c0a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:17.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.927+0000 7f9180ff9640 1 -- 172.21.15.112:0/3903285085 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1903+0+0 (secure 0 0 0) 0x7f91780eeb00 con 0x7f9184101cf0 2024-09-19T04:08:17.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.928+0000 7f91837fe640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f916c0a53f0 0x7f916c0a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:17.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.928+0000 7f91837fe640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f916c0a53f0 0x7f916c0a78b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9184101b50 tx=0x7f9174005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:17.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:17.935+0000 7f9180ff9640 1 -- 172.21.15.112:0/3903285085 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f91780bd210 con 0x7f9184101cf0 2024-09-19T04:08:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:18.114+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9184107b40 con 0x7f916c0a53f0 2024-09-19T04:08:18.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:08:18.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:08:18.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:18.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='osd.1 [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:08:18.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:08:18.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi104 2024-09-19T04:08:18.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:08:18.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:18.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='osd.1 [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi104 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:18.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:18.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:18.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:08:18.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:08:18.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:08:18.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:08:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:08:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='osd.1 [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:08:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:08:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi104 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:08:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:19.354 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:19 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1[38626]: 2024-09-19T04:08:19.077+0000 7fb5b579a640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T04:08:19.354 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:19 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1[38626]: 2024-09-19T04:08:19.081+0000 7fb5addcc640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:08:19.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:19 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:08:19.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:19 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:08:19.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:19 smithi104 ceph-mon[34145]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:08:19.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:19 smithi104 ceph-mon[34145]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:08:19.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:19 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:19.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:19 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:19.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:19 smithi104 ceph-mon[34145]: from='osd.1 ' entity='osd.1' 2024-09-19T04:08:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:19 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:08:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:19 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:08:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:19 smithi112 ceph-mon[35051]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:08:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:19 smithi112 ceph-mon[35051]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:08:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:19 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:19 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:19 smithi112 ceph-mon[35051]: from='osd.1 ' entity='osd.1' 2024-09-19T04:08:19.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:19 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:08:19.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:19 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:08:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:19 smithi094 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:08:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:19 smithi094 ceph-mon[30972]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:08:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:19 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:19 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:19 smithi094 ceph-mon[30972]: from='osd.1 ' entity='osd.1' 2024-09-19T04:08:20.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:20.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:20 smithi104 ceph-mon[34145]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:20.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:20 smithi104 ceph-mon[34145]: osd.1 [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] boot 2024-09-19T04:08:20.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:20 smithi104 ceph-mon[34145]: osdmap e12: 2 total, 2 up, 2 in 2024-09-19T04:08:20.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:20.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:20.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:20 smithi112 ceph-mon[35051]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:20.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:20 smithi112 ceph-mon[35051]: osd.1 [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] boot 2024-09-19T04:08:20.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:20 smithi112 ceph-mon[35051]: osdmap e12: 2 total, 2 up, 2 in 2024-09-19T04:08:20.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:20.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:20.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:20 smithi094 ceph-mon[30972]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:08:20.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:20 smithi094 ceph-mon[30972]: osd.1 [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] boot 2024-09-19T04:08:20.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:20 smithi094 ceph-mon[30972]: osdmap e12: 2 total, 2 up, 2 in 2024-09-19T04:08:20.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:08:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:21 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/3143470164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ebda5b-3f73-46b3-b026-a9d41639aa61"}]: dispatch 2024-09-19T04:08:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:21 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ebda5b-3f73-46b3-b026-a9d41639aa61"}]: dispatch 2024-09-19T04:08:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:21 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92ebda5b-3f73-46b3-b026-a9d41639aa61"}]': finished 2024-09-19T04:08:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:21 smithi104 ceph-mon[34145]: osdmap e13: 3 total, 2 up, 3 in 2024-09-19T04:08:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:21 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/531263962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:08:21.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:21 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/3143470164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ebda5b-3f73-46b3-b026-a9d41639aa61"}]: dispatch 2024-09-19T04:08:21.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:21 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ebda5b-3f73-46b3-b026-a9d41639aa61"}]: dispatch 2024-09-19T04:08:21.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:21 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92ebda5b-3f73-46b3-b026-a9d41639aa61"}]': finished 2024-09-19T04:08:21.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:21 smithi094 ceph-mon[30972]: osdmap e13: 3 total, 2 up, 3 in 2024-09-19T04:08:21.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:21.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:21 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/531263962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:08:21.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:21 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/3143470164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ebda5b-3f73-46b3-b026-a9d41639aa61"}]: dispatch 2024-09-19T04:08:21.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:21 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ebda5b-3f73-46b3-b026-a9d41639aa61"}]: dispatch 2024-09-19T04:08:21.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:21 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92ebda5b-3f73-46b3-b026-a9d41639aa61"}]': finished 2024-09-19T04:08:21.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:21 smithi112 ceph-mon[35051]: osdmap e13: 3 total, 2 up, 3 in 2024-09-19T04:08:21.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:21.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:21 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/531263962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:08:22.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:22 smithi112 ceph-mon[35051]: pgmap v59: 0 pgs: ; 0 B data, 465 MiB used, 178 GiB / 179 GiB avail 2024-09-19T04:08:22.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:22 smithi094 ceph-mon[30972]: pgmap v59: 0 pgs: ; 0 B data, 465 MiB used, 178 GiB / 179 GiB avail 2024-09-19T04:08:22.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:22 smithi104 ceph-mon[34145]: pgmap v59: 0 pgs: ; 0 B data, 465 MiB used, 178 GiB / 179 GiB avail 2024-09-19T04:08:25.385 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:25 smithi112 ceph-mon[35051]: pgmap v60: 0 pgs: ; 0 B data, 465 MiB used, 178 GiB / 179 GiB avail 2024-09-19T04:08:25.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:25 smithi094 ceph-mon[30972]: pgmap v60: 0 pgs: ; 0 B data, 465 MiB used, 178 GiB / 179 GiB avail 2024-09-19T04:08:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:25 smithi104 ceph-mon[34145]: pgmap v60: 0 pgs: ; 0 B data, 465 MiB used, 178 GiB / 179 GiB avail 2024-09-19T04:08:27.368 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:27 smithi112 ceph-mon[35051]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:27.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:27 smithi094 ceph-mon[30972]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:27 smithi104 ceph-mon[34145]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:29.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:29 smithi112 ceph-mon[35051]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:29.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:08:29.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:29.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:29 smithi094 ceph-mon[30972]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:29.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:08:29.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:29 smithi104 ceph-mon[34145]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:08:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:30.172 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:30 smithi112 ceph-mon[35051]: Deploying daemon osd.2 on smithi112 2024-09-19T04:08:30.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:30 smithi094 ceph-mon[30972]: Deploying daemon osd.2 on smithi112 2024-09-19T04:08:30.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:30 smithi104 ceph-mon[34145]: Deploying daemon osd.2 on smithi112 2024-09-19T04:08:31.490 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:31 smithi112 ceph-mon[35051]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:31.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:31 smithi094 ceph-mon[30972]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:31.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:31 smithi104 ceph-mon[34145]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:33.496 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:33 smithi112 ceph-mon[35051]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:33.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:33 smithi094 ceph-mon[30972]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:33.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:33 smithi104 ceph-mon[34145]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:35.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:35 smithi112 ceph-mon[35051]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:35.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:35 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:35.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:35 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:35.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:35 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:35.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:35 smithi094 ceph-mon[30972]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:35.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:35 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:35.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:35 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:35.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:35 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:35 smithi104 ceph-mon[34145]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:35 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:35 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:35 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:37.353 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:37 smithi112 ceph-mon[35051]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:37.354 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:37.354 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:37.354 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:37.354 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:37.354 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:37.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:37 smithi094 ceph-mon[30972]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:37.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:37 smithi104 ceph-mon[34145]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:38.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.963+0000 7f9180ff9640 1 -- 172.21.15.112:0/3903285085 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9184107b40 con 0x7f916c0a53f0 2024-09-19T04:08:38.965 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 2 on host 'smithi112' 2024-09-19T04:08:38.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f916c0a53f0 msgr2=0x7f916c0a78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:38.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f916c0a53f0 0x7f916c0a78b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9184101b50 tx=0x7f9174005f90 comp rx=0 tx=0).stop 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 msgr2=0x7f9184197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 0x7f9184197270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9178005940 tx=0x7f91780042f0 comp rx=0 tx=0).stop 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 shutdown_connections 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f916c0a53f0 0x7f916c0a78b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91841026b0 0x7f918419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9184101cf0 0x7f9184197270 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 --2- 172.21.15.112:0/3903285085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184100af0 0x7f9184196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 >> 172.21.15.112:0/3903285085 conn(0x7f91840fc280 msgr2=0x7f9184105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 shutdown_connections 2024-09-19T04:08:38.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:08:38.966+0000 7f9189d3c640 1 -- 172.21.15.112:0/3903285085 wait complete. 2024-09-19T04:08:39.221 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:39 smithi112 ceph-mon[35051]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:39.221 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:39 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:39.221 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:39 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:39.221 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:39 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:39.406 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:39 smithi104 ceph-mon[34145]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:39.406 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:39 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:39.406 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:39 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:39.407 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:39 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:39.489 DEBUG:teuthology.orchestra.run.smithi112:osd.2> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T04:08:39.492 INFO:tasks.cephadm:Deploying osd.3 on smithi094 with /dev/vg_nvme/lv_3... 2024-09-19T04:08:39.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:08:39.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:39 smithi094 ceph-mon[30972]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:39.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:39 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:39.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:39 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:39.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:39 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:39.734 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:08:41.264 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:41 smithi112 ceph-mon[35051]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:41.504 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:41 smithi094 ceph-mon[30972]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:41.535 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:41 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2[40044]: 2024-09-19T04:08:41.381+0000 7f020f039640 -1 osd.2 0 log_to_monitors true 2024-09-19T04:08:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:41 smithi104 ceph-mon[34145]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:42.234 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:42 smithi094 ceph-mon[30972]: from='osd.2 [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:08:42.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:42 smithi094 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:08:42.238 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:08:42.267 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-09-19T04:08:42.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:42 smithi112 ceph-mon[35051]: from='osd.2 [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:08:42.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:42 smithi112 ceph-mon[35051]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:08:42.557 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:08:42.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:42 smithi104 ceph-mon[34145]: from='osd.2 [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:08:42.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:42 smithi104 ceph-mon[34145]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:08:43.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.046+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/2679609981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 msgr2=0x7f3a68100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:43.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.046+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/2679609981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 0x7f3a68100ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009900 tx=0x7f3a5c02a410 comp rx=0 tx=0).stop 2024-09-19T04:08:43.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.046+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/2679609981 shutdown_connections 2024-09-19T04:08:43.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.046+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/2679609981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a68102690 0x7f3a68108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.046+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/2679609981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a68101cd0 0x7f3a68102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.046+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/2679609981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 0x7f3a68100ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.046+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/2679609981 >> 172.21.15.94:0/2679609981 conn(0x7f3a680fc280 msgr2=0x7f3a680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.046+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/2679609981 shutdown_connections 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.046+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/2679609981 wait complete. 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6f1c2640 1 Processor -- start 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6f1c2640 1 -- start start 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6f1c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 0x7f3a68197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6f1c2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a68101cd0 0x7f3a681983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6f1c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a68102690 0x7f3a6819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6f1c2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810c600 con 0x7f3a68100ad0 2024-09-19T04:08:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6f1c2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810c480 con 0x7f3a68101cd0 2024-09-19T04:08:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6f1c2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810c780 con 0x7f3a68102690 2024-09-19T04:08:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6cf5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 0x7f3a68197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6cf5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 0x7f3a68197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48270/0 (socket says 172.21.15.94:48270) 2024-09-19T04:08:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6cf5e640 1 -- 172.21.15.94:0/1538569027 learned_addr learned my addr 172.21.15.94:0/1538569027 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:08:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a67fff640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a68101cd0 0x7f3a681983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6d75f640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a68102690 0x7f3a6819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6cf5e640 1 -- 172.21.15.94:0/1538569027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a68102690 msgr2=0x7f3a6819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6cf5e640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a68102690 0x7f3a6819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6cf5e640 1 -- 172.21.15.94:0/1538569027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a68101cd0 msgr2=0x7f3a681983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6cf5e640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a68101cd0 0x7f3a681983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.047+0000 7f3a6cf5e640 1 -- 172.21.15.94:0/1538569027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6819fac0 con 0x7f3a68100ad0 2024-09-19T04:08:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.048+0000 7f3a6cf5e640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 0x7f3a68197e90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0314e0 tx=0x7f3a5c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.048+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1538569027 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c03f030 con 0x7f3a68100ad0 2024-09-19T04:08:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.048+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1538569027 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a5c003cb0 con 0x7f3a68100ad0 2024-09-19T04:08:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.048+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1538569027 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c035070 con 0x7f3a68100ad0 2024-09-19T04:08:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.048+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6819fd50 con 0x7f3a68100ad0 2024-09-19T04:08:43.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.048+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a681a0180 con 0x7f3a68100ad0 2024-09-19T04:08:43.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.049+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a681a03e0 con 0x7f3a68100ad0 2024-09-19T04:08:43.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.050+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1538569027 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f3a5c042030 con 0x7f3a68100ad0 2024-09-19T04:08:43.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.051+0000 7f3a65ffb640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3a580a31e0 0x7f3a580a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:43.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.051+0000 7f3a67fff640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3a580a31e0 0x7f3a580a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:43.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.051+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1538569027 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2445+0+0 (secure 0 0 0) 0x7f3a5c0ee620 con 0x7f3a68100ad0 2024-09-19T04:08:43.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.052+0000 7f3a67fff640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3a580a31e0 0x7f3a580a56a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a681993b0 tx=0x7f3a5400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:43.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.055+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1538569027 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a5c0bc4c0 con 0x7f3a68100ad0 2024-09-19T04:08:43.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:43.252+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3a6806a770 con 0x7f3a580a31e0 2024-09-19T04:08:43.535 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2[40044]: 2024-09-19T04:08:43.170+0000 7f020a799640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T04:08:43.535 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2[40044]: 2024-09-19T04:08:43.174+0000 7f0206dd3640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:08:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:08:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T04:08:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='osd.2 [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:08:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:08:43.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi112 2024-09-19T04:08:43.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:08:43.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi112 to 17088M 2024-09-19T04:08:43.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:43.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:43.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.561 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:43.561 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:08:43.561 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T04:08:43.561 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='osd.2 [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi112 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi112 to 17088M 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:43.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:08:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T04:08:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='osd.2 [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi112 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi112 to 17088M 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:43.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:44.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:44 smithi112 ceph-mon[35051]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:08:44.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:44 smithi112 ceph-mon[35051]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:08:44.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:44.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:44.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:44 smithi112 ceph-mon[35051]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:08:44.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:08:44.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:08:44.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:44 smithi094 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:08:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:44 smithi094 ceph-mon[30972]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:08:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:44 smithi094 ceph-mon[30972]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:08:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:08:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:08:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:44 smithi104 ceph-mon[34145]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:08:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:44 smithi104 ceph-mon[34145]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:08:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:44 smithi104 ceph-mon[34145]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:08:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:08:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:08:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:45.326 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:45 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:08:45.326 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:45 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:08:45.326 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:45 smithi094 ceph-mon[30972]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:45.326 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:45 smithi094 ceph-mon[30972]: osd.2 [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] boot 2024-09-19T04:08:45.326 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:45 smithi094 ceph-mon[30972]: osdmap e16: 3 total, 3 up, 3 in 2024-09-19T04:08:45.327 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:45 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:45.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:45 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:08:45.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:45 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:08:45.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:45 smithi112 ceph-mon[35051]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:45.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:45 smithi112 ceph-mon[35051]: osd.2 [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] boot 2024-09-19T04:08:45.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:45 smithi112 ceph-mon[35051]: osdmap e16: 3 total, 3 up, 3 in 2024-09-19T04:08:45.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:45 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:45 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:08:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:45 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:08:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:45 smithi104 ceph-mon[34145]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:08:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:45 smithi104 ceph-mon[34145]: osd.2 [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] boot 2024-09-19T04:08:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:45 smithi104 ceph-mon[34145]: osdmap e16: 3 total, 3 up, 3 in 2024-09-19T04:08:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:45 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:08:46.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:46 smithi094 ceph-mon[30972]: osdmap e17: 3 total, 3 up, 3 in 2024-09-19T04:08:46.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:46 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/446814139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcb93a96-92d9-4cf2-a2f0-f547d49835cf"}]: dispatch 2024-09-19T04:08:46.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:46 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/446814139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dcb93a96-92d9-4cf2-a2f0-f547d49835cf"}]': finished 2024-09-19T04:08:46.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:46 smithi094 ceph-mon[30972]: osdmap e18: 4 total, 3 up, 4 in 2024-09-19T04:08:46.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:46 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:46.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:46 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:08:46.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:46 smithi112 ceph-mon[35051]: osdmap e17: 3 total, 3 up, 3 in 2024-09-19T04:08:46.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:46 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/446814139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcb93a96-92d9-4cf2-a2f0-f547d49835cf"}]: dispatch 2024-09-19T04:08:46.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:46 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/446814139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dcb93a96-92d9-4cf2-a2f0-f547d49835cf"}]': finished 2024-09-19T04:08:46.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:46 smithi112 ceph-mon[35051]: osdmap e18: 4 total, 3 up, 4 in 2024-09-19T04:08:46.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:46 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:46.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:46 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:08:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:46 smithi104 ceph-mon[34145]: osdmap e17: 3 total, 3 up, 3 in 2024-09-19T04:08:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:46 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/446814139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcb93a96-92d9-4cf2-a2f0-f547d49835cf"}]: dispatch 2024-09-19T04:08:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:46 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/446814139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dcb93a96-92d9-4cf2-a2f0-f547d49835cf"}]': finished 2024-09-19T04:08:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:46 smithi104 ceph-mon[34145]: osdmap e18: 4 total, 3 up, 4 in 2024-09-19T04:08:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:46 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:46 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:08:47.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:47 smithi112 ceph-mon[35051]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:47 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/501392298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:08:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:08:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:47 smithi112 ceph-mon[35051]: osdmap e19: 4 total, 3 up, 4 in 2024-09-19T04:08:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:08:47.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:47 smithi094 ceph-mon[30972]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:47.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:47 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/501392298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:08:47.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:08:47.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:47 smithi094 ceph-mon[30972]: osdmap e19: 4 total, 3 up, 4 in 2024-09-19T04:08:47.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:47.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:08:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:47 smithi104 ceph-mon[34145]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:47 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/501392298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:08:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:08:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:47 smithi104 ceph-mon[34145]: osdmap e19: 4 total, 3 up, 4 in 2024-09-19T04:08:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:08:48.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:48 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:08:48.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:48 smithi094 ceph-mon[30972]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T04:08:48.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:48 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:48.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:48 smithi094 ceph-mon[30972]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:48.786 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:08:48 smithi094 sudo[50460]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:08:48.786 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:08:48 smithi094 sudo[50460]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:48.786 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:08:48 smithi094 sudo[50460]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:48.786 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:08:48 smithi094 sudo[50460]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:49.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:48 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:08:49.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:48 smithi112 ceph-mon[35051]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T04:08:49.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:48 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:49.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:48 smithi112 ceph-mon[35051]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:49.057 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:48 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:08:49.057 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:48 smithi104 ceph-mon[34145]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T04:08:49.057 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:48 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:49.057 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:48 smithi104 ceph-mon[34145]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:49.090 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:08:48 smithi094 sudo[50648]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:08:49.090 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:08:48 smithi094 sudo[50648]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:49.090 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:08:48 smithi094 sudo[50648]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:49.090 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:08:48 smithi094 sudo[50648]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:49.311 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:49 smithi104 sudo[42743]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:08:49.311 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:49 smithi104 sudo[42743]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:49.311 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:49 smithi104 sudo[42743]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:49.311 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:49 smithi104 sudo[42743]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:49.604 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:49 smithi104 sudo[42746]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:08:49.604 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:49 smithi104 sudo[42746]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:49.604 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:49 smithi104 sudo[42746]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:49.604 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:08:49 smithi104 sudo[42746]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:49.836 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:49 smithi112 sudo[43959]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:08:49.837 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:49 smithi112 sudo[43959]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:49.837 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:49 smithi112 sudo[43959]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:49.837 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:49 smithi112 sudo[43959]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:49.837 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:49 smithi112 ceph-mon[35051]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:08:49.837 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:49.837 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:49.837 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:50.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:49 smithi094 ceph-mon[30972]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:08:50.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:50.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:50.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:50.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:49 smithi104 ceph-mon[34145]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:08:50.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:08:50.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:50.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:50.284 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:49 smithi112 sudo[43962]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:08:50.285 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:49 smithi112 sudo[43962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:50.285 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:49 smithi112 sudo[43962]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:50.285 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:08:49 smithi112 sudo[43962]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:50.370 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:50 smithi094 sudo[51214]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:08:50.370 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:50 smithi094 sudo[51214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:50.370 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:50 smithi094 sudo[51214]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:50.370 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:50 smithi094 sudo[51214]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:50.790 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:50 smithi094 sudo[51296]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:08:50.790 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:50 smithi094 sudo[51296]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:50.790 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:50 smithi094 sudo[51296]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:50.790 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:50 smithi094 sudo[51296]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:50.864 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:50 smithi112 sudo[43965]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:08:50.864 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:50 smithi112 sudo[43965]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:50.864 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:50 smithi112 sudo[43965]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:50.864 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:50 smithi112 sudo[43965]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:50 smithi112 sudo[43968]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:08:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:50 smithi112 sudo[43968]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:50 smithi112 sudo[43968]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:50 smithi112 sudo[43968]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:51.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 sudo[42749]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:08:51.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 sudo[42749]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:51.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 sudo[42749]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:51.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 sudo[42749]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:51.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:08:51.387+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1538569027 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3a5c0b8b30 con 0x7f3a68100ad0 2024-09-19T04:08:51.661 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:51.661 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:08:51.661 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:51.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:08:51.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:08:51.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:08:51.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:08:51.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:08:51.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:08:51.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:08:51.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 sudo[42752]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:08:51.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 sudo[42752]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:08:51.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 sudo[42752]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:08:51.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 sudo[42752]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:08:51.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:51.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:08:51.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:51.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:08:51.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:08:51.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:08:51.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:08:51.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:08:51.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:08:51.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:08:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:08:52.497 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:52 smithi094 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:08:52.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:52 smithi094 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:08:52.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:52 smithi094 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:08:52.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:08:52.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:08:52.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:08:52.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:52 smithi094 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:08:52.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:52 smithi094 ceph-mon[30972]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:08:52.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:52 smithi094 ceph-mon[30972]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:52 smithi112 ceph-mon[35051]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:08:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:52 smithi112 ceph-mon[35051]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:08:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:52 smithi112 ceph-mon[35051]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:08:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:08:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:08:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:08:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:52 smithi112 ceph-mon[35051]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:08:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:52 smithi112 ceph-mon[35051]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:08:52.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:52 smithi112 ceph-mon[35051]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:52.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:52 smithi104 ceph-mon[34145]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:08:52.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:52 smithi104 ceph-mon[34145]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:08:52.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:52 smithi104 ceph-mon[34145]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:08:52.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:08:52.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:08:52.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:08:52.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:52 smithi104 ceph-mon[34145]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:08:52.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:52 smithi104 ceph-mon[34145]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:08:52.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:52 smithi104 ceph-mon[34145]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:54.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:08:54.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:08:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:54.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:08:54.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:55.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:54 smithi094 ceph-mon[30972]: Deploying daemon osd.3 on smithi094 2024-09-19T04:08:55.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:54 smithi094 ceph-mon[30972]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:55.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:54 smithi112 ceph-mon[35051]: Deploying daemon osd.3 on smithi094 2024-09-19T04:08:55.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:54 smithi112 ceph-mon[35051]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:55.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:54 smithi104 ceph-mon[34145]: Deploying daemon osd.3 on smithi094 2024-09-19T04:08:55.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:54 smithi104 ceph-mon[34145]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:08:57.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:57 smithi094 ceph-mon[30972]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T04:08:57.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:57 smithi112 ceph-mon[35051]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T04:08:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:57 smithi104 ceph-mon[34145]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T04:08:59.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:08:59 smithi094 ceph-mon[30972]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T04:08:59.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:08:59 smithi112 ceph-mon[35051]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T04:08:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:08:59 smithi104 ceph-mon[34145]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T04:09:01.251 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:01 smithi094 ceph-mon[30972]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T04:09:01.251 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:01.251 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:01.251 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:01.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:01 smithi112 ceph-mon[35051]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T04:09:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:01 smithi104 ceph-mon[34145]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T04:09:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:03.433 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:03 smithi094 ceph-mon[30972]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T04:09:03.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:03 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:03.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:03 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:03.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:03 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:03.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:03 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:03.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:03 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:03.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:03 smithi112 ceph-mon[35051]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T04:09:03.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:03 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:03.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:03 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:03.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:03 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:03.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:03 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:03.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:03 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:03 smithi104 ceph-mon[34145]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T04:09:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:03 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:03 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:03 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:03.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:03 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:03.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:03 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:04.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.126+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1538569027 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a6806a770 con 0x7f3a580a31e0 2024-09-19T04:09:04.128 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 3 on host 'smithi094' 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3a580a31e0 msgr2=0x7f3a580a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3a580a31e0 0x7f3a580a56a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a681993b0 tx=0x7f3a5400b390 comp rx=0 tx=0).stop 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 msgr2=0x7f3a68197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 0x7f3a68197e90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0314e0 tx=0x7f3a5c0042e0 comp rx=0 tx=0).stop 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 shutdown_connections 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3a580a31e0 0x7f3a580a56a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a68102690 0x7f3a6819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a68101cd0 0x7f3a681983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 --2- 172.21.15.94:0/1538569027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68100ad0 0x7f3a68197e90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 >> 172.21.15.94:0/1538569027 conn(0x7f3a680fc280 msgr2=0x7f3a6806a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 shutdown_connections 2024-09-19T04:09:04.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:04.128+0000 7f3a6f1c2640 1 -- 172.21.15.94:0/1538569027 wait complete. 2024-09-19T04:09:04.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:04 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:04.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:04 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:04.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:04 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:04.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:04 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:04.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:04 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:04.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:04 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:04 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:04 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:04 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:04.795 DEBUG:teuthology.orchestra.run.smithi094:osd.3> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T04:09:04.797 INFO:tasks.cephadm:Deploying osd.4 on smithi104 with /dev/vg_nvme/lv_3... 2024-09-19T04:09:04.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:09:05.048 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:09:05.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:05 smithi094 ceph-mon[30972]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:09:05.472 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:05 smithi104 ceph-mon[34145]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:09:05.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:05 smithi112 ceph-mon[35051]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:09:06.591 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:09:06 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3[52047]: 2024-09-19T04:09:06.466+0000 7f1c7fa62640 -1 osd.3 0 log_to_monitors true 2024-09-19T04:09:07.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:07 smithi104 ceph-mon[34145]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:09:07.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:07 smithi104 ceph-mon[34145]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:09:07.477 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:09:07.501 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:07 smithi094 ceph-mon[30972]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:09:07.501 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:07 smithi094 ceph-mon[30972]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:09:07.502 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-09-19T04:09:07.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:07 smithi112 ceph-mon[35051]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:09:07.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:07 smithi112 ceph-mon[35051]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:09:07.753 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:09:08.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.193+0000 7f0b294bd640 1 -- 172.21.15.104:0/1746200508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b241026c0 msgr2=0x7f0b24108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.193+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1746200508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b241026c0 0x7f0b24108f50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009ca0 tx=0x7f0b1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.193+0000 7f0b294bd640 1 -- 172.21.15.104:0/1746200508 shutdown_connections 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.193+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1746200508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b241026c0 0x7f0b24108f50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.193+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1746200508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b24101d00 0x7f0b24102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.193+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1746200508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b24100b00 0x7f0b24100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.193+0000 7f0b294bd640 1 -- 172.21.15.104:0/1746200508 >> 172.21.15.104:0/1746200508 conn(0x7f0b240fc270 msgr2=0x7f0b240fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.193+0000 7f0b294bd640 1 -- 172.21.15.104:0/1746200508 shutdown_connections 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.193+0000 7f0b294bd640 1 -- 172.21.15.104:0/1746200508 wait complete. 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b294bd640 1 Processor -- start 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b294bd640 1 -- start start 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b294bd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b24100b00 0x7f0b24104d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b294bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b24101d00 0x7f0b24103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b294bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b241026c0 0x7f0b241038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b294bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2410c610 con 0x7f0b241026c0 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b294bd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2410c490 con 0x7f0b24100b00 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b294bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2410c790 con 0x7f0b24101d00 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b22ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b24100b00 0x7f0b24104d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b22ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b24100b00 0x7f0b24104d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:54384/0 (socket says 172.21.15.104:54384) 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b22ffd640 1 -- 172.21.15.104:0/1998961028 learned_addr learned my addr 172.21.15.104:0/1998961028 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b237fe640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b241026c0 0x7f0b241038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b22ffd640 1 -- 172.21.15.104:0/1998961028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b24101d00 msgr2=0x7f0b24103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.194+0000 7f0b227fc640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b24101d00 0x7f0b24103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b22ffd640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b24101d00 0x7f0b24103360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b22ffd640 1 -- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b241026c0 msgr2=0x7f0b241038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b22ffd640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b241026c0 0x7f0b241038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b22ffd640 1 -- 172.21.15.104:0/1998961028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b24104160 con 0x7f0b24100b00 2024-09-19T04:09:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b237fe640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b241026c0 0x7f0b241038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:09:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b22ffd640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b24100b00 0x7f0b24104d10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800d770 tx=0x7f0b1800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b0bfff640 1 -- 172.21.15.104:0/1998961028 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18017070 con 0x7f0b24100b00 2024-09-19T04:09:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b241a1fd0 con 0x7f0b24100b00 2024-09-19T04:09:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b0bfff640 1 -- 172.21.15.104:0/1998961028 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b18009c30 con 0x7f0b24100b00 2024-09-19T04:09:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b0bfff640 1 -- 172.21.15.104:0/1998961028 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18003d00 con 0x7f0b24100b00 2024-09-19T04:09:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.195+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b241a23d0 con 0x7f0b24100b00 2024-09-19T04:09:08.201 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.196+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b241a26c0 con 0x7f0b24100b00 2024-09-19T04:09:08.201 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.198+0000 7f0b0bfff640 1 -- 172.21.15.104:0/1998961028 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0b18003ea0 con 0x7f0b24100b00 2024-09-19T04:09:08.201 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.199+0000 7f0b0bfff640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0b0c0a3240 0x7f0b0c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:08.201 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.199+0000 7f0b0bfff640 1 -- 172.21.15.104:0/1998961028 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f0b180ce390 con 0x7f0b24100b00 2024-09-19T04:09:08.201 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.199+0000 7f0b227fc640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0b0c0a3240 0x7f0b0c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:08.202 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.199+0000 7f0b227fc640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0b0c0a3240 0x7f0b0c0a5700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005fd0 tx=0x7f0b14005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:08.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.207+0000 7f0b0bfff640 1 -- 172.21.15.104:0/1998961028 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b1809c3f0 con 0x7f0b24100b00 2024-09-19T04:09:08.386 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:08.384+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0b24062630 con 0x7f0b0c0a3240 2024-09-19T04:09:08.387 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:09:08.387 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:09:08.387 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:09:08.387 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:08.387 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.387 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.387 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:08.387 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:08.388 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.388 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:08.388 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:08.388 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.388 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:08 smithi104 ceph-mon[34145]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:09:08.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:09:08.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:09:08.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:09:08.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:08.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:08.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:08.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:08.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:08.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:08 smithi112 ceph-mon[35051]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:09:08.591 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3[52047]: 2024-09-19T04:09:08.156+0000 7f1c7b1c2640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T04:09:08.591 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3[52047]: 2024-09-19T04:09:08.164+0000 7f1c777fc640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:09:08.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:09:08.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:08.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:08 smithi094 ceph-mon[30972]: from='osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:09:09.436 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:09 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi094 2024-09-19T04:09:09.436 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:09 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi094 to 8544M 2024-09-19T04:09:09.436 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:09 smithi112 ceph-mon[35051]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:09:09.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:09 smithi112 ceph-mon[35051]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:09:09.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:09.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:09.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:09:09.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:09:09.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:09.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:09 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi094 2024-09-19T04:09:09.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:09 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi094 to 8544M 2024-09-19T04:09:09.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:09 smithi094 ceph-mon[30972]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:09:09.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:09 smithi094 ceph-mon[30972]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:09:09.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:09.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:09.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:09:09.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:09:09.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:09 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi094 2024-09-19T04:09:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:09 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi094 to 8544M 2024-09-19T04:09:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:09 smithi104 ceph-mon[34145]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:09:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:09 smithi104 ceph-mon[34145]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:09:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:09:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:09:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:10 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:09:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:10 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:09:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:10 smithi104 ceph-mon[34145]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:09:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:10 smithi104 ceph-mon[34145]: osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] boot 2024-09-19T04:09:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:10 smithi104 ceph-mon[34145]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T04:09:10.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:10.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:10 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:09:10.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:10 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:09:10.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:10 smithi112 ceph-mon[35051]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:09:10.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:10.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:10 smithi112 ceph-mon[35051]: osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] boot 2024-09-19T04:09:10.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:10 smithi112 ceph-mon[35051]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T04:09:10.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:10.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:10 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:09:10.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:10 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:09:10.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:10 smithi094 ceph-mon[30972]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:09:10.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:10.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:10 smithi094 ceph-mon[30972]: osd.3 [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] boot 2024-09-19T04:09:10.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:10 smithi094 ceph-mon[30972]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T04:09:10.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:09:11.302 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:11 smithi104 ceph-mon[34145]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:09:11.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:11 smithi104 ceph-mon[34145]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:09:11.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:11 smithi104 ceph-mon[34145]: from='client.? 172.21.15.104:0/411930117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0edd43c9-dec2-4df0-b10e-cf68a6e809b1"}]: dispatch 2024-09-19T04:09:11.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:11 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0edd43c9-dec2-4df0-b10e-cf68a6e809b1"}]: dispatch 2024-09-19T04:09:11.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:11 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0edd43c9-dec2-4df0-b10e-cf68a6e809b1"}]': finished 2024-09-19T04:09:11.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:11 smithi104 ceph-mon[34145]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T04:09:11.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:11 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:11.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:11 smithi112 ceph-mon[35051]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:09:11.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:11 smithi112 ceph-mon[35051]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:09:11.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:11 smithi112 ceph-mon[35051]: from='client.? 172.21.15.104:0/411930117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0edd43c9-dec2-4df0-b10e-cf68a6e809b1"}]: dispatch 2024-09-19T04:09:11.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:11 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0edd43c9-dec2-4df0-b10e-cf68a6e809b1"}]: dispatch 2024-09-19T04:09:11.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:11 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0edd43c9-dec2-4df0-b10e-cf68a6e809b1"}]': finished 2024-09-19T04:09:11.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:11 smithi112 ceph-mon[35051]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T04:09:11.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:11 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:11.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:11 smithi094 ceph-mon[30972]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:09:11.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:11 smithi094 ceph-mon[30972]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:09:11.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:11 smithi094 ceph-mon[30972]: from='client.? 172.21.15.104:0/411930117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0edd43c9-dec2-4df0-b10e-cf68a6e809b1"}]: dispatch 2024-09-19T04:09:11.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:11 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0edd43c9-dec2-4df0-b10e-cf68a6e809b1"}]: dispatch 2024-09-19T04:09:11.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:11 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0edd43c9-dec2-4df0-b10e-cf68a6e809b1"}]': finished 2024-09-19T04:09:11.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:11 smithi094 ceph-mon[30972]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T04:09:11.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:11 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:12 smithi112 ceph-mon[35051]: from='client.? 172.21.15.104:0/1611073980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:09:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:12 smithi112 ceph-mon[35051]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T04:09:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:12 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:12.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:12 smithi094 ceph-mon[30972]: from='client.? 172.21.15.104:0/1611073980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:09:12.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:12 smithi094 ceph-mon[30972]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T04:09:12.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:12 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:12.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:12 smithi104 ceph-mon[34145]: from='client.? 172.21.15.104:0/1611073980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:09:12.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:12 smithi104 ceph-mon[34145]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T04:09:12.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:12 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:13.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:13 smithi112 ceph-mon[35051]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:13.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:13 smithi112 ceph-mon[35051]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:09:13.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:13.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:13 smithi094 ceph-mon[30972]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:13 smithi094 ceph-mon[30972]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:09:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:13 smithi104 ceph-mon[34145]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:13 smithi104 ceph-mon[34145]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:09:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:15.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:15 smithi112 ceph-mon[35051]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:15.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:15 smithi094 ceph-mon[30972]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:15 smithi104 ceph-mon[34145]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:17.476 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:17 smithi104 ceph-mon[34145]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-19T04:09:17.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:17 smithi112 ceph-mon[35051]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-19T04:09:17.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:17 smithi094 ceph-mon[30972]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-19T04:09:19.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:19 smithi104 ceph-mon[34145]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-19T04:09:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:19 smithi112 ceph-mon[35051]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-19T04:09:19.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:19 smithi094 ceph-mon[30972]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-19T04:09:20.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:09:20.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:20 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:20.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:09:20.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:20 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:20.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:09:20.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:20 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:21.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:21 smithi104 ceph-mon[34145]: Deploying daemon osd.4 on smithi104 2024-09-19T04:09:21.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:21 smithi104 ceph-mon[34145]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T04:09:21.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:21 smithi112 ceph-mon[35051]: Deploying daemon osd.4 on smithi104 2024-09-19T04:09:21.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:21 smithi112 ceph-mon[35051]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T04:09:21.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:21 smithi094 ceph-mon[30972]: Deploying daemon osd.4 on smithi104 2024-09-19T04:09:21.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:21 smithi094 ceph-mon[30972]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T04:09:23.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:23 smithi104 ceph-mon[34145]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:09:23.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:23 smithi112 ceph-mon[35051]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:09:23.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:23 smithi094 ceph-mon[30972]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:09:25.459 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:25 smithi104 ceph-mon[34145]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T04:09:25.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:25 smithi112 ceph-mon[35051]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T04:09:25.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:25 smithi094 ceph-mon[30972]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T04:09:26.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:26 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:26.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:26 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:26.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:26 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:26.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:26 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:26.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:26 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:26 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:26.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:26 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:26.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:26 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:26.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:26 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:27 smithi112 ceph-mon[35051]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:09:27.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:27 smithi104 ceph-mon[34145]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:09:27.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:27 smithi094 ceph-mon[30972]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:09:28.998 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.996+0000 7f0b0bfff640 1 -- 172.21.15.104:0/1998961028 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0b24062630 con 0x7f0b0c0a3240 2024-09-19T04:09:28.998 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 4 on host 'smithi104' 2024-09-19T04:09:29.000 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.998+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0b0c0a3240 msgr2=0x7f0b0c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.000 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.998+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0b0c0a3240 0x7f0b0c0a5700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005fd0 tx=0x7f0b14005850 comp rx=0 tx=0).stop 2024-09-19T04:09:29.000 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.998+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b24100b00 msgr2=0x7f0b24104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.000 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.998+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b24100b00 0x7f0b24104d10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800d770 tx=0x7f0b1800dc30 comp rx=0 tx=0).stop 2024-09-19T04:09:29.000 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.998+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 shutdown_connections 2024-09-19T04:09:29.001 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.998+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0b0c0a3240 0x7f0b0c0a5700 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.001 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.998+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b241026c0 0x7f0b241038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.001 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.998+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b24101d00 0x7f0b24103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.001 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.999+0000 7f0b294bd640 1 --2- 172.21.15.104:0/1998961028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b24100b00 0x7f0b24104d10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.001 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.999+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 >> 172.21.15.104:0/1998961028 conn(0x7f0b240fc270 msgr2=0x7f0b240fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:29.001 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.999+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 shutdown_connections 2024-09-19T04:09:29.001 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:09:28.999+0000 7f0b294bd640 1 -- 172.21.15.104:0/1998961028 wait complete. 2024-09-19T04:09:29.302 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:29.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:29.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:29 smithi104 ceph-mon[34145]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:29.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:29.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.303 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:29 smithi094 ceph-mon[30972]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:29 smithi112 ceph-mon[35051]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:29.622 DEBUG:teuthology.orchestra.run.smithi104:osd.4> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T04:09:29.624 INFO:tasks.cephadm:Deploying osd.5 on smithi112 with /dev/vg_nvme/lv_3... 2024-09-19T04:09:29.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:09:29.875 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:09:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:31 smithi112 ceph-mon[35051]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:31.294 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:31 smithi104 ceph-mon[34145]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:31.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:31 smithi094 ceph-mon[30972]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:32.056 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:09:31 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4[45506]: 2024-09-19T04:09:31.789+0000 7f5c8db25640 -1 osd.4 0 log_to_monitors true 2024-09-19T04:09:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:32 smithi104 ceph-mon[34145]: from='osd.4 [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:09:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:32 smithi104 ceph-mon[34145]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:09:32.397 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:32 smithi112 ceph-mon[35051]: from='osd.4 [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:09:32.397 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:32 smithi112 ceph-mon[35051]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:09:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:09:32.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-09-19T04:09:32.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:32 smithi094 ceph-mon[30972]: from='osd.4 [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:09:32.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:32 smithi094 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:09:32.673 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.083+0000 7f2f906b4640 1 -- 172.21.15.112:0/1586156879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88102690 msgr2=0x7f2f88108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.083+0000 7f2f906b4640 1 --2- 172.21.15.112:0/1586156879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88102690 0x7f2f88108f20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009ca0 tx=0x7f2f8002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.083+0000 7f2f906b4640 1 -- 172.21.15.112:0/1586156879 shutdown_connections 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.083+0000 7f2f906b4640 1 --2- 172.21.15.112:0/1586156879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88102690 0x7f2f88108f20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.083+0000 7f2f906b4640 1 --2- 172.21.15.112:0/1586156879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f88101cd0 0x7f2f88102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.083+0000 7f2f906b4640 1 --2- 172.21.15.112:0/1586156879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f88100ad0 0x7f2f88100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.083+0000 7f2f906b4640 1 -- 172.21.15.112:0/1586156879 >> 172.21.15.112:0/1586156879 conn(0x7f2f880fc280 msgr2=0x7f2f880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.083+0000 7f2f906b4640 1 -- 172.21.15.112:0/1586156879 shutdown_connections 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.083+0000 7f2f906b4640 1 -- 172.21.15.112:0/1586156879 wait complete. 2024-09-19T04:09:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f906b4640 1 Processor -- start 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f906b4640 1 -- start start 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f906b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88100ad0 0x7f2f88197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f906b4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f88101cd0 0x7f2f88198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f906b4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f88102690 0x7f2f8819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f906b4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8810c660 con 0x7f2f88101cd0 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f906b4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8810c4e0 con 0x7f2f88102690 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f906b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8810c7e0 con 0x7f2f88100ad0 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f8e450640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88100ad0 0x7f2f88197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f8e450640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88100ad0 0x7f2f88197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56154/0 (socket says 172.21.15.112:56154) 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f8e450640 1 -- 172.21.15.112:0/209278057 learned_addr learned my addr 172.21.15.112:0/209278057 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f8dc4f640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f88101cd0 0x7f2f88198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.084+0000 7f2f8ec51640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f88102690 0x7f2f8819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f8e450640 1 -- 172.21.15.112:0/209278057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f88102690 msgr2=0x7f2f8819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f8e450640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f88102690 0x7f2f8819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f8e450640 1 -- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f88101cd0 msgr2=0x7f2f88198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f8e450640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f88101cd0 0x7f2f88198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f8e450640 1 -- 172.21.15.112:0/209278057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8819fa00 con 0x7f2f88100ad0 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f8dc4f640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f88101cd0 0x7f2f88198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f8e450640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88100ad0 0x7f2f88197dd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400c970 tx=0x7f2f8400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f7f7fe640 1 -- 172.21.15.112:0/209278057 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84007c10 con 0x7f2f88100ad0 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f7f7fe640 1 -- 172.21.15.112:0/209278057 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f84007db0 con 0x7f2f88100ad0 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8819fcf0 con 0x7f2f88100ad0 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f7f7fe640 1 -- 172.21.15.112:0/209278057 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400f7d0 con 0x7f2f88100ad0 2024-09-19T04:09:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.085+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f881a0180 con 0x7f2f88100ad0 2024-09-19T04:09:33.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.086+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f881a0470 con 0x7f2f88100ad0 2024-09-19T04:09:33.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.087+0000 7f2f7f7fe640 1 -- 172.21.15.112:0/209278057 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2f84018480 con 0x7f2f88100ad0 2024-09-19T04:09:33.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.088+0000 7f2f7f7fe640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2f740a3270 0x7f2f740a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:33.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.089+0000 7f2f7f7fe640 1 -- 172.21.15.112:0/209278057 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3805+0+0 (secure 0 0 0) 0x7f2f840ceeb0 con 0x7f2f88100ad0 2024-09-19T04:09:33.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.089+0000 7f2f8dc4f640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2f740a3270 0x7f2f740a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:33.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.089+0000 7f2f8dc4f640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2f740a3270 0x7f2f740a5730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f881992f0 tx=0x7f2f78008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:33.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.092+0000 7f2f7f7fe640 1 -- 172.21.15.112:0/209278057 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f84026080 con 0x7f2f88100ad0 2024-09-19T04:09:33.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:33.277+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2f881a0640 con 0x7f2f740a3270 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='osd.4 [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:33.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.280 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:33.280 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:33.280 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:33 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='osd.4 [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:33.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:33.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:33 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.596 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='osd.4 [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:33.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi104 2024-09-19T04:09:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-19T04:09:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:09:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:09:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:09:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:09:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:09:34.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:34.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:34 smithi112 ceph-mon[35051]: from='osd.4 ' entity='osd.4' 2024-09-19T04:09:34.354 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4[45506]: 2024-09-19T04:09:33.930+0000 7f5c89285640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T04:09:34.354 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:09:33 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4[45506]: 2024-09-19T04:09:33.943+0000 7f5c818b7640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:09:34.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi104 2024-09-19T04:09:34.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-19T04:09:34.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:09:34.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:09:34.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:34.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:34.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:09:34.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:09:34.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:09:34.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:34.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:34 smithi104 ceph-mon[34145]: from='osd.4 ' entity='osd.4' 2024-09-19T04:09:34.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi104 2024-09-19T04:09:34.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-19T04:09:34.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:09:34.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:09:34.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:34.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:34.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:09:34.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:09:34.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:09:34.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:34.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:34 smithi094 ceph-mon[30972]: from='osd.4 ' entity='osd.4' 2024-09-19T04:09:35.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:35 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:09:35.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:35 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:09:35.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:35 smithi112 ceph-mon[35051]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:35.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:35 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:35.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:35 smithi112 ceph-mon[35051]: osd.4 [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] boot 2024-09-19T04:09:35.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:35 smithi112 ceph-mon[35051]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T04:09:35.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:35 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:35.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:35 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:09:35.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:35 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:09:35.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:35 smithi094 ceph-mon[30972]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:35.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:35 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:35.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:35 smithi094 ceph-mon[30972]: osd.4 [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] boot 2024-09-19T04:09:35.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:35 smithi094 ceph-mon[30972]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T04:09:35.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:35 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:35 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:09:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:35 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:09:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:35 smithi104 ceph-mon[34145]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:09:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:35 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:35 smithi104 ceph-mon[34145]: osd.4 [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] boot 2024-09-19T04:09:35.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:35 smithi104 ceph-mon[34145]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T04:09:35.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:35 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:09:36.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:36 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/2063375093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abe550dd-07b4-46e6-a55e-b65a9b15c373"}]: dispatch 2024-09-19T04:09:36.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:36 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abe550dd-07b4-46e6-a55e-b65a9b15c373"}]: dispatch 2024-09-19T04:09:36.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:36 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abe550dd-07b4-46e6-a55e-b65a9b15c373"}]': finished 2024-09-19T04:09:36.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:36 smithi112 ceph-mon[35051]: osdmap e32: 6 total, 5 up, 6 in 2024-09-19T04:09:36.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:36.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:36 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/4087274738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:09:36.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:36 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/2063375093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abe550dd-07b4-46e6-a55e-b65a9b15c373"}]: dispatch 2024-09-19T04:09:36.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:36 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abe550dd-07b4-46e6-a55e-b65a9b15c373"}]: dispatch 2024-09-19T04:09:36.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:36 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abe550dd-07b4-46e6-a55e-b65a9b15c373"}]': finished 2024-09-19T04:09:36.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:36 smithi094 ceph-mon[30972]: osdmap e32: 6 total, 5 up, 6 in 2024-09-19T04:09:36.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:36.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:36 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/4087274738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:09:36.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:36 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/2063375093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abe550dd-07b4-46e6-a55e-b65a9b15c373"}]: dispatch 2024-09-19T04:09:36.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:36 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abe550dd-07b4-46e6-a55e-b65a9b15c373"}]: dispatch 2024-09-19T04:09:36.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:36 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abe550dd-07b4-46e6-a55e-b65a9b15c373"}]': finished 2024-09-19T04:09:36.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:36 smithi104 ceph-mon[34145]: osdmap e32: 6 total, 5 up, 6 in 2024-09-19T04:09:36.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:36.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:36 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/4087274738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:09:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:37 smithi112 ceph-mon[35051]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:37 smithi112 ceph-mon[35051]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T04:09:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:37.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:37 smithi094 ceph-mon[30972]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:37.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:37 smithi094 ceph-mon[30972]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T04:09:37.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:37 smithi104 ceph-mon[34145]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:37 smithi104 ceph-mon[34145]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T04:09:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:39.424 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:39 smithi104 ceph-mon[34145]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:39.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:39 smithi112 ceph-mon[35051]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:39.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:39 smithi094 ceph-mon[30972]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:41.391 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:41 smithi112 ceph-mon[35051]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:41.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:41 smithi094 ceph-mon[30972]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:41 smithi104 ceph-mon[34145]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:43 smithi112 ceph-mon[35051]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:43.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:43 smithi094 ceph-mon[30972]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:43 smithi104 ceph-mon[34145]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:44.491 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:09:44.491 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:44.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:09:44.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:09:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:45.489 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:45 smithi112 ceph-mon[35051]: Deploying daemon osd.5 on smithi112 2024-09-19T04:09:45.489 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:45 smithi112 ceph-mon[35051]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:45 smithi094 ceph-mon[30972]: Deploying daemon osd.5 on smithi112 2024-09-19T04:09:45.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:45 smithi094 ceph-mon[30972]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:45 smithi104 ceph-mon[34145]: Deploying daemon osd.5 on smithi112 2024-09-19T04:09:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:45 smithi104 ceph-mon[34145]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:47.331 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:47 smithi112 ceph-mon[35051]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:47 smithi094 ceph-mon[30972]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:47 smithi104 ceph-mon[34145]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:49 smithi112 ceph-mon[35051]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:49.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:49 smithi094 ceph-mon[30972]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:49 smithi104 ceph-mon[34145]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:50.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:50 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:50.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:50 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:50.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:50 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:50.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:50 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:50.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:50 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:50.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:50 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:50 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:50 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:50 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:51.472 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:51 smithi112 ceph-mon[35051]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:51.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:51 smithi094 ceph-mon[30972]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:51 smithi104 ceph-mon[34145]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:53.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.335+0000 7f2f7f7fe640 1 -- 172.21.15.112:0/209278057 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2f881a0640 con 0x7f2f740a3270 2024-09-19T04:09:53.337 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 5 on host 'smithi112' 2024-09-19T04:09:53.337 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:53 smithi112 ceph-mon[35051]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:53.337 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:53.337 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:53.337 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:53.337 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:53.338 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2f740a3270 msgr2=0x7f2f740a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2f740a3270 0x7f2f740a5730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f881992f0 tx=0x7f2f78008040 comp rx=0 tx=0).stop 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88100ad0 msgr2=0x7f2f88197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88100ad0 0x7f2f88197dd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400c970 tx=0x7f2f8400ce30 comp rx=0 tx=0).stop 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 shutdown_connections 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2f740a3270 0x7f2f740a5730 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f88102690 0x7f2f8819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f88101cd0 0x7f2f88198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 --2- 172.21.15.112:0/209278057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f88100ad0 0x7f2f88197dd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 >> 172.21.15.112:0/209278057 conn(0x7f2f880fc280 msgr2=0x7f2f88069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 shutdown_connections 2024-09-19T04:09:53.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:09:53.337+0000 7f2f906b4640 1 -- 172.21.15.112:0/209278057 wait complete. 2024-09-19T04:09:53.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:53 smithi094 ceph-mon[30972]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:53.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:53 smithi104 ceph-mon[34145]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:53.927 DEBUG:teuthology.orchestra.run.smithi112:osd.5> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T04:09:53.930 INFO:tasks.cephadm:Deploying osd.6 on smithi094 with /dev/vg_nvme/lv_2... 2024-09-19T04:09:53.930 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:09:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:54.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:54 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:54.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:54 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:54.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:54 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:54.537 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:54.537 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:54.537 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:54.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:54 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:54.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:54 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:54.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:54 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:55.314 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:55 smithi112 ceph-mon[35051]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:55.539 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:55 smithi094 ceph-mon[30972]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:55 smithi104 ceph-mon[34145]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:56.398 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:56 smithi112 ceph-mon[35051]: from='osd.5 [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:09:56.399 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:56 smithi112 ceph-mon[35051]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:09:56.399 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:09:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:09:56.179+0000 7f0a3b213640 -1 osd.5 0 log_to_monitors true 2024-09-19T04:09:56.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:56 smithi094 ceph-mon[30972]: from='osd.5 [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:09:56.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:56 smithi094 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:09:56.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:56 smithi104 ceph-mon[34145]: from='osd.5 [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:09:56.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:56 smithi104 ceph-mon[34145]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:09:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:09:56.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:57.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:57.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:09:57.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:09:57.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:57.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='osd.5 [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:09:57.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:09:57.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:57.356 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:57.356 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.356 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:57.356 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:57.356 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 -- 172.21.15.94:0/2207677805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8381037c0 msgr2=0x7fa83810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:57.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/2207677805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8381037c0 0x7fa83810a050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa830009900 tx=0x7fa83002a410 comp rx=0 tx=0).stop 2024-09-19T04:09:57.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 -- 172.21.15.94:0/2207677805 shutdown_connections 2024-09-19T04:09:57.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/2207677805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8381037c0 0x7fa83810a050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:57.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/2207677805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838102e00 0x7fa838103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:57.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/2207677805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa838101c00 0x7fa838102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:57.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 -- 172.21.15.94:0/2207677805 >> 172.21.15.94:0/2207677805 conn(0x7fa8380fd390 msgr2=0x7fa8380ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 -- 172.21.15.94:0/2207677805 shutdown_connections 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 -- 172.21.15.94:0/2207677805 wait complete. 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.437+0000 7fa83d2a9640 1 Processor -- start 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa83d2a9640 1 -- start start 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa83d2a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa838101c00 0x7fa838197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa83d2a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838102e00 0x7fa8381983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa83d2a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8381037c0 0x7fa83819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa83d2a9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810d7b0 con 0x7fa8381037c0 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa83d2a9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810d630 con 0x7fa838102e00 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa83d2a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810d930 con 0x7fa838101c00 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa836ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa838101c00 0x7fa838197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8381037c0 0x7fa83819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:57.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa836ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa838101c00 0x7fa838197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:59670/0 (socket says 172.21.15.94:59670) 2024-09-19T04:09:57.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa836ffd640 1 -- 172.21.15.94:0/1896298448 learned_addr learned my addr 172.21.15.94:0/1896298448 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:09:57.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa8367fc640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838102e00 0x7fa8381983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:57.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa836ffd640 1 -- 172.21.15.94:0/1896298448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838102e00 msgr2=0x7fa8381983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:57.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa836ffd640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838102e00 0x7fa8381983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:57.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa836ffd640 1 -- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8381037c0 msgr2=0x7fa83819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa836ffd640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8381037c0 0x7fa83819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa836ffd640 1 -- 172.21.15.94:0/1896298448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83819faa0 con 0x7fa838101c00 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa8377fe640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8381037c0 0x7fa83819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.438+0000 7fa836ffd640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa838101c00 0x7fa838197e70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa82800eaa0 tx=0x7fa82800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.439+0000 7fa82ffff640 1 -- 172.21.15.94:0/1896298448 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828016070 con 0x7fa838101c00 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.439+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83819fd90 con 0x7fa838101c00 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.439+0000 7fa82ffff640 1 -- 172.21.15.94:0/1896298448 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa828003e20 con 0x7fa838101c00 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.439+0000 7fa82ffff640 1 -- 172.21.15.94:0/1896298448 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8280116d0 con 0x7fa838101c00 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.439+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8381a0240 con 0x7fa838101c00 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.440+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa838109020 con 0x7fa838101c00 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.441+0000 7fa82ffff640 1 -- 172.21.15.94:0/1896298448 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa828019070 con 0x7fa838101c00 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.441+0000 7fa82ffff640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8240a3270 0x7fa8240a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:57.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.441+0000 7fa8367fc640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8240a3270 0x7fa8240a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:57.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.441+0000 7fa82ffff640 1 -- 172.21.15.94:0/1896298448 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4253+0+0 (secure 0 0 0) 0x7fa8280cee30 con 0x7fa838101c00 2024-09-19T04:09:57.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.442+0000 7fa8367fc640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8240a3270 0x7fa8240a5730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa838199390 tx=0x7fa820008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:57.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.448+0000 7fa82ffff640 1 -- 172.21.15.94:0/1896298448 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa82809ca90 con 0x7fa838101c00 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='osd.5 [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:57.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:57.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:09:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:09:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='osd.5 [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:57.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:09:57.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:09:57.621+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa838062630 con 0x7fa8240a3270 2024-09-19T04:09:58.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi112 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi112 to 8544M 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: from='osd.5 ' entity='osd.5' 2024-09-19T04:09:58.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:58.535 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:09:58.048+0000 7f0a36973640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T04:09:58.535 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:09:58.058+0000 7f0a2efa5640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:09:58.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi112 2024-09-19T04:09:58.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi112 to 8544M 2024-09-19T04:09:58.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:09:58.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:09:58.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:58.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:58.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:09:58.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:09:58.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:58.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: from='osd.5 ' entity='osd.5' 2024-09-19T04:09:58.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:58.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi112 2024-09-19T04:09:58.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi112 to 8544M 2024-09-19T04:09:58.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:09:58.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:09:58.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:58.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:58.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:09:58.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:09:58.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:58.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: from='osd.5 ' entity='osd.5' 2024-09-19T04:09:58.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:59 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:09:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:59 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:09:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:59 smithi112 ceph-mon[35051]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:09:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:59 smithi112 ceph-mon[35051]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:59 smithi112 ceph-mon[35051]: osd.5 [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] boot 2024-09-19T04:09:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:59 smithi112 ceph-mon[35051]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T04:09:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:09:59 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:59.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:59 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:09:59.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:59 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:09:59.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:59 smithi094 ceph-mon[30972]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:09:59.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:59 smithi094 ceph-mon[30972]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:59.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:59 smithi094 ceph-mon[30972]: osd.5 [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] boot 2024-09-19T04:09:59.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:59 smithi094 ceph-mon[30972]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T04:09:59.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:09:59 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:09:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:59 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:09:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:59 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:09:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:59 smithi104 ceph-mon[34145]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:09:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:59 smithi104 ceph-mon[34145]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:09:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:59 smithi104 ceph-mon[34145]: osd.5 [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] boot 2024-09-19T04:09:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:59 smithi104 ceph-mon[34145]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T04:09:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:09:59 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:10:00.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:00 smithi094 ceph-mon[30972]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:10:00.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:00 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2661058180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26a90049-fa63-414c-a4e2-6a779aa15473"}]: dispatch 2024-09-19T04:10:00.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:00 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2661058180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26a90049-fa63-414c-a4e2-6a779aa15473"}]': finished 2024-09-19T04:10:00.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:00 smithi094 ceph-mon[30972]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T04:10:00.483 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:00 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:00.483 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:00 smithi094 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T04:10:00.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:00 smithi112 ceph-mon[35051]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:10:00.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:00 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2661058180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26a90049-fa63-414c-a4e2-6a779aa15473"}]: dispatch 2024-09-19T04:10:00.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:00 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2661058180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26a90049-fa63-414c-a4e2-6a779aa15473"}]': finished 2024-09-19T04:10:00.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:00 smithi112 ceph-mon[35051]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T04:10:00.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:00 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:00.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:00 smithi112 ceph-mon[35051]: overall HEALTH_OK 2024-09-19T04:10:00.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:00 smithi104 ceph-mon[34145]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:10:00.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:00 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2661058180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26a90049-fa63-414c-a4e2-6a779aa15473"}]: dispatch 2024-09-19T04:10:00.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:00 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2661058180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26a90049-fa63-414c-a4e2-6a779aa15473"}]': finished 2024-09-19T04:10:00.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:00 smithi104 ceph-mon[34145]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T04:10:00.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:00 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:00.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:00 smithi104 ceph-mon[34145]: overall HEALTH_OK 2024-09-19T04:10:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:01 smithi112 ceph-mon[35051]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:01 smithi112 ceph-mon[35051]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T04:10:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:01 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1621121823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:10:01.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:01 smithi094 ceph-mon[30972]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:01 smithi094 ceph-mon[30972]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T04:10:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:01 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1621121823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:10:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:01 smithi104 ceph-mon[34145]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:01 smithi104 ceph-mon[34145]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T04:10:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:01 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1621121823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:10:02.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:02 smithi112 ceph-mon[35051]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T04:10:02.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:02 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:02.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:02 smithi112 ceph-mon[35051]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-09-19T04:10:02.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:02 smithi094 ceph-mon[30972]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T04:10:02.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:02 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:02.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:02 smithi094 ceph-mon[30972]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-09-19T04:10:02.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:02 smithi104 ceph-mon[34145]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T04:10:02.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:02 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:02.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:02 smithi104 ceph-mon[34145]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-09-19T04:10:05.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:05 smithi094 ceph-mon[30972]: pgmap v137: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-19T04:10:05.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:05 smithi112 ceph-mon[35051]: pgmap v137: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-19T04:10:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:05 smithi104 ceph-mon[34145]: pgmap v137: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-19T04:10:07.439 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:07 smithi094 ceph-mon[30972]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T04:10:07.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:07 smithi112 ceph-mon[35051]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T04:10:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:07 smithi104 ceph-mon[34145]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T04:10:09.406 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:09 smithi104 ceph-mon[34145]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:10:09.406 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:10:09.407 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:09.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:09 smithi112 ceph-mon[35051]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:10:09.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:10:09.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:09.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:09 smithi094 ceph-mon[30972]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:10:09.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:10:09.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:10.423 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:10 smithi094 ceph-mon[30972]: Deploying daemon osd.6 on smithi094 2024-09-19T04:10:10.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:10 smithi112 ceph-mon[35051]: Deploying daemon osd.6 on smithi094 2024-09-19T04:10:10.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:10 smithi104 ceph-mon[34145]: Deploying daemon osd.6 on smithi094 2024-09-19T04:10:11.332 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:11 smithi094 ceph-mon[30972]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:11.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:11 smithi112 ceph-mon[35051]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:11.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:11 smithi104 ceph-mon[34145]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:13.468 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:13 smithi094 ceph-mon[30972]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:13.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:13 smithi112 ceph-mon[35051]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:13 smithi104 ceph-mon[34145]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:15.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:15 smithi094 ceph-mon[30972]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:15.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:15.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:15.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:15 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:15.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:15 smithi112 ceph-mon[35051]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:15.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:15.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:15.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:15 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:15 smithi104 ceph-mon[34145]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:15 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:15 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:15 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:17.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:17 smithi094 ceph-mon[30972]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:17.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:17 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:17.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:17 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:17.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:17 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:17.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:17 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:17.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:17 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:17.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:17 smithi112 ceph-mon[35051]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:17.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:17 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:17.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:17 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:17.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:17 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:17.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:17 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:17.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:17 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:17 smithi104 ceph-mon[34145]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:17 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:18.046 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 6 on host 'smithi094' 2024-09-19T04:10:18.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.044+0000 7fa82ffff640 1 -- 172.21.15.94:0/1896298448 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa838062630 con 0x7fa8240a3270 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8240a3270 msgr2=0x7fa8240a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8240a3270 0x7fa8240a5730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa838199390 tx=0x7fa820008040 comp rx=0 tx=0).stop 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa838101c00 msgr2=0x7fa838197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa838101c00 0x7fa838197e70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa82800eaa0 tx=0x7fa82800ef60 comp rx=0 tx=0).stop 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 shutdown_connections 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8240a3270 0x7fa8240a5730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8381037c0 0x7fa83819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838102e00 0x7fa8381983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 --2- 172.21.15.94:0/1896298448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa838101c00 0x7fa838197e70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.046+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 >> 172.21.15.94:0/1896298448 conn(0x7fa8380fd390 msgr2=0x7fa8381061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.047+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 shutdown_connections 2024-09-19T04:10:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:10:18.047+0000 7fa83d2a9640 1 -- 172.21.15.94:0/1896298448 wait complete. 2024-09-19T04:10:18.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:18.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:18.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:18.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:18.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:18.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:18.726 DEBUG:teuthology.orchestra.run.smithi094:osd.6> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T04:10:18.729 INFO:tasks.cephadm:Deploying osd.7 on smithi104 with /dev/vg_nvme/lv_2... 2024-09-19T04:10:18.729 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:10:18.980 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:10:19.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:19 smithi094 ceph-mon[30972]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:19.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:19 smithi104 ceph-mon[34145]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:19 smithi112 ceph-mon[35051]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:20.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:10:20 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:10:20.404+0000 7fe816167640 -1 osd.6 0 log_to_monitors true 2024-09-19T04:10:21.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:21 smithi104 ceph-mon[34145]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:21.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:21 smithi104 ceph-mon[34145]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:10:21.516 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:10:21.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:21 smithi112 ceph-mon[35051]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:21.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:21 smithi112 ceph-mon[35051]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:10:21.542 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-09-19T04:10:21.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:21 smithi094 ceph-mon[30972]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:21.563 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:21 smithi094 ceph-mon[30972]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:10:21.802 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:10:22.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.275+0000 7f39bb3a5640 1 -- 172.21.15.104:0/2416664062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b4101c20 msgr2=0x7f39b4102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.275+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/2416664062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b4101c20 0x7f39b4102020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009900 tx=0x7f39b002a410 comp rx=0 tx=0).stop 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.275+0000 7f39bb3a5640 1 -- 172.21.15.104:0/2416664062 shutdown_connections 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.275+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/2416664062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39b41037e0 0x7f39b410a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.275+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/2416664062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b4102e20 0x7f39b41032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.275+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/2416664062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b4101c20 0x7f39b4102020 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.275+0000 7f39bb3a5640 1 -- 172.21.15.104:0/2416664062 >> 172.21.15.104:0/2416664062 conn(0x7f39b40fd3b0 msgr2=0x7f39b40ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.275+0000 7f39bb3a5640 1 -- 172.21.15.104:0/2416664062 shutdown_connections 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.275+0000 7f39bb3a5640 1 -- 172.21.15.104:0/2416664062 wait complete. 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39bb3a5640 1 Processor -- start 2024-09-19T04:10:22.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39bb3a5640 1 -- start start 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39bb3a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39b4101c20 0x7f39b4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39bb3a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b4102e20 0x7f39b4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39bb3a5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b41037e0 0x7f39b419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39bb3a5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39b410d760 con 0x7f39b4102e20 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39bb3a5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39b410d5e0 con 0x7f39b41037e0 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39bb3a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39b410d8e0 con 0x7f39b4101c20 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39b9942640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b41037e0 0x7f39b419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39b8940640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b4102e20 0x7f39b4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39b9942640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b41037e0 0x7f39b419f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35030/0 (socket says 172.21.15.104:35030) 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39b9942640 1 -- 172.21.15.104:0/1869573331 learned_addr learned my addr 172.21.15.104:0/1869573331 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.276+0000 7f39b9141640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39b4101c20 0x7f39b4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39b9942640 1 -- 172.21.15.104:0/1869573331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39b4101c20 msgr2=0x7f39b4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39b9942640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39b4101c20 0x7f39b4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:22.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39b9942640 1 -- 172.21.15.104:0/1869573331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b4102e20 msgr2=0x7f39b4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:22.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39b9942640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b4102e20 0x7f39b4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:22.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39b9942640 1 -- 172.21.15.104:0/1869573331 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b419fa50 con 0x7f39b41037e0 2024-09-19T04:10:22.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39b9942640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b41037e0 0x7f39b419f3e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00eaa0 tx=0x7f39ac00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:22.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39aa7fc640 1 -- 172.21.15.104:0/1869573331 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00cdb0 con 0x7f39b41037e0 2024-09-19T04:10:22.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39aa7fc640 1 -- 172.21.15.104:0/1869573331 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39ac005020 con 0x7f39b41037e0 2024-09-19T04:10:22.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b419fd40 con 0x7f39b41037e0 2024-09-19T04:10:22.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39aa7fc640 1 -- 172.21.15.104:0/1869573331 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0116d0 con 0x7f39b41037e0 2024-09-19T04:10:22.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.277+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b41a0270 con 0x7f39b41037e0 2024-09-19T04:10:22.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.278+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b41090c0 con 0x7f39b41037e0 2024-09-19T04:10:22.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.280+0000 7f39aa7fc640 1 -- 172.21.15.104:0/1869573331 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f39ac0051d0 con 0x7f39b41037e0 2024-09-19T04:10:22.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.281+0000 7f39aa7fc640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f39a00a32f0 0x7f39a00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:22.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.281+0000 7f39aa7fc640 1 -- 172.21.15.104:0/1869573331 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4685+0+0 (secure 0 0 0) 0x7f39ac0ceee0 con 0x7f39b41037e0 2024-09-19T04:10:22.284 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.281+0000 7f39b9141640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f39a00a32f0 0x7f39a00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:22.284 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.282+0000 7f39b9141640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f39a00a32f0 0x7f39a00a57b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f39b0004770 tx=0x7f39b0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:22.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.285+0000 7f39aa7fc640 1 -- 172.21.15.104:0/1869573331 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f39ac016030 con 0x7f39b41037e0 2024-09-19T04:10:22.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:22.466+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f39b4062630 con 0x7f39a00a32f0 2024-09-19T04:10:22.469 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:10:22.469 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:22 smithi104 ceph-mon[34145]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:22.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:22.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:22 smithi112 ceph-mon[35051]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:10:22.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:10:22.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-mon[30972]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:10:23.179 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:10:22.861+0000 7fe8118c7640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T04:10:23.179 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:10:22 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:10:22.872+0000 7fe809ef9640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:10:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi094 2024-09-19T04:10:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi094 to 5696M 2024-09-19T04:10:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:10:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:10:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:10:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:23.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:23 smithi112 ceph-mon[35051]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' 2024-09-19T04:10:23.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi094 2024-09-19T04:10:23.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi094 to 5696M 2024-09-19T04:10:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:10:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:10:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:10:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:23.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:23 smithi094 ceph-mon[30972]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' 2024-09-19T04:10:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi094 2024-09-19T04:10:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi094 to 5696M 2024-09-19T04:10:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:10:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:10:23.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:10:23.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:23.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:23 smithi104 ceph-mon[34145]: from='osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966]' entity='osd.6' 2024-09-19T04:10:24.524 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:24 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:10:24.524 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:24 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:10:24.524 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:24 smithi104 ceph-mon[34145]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:24.524 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:24 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:24.524 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:24 smithi104 ceph-mon[34145]: osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] boot 2024-09-19T04:10:24.524 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:24 smithi104 ceph-mon[34145]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T04:10:24.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:24 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:24.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:24 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:10:24.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:24 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:10:24.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:24 smithi112 ceph-mon[35051]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:24.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:24 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:24.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:24 smithi112 ceph-mon[35051]: osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] boot 2024-09-19T04:10:24.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:24 smithi112 ceph-mon[35051]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T04:10:24.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:24 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:24.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:24 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:10:24.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:24 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:10:24.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:24 smithi094 ceph-mon[30972]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:24.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:24 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:24.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:24 smithi094 ceph-mon[30972]: osd.6 [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] boot 2024-09-19T04:10:24.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:24 smithi094 ceph-mon[30972]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T04:10:24.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:24 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:10:25.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:25 smithi104 ceph-mon[34145]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:25.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:25 smithi104 ceph-mon[34145]: from='client.? 172.21.15.104:0/41208341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "699d883d-3aa9-4430-80ef-5a3de8cd773f"}]: dispatch 2024-09-19T04:10:25.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:25 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "699d883d-3aa9-4430-80ef-5a3de8cd773f"}]: dispatch 2024-09-19T04:10:25.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:25 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "699d883d-3aa9-4430-80ef-5a3de8cd773f"}]': finished 2024-09-19T04:10:25.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:25 smithi104 ceph-mon[34145]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T04:10:25.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:25.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:25 smithi112 ceph-mon[35051]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:25 smithi112 ceph-mon[35051]: from='client.? 172.21.15.104:0/41208341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "699d883d-3aa9-4430-80ef-5a3de8cd773f"}]: dispatch 2024-09-19T04:10:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:25 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "699d883d-3aa9-4430-80ef-5a3de8cd773f"}]: dispatch 2024-09-19T04:10:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:25 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "699d883d-3aa9-4430-80ef-5a3de8cd773f"}]': finished 2024-09-19T04:10:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:25 smithi112 ceph-mon[35051]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T04:10:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:25.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:25 smithi094 ceph-mon[30972]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:10:25.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:25 smithi094 ceph-mon[30972]: from='client.? 172.21.15.104:0/41208341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "699d883d-3aa9-4430-80ef-5a3de8cd773f"}]: dispatch 2024-09-19T04:10:25.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:25 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "699d883d-3aa9-4430-80ef-5a3de8cd773f"}]: dispatch 2024-09-19T04:10:25.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:25 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "699d883d-3aa9-4430-80ef-5a3de8cd773f"}]': finished 2024-09-19T04:10:25.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:25 smithi094 ceph-mon[30972]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T04:10:25.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:26.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:26 smithi112 ceph-mon[35051]: from='client.? 172.21.15.104:0/2829782415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:10:26.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:26 smithi094 ceph-mon[30972]: from='client.? 172.21.15.104:0/2829782415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:10:26.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:26 smithi104 ceph-mon[34145]: from='client.? 172.21.15.104:0/2829782415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:10:27.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:27 smithi112 ceph-mon[35051]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:27.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:27 smithi094 ceph-mon[30972]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:27 smithi104 ceph-mon[34145]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:29 smithi112 ceph-mon[35051]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:29.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:29 smithi094 ceph-mon[30972]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:29 smithi104 ceph-mon[34145]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:31.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:31 smithi104 ceph-mon[34145]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:31.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:31 smithi112 ceph-mon[35051]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:31.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:31 smithi094 ceph-mon[30972]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:33.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:33 smithi094 ceph-mon[30972]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:33.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:33 smithi112 ceph-mon[35051]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:33.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:33 smithi104 ceph-mon[34145]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:34.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:10:34.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:34.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:10:34.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:34.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:10:34.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:35.477 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:35 smithi104 ceph-mon[34145]: Deploying daemon osd.7 on smithi104 2024-09-19T04:10:35.477 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:35 smithi104 ceph-mon[34145]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:35.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:35 smithi112 ceph-mon[35051]: Deploying daemon osd.7 on smithi104 2024-09-19T04:10:35.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:35 smithi112 ceph-mon[35051]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:35.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:35 smithi094 ceph-mon[30972]: Deploying daemon osd.7 on smithi104 2024-09-19T04:10:35.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:35 smithi094 ceph-mon[30972]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:37.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:37 smithi112 ceph-mon[35051]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:37.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:37 smithi094 ceph-mon[30972]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:37 smithi104 ceph-mon[34145]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:39.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:39 smithi104 ceph-mon[34145]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:39.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:39 smithi112 ceph-mon[35051]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:39.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:39 smithi094 ceph-mon[30972]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:40.215 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:40 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:40.215 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:40 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:40.215 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:40 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:40.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:40 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:40 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:40 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:40.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:40 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:40.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:40 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:40.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:40 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:41.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:41 smithi112 ceph-mon[35051]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:41.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:41 smithi094 ceph-mon[30972]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:41 smithi104 ceph-mon[34145]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.277+0000 7f39aa7fc640 1 -- 172.21.15.104:0/1869573331 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f39b4062630 con 0x7f39a00a32f0 2024-09-19T04:10:43.279 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 7 on host 'smithi104' 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f39a00a32f0 msgr2=0x7f39a00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f39a00a32f0 0x7f39a00a57b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f39b0004770 tx=0x7f39b0000bc0 comp rx=0 tx=0).stop 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b41037e0 msgr2=0x7f39b419f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b41037e0 0x7f39b419f3e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00eaa0 tx=0x7f39ac00ef60 comp rx=0 tx=0).stop 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 shutdown_connections 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f39a00a32f0 0x7f39a00a57b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b41037e0 0x7f39b419f3e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b4102e20 0x7f39b4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 --2- 172.21.15.104:0/1869573331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39b4101c20 0x7f39b4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 >> 172.21.15.104:0/1869573331 conn(0x7f39b40fd3b0 msgr2=0x7f39b4106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 shutdown_connections 2024-09-19T04:10:43.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:10:43.279+0000 7f39bb3a5640 1 -- 172.21.15.104:0/1869573331 wait complete. 2024-09-19T04:10:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:43 smithi112 ceph-mon[35051]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:43 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:43 smithi104 ceph-mon[34145]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:43.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:43.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:43.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:43.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:43 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:43 smithi094 ceph-mon[30972]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:43.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:43.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:43.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:43.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:43 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:43.876 DEBUG:teuthology.orchestra.run.smithi104:osd.7> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T04:10:43.878 INFO:tasks.cephadm:Deploying osd.8 on smithi112 with /dev/vg_nvme/lv_2... 2024-09-19T04:10:43.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:10:44.128 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:10:44.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:44 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:44.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:44 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:44.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:44 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:45.340 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:45 smithi112 ceph-mon[35051]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:45.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:45 smithi104 ceph-mon[34145]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:45 smithi094 ceph-mon[30972]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:45.855 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:10:45 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:10:45.589+0000 7fe4d9b68640 -1 osd.7 0 log_to_monitors true 2024-09-19T04:10:46.312 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:10:46.338 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-09-19T04:10:46.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:46 smithi104 ceph-mon[34145]: from='osd.7 [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:10:46.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:46 smithi104 ceph-mon[34145]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:10:46.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:46 smithi112 ceph-mon[35051]: from='osd.7 [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:10:46.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:46 smithi112 ceph-mon[35051]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:10:46.586 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:10:46.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:46 smithi094 ceph-mon[30972]: from='osd.7 [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:10:46.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:46 smithi094 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:10:47.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.004+0000 7f82dfb0e640 1 -- 172.21.15.112:0/3503213706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8101cd0 msgr2=0x7f82d8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:47.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.004+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/3503213706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8101cd0 0x7f82d8102150 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009900 tx=0x7f82d402a410 comp rx=0 tx=0).stop 2024-09-19T04:10:47.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.004+0000 7f82dfb0e640 1 -- 172.21.15.112:0/3503213706 shutdown_connections 2024-09-19T04:10:47.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.004+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/3503213706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d8102690 0x7f82d8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:47.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.004+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/3503213706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8101cd0 0x7f82d8102150 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.004+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/3503213706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82d8100ad0 0x7f82d8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.004+0000 7f82dfb0e640 1 -- 172.21.15.112:0/3503213706 >> 172.21.15.112:0/3503213706 conn(0x7f82d80fc280 msgr2=0x7f82d80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.004+0000 7f82dfb0e640 1 -- 172.21.15.112:0/3503213706 shutdown_connections 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.004+0000 7f82dfb0e640 1 -- 172.21.15.112:0/3503213706 wait complete. 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dfb0e640 1 Processor -- start 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dfb0e640 1 -- start start 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dfb0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82d8100ad0 0x7f82d8196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dfb0e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d8101cd0 0x7f82d8197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dfb0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8102690 0x7f82d819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dfb0e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82d810c630 con 0x7f82d8100ad0 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dfb0e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82d810c4b0 con 0x7f82d8101cd0 2024-09-19T04:10:47.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dfb0e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82d810c7b0 con 0x7f82d8102690 2024-09-19T04:10:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82de0ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8102690 0x7f82d819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82de0ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8102690 0x7f82d819e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50320/0 (socket says 172.21.15.112:50320) 2024-09-19T04:10:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82de0ab640 1 -- 172.21.15.112:0/1032287936 learned_addr learned my addr 172.21.15.112:0/1032287936 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:10:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dd0a9640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d8101cd0 0x7f82d8197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.005+0000 7f82dd8aa640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82d8100ad0 0x7f82d8196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82de0ab640 1 -- 172.21.15.112:0/1032287936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d8101cd0 msgr2=0x7f82d8197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82de0ab640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d8101cd0 0x7f82d8197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82de0ab640 1 -- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82d8100ad0 msgr2=0x7f82d8196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82de0ab640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82d8100ad0 0x7f82d8196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82de0ab640 1 -- 172.21.15.112:0/1032287936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d819e950 con 0x7f82d8102690 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82de0ab640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8102690 0x7f82d819e2b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009fd0 tx=0x7f82d000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82caffd640 1 -- 172.21.15.112:0/1032287936 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0017070 con 0x7f82d8102690 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82caffd640 1 -- 172.21.15.112:0/1032287936 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82d0003e20 con 0x7f82d8102690 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d819ec40 con 0x7f82d8102690 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82caffd640 1 -- 172.21.15.112:0/1032287936 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0010460 con 0x7f82d8102690 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.006+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d819f130 con 0x7f82d8102690 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.007+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d8107f60 con 0x7f82d8102690 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.009+0000 7f82caffd640 1 -- 172.21.15.112:0/1032287936 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f82d0003970 con 0x7f82d8102690 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.010+0000 7f82caffd640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f82c40a3240 0x7f82c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.010+0000 7f82caffd640 1 -- 172.21.15.112:0/1032287936 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5101+0+0 (secure 0 0 0) 0x7f82d00ce770 con 0x7f82d8102690 2024-09-19T04:10:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.010+0000 7f82dd8aa640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f82c40a3240 0x7f82c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:47.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.010+0000 7f82dd8aa640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f82c40a3240 0x7f82c40a5700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82cc005fd0 tx=0x7f82cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:47.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.014+0000 7f82caffd640 1 -- 172.21.15.112:0/1032287936 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f82d009c120 con 0x7f82d8102690 2024-09-19T04:10:47.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:10:47.196+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f82d8062630 con 0x7f82c40a3240 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='osd.7 [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:47.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:47.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:10:47.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:10:47.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:47 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:10:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='osd.7 [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:10:47.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:10:47.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:47 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:47.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:10:47.297+0000 7fe4d52c8640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T04:10:47.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:10:47.306+0000 7fe4d1902640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:10:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='osd.7 [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:47.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:10:47.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:10:47.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:10:47.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:10:47.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:47 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:48 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi104 2024-09-19T04:10:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:48 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-19T04:10:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:48 smithi112 ceph-mon[35051]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:48 smithi112 ceph-mon[35051]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:10:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:48 smithi112 ceph-mon[35051]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T04:10:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:48 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:48 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:48.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:48 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi104 2024-09-19T04:10:48.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:48 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-19T04:10:48.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:48 smithi094 ceph-mon[30972]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:48.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:48 smithi094 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:10:48.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:48 smithi094 ceph-mon[30972]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T04:10:48.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:48 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:48.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:48 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:48 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi104 2024-09-19T04:10:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:48 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-19T04:10:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:48 smithi104 ceph-mon[34145]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:48 smithi104 ceph-mon[34145]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:10:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:48 smithi104 ceph-mon[34145]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T04:10:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:48 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:48 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:49 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:10:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:49 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:10:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:49 smithi112 ceph-mon[35051]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:49 smithi112 ceph-mon[35051]: osd.7 [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] boot 2024-09-19T04:10:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:49 smithi112 ceph-mon[35051]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T04:10:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:49.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:49 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:10:49.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:49 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:10:49.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:49 smithi094 ceph-mon[30972]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:49.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:49.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:49 smithi094 ceph-mon[30972]: osd.7 [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] boot 2024-09-19T04:10:49.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:49 smithi094 ceph-mon[30972]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T04:10:49.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:49 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:10:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:49 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:10:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:49 smithi104 ceph-mon[34145]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:10:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:49 smithi104 ceph-mon[34145]: osd.7 [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] boot 2024-09-19T04:10:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:49 smithi104 ceph-mon[34145]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T04:10:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:10:50.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:50 smithi112 ceph-mon[35051]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T04:10:50.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:50 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/649005299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ed7d18-c753-4ddc-b1cc-518b5c67923d"}]: dispatch 2024-09-19T04:10:50.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:50 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ed7d18-c753-4ddc-b1cc-518b5c67923d"}]: dispatch 2024-09-19T04:10:50.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:50 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4ed7d18-c753-4ddc-b1cc-518b5c67923d"}]': finished 2024-09-19T04:10:50.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:50 smithi112 ceph-mon[35051]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T04:10:50.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:50 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:10:50.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:50 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/193141971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:10:50.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:50 smithi094 ceph-mon[30972]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T04:10:50.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:50 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/649005299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ed7d18-c753-4ddc-b1cc-518b5c67923d"}]: dispatch 2024-09-19T04:10:50.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:50 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ed7d18-c753-4ddc-b1cc-518b5c67923d"}]: dispatch 2024-09-19T04:10:50.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:50 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4ed7d18-c753-4ddc-b1cc-518b5c67923d"}]': finished 2024-09-19T04:10:50.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:50 smithi094 ceph-mon[30972]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T04:10:50.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:50 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:10:50.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:50 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/193141971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:10:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:50 smithi104 ceph-mon[34145]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T04:10:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:50 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/649005299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ed7d18-c753-4ddc-b1cc-518b5c67923d"}]: dispatch 2024-09-19T04:10:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:50 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ed7d18-c753-4ddc-b1cc-518b5c67923d"}]: dispatch 2024-09-19T04:10:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:50 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4ed7d18-c753-4ddc-b1cc-518b5c67923d"}]': finished 2024-09-19T04:10:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:50 smithi104 ceph-mon[34145]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T04:10:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:50 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:10:50.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:50 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/193141971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:10:51.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:51 smithi094 ceph-mon[30972]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:51.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:51 smithi094 ceph-mon[30972]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T04:10:51.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:10:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:51 smithi104 ceph-mon[34145]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:51 smithi104 ceph-mon[34145]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T04:10:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:10:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:51 smithi112 ceph-mon[35051]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:51 smithi112 ceph-mon[35051]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T04:10:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:10:52.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:52 smithi112 ceph-mon[35051]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T04:10:52.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:10:52.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:52 smithi112 ceph-mon[35051]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:52.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:52 smithi094 ceph-mon[30972]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T04:10:52.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:10:52.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:52 smithi094 ceph-mon[30972]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:52.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:52 smithi104 ceph-mon[34145]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T04:10:52.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:10:52.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:52 smithi104 ceph-mon[34145]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:55.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:55 smithi112 ceph-mon[35051]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:55 smithi094 ceph-mon[30972]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:55 smithi104 ceph-mon[34145]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:57.269 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:57 smithi112 ceph-mon[35051]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:57 smithi094 ceph-mon[30972]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:57 smithi104 ceph-mon[34145]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:59 smithi112 ceph-mon[35051]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:59 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:10:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:59 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:10:59 smithi112 ceph-mon[35051]: Deploying daemon osd.8 on smithi112 2024-09-19T04:10:59.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:59 smithi094 ceph-mon[30972]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:59.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:59 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:10:59.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:59 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:59.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:10:59 smithi094 ceph-mon[30972]: Deploying daemon osd.8 on smithi112 2024-09-19T04:10:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:59 smithi104 ceph-mon[34145]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:10:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:59 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:10:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:59 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:10:59 smithi104 ceph-mon[34145]: Deploying daemon osd.8 on smithi112 2024-09-19T04:11:01.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:01 smithi112 ceph-mon[35051]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:01.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:01 smithi094 ceph-mon[30972]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:01.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:01 smithi104 ceph-mon[34145]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:03.425 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:03 smithi112 ceph-mon[35051]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:03.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:03 smithi094 ceph-mon[30972]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:03 smithi104 ceph-mon[34145]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:04.279 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:04 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:04.280 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:04 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:04.280 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:04 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:04.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:04 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:04.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:04 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:04.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:04 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:04 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:04.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:04 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:04.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:04 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:05.453 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:05 smithi112 ceph-mon[35051]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:05.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:05 smithi094 ceph-mon[30972]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:05 smithi104 ceph-mon[34145]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.752+0000 7f82caffd640 1 -- 172.21.15.112:0/1032287936 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f82d8062630 con 0x7f82c40a3240 2024-09-19T04:11:06.755 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 8 on host 'smithi112' 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f82c40a3240 msgr2=0x7f82c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f82c40a3240 0x7f82c40a5700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82cc005fd0 tx=0x7f82cc005850 comp rx=0 tx=0).stop 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8102690 msgr2=0x7f82d819e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8102690 0x7f82d819e2b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009fd0 tx=0x7f82d000b260 comp rx=0 tx=0).stop 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 shutdown_connections 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f82c40a3240 0x7f82c40a5700 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82d8102690 0x7f82d819e2b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d8101cd0 0x7f82d8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 --2- 172.21.15.112:0/1032287936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82d8100ad0 0x7f82d8196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 >> 172.21.15.112:0/1032287936 conn(0x7f82d80fc280 msgr2=0x7f82d8105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:06.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 shutdown_connections 2024-09-19T04:11:06.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:11:06.754+0000 7f82dfb0e640 1 -- 172.21.15.112:0/1032287936 wait complete. 2024-09-19T04:11:07.347 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:07 smithi112 ceph-mon[35051]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:07.347 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:07.347 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:07.375 DEBUG:teuthology.orchestra.run.smithi112:osd.8> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T04:11:07.377 INFO:tasks.cephadm:Deploying osd.9 on smithi094 with /dev/vg_nvme/lv_1... 2024-09-19T04:11:07.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:11:07.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:07 smithi094 ceph-mon[30972]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:07.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:07.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:07 smithi104 ceph-mon[34145]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:07 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:07 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:07.648 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:08.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:08.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:08.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:08 smithi112 ceph-mon[35051]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:08.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:08.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:08.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:08 smithi094 ceph-mon[30972]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:08 smithi104 ceph-mon[34145]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:09.912 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:11:09 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:11:09.848+0000 7f3e81c23640 -1 osd.8 0 log_to_monitors true 2024-09-19T04:11:10.202 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:09 smithi112 ceph-mon[35051]: from='osd.8 [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:11:10.203 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:09 smithi112 ceph-mon[35051]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:11:10.268 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:09 smithi094 ceph-mon[30972]: from='osd.8 [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:11:10.268 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:09 smithi094 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:11:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:11:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:09 smithi104 ceph-mon[34145]: from='osd.8 [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:11:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:09 smithi104 ceph-mon[34145]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:11:10.369 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_1 2024-09-19T04:11:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:11.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.086+0000 7f234b4ba640 1 -- 172.21.15.94:0/3545243737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2344102690 msgr2=0x7f2344104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:11.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.086+0000 7f234b4ba640 1 --2- 172.21.15.94:0/3545243737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2344102690 0x7f2344104a80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2334009900 tx=0x7f233402a410 comp rx=0 tx=0).stop 2024-09-19T04:11:11.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.086+0000 7f234b4ba640 1 -- 172.21.15.94:0/3545243737 shutdown_connections 2024-09-19T04:11:11.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.086+0000 7f234b4ba640 1 --2- 172.21.15.94:0/3545243737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23441078f0 0x7f2344109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:11.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.086+0000 7f234b4ba640 1 --2- 172.21.15.94:0/3545243737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2344104fc0 0x7f23441073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.086+0000 7f234b4ba640 1 --2- 172.21.15.94:0/3545243737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2344102690 0x7f2344104a80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.086+0000 7f234b4ba640 1 -- 172.21.15.94:0/3545243737 >> 172.21.15.94:0/3545243737 conn(0x7f23440fc270 msgr2=0x7f23440fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.086+0000 7f234b4ba640 1 -- 172.21.15.94:0/3545243737 shutdown_connections 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.086+0000 7f234b4ba640 1 -- 172.21.15.94:0/3545243737 wait complete. 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f234b4ba640 1 Processor -- start 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f234b4ba640 1 -- start start 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f234b4ba640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2344102690 0x7f2344197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f234b4ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2344104fc0 0x7f2344198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f234b4ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23441078f0 0x7f234419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f234b4ba640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f234410d750 con 0x7f23441078f0 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f234b4ba640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f234410d5d0 con 0x7f2344102690 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f234b4ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f234410d8d0 con 0x7f2344104fc0 2024-09-19T04:11:11.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f2349256640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2344102690 0x7f2344197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f2349a57640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23441078f0 0x7f234419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f2348a55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2344104fc0 0x7f2344198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f2349256640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2344102690 0x7f2344197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:35252/0 (socket says 172.21.15.94:35252) 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f2349256640 1 -- 172.21.15.94:0/1833619056 learned_addr learned my addr 172.21.15.94:0/1833619056 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.087+0000 7f2349a57640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23441078f0 0x7f234419f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43562/0 (socket says 172.21.15.94:43562) 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f2349256640 1 -- 172.21.15.94:0/1833619056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2344104fc0 msgr2=0x7f2344198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f2349256640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2344104fc0 0x7f2344198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f2349256640 1 -- 172.21.15.94:0/1833619056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23441078f0 msgr2=0x7f234419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f2349256640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23441078f0 0x7f234419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f2349256640 1 -- 172.21.15.94:0/1833619056 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234419fb30 con 0x7f2344102690 2024-09-19T04:11:11.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f2349256640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2344102690 0x7f2344197f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2334005940 tx=0x7f2334004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f233a7fc640 1 -- 172.21.15.94:0/1833619056 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233403f030 con 0x7f2344102690 2024-09-19T04:11:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f233a7fc640 1 -- 172.21.15.94:0/1833619056 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2334003cb0 con 0x7f2344102690 2024-09-19T04:11:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234419fdc0 con 0x7f2344102690 2024-09-19T04:11:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f233a7fc640 1 -- 172.21.15.94:0/1833619056 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334035070 con 0x7f2344102690 2024-09-19T04:11:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.088+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23441a0210 con 0x7f2344102690 2024-09-19T04:11:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.090+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23440fdd60 con 0x7f2344102690 2024-09-19T04:11:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.090+0000 7f233a7fc640 1 -- 172.21.15.94:0/1833619056 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2334042030 con 0x7f2344102690 2024-09-19T04:11:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.091+0000 7f233a7fc640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f23300a32f0 0x7f23300a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.091+0000 7f2348a55640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f23300a32f0 0x7f23300a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.091+0000 7f233a7fc640 1 -- 172.21.15.94:0/1833619056 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5549+0+0 (secure 0 0 0) 0x7f23340ef280 con 0x7f2344102690 2024-09-19T04:11:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.091+0000 7f2348a55640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f23300a32f0 0x7f23300a57b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2344199420 tx=0x7f234000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:11.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.095+0000 7f233a7fc640 1 -- 172.21.15.94:0/1833619056 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f23340bc420 con 0x7f2344102690 2024-09-19T04:11:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:11.277+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2344062630 con 0x7f23300a32f0 2024-09-19T04:11:11.278 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='osd.8 [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi112 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi112 to 5696M 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:11.280 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:11.280 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.280 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:11.280 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:11.280 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:11.280 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.284 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:11:10.921+0000 7f3e7d383640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T04:11:11.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:11:10.930+0000 7f3e799bd640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:11:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:11:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T04:11:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='osd.8 [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:11:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:11:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi112 2024-09-19T04:11:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi112 to 5696M 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:11.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='osd.8 [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi112 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:11.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi112 to 5696M 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:11.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:12.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:11 smithi094 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:11:12.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:11 smithi094 ceph-mon[30972]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T04:11:12.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:11 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:12.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:11 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:12.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:11 smithi094 ceph-mon[30972]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:12.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:11 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:11:12.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:11 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:11:12.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:11 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:11 smithi112 ceph-mon[35051]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:11:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:11 smithi112 ceph-mon[35051]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T04:11:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:11 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:11 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:11 smithi112 ceph-mon[35051]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:11 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:11:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:11 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:11:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:11 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:11 smithi104 ceph-mon[34145]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:11:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:11 smithi104 ceph-mon[34145]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T04:11:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:11 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:11 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:11 smithi104 ceph-mon[34145]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:11 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:11:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:11 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:11:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:11 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:13.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:12 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:11:13.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:12 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:11:13.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:12 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:13.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:12 smithi094 ceph-mon[30972]: osd.8 [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] boot 2024-09-19T04:11:13.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:12 smithi094 ceph-mon[30972]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T04:11:13.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:12 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:13.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:12 smithi094 ceph-mon[30972]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:13.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:12 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:11:13.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:12 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:11:13.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:12 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:13.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:12 smithi112 ceph-mon[35051]: osd.8 [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] boot 2024-09-19T04:11:13.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:12 smithi112 ceph-mon[35051]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T04:11:13.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:12 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:13.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:12 smithi112 ceph-mon[35051]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:13.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:12 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:11:13.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:12 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:11:13.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:12 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:13.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:12 smithi104 ceph-mon[34145]: osd.8 [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] boot 2024-09-19T04:11:13.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:12 smithi104 ceph-mon[34145]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T04:11:13.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:12 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:11:13.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:12 smithi104 ceph-mon[34145]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:11:14.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:13 smithi112 ceph-mon[35051]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T04:11:14.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:13 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1594648422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4dd28b1-8e3e-4871-9391-4d74b4c49b47"}]: dispatch 2024-09-19T04:11:14.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:13 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1594648422' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4dd28b1-8e3e-4871-9391-4d74b4c49b47"}]': finished 2024-09-19T04:11:14.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:13 smithi112 ceph-mon[35051]: osdmap e56: 10 total, 9 up, 10 in 2024-09-19T04:11:14.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:14.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:13 smithi094 ceph-mon[30972]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T04:11:14.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:13 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1594648422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4dd28b1-8e3e-4871-9391-4d74b4c49b47"}]: dispatch 2024-09-19T04:11:14.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:13 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1594648422' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4dd28b1-8e3e-4871-9391-4d74b4c49b47"}]': finished 2024-09-19T04:11:14.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:13 smithi094 ceph-mon[30972]: osdmap e56: 10 total, 9 up, 10 in 2024-09-19T04:11:14.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:14.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:13 smithi104 ceph-mon[34145]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T04:11:14.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:13 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1594648422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4dd28b1-8e3e-4871-9391-4d74b4c49b47"}]: dispatch 2024-09-19T04:11:14.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:13 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1594648422' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4dd28b1-8e3e-4871-9391-4d74b4c49b47"}]': finished 2024-09-19T04:11:14.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:13 smithi104 ceph-mon[34145]: osdmap e56: 10 total, 9 up, 10 in 2024-09-19T04:11:14.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:15.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:14 smithi094 ceph-mon[30972]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:15.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:14 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1899003687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:11:15.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:14 smithi094 ceph-mon[30972]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T04:11:15.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:14 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:15.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:14 smithi112 ceph-mon[35051]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:15.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:14 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1899003687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:11:15.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:14 smithi112 ceph-mon[35051]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T04:11:15.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:14 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:15.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:14 smithi104 ceph-mon[34145]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:15.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:14 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1899003687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:11:15.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:14 smithi104 ceph-mon[34145]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T04:11:15.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:14 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:16 smithi094 ceph-mon[30972]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T04:11:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:16 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:17.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:16 smithi094 ceph-mon[30972]: pgmap v191: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:11:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:16 smithi104 ceph-mon[34145]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T04:11:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:16 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:16 smithi104 ceph-mon[34145]: pgmap v191: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:11:17.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:16 smithi112 ceph-mon[35051]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T04:11:17.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:16 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:17.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:16 smithi112 ceph-mon[35051]: pgmap v191: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:11:19.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:19 smithi094 ceph-mon[30972]: pgmap v192: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T04:11:19.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:19 smithi112 ceph-mon[35051]: pgmap v192: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T04:11:19.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:19 smithi104 ceph-mon[34145]: pgmap v192: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T04:11:21.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:21 smithi112 ceph-mon[35051]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T04:11:21.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:21 smithi104 ceph-mon[34145]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T04:11:21.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:21 smithi094 ceph-mon[30972]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T04:11:23.399 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:23 smithi094 ceph-mon[30972]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:11:23.399 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:11:23.399 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:23 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:23 smithi112 ceph-mon[35051]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:11:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:11:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:23 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:23 smithi104 ceph-mon[34145]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:11:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:11:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:23 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:24.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:24 smithi094 ceph-mon[30972]: Deploying daemon osd.9 on smithi094 2024-09-19T04:11:24.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:24 smithi112 ceph-mon[35051]: Deploying daemon osd.9 on smithi094 2024-09-19T04:11:24.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:24 smithi104 ceph-mon[34145]: Deploying daemon osd.9 on smithi094 2024-09-19T04:11:25.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:25 smithi112 ceph-mon[35051]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:25.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:25 smithi094 ceph-mon[30972]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:25 smithi104 ceph-mon[34145]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:27.489 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:27 smithi094 ceph-mon[30972]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:27.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:27 smithi112 ceph-mon[35051]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:27 smithi104 ceph-mon[34145]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:29.502 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:29 smithi094 ceph-mon[30972]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:29.502 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:29.502 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:29.502 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:29 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:29.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:29 smithi112 ceph-mon[35051]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:29.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:29.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:29 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:29 smithi104 ceph-mon[34145]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:29 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:31.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:31 smithi094 ceph-mon[30972]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:31.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:31 smithi112 ceph-mon[35051]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:31.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:31 smithi104 ceph-mon[34145]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:32.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.463+0000 7f233a7fc640 1 -- 172.21.15.94:0/1833619056 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2344062630 con 0x7f23300a32f0 2024-09-19T04:11:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 9 on host 'smithi094' 2024-09-19T04:11:32.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f23300a32f0 msgr2=0x7f23300a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:32.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f23300a32f0 0x7f23300a57b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2344199420 tx=0x7f234000b410 comp rx=0 tx=0).stop 2024-09-19T04:11:32.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2344102690 msgr2=0x7f2344197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:32.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2344102690 0x7f2344197f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2334005940 tx=0x7f2334004340 comp rx=0 tx=0).stop 2024-09-19T04:11:32.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 shutdown_connections 2024-09-19T04:11:32.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f23300a32f0 0x7f23300a57b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:32.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23441078f0 0x7f234419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:32.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2344104fc0 0x7f2344198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:32.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 --2- 172.21.15.94:0/1833619056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2344102690 0x7f2344197f00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:32.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 >> 172.21.15.94:0/1833619056 conn(0x7f23440fc270 msgr2=0x7f23441030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:32.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 shutdown_connections 2024-09-19T04:11:32.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:11:32.466+0000 7f234b4ba640 1 -- 172.21.15.94:0/1833619056 wait complete. 2024-09-19T04:11:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:32 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:32 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:32 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:32 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:32 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:32 smithi112 ceph-mon[35051]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:32 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:32 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:32 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:32 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:32 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:32 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:32.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:32 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:32.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:32 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:32 smithi094 ceph-mon[30972]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:32.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:32 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:32.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:32 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:32 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:32 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:32 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:32 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:32.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:32 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:32.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:32 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:32 smithi104 ceph-mon[34145]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:32.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:32 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:32.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:32 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:32.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:32 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:33.096 DEBUG:teuthology.orchestra.run.smithi094:osd.9> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T04:11:33.099 INFO:tasks.cephadm:Deploying osd.10 on smithi104 with /dev/vg_nvme/lv_1... 2024-09-19T04:11:33.099 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:11:33.353 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:11:35.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:35 smithi094 ceph-mon[30972]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:35.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:35 smithi094 ceph-mon[30972]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:11:35.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:11:35 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:11:35.119+0000 7f17db41d640 -1 osd.9 0 log_to_monitors true 2024-09-19T04:11:35.378 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:35 smithi104 ceph-mon[34145]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:35.378 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:35 smithi104 ceph-mon[34145]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:11:35.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:35 smithi112 ceph-mon[35051]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:35.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:35 smithi112 ceph-mon[35051]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:11:35.983 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-19T04:11:36.009 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2024-09-19T04:11:36.261 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:11:36.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:36 smithi104 ceph-mon[34145]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:11:36.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:36 smithi104 ceph-mon[34145]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T04:11:36.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:36 smithi104 ceph-mon[34145]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:11:36.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:36.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:36 smithi104 ceph-mon[34145]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:11:36.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:36 smithi112 ceph-mon[35051]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:11:36.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:36 smithi112 ceph-mon[35051]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T04:11:36.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:36 smithi112 ceph-mon[35051]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:11:36.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:36.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:36 smithi112 ceph-mon[35051]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:11:36.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:36 smithi094 ceph-mon[30972]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:11:36.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:36 smithi094 ceph-mon[30972]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T04:11:36.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:36 smithi094 ceph-mon[30972]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-09-19T04:11:36.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:36.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:36 smithi094 ceph-mon[30972]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-09-19T04:11:36.748 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.737+0000 7fbc5984e640 1 -- 172.21.15.104:0/456099024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54100b10 msgr2=0x7fbc54100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:36.748 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.737+0000 7fbc5984e640 1 --2- 172.21.15.104:0/456099024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54100b10 0x7fbc54100f10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009900 tx=0x7fbc4402a410 comp rx=0 tx=0).stop 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.737+0000 7fbc5984e640 1 -- 172.21.15.104:0/456099024 shutdown_connections 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 --2- 172.21.15.104:0/456099024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc541026d0 0x7fbc54108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 --2- 172.21.15.104:0/456099024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc54101d10 0x7fbc54102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 --2- 172.21.15.104:0/456099024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54100b10 0x7fbc54100f10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 -- 172.21.15.104:0/456099024 >> 172.21.15.104:0/456099024 conn(0x7fbc540fc2a0 msgr2=0x7fbc540fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 -- 172.21.15.104:0/456099024 shutdown_connections 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 -- 172.21.15.104:0/456099024 wait complete. 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 Processor -- start 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 -- start start 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100b10 0x7fbc54196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54101d10 0x7fbc541972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc541026d0 0x7fbc5419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:36.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5410c640 con 0x7fbc54100b10 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5410c4c0 con 0x7fbc54101d10 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.738+0000 7fbc5984e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5410c7c0 con 0x7fbc541026d0 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc527fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54101d10 0x7fbc541972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc52ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100b10 0x7fbc54196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc527fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54101d10 0x7fbc541972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49144/0 (socket says 172.21.15.104:49144) 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc527fc640 1 -- 172.21.15.104:0/867337898 learned_addr learned my addr 172.21.15.104:0/867337898 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc537fe640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc541026d0 0x7fbc5419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc527fc640 1 -- 172.21.15.104:0/867337898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc541026d0 msgr2=0x7fbc5419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc527fc640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc541026d0 0x7fbc5419e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc527fc640 1 -- 172.21.15.104:0/867337898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100b10 msgr2=0x7fbc54196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc527fc640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100b10 0x7fbc54196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc527fc640 1 -- 172.21.15.104:0/867337898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5419e9c0 con 0x7fbc54101d10 2024-09-19T04:11:36.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc527fc640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54101d10 0x7fbc541972e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800d770 tx=0x7fbc4800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc5884c640 1 -- 172.21.15.104:0/867337898 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48017070 con 0x7fbc54101d10 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5419ecb0 con 0x7fbc54101d10 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc5884c640 1 -- 172.21.15.104:0/867337898 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc480054d0 con 0x7fbc54101d10 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc5884c640 1 -- 172.21.15.104:0/867337898 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48004cf0 con 0x7fbc54101d10 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.739+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5419f170 con 0x7fbc54101d10 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.740+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc54107fb0 con 0x7fbc54101d10 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.741+0000 7fbc5884c640 1 -- 172.21.15.104:0/867337898 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbc48004e90 con 0x7fbc54101d10 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.742+0000 7fbc5884c640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbc3c0a3240 0x7fbc3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.742+0000 7fbc5884c640 1 -- 172.21.15.104:0/867337898 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5981+0+0 (secure 0 0 0) 0x7fbc480ce9b0 con 0x7fbc54101d10 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.742+0000 7fbc52ffd640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbc3c0a3240 0x7fbc3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.743+0000 7fbc52ffd640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbc3c0a3240 0x7fbc3c0a5700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc44004770 tx=0x7fbc44000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.746+0000 7fbc5884c640 1 -- 172.21.15.104:0/867337898 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbc4809b9a0 con 0x7fbc54101d10 2024-09-19T04:11:36.924 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:36.923+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbc54062630 con 0x7fbc3c0a3240 2024-09-19T04:11:37.185 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:11:36 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:11:36.847+0000 7f17d6b7d640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T04:11:37.185 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:11:36 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:11:36.858+0000 7f17cf1af640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi094 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.467 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi094 to 4272M 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:37.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:37 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T04:11:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi094 2024-09-19T04:11:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi094 to 4272M 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:11:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:37.537 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:37 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T04:11:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi094 2024-09-19T04:11:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi094 to 4272M 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533]' entity='osd.9' 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:37.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:37 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:38.296 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:38 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:11:38.296 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:38 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:11:38.297 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:38 smithi104 ceph-mon[34145]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:38.297 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:38 smithi104 ceph-mon[34145]: osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] boot 2024-09-19T04:11:38.297 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:38 smithi104 ceph-mon[34145]: osdmap e61: 10 total, 10 up, 10 in 2024-09-19T04:11:38.297 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:38 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:38.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:38 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:11:38.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:38 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:11:38.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:38 smithi112 ceph-mon[35051]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:38.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:38 smithi112 ceph-mon[35051]: osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] boot 2024-09-19T04:11:38.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:38 smithi112 ceph-mon[35051]: osdmap e61: 10 total, 10 up, 10 in 2024-09-19T04:11:38.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:38 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:38.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:38 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:11:38.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:38 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:11:38.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:38 smithi094 ceph-mon[30972]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:38.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:38 smithi094 ceph-mon[30972]: osd.9 [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] boot 2024-09-19T04:11:38.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:38 smithi094 ceph-mon[30972]: osdmap e61: 10 total, 10 up, 10 in 2024-09-19T04:11:38.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:38 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:11:39.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:39 smithi104 ceph-mon[34145]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:39.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:39 smithi104 ceph-mon[34145]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T04:11:39.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:39 smithi112 ceph-mon[35051]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:39 smithi112 ceph-mon[35051]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T04:11:39.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:39 smithi094 ceph-mon[30972]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:11:39.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:39 smithi094 ceph-mon[30972]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T04:11:40.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:40 smithi112 ceph-mon[35051]: from='client.? 172.21.15.104:0/1288167040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a22e7-d51e-499d-8154-301d97f3790f"}]: dispatch 2024-09-19T04:11:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:40 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a22e7-d51e-499d-8154-301d97f3790f"}]: dispatch 2024-09-19T04:11:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:40 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab8a22e7-d51e-499d-8154-301d97f3790f"}]': finished 2024-09-19T04:11:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:40 smithi112 ceph-mon[35051]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T04:11:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:40 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:11:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:40 smithi112 ceph-mon[35051]: from='client.? 172.21.15.104:0/3856165301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:11:40.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:40 smithi094 ceph-mon[30972]: from='client.? 172.21.15.104:0/1288167040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a22e7-d51e-499d-8154-301d97f3790f"}]: dispatch 2024-09-19T04:11:40.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:40 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a22e7-d51e-499d-8154-301d97f3790f"}]: dispatch 2024-09-19T04:11:40.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:40 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab8a22e7-d51e-499d-8154-301d97f3790f"}]': finished 2024-09-19T04:11:40.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:40 smithi094 ceph-mon[30972]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T04:11:40.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:40 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:11:40.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:40 smithi094 ceph-mon[30972]: from='client.? 172.21.15.104:0/3856165301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:11:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:40 smithi104 ceph-mon[34145]: from='client.? 172.21.15.104:0/1288167040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a22e7-d51e-499d-8154-301d97f3790f"}]: dispatch 2024-09-19T04:11:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:40 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a22e7-d51e-499d-8154-301d97f3790f"}]: dispatch 2024-09-19T04:11:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:40 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab8a22e7-d51e-499d-8154-301d97f3790f"}]': finished 2024-09-19T04:11:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:40 smithi104 ceph-mon[34145]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T04:11:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:40 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:11:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:40 smithi104 ceph-mon[34145]: from='client.? 172.21.15.104:0/3856165301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:11:41.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:41 smithi112 ceph-mon[35051]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:41.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:41 smithi094 ceph-mon[30972]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:41 smithi104 ceph-mon[34145]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:43.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:43 smithi112 ceph-mon[35051]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:43.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:43 smithi094 ceph-mon[30972]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:43 smithi104 ceph-mon[34145]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:45.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:45 smithi112 ceph-mon[35051]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:45.556 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:45 smithi104 ceph-mon[34145]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:45 smithi094 ceph-mon[30972]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:47.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:47 smithi112 ceph-mon[35051]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:47 smithi094 ceph-mon[30972]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:47 smithi104 ceph-mon[34145]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:48.510 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:48 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:11:48.510 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:48 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:48.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:48 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:11:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:48 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:48.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:48 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:11:48.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:48 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:49.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:49 smithi104 ceph-mon[34145]: Deploying daemon osd.10 on smithi104 2024-09-19T04:11:49.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:49 smithi104 ceph-mon[34145]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:49.492 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:49 smithi094 ceph-mon[30972]: Deploying daemon osd.10 on smithi104 2024-09-19T04:11:49.492 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:49 smithi094 ceph-mon[30972]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:49.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:49 smithi112 ceph-mon[35051]: Deploying daemon osd.10 on smithi104 2024-09-19T04:11:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:49 smithi112 ceph-mon[35051]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:51.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:51 smithi104 ceph-mon[34145]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:51.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:51 smithi112 ceph-mon[35051]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:51.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:51 smithi094 ceph-mon[30972]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:53.462 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:53 smithi104 ceph-mon[34145]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:53.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:53 smithi112 ceph-mon[35051]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:53.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:53 smithi094 ceph-mon[30972]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:54.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:54 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:54.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:54 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:54.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:54 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:54.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:54 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:54.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:54 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:54.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:54 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:54.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:54.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:54.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:55.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:55 smithi104 ceph-mon[34145]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:55.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:55 smithi112 ceph-mon[35051]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:55 smithi094 ceph-mon[30972]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:57.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:57 smithi104 ceph-mon[34145]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:57.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:57.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:57.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:57.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:57.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:57 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:57.352 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.350+0000 7fbc5884c640 1 -- 172.21.15.104:0/867337898 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbc54062630 con 0x7fbc3c0a3240 2024-09-19T04:11:57.352 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 10 on host 'smithi104' 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbc3c0a3240 msgr2=0x7fbc3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbc3c0a3240 0x7fbc3c0a5700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc44004770 tx=0x7fbc44000bc0 comp rx=0 tx=0).stop 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54101d10 msgr2=0x7fbc541972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54101d10 0x7fbc541972e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800d770 tx=0x7fbc4800dc30 comp rx=0 tx=0).stop 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 shutdown_connections 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbc3c0a3240 0x7fbc3c0a5700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc541026d0 0x7fbc5419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc54101d10 0x7fbc541972e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 --2- 172.21.15.104:0/867337898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100b10 0x7fbc54196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 >> 172.21.15.104:0/867337898 conn(0x7fbc540fc2a0 msgr2=0x7fbc54105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:57.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 shutdown_connections 2024-09-19T04:11:57.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-19T04:11:57.352+0000 7fbc5984e640 1 -- 172.21.15.104:0/867337898 wait complete. 2024-09-19T04:11:57.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:57 smithi112 ceph-mon[35051]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:57 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:57 smithi094 ceph-mon[30972]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:57 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:57.962 DEBUG:teuthology.orchestra.run.smithi104:osd.10> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T04:11:57.965 INFO:tasks.cephadm:Deploying osd.11 on smithi112 with /dev/vg_nvme/lv_1... 2024-09-19T04:11:57.965 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:11:58.219 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:11:58.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:58.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:58.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:58.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:58.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:58.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:58.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:11:59.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:11:59 smithi104 ceph-mon[34145]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:59.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:11:59 smithi112 ceph-mon[35051]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:11:59.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:11:59 smithi094 ceph-mon[30972]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:12:00.225 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:11:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:11:59.926+0000 7f57fd934640 -1 osd.10 0 log_to_monitors true 2024-09-19T04:12:00.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:00 smithi104 ceph-mon[34145]: from='osd.10 [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:12:00.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:00 smithi104 ceph-mon[34145]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:12:00.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:00 smithi112 ceph-mon[35051]: from='osd.10 [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:12:00.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:00 smithi112 ceph-mon[35051]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:12:00.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:00 smithi094 ceph-mon[30972]: from='osd.10 [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:12:00.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:00 smithi094 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:12:00.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-19T04:12:00.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-09-19T04:12:00.902 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:12:01.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:12:01.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:12:01.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T04:12:01.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:01.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='osd.10 [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:01.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.370+0000 7fd4c1657640 1 -- 172.21.15.112:0/312005838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc1026a0 msgr2=0x7fd4bc104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:01.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.370+0000 7fd4c1657640 1 --2- 172.21.15.112:0/312005838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc1026a0 0x7fd4bc104a90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009900 tx=0x7fd4ac02a410 comp rx=0 tx=0).stop 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.370+0000 7fd4c1657640 1 -- 172.21.15.112:0/312005838 shutdown_connections 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.370+0000 7fd4c1657640 1 --2- 172.21.15.112:0/312005838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4bc107900 0x7fd4bc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.370+0000 7fd4c1657640 1 --2- 172.21.15.112:0/312005838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4bc104fd0 0x7fd4bc1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.370+0000 7fd4c1657640 1 --2- 172.21.15.112:0/312005838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc1026a0 0x7fd4bc104a90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.370+0000 7fd4c1657640 1 -- 172.21.15.112:0/312005838 >> 172.21.15.112:0/312005838 conn(0x7fd4bc0fc2a0 msgr2=0x7fd4bc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.370+0000 7fd4c1657640 1 -- 172.21.15.112:0/312005838 shutdown_connections 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.370+0000 7fd4c1657640 1 -- 172.21.15.112:0/312005838 wait complete. 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4c1657640 1 Processor -- start 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4c1657640 1 -- start start 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4c1657640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4bc1026a0 0x7fd4bc197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4c1657640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc104fd0 0x7fd4bc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4c1657640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4bc107900 0x7fd4bc19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4c1657640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc10d7f0 con 0x7fd4bc1026a0 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4c1657640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc10d670 con 0x7fd4bc107900 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4c1657640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc10d970 con 0x7fd4bc104fd0 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4ba7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc104fd0 0x7fd4bc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4ba7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc104fd0 0x7fd4bc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59186/0 (socket says 172.21.15.112:59186) 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4ba7fc640 1 -- 172.21.15.112:0/4135736008 learned_addr learned my addr 172.21.15.112:0/4135736008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.371+0000 7fd4bb7fe640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4bc107900 0x7fd4bc19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4baffd640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4bc1026a0 0x7fd4bc197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4ba7fc640 1 -- 172.21.15.112:0/4135736008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4bc107900 msgr2=0x7fd4bc19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4ba7fc640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4bc107900 0x7fd4bc19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4ba7fc640 1 -- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4bc1026a0 msgr2=0x7fd4bc197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4ba7fc640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4bc1026a0 0x7fd4bc197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4ba7fc640 1 -- 172.21.15.112:0/4135736008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc19fae0 con 0x7fd4bc104fd0 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4ba7fc640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc104fd0 0x7fd4bc1983f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b450 tx=0x7fd4b000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4a3fff640 1 -- 172.21.15.112:0/4135736008 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0015020 con 0x7fd4bc104fd0 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4a3fff640 1 -- 172.21.15.112:0/4135736008 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4b0011d40 con 0x7fd4bc104fd0 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4bc19fdd0 con 0x7fd4bc104fd0 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4a3fff640 1 -- 172.21.15.112:0/4135736008 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000f340 con 0x7fd4bc104fd0 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.372+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4bc1a0300 con 0x7fd4bc104fd0 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.373+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4bc0fdd90 con 0x7fd4bc104fd0 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.374+0000 7fd4a3fff640 1 -- 172.21.15.112:0/4135736008 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd4b0018020 con 0x7fd4bc104fd0 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.375+0000 7fd4a3fff640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd4a40a3270 0x7fd4a40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:01.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.375+0000 7fd4a3fff640 1 -- 172.21.15.112:0/4135736008 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6414+0+0 (secure 0 0 0) 0x7fd4b00cec00 con 0x7fd4bc104fd0 2024-09-19T04:12:01.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.375+0000 7fd4baffd640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd4a40a3270 0x7fd4a40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:01.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.375+0000 7fd4baffd640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd4a40a3270 0x7fd4a40a5730 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac000c00 tx=0x7fd4ac0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:01.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.379+0000 7fd4a3fff640 1 -- 172.21.15.112:0/4135736008 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd4b00d5080 con 0x7fd4bc104fd0 2024-09-19T04:12:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:01.555+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd4bc062630 con 0x7fd4a40a3270 2024-09-19T04:12:01.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='osd.10 [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:01.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:01.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:12:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:12:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T04:12:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='osd.10 [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:12:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-19T04:12:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:01.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:02.354 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:12:02.056+0000 7f57f9094640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T04:12:02.354 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:12:02.073+0000 7f57f16c6640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi104 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:02.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:02 smithi104 ceph-mon[34145]: from='osd.10 ' entity='osd.10' 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi104 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:02 smithi112 ceph-mon[35051]: from='osd.10 ' entity='osd.10' 2024-09-19T04:12:02.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi104 2024-09-19T04:12:02.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-19T04:12:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-19T04:12:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T04:12:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:02 smithi094 ceph-mon[30972]: from='osd.10 ' entity='osd.10' 2024-09-19T04:12:03.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:03 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:12:03.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:03 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:12:03.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:03 smithi112 ceph-mon[35051]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:03.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:03 smithi112 ceph-mon[35051]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:12:03.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:03 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:03.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:03 smithi112 ceph-mon[35051]: osd.10 [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] boot 2024-09-19T04:12:03.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:03 smithi112 ceph-mon[35051]: osdmap e66: 11 total, 11 up, 11 in 2024-09-19T04:12:03.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:03 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:03.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:03 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:12:03.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:03 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:12:03.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:03 smithi094 ceph-mon[30972]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:03.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:03 smithi094 ceph-mon[30972]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:12:03.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:03 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:03.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:03 smithi094 ceph-mon[30972]: osd.10 [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] boot 2024-09-19T04:12:03.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:03 smithi094 ceph-mon[30972]: osdmap e66: 11 total, 11 up, 11 in 2024-09-19T04:12:03.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:03 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:03 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:12:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:03 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:12:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:03 smithi104 ceph-mon[34145]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:03 smithi104 ceph-mon[34145]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:12:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:03 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:03 smithi104 ceph-mon[34145]: osd.10 [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] boot 2024-09-19T04:12:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:03 smithi104 ceph-mon[34145]: osdmap e66: 11 total, 11 up, 11 in 2024-09-19T04:12:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:03 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:12:04.428 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:04 smithi112 ceph-mon[35051]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T04:12:04.428 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:04 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/2598280858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47a94e97-ed76-4aa0-918c-a813e88ca539"}]: dispatch 2024-09-19T04:12:04.428 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:04 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47a94e97-ed76-4aa0-918c-a813e88ca539"}]: dispatch 2024-09-19T04:12:04.428 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:04 smithi112 ceph-mon[35051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47a94e97-ed76-4aa0-918c-a813e88ca539"}]': finished 2024-09-19T04:12:04.429 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:04 smithi112 ceph-mon[35051]: osdmap e68: 12 total, 11 up, 12 in 2024-09-19T04:12:04.429 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:04 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:04.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:04 smithi094 ceph-mon[30972]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T04:12:04.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:04 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/2598280858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47a94e97-ed76-4aa0-918c-a813e88ca539"}]: dispatch 2024-09-19T04:12:04.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:04 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47a94e97-ed76-4aa0-918c-a813e88ca539"}]: dispatch 2024-09-19T04:12:04.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:04 smithi094 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47a94e97-ed76-4aa0-918c-a813e88ca539"}]': finished 2024-09-19T04:12:04.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:04 smithi094 ceph-mon[30972]: osdmap e68: 12 total, 11 up, 12 in 2024-09-19T04:12:04.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:04 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:04 smithi104 ceph-mon[34145]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T04:12:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:04 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/2598280858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47a94e97-ed76-4aa0-918c-a813e88ca539"}]: dispatch 2024-09-19T04:12:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:04 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47a94e97-ed76-4aa0-918c-a813e88ca539"}]: dispatch 2024-09-19T04:12:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:04 smithi104 ceph-mon[34145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47a94e97-ed76-4aa0-918c-a813e88ca539"}]': finished 2024-09-19T04:12:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:04 smithi104 ceph-mon[34145]: osdmap e68: 12 total, 11 up, 12 in 2024-09-19T04:12:04.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:04 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:05.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:05 smithi112 ceph-mon[35051]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:05.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:05 smithi112 ceph-mon[35051]: from='client.? 172.21.15.112:0/1458181883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:05.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:05 smithi112 ceph-mon[35051]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T04:12:05.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:05 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:05.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:05 smithi094 ceph-mon[30972]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:05 smithi094 ceph-mon[30972]: from='client.? 172.21.15.112:0/1458181883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:05 smithi094 ceph-mon[30972]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T04:12:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:05 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:05 smithi104 ceph-mon[34145]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:05 smithi104 ceph-mon[34145]: from='client.? 172.21.15.112:0/1458181883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:05 smithi104 ceph-mon[34145]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T04:12:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:05 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:06 smithi112 ceph-mon[35051]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T04:12:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:06 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:06 smithi112 ceph-mon[35051]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-09-19T04:12:07.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:06 smithi094 ceph-mon[30972]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T04:12:07.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:06 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:07.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:06 smithi094 ceph-mon[30972]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-09-19T04:12:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:06 smithi104 ceph-mon[34145]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T04:12:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:06 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:06 smithi104 ceph-mon[34145]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-09-19T04:12:09.263 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:09 smithi112 ceph-mon[35051]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-19T04:12:09.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:09 smithi094 ceph-mon[30972]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-19T04:12:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:09 smithi104 ceph-mon[34145]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-19T04:12:11.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:11 smithi112 ceph-mon[35051]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T04:12:11.424 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:11 smithi104 ceph-mon[34145]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T04:12:11.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:11 smithi094 ceph-mon[30972]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T04:12:13.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:13 smithi112 ceph-mon[35051]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:12:13.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:12:13.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:13 smithi094 ceph-mon[30972]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:12:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:12:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:13 smithi104 ceph-mon[34145]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:12:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:12:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:14.308 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:14 smithi112 ceph-mon[35051]: Deploying daemon osd.11 on smithi112 2024-09-19T04:12:14.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:14 smithi094 ceph-mon[30972]: Deploying daemon osd.11 on smithi112 2024-09-19T04:12:14.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:14 smithi104 ceph-mon[34145]: Deploying daemon osd.11 on smithi112 2024-09-19T04:12:15.481 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:15 smithi112 ceph-mon[35051]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:15.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:15 smithi094 ceph-mon[30972]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:15 smithi104 ceph-mon[34145]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:17.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:17 smithi112 ceph-mon[35051]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:17.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:17 smithi094 ceph-mon[30972]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:17 smithi104 ceph-mon[34145]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:18.491 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:18.491 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:18.491 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:18 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:18.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:18.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:18.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:18 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:18 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:19.411 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:19 smithi112 ceph-mon[35051]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:19.492 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:19 smithi094 ceph-mon[30972]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:19.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:19 smithi104 ceph-mon[34145]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:21.514 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:21 smithi112 ceph-mon[35051]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:21.514 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:21.514 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:21.515 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:21.515 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:21.515 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:21.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.551+0000 7fd4a3fff640 1 -- 172.21.15.112:0/4135736008 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd4bc062630 con 0x7fd4a40a3270 2024-09-19T04:12:21.554 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 11 on host 'smithi112' 2024-09-19T04:12:21.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd4a40a3270 msgr2=0x7fd4a40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd4a40a3270 0x7fd4a40a5730 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac000c00 tx=0x7fd4ac0314a0 comp rx=0 tx=0).stop 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc104fd0 msgr2=0x7fd4bc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc104fd0 0x7fd4bc1983f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b450 tx=0x7fd4b000b910 comp rx=0 tx=0).stop 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 shutdown_connections 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd4a40a3270 0x7fd4a40a5730 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4bc107900 0x7fd4bc19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4bc104fd0 0x7fd4bc1983f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 --2- 172.21.15.112:0/4135736008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4bc1026a0 0x7fd4bc197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 >> 172.21.15.112:0/4135736008 conn(0x7fd4bc0fc2a0 msgr2=0x7fd4bc102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 shutdown_connections 2024-09-19T04:12:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:21.554+0000 7fd4c1657640 1 -- 172.21.15.112:0/4135736008 wait complete. 2024-09-19T04:12:21.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:21 smithi094 ceph-mon[30972]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:21.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:21.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:21.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:21.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:21.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:21 smithi104 ceph-mon[34145]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:21.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:22.195 DEBUG:teuthology.orchestra.run.smithi112:osd.11> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T04:12:22.198 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T04:12:22.198 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:12:22.452 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:22.492 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:22.492 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:22.492 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:22.573 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:22.573 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:22.573 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:22.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:22.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:22.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 -- 172.21.15.94:0/3960995423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c101d10 msgr2=0x7f611c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 --2- 172.21.15.94:0/3960995423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c101d10 0x7f611c102190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f610c009900 tx=0x7f610c02a410 comp rx=0 tx=0).stop 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 -- 172.21.15.94:0/3960995423 shutdown_connections 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 --2- 172.21.15.94:0/3960995423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611c1026d0 0x7f611c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 --2- 172.21.15.94:0/3960995423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c101d10 0x7f611c102190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 --2- 172.21.15.94:0/3960995423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c100b10 0x7f611c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 -- 172.21.15.94:0/3960995423 >> 172.21.15.94:0/3960995423 conn(0x7f611c0fc2a0 msgr2=0x7f611c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 -- 172.21.15.94:0/3960995423 shutdown_connections 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 -- 172.21.15.94:0/3960995423 wait complete. 2024-09-19T04:12:22.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 Processor -- start 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 -- start start 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.880+0000 7f6124594640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c100b10 0x7f611c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6124594640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c101d10 0x7f611c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6124594640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611c1026d0 0x7f611c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6124594640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f611c10c740 con 0x7f611c100b10 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6124594640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f611c10c5c0 con 0x7f611c1026d0 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6124594640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f611c10c8c0 con 0x7f611c101d10 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6122330640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c100b10 0x7f611c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6122330640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c100b10 0x7f611c06b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57322/0 (socket says 172.21.15.94:57322) 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6122330640 1 -- 172.21.15.94:0/1544645844 learned_addr learned my addr 172.21.15.94:0/1544645844 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6121b2f640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c101d10 0x7f611c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:22.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6122330640 1 -- 172.21.15.94:0/1544645844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c101d10 msgr2=0x7f611c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6122330640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c101d10 0x7f611c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6122330640 1 -- 172.21.15.94:0/1544645844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611c1026d0 msgr2=0x7f611c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6122330640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611c1026d0 0x7f611c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6122330640 1 -- 172.21.15.94:0/1544645844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611c1a1e60 con 0x7f611c100b10 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6122330640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c100b10 0x7f611c06b2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f611800b450 tx=0x7f611800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f61137fe640 1 -- 172.21.15.94:0/1544645844 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118015020 con 0x7f611c100b10 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f61137fe640 1 -- 172.21.15.94:0/1544645844 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6118011d40 con 0x7f611c100b10 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611c1a20f0 con 0x7f611c100b10 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f61137fe640 1 -- 172.21.15.94:0/1544645844 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611800f340 con 0x7f611c100b10 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.881+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611c1a2680 con 0x7f611c100b10 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.882+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f611c108050 con 0x7f611c100b10 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.883+0000 7f61137fe640 1 -- 172.21.15.94:0/1544645844 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6118018020 con 0x7f611c100b10 2024-09-19T04:12:22.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.884+0000 7f61137fe640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61080a3270 0x7f61080a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:22.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.884+0000 7f61137fe640 1 -- 172.21.15.94:0/1544645844 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6809+0+0 (secure 0 0 0) 0x7f61180ced90 con 0x7f611c100b10 2024-09-19T04:12:22.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.884+0000 7f6121b2f640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61080a3270 0x7f61080a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:22.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.884+0000 7f6121b2f640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61080a3270 0x7f61080a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f610c000c00 tx=0x7f610c005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:22.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:22.888+0000 7f61137fe640 1 -- 172.21.15.94:0/1544645844 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f611801f020 con 0x7f611c100b10 2024-09-19T04:12:23.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.053+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f611c02da30 con 0x7f611c100b10 2024-09-19T04:12:23.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.054+0000 7f61137fe640 1 -- 172.21.15.94:0/1544645844 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f611800d040 con 0x7f611c100b10 2024-09-19T04:12:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61080a3270 msgr2=0x7f61080a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61080a3270 0x7f61080a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f610c000c00 tx=0x7f610c005e40 comp rx=0 tx=0).stop 2024-09-19T04:12:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c100b10 msgr2=0x7f611c06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c100b10 0x7f611c06b2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f611800b450 tx=0x7f611800b910 comp rx=0 tx=0).stop 2024-09-19T04:12:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 shutdown_connections 2024-09-19T04:12:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61080a3270 0x7f61080a5730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611c1026d0 0x7f611c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c101d10 0x7f611c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 --2- 172.21.15.94:0/1544645844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611c100b10 0x7f611c06b2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 >> 172.21.15.94:0/1544645844 conn(0x7f611c0fc2a0 msgr2=0x7f611c105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 shutdown_connections 2024-09-19T04:12:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:23.056+0000 7f6124594640 1 -- 172.21.15.94:0/1544645844 wait complete. 2024-09-19T04:12:23.451 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:23 smithi112 ceph-mon[35051]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:23.451 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:23 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1544645844' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:12:23.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:23 smithi094 ceph-mon[30972]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:23.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:23 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1544645844' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:12:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:23 smithi104 ceph-mon[34145]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:23 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1544645844' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:12:24.066 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726719122,"num_in_osds":12,"osd_in_since":1726719123,"num_remapped_pgs":0} 2024-09-19T04:12:24.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:24 smithi094 ceph-mon[30972]: from='osd.11 [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:12:24.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:24 smithi094 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:12:24.531 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:24 smithi112 ceph-mon[35051]: from='osd.11 [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:12:24.531 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:24 smithi112 ceph-mon[35051]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:12:24.532 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:12:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:12:24.133+0000 7f816edb4640 -1 osd.11 0 log_to_monitors true 2024-09-19T04:12:24.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:24 smithi104 ceph-mon[34145]: from='osd.11 [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:12:24.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:24 smithi104 ceph-mon[34145]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:12:25.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:12:25.315 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:25.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:25.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:12:25.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='osd.11 [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:25.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:25 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:12:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T04:12:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='osd.11 [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:12:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:12:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:25.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:25.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:12:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T04:12:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='osd.11 [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:25.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:25 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:25.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.762+0000 7f3c5243a640 1 -- 172.21.15.94:0/1670938759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 msgr2=0x7f3c4c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.762+0000 7f3c5243a640 1 --2- 172.21.15.94:0/1670938759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 0x7f3c4c108f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009ca0 tx=0x7f3c4402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.762+0000 7f3c5243a640 1 -- 172.21.15.94:0/1670938759 shutdown_connections 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.762+0000 7f3c5243a640 1 --2- 172.21.15.94:0/1670938759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 0x7f3c4c108f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.762+0000 7f3c5243a640 1 --2- 172.21.15.94:0/1670938759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c101cd0 0x7f3c4c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.762+0000 7f3c5243a640 1 --2- 172.21.15.94:0/1670938759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c100ad0 0x7f3c4c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.762+0000 7f3c5243a640 1 -- 172.21.15.94:0/1670938759 >> 172.21.15.94:0/1670938759 conn(0x7f3c4c0fc280 msgr2=0x7f3c4c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.762+0000 7f3c5243a640 1 -- 172.21.15.94:0/1670938759 shutdown_connections 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.762+0000 7f3c5243a640 1 -- 172.21.15.94:0/1670938759 wait complete. 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c5243a640 1 Processor -- start 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c5243a640 1 -- start start 2024-09-19T04:12:25.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c5243a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c100ad0 0x7f3c4c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c5243a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c101cd0 0x7f3c4c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c5243a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 0x7f3c4c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c5243a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c10c6b0 con 0x7f3c4c102690 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c5243a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c10c530 con 0x7f3c4c101cd0 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c5243a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c10c830 con 0x7f3c4c100ad0 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c509d7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 0x7f3c4c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c100ad0 0x7f3c4c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c100ad0 0x7f3c4c19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48248/0 (socket says 172.21.15.94:48248) 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c509d7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 0x7f3c4c1a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57346/0 (socket says 172.21.15.94:57346) 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c4bfff640 1 -- 172.21.15.94:0/3830873468 learned_addr learned my addr 172.21.15.94:0/3830873468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:25.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c4b7fe640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c101cd0 0x7f3c4c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c509d7640 1 -- 172.21.15.94:0/3830873468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c100ad0 msgr2=0x7f3c4c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c509d7640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c100ad0 0x7f3c4c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.763+0000 7f3c509d7640 1 -- 172.21.15.94:0/3830873468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c101cd0 msgr2=0x7f3c4c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c509d7640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c101cd0 0x7f3c4c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c509d7640 1 -- 172.21.15.94:0/3830873468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c1a1cb0 con 0x7f3c4c102690 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c4bfff640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c100ad0 0x7f3c4c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c4b7fe640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c101cd0 0x7f3c4c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c509d7640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 0x7f3c4c1a1640 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c44031040 tx=0x7f3c44003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c497fa640 1 -- 172.21.15.94:0/3830873468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44004040 con 0x7f3c4c102690 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c1a1f40 con 0x7f3c4c102690 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c497fa640 1 -- 172.21.15.94:0/3830873468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c44004690 con 0x7f3c4c102690 2024-09-19T04:12:25.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c497fa640 1 -- 172.21.15.94:0/3830873468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44030c30 con 0x7f3c4c102690 2024-09-19T04:12:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.764+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c1a23f0 con 0x7f3c4c102690 2024-09-19T04:12:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.765+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c1a2650 con 0x7f3c4c102690 2024-09-19T04:12:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.766+0000 7f3c497fa640 1 -- 172.21.15.94:0/3830873468 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3c440041e0 con 0x7f3c4c102690 2024-09-19T04:12:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.767+0000 7f3c497fa640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3c340a3270 0x7f3c340a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.767+0000 7f3c4bfff640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3c340a3270 0x7f3c340a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.767+0000 7f3c497fa640 1 -- 172.21.15.94:0/3830873468 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6847+0+0 (secure 0 0 0) 0x7f3c440f07d0 con 0x7f3c4c102690 2024-09-19T04:12:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.767+0000 7f3c4bfff640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3c340a3270 0x7f3c340a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009e00 tx=0x7f3c3c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.771+0000 7f3c497fa640 1 -- 172.21.15.94:0/3830873468 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c440bdab0 con 0x7f3c4c102690 2024-09-19T04:12:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.936+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3c4c1a29e0 con 0x7f3c4c102690 2024-09-19T04:12:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.937+0000 7f3c497fa640 1 -- 172.21.15.94:0/3830873468 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f3c440bd3b0 con 0x7f3c4c102690 2024-09-19T04:12:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3c340a3270 msgr2=0x7f3c340a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3c340a3270 0x7f3c340a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009e00 tx=0x7f3c3c009190 comp rx=0 tx=0).stop 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 msgr2=0x7f3c4c1a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 0x7f3c4c1a1640 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c44031040 tx=0x7f3c44003ba0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 shutdown_connections 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f3c340a3270 0x7f3c340a5730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102690 0x7f3c4c1a1640 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c101cd0 0x7f3c4c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 --2- 172.21.15.94:0/3830873468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c100ad0 0x7f3c4c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 >> 172.21.15.94:0/3830873468 conn(0x7f3c4c0fc280 msgr2=0x7f3c4c06a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 shutdown_connections 2024-09-19T04:12:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:25.939+0000 7f3c5243a640 1 -- 172.21.15.94:0/3830873468 wait complete. 2024-09-19T04:12:26.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:12:25 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:12:25.993+0000 7f816a514640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T04:12:26.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:12:26.009+0000 7f8162b46640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:12:26.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-mon[35051]: Detected new or changed devices on smithi112 2024-09-19T04:12:26.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi112 to 4272M 2024-09-19T04:12:26.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-mon[35051]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:12:26.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-mon[35051]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T04:12:26.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:26.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:26.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3830873468' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:12:26.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-mon[35051]: from='osd.11 ' entity='osd.11' 2024-09-19T04:12:26.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:26 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:26 smithi094 ceph-mon[30972]: Detected new or changed devices on smithi112 2024-09-19T04:12:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:26 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi112 to 4272M 2024-09-19T04:12:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:26 smithi094 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:12:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:26 smithi094 ceph-mon[30972]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T04:12:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:26 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:26 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:26 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3830873468' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:12:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:26 smithi094 ceph-mon[30972]: from='osd.11 ' entity='osd.11' 2024-09-19T04:12:26.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:26 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:26.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:26 smithi104 ceph-mon[34145]: Detected new or changed devices on smithi112 2024-09-19T04:12:26.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:26 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi112 to 4272M 2024-09-19T04:12:26.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:26 smithi104 ceph-mon[34145]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-19T04:12:26.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:26 smithi104 ceph-mon[34145]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T04:12:26.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:26 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:26.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:26 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:26.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:26 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3830873468' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:12:26.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:26 smithi104 ceph-mon[34145]: from='osd.11 ' entity='osd.11' 2024-09-19T04:12:26.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:26 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:26.746 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726719122,"num_in_osds":12,"osd_in_since":1726719123,"num_remapped_pgs":0} 2024-09-19T04:12:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:27 smithi112 ceph-mon[35051]: purged_snaps scrub starts 2024-09-19T04:12:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:27 smithi112 ceph-mon[35051]: purged_snaps scrub ok 2024-09-19T04:12:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:27 smithi112 ceph-mon[35051]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:27 smithi112 ceph-mon[35051]: osd.11 [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] boot 2024-09-19T04:12:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:27 smithi112 ceph-mon[35051]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T04:12:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:27 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:27.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:27 smithi094 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T04:12:27.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:27 smithi094 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T04:12:27.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:27 smithi094 ceph-mon[30972]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:27.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:27 smithi094 ceph-mon[30972]: osd.11 [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] boot 2024-09-19T04:12:27.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:27 smithi094 ceph-mon[30972]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T04:12:27.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:27 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:27 smithi104 ceph-mon[34145]: purged_snaps scrub starts 2024-09-19T04:12:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:27 smithi104 ceph-mon[34145]: purged_snaps scrub ok 2024-09-19T04:12:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:27 smithi104 ceph-mon[34145]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:12:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:27 smithi104 ceph-mon[34145]: osd.11 [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] boot 2024-09-19T04:12:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:27 smithi104 ceph-mon[34145]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T04:12:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:27 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:12:27.747 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:12:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:28.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 -- 172.21.15.94:0/55034234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 msgr2=0x7f091410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:28.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 --2- 172.21.15.94:0/55034234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 0x7f091410a050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f090c009ca0 tx=0x7f090c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 -- 172.21.15.94:0/55034234 shutdown_connections 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 --2- 172.21.15.94:0/55034234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 0x7f091410a050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 --2- 172.21.15.94:0/55034234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0914102e00 0x7f0914103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 --2- 172.21.15.94:0/55034234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0914101c00 0x7f0914102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 -- 172.21.15.94:0/55034234 >> 172.21.15.94:0/55034234 conn(0x7f09140fd390 msgr2=0x7f09140ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 -- 172.21.15.94:0/55034234 shutdown_connections 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 -- 172.21.15.94:0/55034234 wait complete. 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 Processor -- start 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.435+0000 7f091a691640 1 -- start start 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f091a691640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0914101c00 0x7f0914197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f091a691640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0914102e00 0x7f09141983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:28.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f091a691640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 0x7f091419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f091a691640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f091410d7b0 con 0x7f09141037c0 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f091a691640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f091410d630 con 0x7f0914101c00 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f091a691640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f091410d930 con 0x7f0914102e00 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0918c2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 0x7f091419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0918c2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 0x7f091419f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57362/0 (socket says 172.21.15.94:57362) 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0918c2e640 1 -- 172.21.15.94:0/1244885364 learned_addr learned my addr 172.21.15.94:0/1244885364 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0913fff640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0914101c00 0x7f0914197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f09137fe640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0914102e00 0x7f09141983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0918c2e640 1 -- 172.21.15.94:0/1244885364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0914102e00 msgr2=0x7f09141983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0918c2e640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0914102e00 0x7f09141983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0918c2e640 1 -- 172.21.15.94:0/1244885364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0914101c00 msgr2=0x7f0914197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0918c2e640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0914101c00 0x7f0914197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0918c2e640 1 -- 172.21.15.94:0/1244885364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091419fa90 con 0x7f09141037c0 2024-09-19T04:12:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.436+0000 7f0918c2e640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 0x7f091419f420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f090c009dc0 tx=0x7f090c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.437+0000 7f09117fa640 1 -- 172.21.15.94:0/1244885364 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c032ec0 con 0x7f09141037c0 2024-09-19T04:12:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.437+0000 7f09117fa640 1 -- 172.21.15.94:0/1244885364 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f090c0044f0 con 0x7f09141037c0 2024-09-19T04:12:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.437+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091419fd20 con 0x7f09141037c0 2024-09-19T04:12:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.437+0000 7f09117fa640 1 -- 172.21.15.94:0/1244885364 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c030d40 con 0x7f09141037c0 2024-09-19T04:12:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.437+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09141a0250 con 0x7f09141037c0 2024-09-19T04:12:28.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.438+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0914109050 con 0x7f09141037c0 2024-09-19T04:12:28.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.440+0000 7f09117fa640 1 -- 172.21.15.94:0/1244885364 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f090c004040 con 0x7f09141037c0 2024-09-19T04:12:28.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.440+0000 7f09117fa640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08fc0a3490 0x7f08fc0a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:28.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.440+0000 7f0913fff640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08fc0a3490 0x7f08fc0a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:28.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.440+0000 7f09117fa640 1 -- 172.21.15.94:0/1244885364 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f090c0f0b80 con 0x7f09141037c0 2024-09-19T04:12:28.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.441+0000 7f0913fff640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08fc0a3490 0x7f08fc0a5950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0904005fd0 tx=0x7f090400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.443+0000 7f09117fa640 1 -- 172.21.15.94:0/1244885364 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f090c0bdd40 con 0x7f09141037c0 2024-09-19T04:12:28.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.611+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0914062df0 con 0x7f09141037c0 2024-09-19T04:12:28.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.611+0000 7f09117fa640 1 -- 172.21.15.94:0/1244885364 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f090c0b9030 con 0x7f09141037c0 2024-09-19T04:12:28.613 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:28.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08fc0a3490 msgr2=0x7f08fc0a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:28.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08fc0a3490 0x7f08fc0a5950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0904005fd0 tx=0x7f090400b390 comp rx=0 tx=0).stop 2024-09-19T04:12:28.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 msgr2=0x7f091419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:28.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 0x7f091419f420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f090c009dc0 tx=0x7f090c003ba0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 shutdown_connections 2024-09-19T04:12:28.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08fc0a3490 0x7f08fc0a5950 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09141037c0 0x7f091419f420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0914102e00 0x7f09141983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 --2- 172.21.15.94:0/1244885364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0914101c00 0x7f0914197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:28.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.613+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 >> 172.21.15.94:0/1244885364 conn(0x7f09140fd390 msgr2=0x7f0914106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:28.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.614+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 shutdown_connections 2024-09-19T04:12:28.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:28.614+0000 7f091a691640 1 -- 172.21.15.94:0/1244885364 wait complete. 2024-09-19T04:12:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:29 smithi112 ceph-mon[35051]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T04:12:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:29 smithi112 ceph-mon[35051]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:29 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1244885364' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:12:29.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:29 smithi094 ceph-mon[30972]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T04:12:29.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:29 smithi094 ceph-mon[30972]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:29.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:29 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1244885364' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:12:29.313 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1726719146,"num_in_osds":12,"osd_in_since":1726719123,"num_remapped_pgs":0} 2024-09-19T04:12:29.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:12:29.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:29 smithi104 ceph-mon[34145]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T04:12:29.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:29 smithi104 ceph-mon[34145]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:29.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:29 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1244885364' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:12:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.017+0000 7fb7db335640 1 -- 172.21.15.94:0/2798115026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d4105ec0 msgr2=0x7fb7d4106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.017+0000 7fb7db335640 1 --2- 172.21.15.94:0/2798115026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d4105ec0 0x7fb7d4106340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009ca0 tx=0x7fb7cc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.017+0000 7fb7db335640 1 -- 172.21.15.94:0/2798115026 shutdown_connections 2024-09-19T04:12:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.017+0000 7fb7db335640 1 --2- 172.21.15.94:0/2798115026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d4105ec0 0x7fb7d4106340 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.017+0000 7fb7db335640 1 --2- 172.21.15.94:0/2798115026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d41011e0 0x7fb7d4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.017+0000 7fb7db335640 1 --2- 172.21.15.94:0/2798115026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7d41008c0 0x7fb7d4100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.017+0000 7fb7db335640 1 -- 172.21.15.94:0/2798115026 >> 172.21.15.94:0/2798115026 conn(0x7fb7d40fc7d0 msgr2=0x7fb7d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.017+0000 7fb7db335640 1 -- 172.21.15.94:0/2798115026 shutdown_connections 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.017+0000 7fb7db335640 1 -- 172.21.15.94:0/2798115026 wait complete. 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7db335640 1 Processor -- start 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7db335640 1 -- start start 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7db335640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7d41008c0 0x7fb7d41a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7db335640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d41011e0 0x7fb7d41a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7db335640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4105ec0 0x7fb7d419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7db335640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4118590 con 0x7fb7d41011e0 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7db335640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4118410 con 0x7fb7d41008c0 2024-09-19T04:12:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7db335640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4118710 con 0x7fb7d4105ec0 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d88d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d41011e0 0x7fb7d41a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d88d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d41011e0 0x7fb7d41a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57372/0 (socket says 172.21.15.94:57372) 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d88d0640 1 -- 172.21.15.94:0/3061053143 learned_addr learned my addr 172.21.15.94:0/3061053143 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d90d1640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7d41008c0 0x7fb7d41a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d88d0640 1 -- 172.21.15.94:0/3061053143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4105ec0 msgr2=0x7fb7d419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d88d0640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4105ec0 0x7fb7d419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d88d0640 1 -- 172.21.15.94:0/3061053143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7d41008c0 msgr2=0x7fb7d41a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d88d0640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7d41008c0 0x7fb7d41a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d88d0640 1 -- 172.21.15.94:0/3061053143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d419ef80 con 0x7fb7d41011e0 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d90d1640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7d41008c0 0x7fb7d41a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.018+0000 7fb7d88d0640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d41011e0 0x7fb7d41a4ba0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400c920 tx=0x7fb7c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.019+0000 7fb7ca7fc640 1 -- 172.21.15.94:0/3061053143 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4007c70 con 0x7fb7d41011e0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.019+0000 7fb7ca7fc640 1 -- 172.21.15.94:0/3061053143 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7c4007e10 con 0x7fb7d41011e0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.019+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d419f270 con 0x7fb7d41011e0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.019+0000 7fb7ca7fc640 1 -- 172.21.15.94:0/3061053143 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4018820 con 0x7fb7d41011e0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.019+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d41ab4c0 con 0x7fb7d41011e0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.020+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d406a850 con 0x7fb7d41011e0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.021+0000 7fb7ca7fc640 1 -- 172.21.15.94:0/3061053143 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb7c40189c0 con 0x7fb7d41011e0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.021+0000 7fb7ca7fc640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb7c00a32c0 0x7fb7c00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.021+0000 7fb7ca7fc640 1 -- 172.21.15.94:0/3061053143 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fb7c40cf5f0 con 0x7fb7d41011e0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.021+0000 7fb7d90d1640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb7c00a32c0 0x7fb7c00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.022+0000 7fb7d90d1640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb7c00a32c0 0x7fb7c00a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006fd0 tx=0x7fb7d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:30.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.025+0000 7fb7ca7fc640 1 -- 172.21.15.94:0/3061053143 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb7c4014030 con 0x7fb7d41011e0 2024-09-19T04:12:30.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.204+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7d41ab780 con 0x7fb7d41011e0 2024-09-19T04:12:30.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.205+0000 7fb7ca7fc640 1 -- 172.21.15.94:0/3061053143 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18493 (secure 0 0 0) 0x7fb7c400de20 con 0x7fb7d41011e0 2024-09-19T04:12:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:30.207 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":74,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:12:28.002612+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:12:30.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb7c00a32c0 msgr2=0x7fb7c00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:30.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb7c00a32c0 0x7fb7c00a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006fd0 tx=0x7fb7d0008040 comp rx=0 tx=0).stop 2024-09-19T04:12:30.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d41011e0 msgr2=0x7fb7d41a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d41011e0 0x7fb7d41a4ba0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400c920 tx=0x7fb7c400cde0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 shutdown_connections 2024-09-19T04:12:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb7c00a32c0 0x7fb7c00a5780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4105ec0 0x7fb7d419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7d41011e0 0x7fb7d41a4ba0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 --2- 172.21.15.94:0/3061053143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7d41008c0 0x7fb7d41a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 >> 172.21.15.94:0/3061053143 conn(0x7fb7d40fc7d0 msgr2=0x7fb7d4103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 shutdown_connections 2024-09-19T04:12:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:30.208+0000 7fb7db335640 1 -- 172.21.15.94:0/3061053143 wait complete. 2024-09-19T04:12:30.854 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:08:46.366796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:12:30.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T04:12:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:31.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:31 smithi094 ceph-mon[30972]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:31.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:31 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3061053143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:12:31.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:31 smithi112 ceph-mon[35051]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:31.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:31 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3061053143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:12:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 -- 172.21.15.94:0/348437687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 msgr2=0x7fd3b410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/348437687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 0x7fd3b410eaa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009900 tx=0x7fd3ac02a410 comp rx=0 tx=0).stop 2024-09-19T04:12:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 -- 172.21.15.94:0/348437687 shutdown_connections 2024-09-19T04:12:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/348437687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 0x7fd3b410eaa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/348437687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b41027a0 0x7fd3b41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/348437687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3b4101df0 0x7fd3b41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 -- 172.21.15.94:0/348437687 >> 172.21.15.94:0/348437687 conn(0x7fd3b40fd8f0 msgr2=0x7fd3b40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 -- 172.21.15.94:0/348437687 shutdown_connections 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 -- 172.21.15.94:0/348437687 wait complete. 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 Processor -- start 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.536+0000 7fd3bb7db640 1 -- start start 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3bb7db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b4101df0 0x7fd3b4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3bb7db640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3b41027a0 0x7fd3b4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3bb7db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 0x7fd3b419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3bb7db640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4111e30 con 0x7fd3b4106f20 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3bb7db640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4111cb0 con 0x7fd3b41027a0 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3bb7db640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4111fb0 con 0x7fd3b4101df0 2024-09-19T04:12:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9d78640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 0x7fd3b419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9d78640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 0x7fd3b419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57388/0 (socket says 172.21.15.94:57388) 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9d78640 1 -- 172.21.15.94:0/3686526521 learned_addr learned my addr 172.21.15.94:0/3686526521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9577640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b4101df0 0x7fd3b4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b8d76640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3b41027a0 0x7fd3b4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9d78640 1 -- 172.21.15.94:0/3686526521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b4101df0 msgr2=0x7fd3b4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9d78640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b4101df0 0x7fd3b4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9d78640 1 -- 172.21.15.94:0/3686526521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3b41027a0 msgr2=0x7fd3b4198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9d78640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3b41027a0 0x7fd3b4198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9d78640 1 -- 172.21.15.94:0/3686526521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b419d300 con 0x7fd3b4106f20 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b8d76640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3b41027a0 0x7fd3b4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:12:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b9d78640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 0x7fd3b419cc20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac004770 tx=0x7fd3ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b27fc640 1 -- 172.21.15.94:0/3686526521 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac03f030 con 0x7fd3b4106f20 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b419d590 con 0x7fd3b4106f20 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b27fc640 1 -- 172.21.15.94:0/3686526521 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3ac0318e0 con 0x7fd3b4106f20 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3b27fc640 1 -- 172.21.15.94:0/3686526521 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac035070 con 0x7fd3b4106f20 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.537+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b41a4ed0 con 0x7fd3b4106f20 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.538+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b41021d0 con 0x7fd3b4106f20 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.539+0000 7fd3b27fc640 1 -- 172.21.15.94:0/3686526521 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd3ac042030 con 0x7fd3b4106f20 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.540+0000 7fd3b27fc640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd39c0a3240 0x7fd39c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.540+0000 7fd3b9577640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd39c0a3240 0x7fd39c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.540+0000 7fd3b27fc640 1 -- 172.21.15.94:0/3686526521 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fd3ac0efb10 con 0x7fd3b4106f20 2024-09-19T04:12:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.540+0000 7fd3b9577640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd39c0a3240 0x7fd39c0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4005fd0 tx=0x7fd3a4009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.544+0000 7fd3b27fc640 1 -- 172.21.15.94:0/3686526521 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd3ac0bccd0 con 0x7fd3b4106f20 2024-09-19T04:12:31.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:31 smithi104 ceph-mon[34145]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:31.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:31 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3061053143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:12:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.709+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3b410a100 con 0x7fd3b4106f20 2024-09-19T04:12:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.709+0000 7fd3b27fc640 1 -- 172.21.15.94:0/3686526521 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fd3ac0bc5d0 con 0x7fd3b4106f20 2024-09-19T04:12:31.711 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-19T04:12:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.711+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd39c0a3240 msgr2=0x7fd39c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.711+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd39c0a3240 0x7fd39c0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4005fd0 tx=0x7fd3a4009760 comp rx=0 tx=0).stop 2024-09-19T04:12:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.711+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 msgr2=0x7fd3b419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.711+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 0x7fd3b419cc20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac004770 tx=0x7fd3ac004270 comp rx=0 tx=0).stop 2024-09-19T04:12:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.712+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 shutdown_connections 2024-09-19T04:12:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.712+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd39c0a3240 0x7fd39c0a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.712+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b4106f20 0x7fd3b419cc20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.712+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3b41027a0 0x7fd3b4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.712+0000 7fd3bb7db640 1 --2- 172.21.15.94:0/3686526521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b4101df0 0x7fd3b4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.712+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 >> 172.21.15.94:0/3686526521 conn(0x7fd3b40fd8f0 msgr2=0x7fd3b40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.712+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 shutdown_connections 2024-09-19T04:12:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:31.712+0000 7fd3bb7db640 1 -- 172.21.15.94:0/3686526521 wait complete. 2024-09-19T04:12:32.308 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:32 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3686526521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:12:32.335 INFO:tasks.cephadm:Adding mds.a on smithi094 2024-09-19T04:12:32.335 INFO:tasks.cephadm:Adding mds.d on smithi094 2024-09-19T04:12:32.335 INFO:tasks.cephadm:Adding mds.g on smithi094 2024-09-19T04:12:32.335 INFO:tasks.cephadm:Adding mds.j on smithi094 2024-09-19T04:12:32.335 INFO:tasks.cephadm:Adding mds.b on smithi104 2024-09-19T04:12:32.336 INFO:tasks.cephadm:Adding mds.e on smithi104 2024-09-19T04:12:32.336 INFO:tasks.cephadm:Adding mds.h on smithi104 2024-09-19T04:12:32.336 INFO:tasks.cephadm:Adding mds.k on smithi104 2024-09-19T04:12:32.336 INFO:tasks.cephadm:Adding mds.c on smithi112 2024-09-19T04:12:32.336 INFO:tasks.cephadm:Adding mds.f on smithi112 2024-09-19T04:12:32.336 INFO:tasks.cephadm:Adding mds.i on smithi112 2024-09-19T04:12:32.336 INFO:tasks.cephadm:Adding mds.l on smithi112 2024-09-19T04:12:32.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l' 2024-09-19T04:12:32.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:32 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3686526521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:12:32.595 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:12:32.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:32 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3686526521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:12:33.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.018+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3243305797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e8101d10 msgr2=0x7fe8e8102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:33.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.018+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3243305797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e8101d10 0x7fe8e8102190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009900 tx=0x7fe8d802a410 comp rx=0 tx=0).stop 2024-09-19T04:12:33.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.018+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3243305797 shutdown_connections 2024-09-19T04:12:33.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.018+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3243305797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8e81026d0 0x7fe8e8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:33.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.018+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3243305797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e8101d10 0x7fe8e8102190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:33.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.018+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3243305797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e8100b10 0x7fe8e8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.018+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3243305797 >> 172.21.15.112:0/3243305797 conn(0x7fe8e80fc2a0 msgr2=0x7fe8e80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.018+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3243305797 shutdown_connections 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.018+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3243305797 wait complete. 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8efa8d640 1 Processor -- start 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8efa8d640 1 -- start start 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8efa8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e8100b10 0x7fe8e8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8efa8d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8e8101d10 0x7fe8e81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8efa8d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e81026d0 0x7fe8e819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8efa8d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e810d770 con 0x7fe8e8100b10 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8efa8d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e810d5f0 con 0x7fe8e8101d10 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8efa8d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e810d8f0 con 0x7fe8e81026d0 2024-09-19T04:12:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ee02a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e81026d0 0x7fe8e819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ee02a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e81026d0 0x7fe8e819f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56740/0 (socket says 172.21.15.112:56740) 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ee02a640 1 -- 172.21.15.112:0/3189076015 learned_addr learned my addr 172.21.15.112:0/3189076015 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ed028640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8e8101d10 0x7fe8e81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ed829640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e8100b10 0x7fe8e8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ee02a640 1 -- 172.21.15.112:0/3189076015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8e8101d10 msgr2=0x7fe8e81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ee02a640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8e8101d10 0x7fe8e81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ee02a640 1 -- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e8100b10 msgr2=0x7fe8e8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ee02a640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e8100b10 0x7fe8e8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ee02a640 1 -- 172.21.15.112:0/3189076015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e819faa0 con 0x7fe8e81026d0 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ed028640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8e8101d10 0x7fe8e81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:12:33.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ed829640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e8100b10 0x7fe8e8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.019+0000 7fe8ee02a640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e81026d0 0x7fe8e819f430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000cad0 tx=0x7fe8e000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.020+0000 7fe8deffd640 1 -- 172.21.15.112:0/3189076015 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000ae20 con 0x7fe8e81026d0 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.020+0000 7fe8deffd640 1 -- 172.21.15.112:0/3189076015 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e0005020 con 0x7fe8e81026d0 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.020+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e819fd90 con 0x7fe8e81026d0 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.020+0000 7fe8deffd640 1 -- 172.21.15.112:0/3189076015 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000f6b0 con 0x7fe8e81026d0 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.020+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e81a0220 con 0x7fe8e81026d0 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.021+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e8107fb0 con 0x7fe8e81026d0 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.022+0000 7fe8deffd640 1 -- 172.21.15.112:0/3189076015 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe8e00051d0 con 0x7fe8e81026d0 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.023+0000 7fe8deffd640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8d40a3270 0x7fe8d40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.023+0000 7fe8deffd640 1 -- 172.21.15.112:0/3189076015 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fe8e00cf960 con 0x7fe8e81026d0 2024-09-19T04:12:33.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.023+0000 7fe8ed829640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8d40a3270 0x7fe8d40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:33.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.023+0000 7fe8ed829640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8d40a3270 0x7fe8d40a5730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8101b70 tx=0x7fe8e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:33.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.026+0000 7fe8deffd640 1 -- 172.21.15.112:0/3189076015 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8e0014030 con 0x7fe8e81026d0 2024-09-19T04:12:33.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.203+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l", "target": ["mon-mgr", ""]}) -- 0x7fe8e8062730 con 0x7fe8d40a3270 2024-09-19T04:12:33.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.210+0000 7fe8deffd640 1 -- 172.21.15.112:0/3189076015 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe8e8062730 con 0x7fe8d40a3270 2024-09-19T04:12:33.211 INFO:teuthology.orchestra.run.smithi112.stdout:Scheduled mds.all update... 2024-09-19T04:12:33.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8d40a3270 msgr2=0x7fe8d40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8d40a3270 0x7fe8d40a5730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8101b70 tx=0x7fe8e4005f90 comp rx=0 tx=0).stop 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e81026d0 msgr2=0x7fe8e819f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e81026d0 0x7fe8e819f430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000cad0 tx=0x7fe8e000cf90 comp rx=0 tx=0).stop 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 shutdown_connections 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8d40a3270 0x7fe8d40a5730 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e81026d0 0x7fe8e819f430 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8e8101d10 0x7fe8e81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 --2- 172.21.15.112:0/3189076015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e8100b10 0x7fe8e8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 >> 172.21.15.112:0/3189076015 conn(0x7fe8e80fc2a0 msgr2=0x7fe8e8105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 shutdown_connections 2024-09-19T04:12:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-19T04:12:33.212+0000 7fe8efa8d640 1 -- 172.21.15.112:0/3189076015 wait complete. 2024-09-19T04:12:33.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:33 smithi112 ceph-mon[35051]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:33.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:33 smithi094 ceph-mon[30972]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:33.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:33 smithi104 ceph-mon[34145]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:33.780 DEBUG:teuthology.orchestra.run.smithi094:mds.a> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T04:12:33.783 DEBUG:teuthology.orchestra.run.smithi094:mds.d> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T04:12:33.826 DEBUG:teuthology.orchestra.run.smithi094:mds.g> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T04:12:33.869 DEBUG:teuthology.orchestra.run.smithi094:mds.j> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T04:12:33.913 DEBUG:teuthology.orchestra.run.smithi104:mds.b> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T04:12:33.916 DEBUG:teuthology.orchestra.run.smithi104:mds.e> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T04:12:33.960 DEBUG:teuthology.orchestra.run.smithi104:mds.h> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T04:12:34.003 DEBUG:teuthology.orchestra.run.smithi104:mds.k> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T04:12:34.046 DEBUG:teuthology.orchestra.run.smithi112:mds.c> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T04:12:34.049 DEBUG:teuthology.orchestra.run.smithi112:mds.f> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T04:12:34.093 DEBUG:teuthology.orchestra.run.smithi112:mds.i> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T04:12:34.136 DEBUG:teuthology.orchestra.run.smithi112:mds.l> sudo journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T04:12:34.179 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T04:12:34.179 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:12:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l;count:12 2024-09-19T04:12:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:34.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:34.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:34.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:34.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:34 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:34.544 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:34.544 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l;count:12 2024-09-19T04:12:34.544 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:34.544 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:34.544 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:34.544 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:34.544 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:34.544 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:34.545 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:34.545 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:34 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:34.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:34.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l;count:12 2024-09-19T04:12:34.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:34.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:34.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:34.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:34.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:34.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:34.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:34.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:34 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:34.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.897+0000 7f02e0b14640 1 -- 172.21.15.94:0/4076896946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 msgr2=0x7f02dc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:34.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 --2- 172.21.15.94:0/4076896946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 0x7f02dc10ec30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009cd0 tx=0x7f02cc02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:12:34.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 -- 172.21.15.94:0/4076896946 shutdown_connections 2024-09-19T04:12:34.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 --2- 172.21.15.94:0/4076896946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 0x7f02dc10ec30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:34.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 --2- 172.21.15.94:0/4076896946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02dc103740 0x7f02dc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:34.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 --2- 172.21.15.94:0/4076896946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc102d90 0x7f02dc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:34.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 -- 172.21.15.94:0/4076896946 >> 172.21.15.94:0/4076896946 conn(0x7f02dc0fc7d0 msgr2=0x7f02dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:34.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 -- 172.21.15.94:0/4076896946 shutdown_connections 2024-09-19T04:12:34.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 -- 172.21.15.94:0/4076896946 wait complete. 2024-09-19T04:12:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 Processor -- start 2024-09-19T04:12:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.898+0000 7f02e0b14640 1 -- start start 2024-09-19T04:12:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02e0b14640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02dc102d90 0x7f02dc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02e0b14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc103740 0x7f02dc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02e0b14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 0x7f02dc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02e0b14640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc111fa0 con 0x7f02dc108290 2024-09-19T04:12:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02e0b14640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc111e20 con 0x7f02dc102d90 2024-09-19T04:12:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02e0b14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc112120 con 0x7f02dc103740 2024-09-19T04:12:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02dad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 0x7f02dc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02dad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 0x7f02dc19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41418/0 (socket says 172.21.15.94:41418) 2024-09-19T04:12:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02dad9d640 1 -- 172.21.15.94:0/2419394683 learned_addr learned my addr 172.21.15.94:0/2419394683 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02da59c640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02dc102d90 0x7f02dc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02d9d9b640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc103740 0x7f02dc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02dad9d640 1 -- 172.21.15.94:0/2419394683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc103740 msgr2=0x7f02dc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02dad9d640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc103740 0x7f02dc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02dad9d640 1 -- 172.21.15.94:0/2419394683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02dc102d90 msgr2=0x7f02dc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02dad9d640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02dc102d90 0x7f02dc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02dad9d640 1 -- 172.21.15.94:0/2419394683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc19d4b0 con 0x7f02dc108290 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02dad9d640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 0x7f02dc19cdd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f02cc005cc0 tx=0x7f02cc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02d37fe640 1 -- 172.21.15.94:0/2419394683 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc004240 con 0x7f02dc108290 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02dc19d740 con 0x7f02dc108290 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02d37fe640 1 -- 172.21.15.94:0/2419394683 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02cc0305a0 con 0x7f02dc108290 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.899+0000 7f02d37fe640 1 -- 172.21.15.94:0/2419394683 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc0466c0 con 0x7f02dc108290 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.900+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02dc1a5080 con 0x7f02dc108290 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.901+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02dc06a7f0 con 0x7f02dc108290 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.902+0000 7f02d37fe640 1 -- 172.21.15.94:0/2419394683 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f02cc030ac0 con 0x7f02dc108290 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.902+0000 7f02d37fe640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f02c80a3240 0x7f02c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.902+0000 7f02da59c640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f02c80a3240 0x7f02c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:34.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.902+0000 7f02d37fe640 1 -- 172.21.15.94:0/2419394683 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f02cc02e030 con 0x7f02dc108290 2024-09-19T04:12:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.903+0000 7f02da59c640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f02c80a3240 0x7f02c80a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009990 tx=0x7f02d4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:34.906+0000 7f02d37fe640 1 -- 172.21.15.94:0/2419394683 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f02cc04b060 con 0x7f02dc108290 2024-09-19T04:12:35.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.107+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02dc10a2a0 con 0x7f02dc108290 2024-09-19T04:12:35.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.107+0000 7f02d37fe640 1 -- 172.21.15.94:0/2419394683 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f02cc0eee20 con 0x7f02dc108290 2024-09-19T04:12:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:35.109 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":1,"btime":"2024-09-19T04:05:34:057432+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T04:12:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 1 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f02c80a3240 msgr2=0x7f02c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f02c80a3240 0x7f02c80a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009990 tx=0x7f02d4009110 comp rx=0 tx=0).stop 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 msgr2=0x7f02dc19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 0x7f02dc19cdd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f02cc005cc0 tx=0x7f02cc003ba0 comp rx=0 tx=0).stop 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 shutdown_connections 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f02c80a3240 0x7f02c80a5700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02dc108290 0x7f02dc19cdd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc103740 0x7f02dc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 --2- 172.21.15.94:0/2419394683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02dc102d90 0x7f02dc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:35.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 >> 172.21.15.94:0/2419394683 conn(0x7f02dc0fc7d0 msgr2=0x7f02dc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:35.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 shutdown_connections 2024-09-19T04:12:35.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:35.109+0000 7f02e0b14640 1 -- 172.21.15.94:0/2419394683 wait complete. 2024-09-19T04:12:35.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:35 smithi112 ceph-mon[35051]: Deploying daemon mds.l on smithi112 2024-09-19T04:12:35.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:35 smithi112 ceph-mon[35051]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:35.506 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:35 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2419394683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:35.506 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:12:35 smithi112 systemd[1]: Starting Ceph mds.l for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:35.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:35 smithi094 ceph-mon[30972]: Deploying daemon mds.l on smithi112 2024-09-19T04:12:35.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:35 smithi094 ceph-mon[30972]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:35.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:35 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2419394683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:35 smithi104 ceph-mon[34145]: Deploying daemon mds.l on smithi112 2024-09-19T04:12:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:35 smithi104 ceph-mon[34145]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:35 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2419394683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:35.785 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:12:35 smithi112 podman[64514]: 2024-09-19 04:12:35.618148664 +0000 UTC m=+0.015648896 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:35.785 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:12:35 smithi112 podman[64514]: 2024-09-19 04:12:35.741566521 +0000 UTC m=+0.139066741 container create c9740431c0dfe128be9d4e1d07759377ff433883ed2c366077bb028cdf9aa549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T04:12:36.285 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:12:35 smithi112 podman[64514]: 2024-09-19 04:12:35.960803452 +0000 UTC m=+0.358303680 container init c9740431c0dfe128be9d4e1d07759377ff433883ed2c366077bb028cdf9aa549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:12:36.285 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:12:35 smithi112 podman[64514]: 2024-09-19 04:12:35.963417592 +0000 UTC m=+0.360917813 container start c9740431c0dfe128be9d4e1d07759377ff433883ed2c366077bb028cdf9aa549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T04:12:36.285 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:12:35 smithi112 ceph-mds[64533]: -- 172.21.15.112:0/1472645877 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e11abf860 con 0x559e11a9d000 2024-09-19T04:12:36.285 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:12:35 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l[64529]: starting mds.l at 2024-09-19T04:12:36.285 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:12:36 smithi112 bash[64514]: c9740431c0dfe128be9d4e1d07759377ff433883ed2c366077bb028cdf9aa549 2024-09-19T04:12:36.285 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:12:36 smithi112 systemd[1]: Started Ceph mds.l for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.112:6832/1795597710,v1:172.21.15.112:6833/1795597710] up:boot 2024-09-19T04:12:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: fsmap 1 up:standby 2024-09-19T04:12:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:12:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: Deploying daemon mds.k on smithi104 2024-09-19T04:12:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:36 smithi104 ceph-mon[34145]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:37.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.112:6832/1795597710,v1:172.21.15.112:6833/1795597710] up:boot 2024-09-19T04:12:37.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: fsmap 1 up:standby 2024-09-19T04:12:37.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:12:37.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:37.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:37.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:37.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:37.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:37.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:37.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: Deploying daemon mds.k on smithi104 2024-09-19T04:12:37.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:36 smithi112 ceph-mon[35051]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:37.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.112:6832/1795597710,v1:172.21.15.112:6833/1795597710] up:boot 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: fsmap 1 up:standby 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: Deploying daemon mds.k on smithi104 2024-09-19T04:12:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:36 smithi094 ceph-mon[30972]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:37.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:12:38.060 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:38.317 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:12:38 smithi104 systemd[1]: Starting Ceph mds.k for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:38.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.564+0000 7fc6101b5640 1 -- 172.21.15.94:0/846160298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 msgr2=0x7fc608103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:38.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.564+0000 7fc6101b5640 1 --2- 172.21.15.94:0/846160298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 0x7fc608103110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc600009930 tx=0x7fc60002a420 comp rx=0 tx=0).stop 2024-09-19T04:12:38.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.564+0000 7fc6101b5640 1 -- 172.21.15.94:0/846160298 shutdown_connections 2024-09-19T04:12:38.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.564+0000 7fc6101b5640 1 --2- 172.21.15.94:0/846160298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6081082a0 0x7fc60810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.564+0000 7fc6101b5640 1 --2- 172.21.15.94:0/846160298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6081036e0 0x7fc608107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.565+0000 7fc6101b5640 1 --2- 172.21.15.94:0/846160298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 0x7fc608103110 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.565+0000 7fc6101b5640 1 -- 172.21.15.94:0/846160298 >> 172.21.15.94:0/846160298 conn(0x7fc6080fc7d0 msgr2=0x7fc6080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.565+0000 7fc6101b5640 1 -- 172.21.15.94:0/846160298 shutdown_connections 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.565+0000 7fc6101b5640 1 -- 172.21.15.94:0/846160298 wait complete. 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.565+0000 7fc6101b5640 1 Processor -- start 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.565+0000 7fc6101b5640 1 -- start start 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc6101b5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 0x7fc608198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc6101b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6081036e0 0x7fc608198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc6101b5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6081082a0 0x7fc60819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc6101b5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc608111f50 con 0x7fc608102d30 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc6101b5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc608111dd0 con 0x7fc6081082a0 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc6101b5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6081120d0 con 0x7fc6081036e0 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60df51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 0x7fc608198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60df51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 0x7fc608198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41432/0 (socket says 172.21.15.94:41432) 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60df51640 1 -- 172.21.15.94:0/92088630 learned_addr learned my addr 172.21.15.94:0/92088630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60e752640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6081082a0 0x7fc60819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60d750640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6081036e0 0x7fc608198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60df51640 1 -- 172.21.15.94:0/92088630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6081036e0 msgr2=0x7fc608198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60df51640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6081036e0 0x7fc608198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60df51640 1 -- 172.21.15.94:0/92088630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6081082a0 msgr2=0x7fc60819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60df51640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6081082a0 0x7fc60819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60df51640 1 -- 172.21.15.94:0/92088630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60819d4c0 con 0x7fc608102d30 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60e752640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6081082a0 0x7fc60819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60d750640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6081036e0 0x7fc608198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc60df51640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 0x7fc608198510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc600004770 tx=0x7fc6000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc606ffd640 1 -- 172.21.15.94:0/92088630 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60003f070 con 0x7fc608102d30 2024-09-19T04:12:38.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc606ffd640 1 -- 172.21.15.94:0/92088630 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc60003b460 con 0x7fc608102d30 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60819d750 con 0x7fc608102d30 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc606ffd640 1 -- 172.21.15.94:0/92088630 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600035070 con 0x7fc608102d30 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.566+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6081008c0 con 0x7fc608102d30 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.567+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60806a850 con 0x7fc608102d30 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.568+0000 7fc606ffd640 1 -- 172.21.15.94:0/92088630 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc60003b610 con 0x7fc608102d30 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.569+0000 7fc606ffd640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc5f00a3240 0x7fc5f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.569+0000 7fc60d750640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc5f00a3240 0x7fc5f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.569+0000 7fc606ffd640 1 -- 172.21.15.94:0/92088630 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fc6000efb80 con 0x7fc608102d30 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.570+0000 7fc60d750640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc5f00a3240 0x7fc5f00a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc6080fef20 tx=0x7fc5fc0082f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:38.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.573+0000 7fc606ffd640 1 -- 172.21.15.94:0/92088630 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc6000bcd40 con 0x7fc608102d30 2024-09-19T04:12:38.579 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:12:38 smithi104 podman[63025]: 2024-09-19 04:12:38.219144768 +0000 UTC m=+0.026616610 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:38.579 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:12:38 smithi104 podman[63025]: 2024-09-19 04:12:38.342915574 +0000 UTC m=+0.150387456 container create 5bd23669d672323b7483b798c4f803cb5255158332d12dd2fb6b8200de107c13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-k, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T04:12:38.579 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:12:38 smithi104 podman[63025]: 2024-09-19 04:12:38.578601884 +0000 UTC m=+0.386073731 container init 5bd23669d672323b7483b798c4f803cb5255158332d12dd2fb6b8200de107c13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T04:12:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.786+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc608103110 con 0x7fc608102d30 2024-09-19T04:12:38.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.787+0000 7fc606ffd640 1 -- 172.21.15.94:0/92088630 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fc6000b8030 con 0x7fc608102d30 2024-09-19T04:12:38.789 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:38.789 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"btime":"2024-09-19T04:12:35:991456+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:12:38.789 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-09-19T04:12:38.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.789+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc5f00a3240 msgr2=0x7fc5f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:38.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.789+0000 7fc6101b5640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc5f00a3240 0x7fc5f00a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc6080fef20 tx=0x7fc5fc0082f0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.789+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 msgr2=0x7fc608198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:38.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.789+0000 7fc6101b5640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 0x7fc608198510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc600004770 tx=0x7fc6000042f0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.790+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 shutdown_connections 2024-09-19T04:12:38.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.790+0000 7fc6101b5640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc5f00a3240 0x7fc5f00a5700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.790+0000 7fc6101b5640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6081082a0 0x7fc60819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.790+0000 7fc6101b5640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6081036e0 0x7fc608198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.790+0000 7fc6101b5640 1 --2- 172.21.15.94:0/92088630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc608102d30 0x7fc608198510 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:38.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.790+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 >> 172.21.15.94:0/92088630 conn(0x7fc6080fc7d0 msgr2=0x7fc608105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:38.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.790+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 shutdown_connections 2024-09-19T04:12:38.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:38.790+0000 7fc6101b5640 1 -- 172.21.15.94:0/92088630 wait complete. 2024-09-19T04:12:38.855 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:12:38 smithi104 podman[63025]: 2024-09-19 04:12:38.581491985 +0000 UTC m=+0.388963823 container start 5bd23669d672323b7483b798c4f803cb5255158332d12dd2fb6b8200de107c13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T04:12:38.905 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:12:38 smithi104 ceph-mds[63044]: -- 172.21.15.104:0/1773453633 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559111c6f860 con 0x559111c4c800 2024-09-19T04:12:38.906 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:12:38 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-k[63040]: starting mds.k at 2024-09-19T04:12:38.906 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:12:38 smithi104 bash[63025]: 5bd23669d672323b7483b798c4f803cb5255158332d12dd2fb6b8200de107c13 2024-09-19T04:12:38.906 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:12:38 smithi104 systemd[1]: Started Ceph mds.k for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:39.484 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:39 smithi094 ceph-mon[30972]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:39.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:39 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:39.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:39 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:39.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:39 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:39.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:39 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:39.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:39 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:39.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:39 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:39.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:39 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/92088630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:39.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:39 smithi112 ceph-mon[35051]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:39 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:39 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:39 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:39 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:39 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:39 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:39 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/92088630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:39 smithi104 ceph-mon[34145]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:39 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:39 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:39 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:39 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:39.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:39 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:39.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:39 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:39.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:39 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/92088630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:40.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:40 smithi112 ceph-mon[35051]: Deploying daemon mds.j on smithi094 2024-09-19T04:12:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:40 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.104:6832/1061074349,v1:172.21.15.104:6833/1061074349] up:boot 2024-09-19T04:12:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:40 smithi112 ceph-mon[35051]: fsmap 2 up:standby 2024-09-19T04:12:40.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:40 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:12:40.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:40 smithi094 ceph-mon[30972]: Deploying daemon mds.j on smithi094 2024-09-19T04:12:40.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:40 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.104:6832/1061074349,v1:172.21.15.104:6833/1061074349] up:boot 2024-09-19T04:12:40.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:40 smithi094 ceph-mon[30972]: fsmap 2 up:standby 2024-09-19T04:12:40.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:40 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:12:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:40 smithi104 ceph-mon[34145]: Deploying daemon mds.j on smithi094 2024-09-19T04:12:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:40 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.104:6832/1061074349,v1:172.21.15.104:6833/1061074349] up:boot 2024-09-19T04:12:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:40 smithi104 ceph-mon[34145]: fsmap 2 up:standby 2024-09-19T04:12:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:40 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:12:41.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:41 smithi094 ceph-mon[30972]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:41.341 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:12:41 smithi094 systemd[1]: Starting Ceph mds.j for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:41.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:41 smithi104 ceph-mon[34145]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:41.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:12:41.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:41 smithi112 ceph-mon[35051]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:41.730 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:12:41 smithi094 podman[70717]: 2024-09-19 04:12:41.36720084 +0000 UTC m=+0.016329842 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:41.730 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:12:41 smithi094 podman[70717]: 2024-09-19 04:12:41.484816123 +0000 UTC m=+0.133945153 container create 8d883c184a160f6d085691989bc4f5c4db033e68a7946f3626bcaace6f5f955e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-j, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T04:12:41.731 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:12:41 smithi094 podman[70717]: 2024-09-19 04:12:41.729553024 +0000 UTC m=+0.378682019 container init 8d883c184a160f6d085691989bc4f5c4db033e68a7946f3626bcaace6f5f955e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T04:12:41.950 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:42.072 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:12:41 smithi094 podman[70717]: 2024-09-19 04:12:41.732747205 +0000 UTC m=+0.381876193 container start 8d883c184a160f6d085691989bc4f5c4db033e68a7946f3626bcaace6f5f955e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:12:42.072 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:12:41 smithi094 ceph-mds[70767]: -- 172.21.15.94:0/469727808 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a398e51860 con 0x55a398e2e800 2024-09-19T04:12:42.073 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:12:41 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-j[70750]: starting mds.j at 2024-09-19T04:12:42.073 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:12:41 smithi094 bash[70717]: 8d883c184a160f6d085691989bc4f5c4db033e68a7946f3626bcaace6f5f955e 2024-09-19T04:12:42.073 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:12:41 smithi094 systemd[1]: Started Ceph mds.j for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:42.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.420+0000 7ff91de16640 1 -- 172.21.15.94:0/3181440035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff918103660 msgr2=0x7ff91810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:42.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.420+0000 7ff91de16640 1 --2- 172.21.15.94:0/3181440035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff918103660 0x7ff91810bb50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009900 tx=0x7ff90c02a480 comp rx=0 tx=0).stop 2024-09-19T04:12:42.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.420+0000 7ff91de16640 1 -- 172.21.15.94:0/3181440035 shutdown_connections 2024-09-19T04:12:42.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.420+0000 7ff91de16640 1 --2- 172.21.15.94:0/3181440035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff91810c170 0x7ff91810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.420+0000 7ff91de16640 1 --2- 172.21.15.94:0/3181440035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff918103660 0x7ff91810bb50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.420+0000 7ff91de16640 1 --2- 172.21.15.94:0/3181440035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff918102d40 0x7ff918103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.421+0000 7ff91de16640 1 -- 172.21.15.94:0/3181440035 >> 172.21.15.94:0/3181440035 conn(0x7ff9180fc7a0 msgr2=0x7ff9180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.421+0000 7ff91de16640 1 -- 172.21.15.94:0/3181440035 shutdown_connections 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.421+0000 7ff91de16640 1 -- 172.21.15.94:0/3181440035 wait complete. 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.421+0000 7ff91de16640 1 Processor -- start 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.421+0000 7ff91de16640 1 -- start start 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff91de16640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918102d40 0x7ff9181983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff91de16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff918103660 0x7ff918198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff91de16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91810c170 0x7ff91819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff91de16640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff918111e80 con 0x7ff91810c170 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff91de16640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff918111d00 con 0x7ff918102d40 2024-09-19T04:12:42.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff91de16640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff918112000 con 0x7ff918103660 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff917fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91810c170 0x7ff91819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff917fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91810c170 0x7ff91819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51068/0 (socket says 172.21.15.94:51068) 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff917fff640 1 -- 172.21.15.94:0/2387740431 learned_addr learned my addr 172.21.15.94:0/2387740431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff9177fe640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918102d40 0x7ff9181983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff916ffd640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff918103660 0x7ff918198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff917fff640 1 -- 172.21.15.94:0/2387740431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff918103660 msgr2=0x7ff918198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff917fff640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff918103660 0x7ff918198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff917fff640 1 -- 172.21.15.94:0/2387740431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918102d40 msgr2=0x7ff9181983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff917fff640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918102d40 0x7ff9181983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff917fff640 1 -- 172.21.15.94:0/2387740431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91819d3a0 con 0x7ff91810c170 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff916ffd640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff918103660 0x7ff918198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:12:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff917fff640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91810c170 0x7ff91819ccc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff91000eaa0 tx=0x7ff91000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff914ff9640 1 -- 172.21.15.94:0/2387740431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910004d20 con 0x7ff91810c170 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff914ff9640 1 -- 172.21.15.94:0/2387740431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff910005020 con 0x7ff91810c170 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff914ff9640 1 -- 172.21.15.94:0/2387740431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100116b0 con 0x7ff91810c170 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.422+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91819d690 con 0x7ff91810c170 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.423+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9181a4f70 con 0x7ff91810c170 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.423+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff918103120 con 0x7ff91810c170 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.425+0000 7ff914ff9640 1 -- 172.21.15.94:0/2387740431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff9100051d0 con 0x7ff91810c170 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.425+0000 7ff914ff9640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff9000a32c0 0x7ff9000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.425+0000 7ff9177fe640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff9000a32c0 0x7ff9000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.425+0000 7ff914ff9640 1 -- 172.21.15.94:0/2387740431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7ff9100cf610 con 0x7ff91810c170 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.426+0000 7ff9177fe640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff9000a32c0 0x7ff9000a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff908009990 tx=0x7ff908009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.429+0000 7ff914ff9640 1 -- 172.21.15.94:0/2387740431 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff910016030 con 0x7ff91810c170 2024-09-19T04:12:42.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.644+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff91802d7c0 con 0x7ff91810c170 2024-09-19T04:12:42.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.644+0000 7ff914ff9640 1 -- 172.21.15.94:0/2387740431 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7ff91009c7d0 con 0x7ff91810c170 2024-09-19T04:12:42.646 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:42.646 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":4,"btime":"2024-09-19T04:12:42:185916+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:12:42.646 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 4 2024-09-19T04:12:42.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.646+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff9000a32c0 msgr2=0x7ff9000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:42.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.646+0000 7ff91de16640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff9000a32c0 0x7ff9000a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff908009990 tx=0x7ff908009110 comp rx=0 tx=0).stop 2024-09-19T04:12:42.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.646+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91810c170 msgr2=0x7ff91819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:42.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.646+0000 7ff91de16640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91810c170 0x7ff91819ccc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff91000eaa0 tx=0x7ff91000eea0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.646+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 shutdown_connections 2024-09-19T04:12:42.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.646+0000 7ff91de16640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff9000a32c0 0x7ff9000a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.647+0000 7ff91de16640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91810c170 0x7ff91819ccc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.647+0000 7ff91de16640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff918103660 0x7ff918198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.647+0000 7ff91de16640 1 --2- 172.21.15.94:0/2387740431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918102d40 0x7ff9181983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:42.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.647+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 >> 172.21.15.94:0/2387740431 conn(0x7ff9180fc7a0 msgr2=0x7ff9180fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:42.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.647+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 shutdown_connections 2024-09-19T04:12:42.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:42.647+0000 7ff91de16640 1 -- 172.21.15.94:0/2387740431 wait complete. 2024-09-19T04:12:43.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: Deploying daemon mds.d on smithi094 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.94:6834/422186833,v1:172.21.15.94:6835/422186833] up:boot 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: fsmap 3 up:standby 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:12:43.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:42 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2387740431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: Deploying daemon mds.d on smithi094 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.94:6834/422186833,v1:172.21.15.94:6835/422186833] up:boot 2024-09-19T04:12:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: fsmap 3 up:standby 2024-09-19T04:12:43.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:12:43.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:42 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2387740431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:43.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: Deploying daemon mds.d on smithi094 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.94:6834/422186833,v1:172.21.15.94:6835/422186833] up:boot 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: fsmap 3 up:standby 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:12:43.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:42 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2387740431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:45.091 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:12:44 smithi094 systemd[1]: Starting Ceph mds.d for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:45.393 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:45 smithi094 ceph-mon[30972]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:45.393 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:12:45 smithi094 podman[71132]: 2024-09-19 04:12:45.02583345 +0000 UTC m=+0.016096326 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:45.393 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:12:45 smithi094 podman[71132]: 2024-09-19 04:12:45.156419707 +0000 UTC m=+0.146682632 container create 094be904acb43a0c7d203a1c1846bbf23336abd159cc359d854329bc8116bc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T04:12:45.393 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:12:45 smithi094 podman[71132]: 2024-09-19 04:12:45.392266779 +0000 UTC m=+0.382529655 container init 094be904acb43a0c7d203a1c1846bbf23336abd159cc359d854329bc8116bc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:12:45.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:45 smithi112 ceph-mon[35051]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:45 smithi104 ceph-mon[34145]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:45.840 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:12:45 smithi094 podman[71132]: 2024-09-19 04:12:45.395079203 +0000 UTC m=+0.385342087 container start 094be904acb43a0c7d203a1c1846bbf23336abd159cc359d854329bc8116bc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:12:45.841 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:12:45 smithi094 ceph-mds[71151]: -- 172.21.15.94:0/3561348639 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4d301f860 con 0x55f4d2ffd400 2024-09-19T04:12:45.841 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:12:45 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-d[71147]: starting mds.d at 2024-09-19T04:12:45.841 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:12:45 smithi094 bash[71132]: 094be904acb43a0c7d203a1c1846bbf23336abd159cc359d854329bc8116bc8d 2024-09-19T04:12:45.841 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:12:45 smithi094 systemd[1]: Started Ceph mds.d for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:45.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:12:46.128 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:46.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 -- 172.21.15.94:0/2247317357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee4102d30 msgr2=0x7fcee4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:46.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 --2- 172.21.15.94:0/2247317357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee4102d30 0x7fcee4103110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009930 tx=0x7fcedc02a440 comp rx=0 tx=0).stop 2024-09-19T04:12:46.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 -- 172.21.15.94:0/2247317357 shutdown_connections 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 --2- 172.21.15.94:0/2247317357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee41082a0 0x7fcee410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 --2- 172.21.15.94:0/2247317357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee41036e0 0x7fcee4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 --2- 172.21.15.94:0/2247317357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee4102d30 0x7fcee4103110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 -- 172.21.15.94:0/2247317357 >> 172.21.15.94:0/2247317357 conn(0x7fcee40fc7d0 msgr2=0x7fcee40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 -- 172.21.15.94:0/2247317357 shutdown_connections 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 -- 172.21.15.94:0/2247317357 wait complete. 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 Processor -- start 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.651+0000 7fceec35a640 1 -- start start 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceec35a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4102d30 0x7fcee4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceec35a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee41036e0 0x7fcee4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceec35a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee41082a0 0x7fcee419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceec35a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4111f90 con 0x7fcee41082a0 2024-09-19T04:12:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceec35a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4111e10 con 0x7fcee41036e0 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceec35a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4112110 con 0x7fcee4102d30 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea8f7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee41082a0 0x7fcee419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea8f7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee41082a0 0x7fcee419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51116/0 (socket says 172.21.15.94:51116) 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea8f7640 1 -- 172.21.15.94:0/738003261 learned_addr learned my addr 172.21.15.94:0/738003261 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fcee98f5640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee41036e0 0x7fcee4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea0f6640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4102d30 0x7fcee4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea8f7640 1 -- 172.21.15.94:0/738003261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4102d30 msgr2=0x7fcee4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea8f7640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4102d30 0x7fcee4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea8f7640 1 -- 172.21.15.94:0/738003261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee41036e0 msgr2=0x7fcee4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea8f7640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee41036e0 0x7fcee4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea8f7640 1 -- 172.21.15.94:0/738003261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee419d500 con 0x7fcee41082a0 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceea8f7640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee41082a0 0x7fcee419ce20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fced400e970 tx=0x7fced400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fcee37fe640 1 -- 172.21.15.94:0/738003261 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced400ccc0 con 0x7fcee41082a0 2024-09-19T04:12:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fcee37fe640 1 -- 172.21.15.94:0/738003261 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fced400ce60 con 0x7fcee41082a0 2024-09-19T04:12:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee419d7f0 con 0x7fcee41082a0 2024-09-19T04:12:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fcee37fe640 1 -- 172.21.15.94:0/738003261 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4010430 con 0x7fcee41082a0 2024-09-19T04:12:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.652+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee41a50d0 con 0x7fcee41082a0 2024-09-19T04:12:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.653+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee406a850 con 0x7fcee41082a0 2024-09-19T04:12:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.655+0000 7fcee37fe640 1 -- 172.21.15.94:0/738003261 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fced4004fb0 con 0x7fcee41082a0 2024-09-19T04:12:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.655+0000 7fcee37fe640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcecc0a3240 0x7fcecc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.655+0000 7fceea0f6640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcecc0a3240 0x7fcecc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.655+0000 7fcee37fe640 1 -- 172.21.15.94:0/738003261 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fced40cfa40 con 0x7fcee41082a0 2024-09-19T04:12:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.656+0000 7fceea0f6640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcecc0a3240 0x7fcecc0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009900 tx=0x7fcedc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:46.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.659+0000 7fcee37fe640 1 -- 172.21.15.94:0/738003261 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fced4054210 con 0x7fcee41082a0 2024-09-19T04:12:46.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: Deploying daemon mds.e on smithi104 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.94:6836/3764912957,v1:172.21.15.94:6837/3764912957] up:boot 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: fsmap 4 up:standby 2024-09-19T04:12:46.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:46 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:12:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: Deploying daemon mds.e on smithi104 2024-09-19T04:12:46.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:46.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.94:6836/3764912957,v1:172.21.15.94:6837/3764912957] up:boot 2024-09-19T04:12:46.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: fsmap 4 up:standby 2024-09-19T04:12:46.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:46 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:12:46.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.860+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcee4103110 con 0x7fcee41082a0 2024-09-19T04:12:46.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.861+0000 7fcee37fe640 1 -- 172.21.15.94:0/738003261 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7fcee4103110 con 0x7fcee41082a0 2024-09-19T04:12:46.862 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:46.862 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":5,"btime":"2024-09-19T04:12:46:177424+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:12:46.862 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 5 2024-09-19T04:12:46.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcecc0a3240 msgr2=0x7fcecc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:46.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcecc0a3240 0x7fcecc0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009900 tx=0x7fcedc000bc0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee41082a0 msgr2=0x7fcee419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:46.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee41082a0 0x7fcee419ce20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fced400e970 tx=0x7fced400ed70 comp rx=0 tx=0).stop 2024-09-19T04:12:46.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 shutdown_connections 2024-09-19T04:12:46.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fcecc0a3240 0x7fcecc0a5700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcee41082a0 0x7fcee419ce20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee41036e0 0x7fcee4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 --2- 172.21.15.94:0/738003261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4102d30 0x7fcee4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:46.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 >> 172.21.15.94:0/738003261 conn(0x7fcee40fc7d0 msgr2=0x7fcee4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:46.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 shutdown_connections 2024-09-19T04:12:46.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:46.863+0000 7fceec35a640 1 -- 172.21.15.94:0/738003261 wait complete. 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: Deploying daemon mds.e on smithi104 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.94:6836/3764912957,v1:172.21.15.94:6837/3764912957] up:boot 2024-09-19T04:12:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: fsmap 4 up:standby 2024-09-19T04:12:47.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:46 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:12:47.556 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:12:47 smithi104 systemd[1]: Starting Ceph mds.e for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:47.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:47 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/738003261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:47 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/738003261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:47.854 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:12:47 smithi104 podman[63337]: 2024-09-19 04:12:47.619751724 +0000 UTC m=+0.109037138 container create 91311a4e8208cd9029a0fc7ae900609f98acdbdcb82d7feddda3d3df797bd415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T04:12:47.854 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:12:47 smithi104 podman[63337]: 2024-09-19 04:12:47.526287881 +0000 UTC m=+0.015573249 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:48.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:47 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/738003261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:48.354 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:12:47 smithi104 podman[63337]: 2024-09-19 04:12:47.863519049 +0000 UTC m=+0.352804426 container init 91311a4e8208cd9029a0fc7ae900609f98acdbdcb82d7feddda3d3df797bd415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:12:48.354 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:12:47 smithi104 podman[63337]: 2024-09-19 04:12:47.866391369 +0000 UTC m=+0.355676744 container start 91311a4e8208cd9029a0fc7ae900609f98acdbdcb82d7feddda3d3df797bd415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T04:12:48.354 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:12:47 smithi104 ceph-mds[63356]: -- 172.21.15.104:0/2525079875 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567a81cf860 con 0x5567a81ac800 2024-09-19T04:12:48.354 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:12:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e[63352]: starting mds.e at 2024-09-19T04:12:48.355 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:12:47 smithi104 bash[63337]: 91311a4e8208cd9029a0fc7ae900609f98acdbdcb82d7feddda3d3df797bd415 2024-09-19T04:12:48.355 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:12:47 smithi104 systemd[1]: Started Ceph mds.e for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: Deploying daemon mds.f on smithi112 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.104:6834/2064839900,v1:172.21.15.104:6835/2064839900] up:boot 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: fsmap 5 up:standby 2024-09-19T04:12:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:49 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:12:49.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:49.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:49.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:49.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:49.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:49.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:49.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: Deploying daemon mds.f on smithi112 2024-09-19T04:12:49.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:49.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.104:6834/2064839900,v1:172.21.15.104:6835/2064839900] up:boot 2024-09-19T04:12:49.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: fsmap 5 up:standby 2024-09-19T04:12:49.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:49 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:12:49.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:49.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:49.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:49.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:49.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:49.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:49.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: Deploying daemon mds.f on smithi112 2024-09-19T04:12:49.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:49.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.104:6834/2064839900,v1:172.21.15.104:6835/2064839900] up:boot 2024-09-19T04:12:49.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: fsmap 5 up:standby 2024-09-19T04:12:49.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:49 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:12:49.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:12:49.914 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:50.035 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:12:49 smithi112 systemd[1]: Starting Ceph mds.f for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:50.314 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:12:50 smithi112 podman[64817]: 2024-09-19 04:12:49.952136589 +0000 UTC m=+0.015468172 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:50.315 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:12:50 smithi112 podman[64817]: 2024-09-19 04:12:50.21180254 +0000 UTC m=+0.275134115 container create 8f1c7824f25b3978db8ce5a3ebf553697e4fde293deca684cd7268b9d513d62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T04:12:50.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 -- 172.21.15.94:0/3915171431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0106f00 msgr2=0x7f19e010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:50.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 --2- 172.21.15.94:0/3915171431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0106f00 0x7f19e010ea80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009900 tx=0x7f19d002a410 comp rx=0 tx=0).stop 2024-09-19T04:12:50.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 -- 172.21.15.94:0/3915171431 shutdown_connections 2024-09-19T04:12:50.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 --2- 172.21.15.94:0/3915171431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0106f00 0x7f19e010ea80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:50.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 --2- 172.21.15.94:0/3915171431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e0102780 0x7f19e01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:50.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 --2- 172.21.15.94:0/3915171431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e0101dd0 0x7f19e01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 -- 172.21.15.94:0/3915171431 >> 172.21.15.94:0/3915171431 conn(0x7f19e00fd8b0 msgr2=0x7f19e00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 -- 172.21.15.94:0/3915171431 shutdown_connections 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 -- 172.21.15.94:0/3915171431 wait complete. 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.447+0000 7f19e878b640 1 Processor -- start 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e878b640 1 -- start start 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e878b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0101dd0 0x7f19e0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e878b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e0102780 0x7f19e01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e878b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e0106f00 0x7f19e019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e878b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0111e10 con 0x7f19e0101dd0 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e878b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0111c90 con 0x7f19e0102780 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e878b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0111f90 con 0x7f19e0106f00 2024-09-19T04:12:50.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6527640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0101dd0 0x7f19e0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6d28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e0106f00 0x7f19e019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6527640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0101dd0 0x7f19e0198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51128/0 (socket says 172.21.15.94:51128) 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6527640 1 -- 172.21.15.94:0/3177973319 learned_addr learned my addr 172.21.15.94:0/3177973319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e5d26640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e0102780 0x7f19e01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6d28640 1 -- 172.21.15.94:0/3177973319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e0102780 msgr2=0x7f19e01987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6d28640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e0102780 0x7f19e01987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6d28640 1 -- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0101dd0 msgr2=0x7f19e0198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6d28640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0101dd0 0x7f19e0198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6d28640 1 -- 172.21.15.94:0/3177973319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e019d220 con 0x7f19e0106f00 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6527640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0101dd0 0x7f19e0198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:12:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.448+0000 7f19e6d28640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e0106f00 0x7f19e019cb40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005940 tx=0x7f19d0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.449+0000 7f19df7fe640 1 -- 172.21.15.94:0/3177973319 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d003f030 con 0x7f19e0106f00 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.449+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e019d4b0 con 0x7f19e0106f00 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.449+0000 7f19df7fe640 1 -- 172.21.15.94:0/3177973319 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19d0004440 con 0x7f19e0106f00 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.449+0000 7f19df7fe640 1 -- 172.21.15.94:0/3177973319 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0035070 con 0x7f19e0106f00 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.449+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e01a4df0 con 0x7f19e0106f00 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.450+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e01021b0 con 0x7f19e0106f00 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.452+0000 7f19df7fe640 1 -- 172.21.15.94:0/3177973319 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f19d0042030 con 0x7f19e0106f00 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.452+0000 7f19df7fe640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f19c80a3240 0x7f19c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.452+0000 7f19e6527640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f19c80a3240 0x7f19c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.452+0000 7f19df7fe640 1 -- 172.21.15.94:0/3177973319 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f19d00efa20 con 0x7f19e0106f00 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.453+0000 7f19e6527640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f19c80a3240 0x7f19c80a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f19d8011500 tx=0x7f19d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.455+0000 7f19df7fe640 1 -- 172.21.15.94:0/3177973319 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f19d00bcbe0 con 0x7f19e0106f00 2024-09-19T04:12:50.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.674+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19e01995d0 con 0x7f19e0106f00 2024-09-19T04:12:50.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.674+0000 7f19df7fe640 1 -- 172.21.15.94:0/3177973319 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7f19d00bc4e0 con 0x7f19e0106f00 2024-09-19T04:12:50.676 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:50.676 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":6,"btime":"2024-09-19T04:12:48:554580+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T04:12:50.677 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 6 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f19c80a3240 msgr2=0x7f19c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f19c80a3240 0x7f19c80a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f19d8011500 tx=0x7f19d8005f90 comp rx=0 tx=0).stop 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e0106f00 msgr2=0x7f19e019cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e0106f00 0x7f19e019cb40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005940 tx=0x7f19d0004040 comp rx=0 tx=0).stop 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 shutdown_connections 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f19c80a3240 0x7f19c80a5700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e0106f00 0x7f19e019cb40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e0102780 0x7f19e01987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 --2- 172.21.15.94:0/3177973319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e0101dd0 0x7f19e0198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 >> 172.21.15.94:0/3177973319 conn(0x7f19e00fd8b0 msgr2=0x7f19e00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 shutdown_connections 2024-09-19T04:12:50.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:50.678+0000 7f19e878b640 1 -- 172.21.15.94:0/3177973319 wait complete. 2024-09-19T04:12:50.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:12:50 smithi112 podman[64817]: 2024-09-19 04:12:50.431392946 +0000 UTC m=+0.494724526 container init 8f1c7824f25b3978db8ce5a3ebf553697e4fde293deca684cd7268b9d513d62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T04:12:50.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:12:50 smithi112 podman[64817]: 2024-09-19 04:12:50.434032247 +0000 UTC m=+0.497363823 container start 8f1c7824f25b3978db8ce5a3ebf553697e4fde293deca684cd7268b9d513d62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T04:12:50.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:12:50 smithi112 ceph-mds[64836]: -- 172.21.15.112:0/3333441517 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a01622f860 con 0x55a01620d400 2024-09-19T04:12:50.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:12:50 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f[64832]: starting mds.f at 2024-09-19T04:12:50.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:12:50 smithi112 bash[64817]: 8f1c7824f25b3978db8ce5a3ebf553697e4fde293deca684cd7268b9d513d62b 2024-09-19T04:12:50.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:12:50 smithi112 systemd[1]: Started Ceph mds.f for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:51 smithi112 ceph-mon[35051]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:51.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:51.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:51 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:51.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:51 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3177973319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:51.437 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:51 smithi094 ceph-mon[30972]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:51.437 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:51.437 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:51.437 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:51.438 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:51.438 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:51.438 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:51 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:51.438 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:51 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3177973319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:51.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:51 smithi104 ceph-mon[34145]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:51.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:51.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:51.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:51.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:51.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:51.466 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:51 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:51.466 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:51 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3177973319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:52.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:52 smithi104 ceph-mon[34145]: Deploying daemon mds.c on smithi112 2024-09-19T04:12:52.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:52 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.112:6834/1098195473,v1:172.21.15.112:6835/1098195473] up:boot 2024-09-19T04:12:52.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:52 smithi104 ceph-mon[34145]: fsmap 6 up:standby 2024-09-19T04:12:52.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:52 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:12:52.489 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:52 smithi112 ceph-mon[35051]: Deploying daemon mds.c on smithi112 2024-09-19T04:12:52.489 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:52 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.112:6834/1098195473,v1:172.21.15.112:6835/1098195473] up:boot 2024-09-19T04:12:52.489 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:52 smithi112 ceph-mon[35051]: fsmap 6 up:standby 2024-09-19T04:12:52.489 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:52 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:12:52.490 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:12:52 smithi112 systemd[1]: Starting Ceph mds.c for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:52.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:52 smithi094 ceph-mon[30972]: Deploying daemon mds.c on smithi112 2024-09-19T04:12:52.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:52 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.112:6834/1098195473,v1:172.21.15.112:6835/1098195473] up:boot 2024-09-19T04:12:52.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:52 smithi094 ceph-mon[30972]: fsmap 6 up:standby 2024-09-19T04:12:52.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:52 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:12:52.744 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:12:52 smithi112 podman[65119]: 2024-09-19 04:12:52.39082459 +0000 UTC m=+0.015695254 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:52.744 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:12:52 smithi112 podman[65119]: 2024-09-19 04:12:52.523784017 +0000 UTC m=+0.148654680 container create fc3ebc87679e78d69e922b17d133249eedacb810eac88531c8e9e92a72813af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T04:12:52.744 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:12:52 smithi112 podman[65119]: 2024-09-19 04:12:52.743170064 +0000 UTC m=+0.368040732 container init fc3ebc87679e78d69e922b17d133249eedacb810eac88531c8e9e92a72813af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T04:12:53.035 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:12:52 smithi112 podman[65119]: 2024-09-19 04:12:52.746072336 +0000 UTC m=+0.370942996 container start fc3ebc87679e78d69e922b17d133249eedacb810eac88531c8e9e92a72813af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T04:12:53.035 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:12:52 smithi112 ceph-mds[65139]: -- 172.21.15.112:0/1894265208 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617c0cb3860 con 0x5617c0c91000 2024-09-19T04:12:53.035 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:12:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c[65135]: starting mds.c at 2024-09-19T04:12:53.035 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:12:52 smithi112 bash[65119]: fc3ebc87679e78d69e922b17d133249eedacb810eac88531c8e9e92a72813af1 2024-09-19T04:12:53.035 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:12:52 smithi112 systemd[1]: Started Ceph mds.c for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:53.440 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:12:53.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:53 smithi104 ceph-mon[34145]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:53.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:53.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:53.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:53.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:53.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:53.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:53 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:53.469 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:53 smithi094 ceph-mon[30972]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:53.469 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:53.470 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:53.470 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:53.470 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:53.470 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:53.470 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:53 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:53 smithi112 ceph-mon[35051]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:53 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:54.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.151+0000 7ff385b5d640 1 -- 172.21.15.94:0/2315079960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 msgr2=0x7ff3801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:54.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.151+0000 7ff385b5d640 1 --2- 172.21.15.94:0/2315079960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 0x7ff3801021c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff370009900 tx=0x7ff37002a410 comp rx=0 tx=0).stop 2024-09-19T04:12:54.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.151+0000 7ff385b5d640 1 -- 172.21.15.94:0/2315079960 shutdown_connections 2024-09-19T04:12:54.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.151+0000 7ff385b5d640 1 --2- 172.21.15.94:0/2315079960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff380106f10 0x7ff38010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:54.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.151+0000 7ff385b5d640 1 --2- 172.21.15.94:0/2315079960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff380102790 0x7ff3801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.151+0000 7ff385b5d640 1 --2- 172.21.15.94:0/2315079960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 0x7ff3801021c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.151+0000 7ff385b5d640 1 -- 172.21.15.94:0/2315079960 >> 172.21.15.94:0/2315079960 conn(0x7ff3800fd8c0 msgr2=0x7ff3800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.151+0000 7ff385b5d640 1 -- 172.21.15.94:0/2315079960 shutdown_connections 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.151+0000 7ff385b5d640 1 -- 172.21.15.94:0/2315079960 wait complete. 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff385b5d640 1 Processor -- start 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff385b5d640 1 -- start start 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff385b5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 0x7ff380198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff385b5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff380102790 0x7ff380198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff385b5d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff380106f10 0x7ff38019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:54.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff385b5d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111e60 con 0x7ff380101de0 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff385b5d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111ce0 con 0x7ff380106f10 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff385b5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111fe0 con 0x7ff380102790 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 0x7ff380198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 0x7ff380198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36408/0 (socket says 172.21.15.94:36408) 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff37f7fe640 1 -- 172.21.15.94:0/3279232042 learned_addr learned my addr 172.21.15.94:0/3279232042 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff37effd640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff380102790 0x7ff380198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff37ffff640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff380106f10 0x7ff38019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff37f7fe640 1 -- 172.21.15.94:0/3279232042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff380102790 msgr2=0x7ff380198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff37f7fe640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff380102790 0x7ff380198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff37f7fe640 1 -- 172.21.15.94:0/3279232042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff380106f10 msgr2=0x7ff38019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:54.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.152+0000 7ff37f7fe640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff380106f10 0x7ff38019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:54.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.153+0000 7ff37f7fe640 1 -- 172.21.15.94:0/3279232042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38019d3d0 con 0x7ff380101de0 2024-09-19T04:12:54.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.153+0000 7ff37ffff640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff380106f10 0x7ff38019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:12:54.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.153+0000 7ff37f7fe640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 0x7ff380198420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff370006010 tx=0x7ff370004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:54.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.153+0000 7ff37cff9640 1 -- 172.21.15.94:0/3279232042 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37003f030 con 0x7ff380101de0 2024-09-19T04:12:54.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.153+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38019d660 con 0x7ff380101de0 2024-09-19T04:12:54.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.153+0000 7ff37cff9640 1 -- 172.21.15.94:0/3279232042 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3700035f0 con 0x7ff380101de0 2024-09-19T04:12:54.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.153+0000 7ff37cff9640 1 -- 172.21.15.94:0/3279232042 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370035070 con 0x7ff380101de0 2024-09-19T04:12:54.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.153+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3801a4fa0 con 0x7ff380101de0 2024-09-19T04:12:54.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.154+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3801032c0 con 0x7ff380101de0 2024-09-19T04:12:54.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.155+0000 7ff37cff9640 1 -- 172.21.15.94:0/3279232042 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff370003b10 con 0x7ff380101de0 2024-09-19T04:12:54.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.156+0000 7ff37cff9640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3680a3240 0x7ff3680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:54.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.156+0000 7ff37effd640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3680a3240 0x7ff3680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:54.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.156+0000 7ff37cff9640 1 -- 172.21.15.94:0/3279232042 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7ff3700f00c0 con 0x7ff380101de0 2024-09-19T04:12:54.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.156+0000 7ff37effd640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3680a3240 0x7ff3680a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff3800ffe30 tx=0x7ff374008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:54.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.160+0000 7ff37cff9640 1 -- 172.21.15.94:0/3279232042 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff3700bd260 con 0x7ff380101de0 2024-09-19T04:12:54.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:54 smithi094 ceph-mon[30972]: Deploying daemon mds.h on smithi104 2024-09-19T04:12:54.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:54 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] up:boot 2024-09-19T04:12:54.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:54 smithi094 ceph-mon[30972]: fsmap 7 up:standby 2024-09-19T04:12:54.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:54 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:12:54.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.368+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff38010a0f0 con 0x7ff380101de0 2024-09-19T04:12:54.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.369+0000 7ff37cff9640 1 -- 172.21.15.94:0/3279232042 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6394 (secure 0 0 0) 0x7ff3700bcb60 con 0x7ff380101de0 2024-09-19T04:12:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:54.371 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"btime":"2024-09-19T04:12:53:187528+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T04:12:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-09-19T04:12:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3680a3240 msgr2=0x7ff3680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3680a3240 0x7ff3680a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff3800ffe30 tx=0x7ff374008040 comp rx=0 tx=0).stop 2024-09-19T04:12:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 msgr2=0x7ff380198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 0x7ff380198420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff370006010 tx=0x7ff370004270 comp rx=0 tx=0).stop 2024-09-19T04:12:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 shutdown_connections 2024-09-19T04:12:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3680a3240 0x7ff3680a5700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff380106f10 0x7ff38019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff380102790 0x7ff380198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 --2- 172.21.15.94:0/3279232042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff380101de0 0x7ff380198420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:54.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 >> 172.21.15.94:0/3279232042 conn(0x7ff3800fd8c0 msgr2=0x7ff3800fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:54.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 shutdown_connections 2024-09-19T04:12:54.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:54.371+0000 7ff385b5d640 1 -- 172.21.15.94:0/3279232042 wait complete. 2024-09-19T04:12:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:54 smithi112 ceph-mon[35051]: Deploying daemon mds.h on smithi104 2024-09-19T04:12:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:54 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] up:boot 2024-09-19T04:12:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:54 smithi112 ceph-mon[35051]: fsmap 7 up:standby 2024-09-19T04:12:54.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:54 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:12:54.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:54 smithi104 ceph-mon[34145]: Deploying daemon mds.h on smithi104 2024-09-19T04:12:54.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:54 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] up:boot 2024-09-19T04:12:54.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:54 smithi104 ceph-mon[34145]: fsmap 7 up:standby 2024-09-19T04:12:54.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:54 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:12:55.030 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:12:54 smithi104 systemd[1]: Starting Ceph mds.h for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:55.308 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:55 smithi104 ceph-mon[34145]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:55.308 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:55 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3279232042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:55.308 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:12:55 smithi104 podman[63639]: 2024-09-19 04:12:54.932073274 +0000 UTC m=+0.016070536 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:55.308 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:12:55 smithi104 podman[63639]: 2024-09-19 04:12:55.063169814 +0000 UTC m=+0.147167075 container create fd7a16416277a40958965506b5d5588a722d16cf5a3806f26437010301204a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T04:12:55.308 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:12:55 smithi104 podman[63639]: 2024-09-19 04:12:55.307400602 +0000 UTC m=+0.391397869 container init fd7a16416277a40958965506b5d5588a722d16cf5a3806f26437010301204a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:12:55.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:55 smithi112 ceph-mon[35051]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:55.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:55 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3279232042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:55 smithi094 ceph-mon[30972]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:55.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:55 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3279232042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:55.604 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:12:55 smithi104 podman[63639]: 2024-09-19 04:12:55.310620905 +0000 UTC m=+0.394618176 container start fd7a16416277a40958965506b5d5588a722d16cf5a3806f26437010301204a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T04:12:55.604 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:12:55 smithi104 ceph-mds[63658]: -- 172.21.15.104:0/1267530818 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55771409f860 con 0x55771407d400 2024-09-19T04:12:55.604 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:12:55 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h[63654]: starting mds.h at 2024-09-19T04:12:55.604 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:12:55 smithi104 bash[63639]: fd7a16416277a40958965506b5d5588a722d16cf5a3806f26437010301204a34 2024-09-19T04:12:55.605 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:12:55 smithi104 systemd[1]: Started Ceph mds.h for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:56.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:56.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:56.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:56.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:56.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:56.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:56.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: Deploying daemon mds.i on smithi112 2024-09-19T04:12:56.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:56.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.104:6836/2682504501,v1:172.21.15.104:6837/2682504501] up:boot 2024-09-19T04:12:56.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: fsmap 8 up:standby 2024-09-19T04:12:56.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:56 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:12:56.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: Deploying daemon mds.i on smithi112 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.104:6836/2682504501,v1:172.21.15.104:6837/2682504501] up:boot 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: fsmap 8 up:standby 2024-09-19T04:12:56.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:56 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:12:56.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:56.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:56.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:56.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:56.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:56.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:56.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: Deploying daemon mds.i on smithi112 2024-09-19T04:12:56.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:56.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.104:6836/2682504501,v1:172.21.15.104:6837/2682504501] up:boot 2024-09-19T04:12:56.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: fsmap 8 up:standby 2024-09-19T04:12:56.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:56 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:12:57.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:12:57.291 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:12:57.505 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:12:57 smithi112 systemd[1]: Starting Ceph mds.i for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:57.785 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:12:57 smithi112 podman[65420]: 2024-09-19 04:12:57.406856047 +0000 UTC m=+0.015795929 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:57.785 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:12:57 smithi112 podman[65420]: 2024-09-19 04:12:57.51101184 +0000 UTC m=+0.119951706 container create 00741c13c1113e5228298c77ce380e31eae0f7c8bddd81b3153c971d7bfbb2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T04:12:57.786 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:12:57 smithi112 podman[65420]: 2024-09-19 04:12:57.713434609 +0000 UTC m=+0.322374482 container init 00741c13c1113e5228298c77ce380e31eae0f7c8bddd81b3153c971d7bfbb2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T04:12:57.786 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:12:57 smithi112 podman[65420]: 2024-09-19 04:12:57.716327412 +0000 UTC m=+0.325267279 container start 00741c13c1113e5228298c77ce380e31eae0f7c8bddd81b3153c971d7bfbb2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T04:12:57.786 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:12:57 smithi112 ceph-mds[65439]: -- 172.21.15.112:0/4160852120 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e7a8c9860 con 0x564e7a8a7000 2024-09-19T04:12:57.786 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:12:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i[65435]: starting mds.i at 2024-09-19T04:12:57.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.808+0000 7f4bfcafd640 1 -- 172.21.15.94:0/3531595151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81082a0 msgr2=0x7f4bf810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:57.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.808+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/3531595151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81082a0 0x7f4bf810ec40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009930 tx=0x7f4bf002a420 comp rx=0 tx=0).stop 2024-09-19T04:12:57.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 -- 172.21.15.94:0/3531595151 shutdown_connections 2024-09-19T04:12:57.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/3531595151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81082a0 0x7f4bf810ec40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:57.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/3531595151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf81036e0 0x7f4bf8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:57.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/3531595151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf8102d30 0x7f4bf8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:57.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 -- 172.21.15.94:0/3531595151 >> 172.21.15.94:0/3531595151 conn(0x7f4bf80fc7d0 msgr2=0x7f4bf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:57.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 -- 172.21.15.94:0/3531595151 shutdown_connections 2024-09-19T04:12:57.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 -- 172.21.15.94:0/3531595151 wait complete. 2024-09-19T04:12:57.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 Processor -- start 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 -- start start 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf8102d30 0x7f4bf8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.809+0000 7f4bfcafd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81036e0 0x7f4bf8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bfcafd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf81082a0 0x7f4bf8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bfcafd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8111f50 con 0x7f4bf81036e0 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bfcafd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8111dd0 con 0x7f4bf81082a0 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bfcafd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf81120d0 con 0x7f4bf8102d30 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81036e0 0x7f4bf8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81036e0 0x7f4bf8100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36418/0 (socket says 172.21.15.94:36418) 2024-09-19T04:12:57.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf5d9b640 1 -- 172.21.15.94:0/1287829179 learned_addr learned my addr 172.21.15.94:0/1287829179 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:12:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf659c640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf8102d30 0x7f4bf8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf6d9d640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf81082a0 0x7f4bf8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf5d9b640 1 -- 172.21.15.94:0/1287829179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf8102d30 msgr2=0x7f4bf8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf5d9b640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf8102d30 0x7f4bf8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf5d9b640 1 -- 172.21.15.94:0/1287829179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf81082a0 msgr2=0x7f4bf8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf5d9b640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf81082a0 0x7f4bf8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf5d9b640 1 -- 172.21.15.94:0/1287829179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf81a68e0 con 0x7f4bf81036e0 2024-09-19T04:12:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bf5d9b640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81036e0 0x7f4bf8100ed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c8c0 tx=0x7f4be000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bef7fe640 1 -- 172.21.15.94:0/1287829179 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0007bb0 con 0x7f4bf81036e0 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.810+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf81a6bd0 con 0x7f4bf81036e0 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.811+0000 7f4bef7fe640 1 -- 172.21.15.94:0/1287829179 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4be0007d50 con 0x7f4bf81036e0 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.811+0000 7f4bef7fe640 1 -- 172.21.15.94:0/1287829179 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be00187c0 con 0x7f4bf81036e0 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.811+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf81a7160 con 0x7f4bf81036e0 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.812+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf8103110 con 0x7f4bf81036e0 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.813+0000 7f4bef7fe640 1 -- 172.21.15.94:0/1287829179 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4be000f450 con 0x7f4bf81036e0 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.813+0000 7f4bef7fe640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4be40a3240 0x7f4be40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.814+0000 7f4bf659c640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4be40a3240 0x7f4be40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.814+0000 7f4bef7fe640 1 -- 172.21.15.94:0/1287829179 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f4be00cf410 con 0x7f4bf81036e0 2024-09-19T04:12:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.814+0000 7f4bf659c640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4be40a3240 0x7f4be40a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005fd0 tx=0x7f4be8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:57.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:57.818+0000 7f4bef7fe640 1 -- 172.21.15.94:0/1287829179 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4be0014030 con 0x7f4bf81036e0 2024-09-19T04:12:58.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.029+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bf810a2b0 con 0x7f4bf81036e0 2024-09-19T04:12:58.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.030+0000 7f4bef7fe640 1 -- 172.21.15.94:0/1287829179 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8052 (secure 0 0 0) 0x7f4be009c5d0 con 0x7f4bf81036e0 2024-09-19T04:12:58.031 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:12:58.031 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"btime":"2024-09-19T04:12:57:794842+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T04:12:58.032 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 10 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4be40a3240 msgr2=0x7f4be40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4be40a3240 0x7f4be40a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005fd0 tx=0x7f4be8009500 comp rx=0 tx=0).stop 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81036e0 msgr2=0x7f4bf8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81036e0 0x7f4bf8100ed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c8c0 tx=0x7f4be000cd80 comp rx=0 tx=0).stop 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 shutdown_connections 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4be40a3240 0x7f4be40a5700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf81082a0 0x7f4bf8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bf81036e0 0x7f4bf8100ed0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 --2- 172.21.15.94:0/1287829179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf8102d30 0x7f4bf8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:58.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 >> 172.21.15.94:0/1287829179 conn(0x7f4bf80fc7d0 msgr2=0x7f4bf8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:58.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 shutdown_connections 2024-09-19T04:12:58.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:12:58.032+0000 7f4bfcafd640 1 -- 172.21.15.94:0/1287829179 wait complete. 2024-09-19T04:12:58.285 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:12:57 smithi112 bash[65420]: 00741c13c1113e5228298c77ce380e31eae0f7c8bddd81b3153c971d7bfbb2ab 2024-09-19T04:12:58.285 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:12:57 smithi112 systemd[1]: Started Ceph mds.i for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] up:boot 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: fsmap 9 up:standby 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: Deploying daemon mds.g on smithi094 2024-09-19T04:12:58.997 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1287829179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:58.998 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:12:58 smithi094 ceph-mon[30972]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] up:boot 2024-09-19T04:12:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: fsmap 9 up:standby 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: Deploying daemon mds.g on smithi094 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1287829179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:59.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:12:58 smithi104 ceph-mon[34145]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] up:boot 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: fsmap 9 up:standby 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: Deploying daemon mds.g on smithi094 2024-09-19T04:12:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1287829179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:12:59.286 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:12:58 smithi112 ceph-mon[35051]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:00.562 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:13:00 smithi094 systemd[1]: Starting Ceph mds.g for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:13:00.710 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:13:00.812 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:13:00 smithi094 podman[71903]: 2024-09-19 04:13:00.463574295 +0000 UTC m=+0.015881469 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:13:00.812 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:13:00 smithi094 podman[71903]: 2024-09-19 04:13:00.583840489 +0000 UTC m=+0.136147662 container create d640449f79613d7332d8e12aeb8bf9c02b5ce4f8d860aba97640d77e62f16ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T04:13:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:01.091 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:13:00 smithi094 podman[71903]: 2024-09-19 04:13:00.811645205 +0000 UTC m=+0.363952379 container init d640449f79613d7332d8e12aeb8bf9c02b5ce4f8d860aba97640d77e62f16ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T04:13:01.091 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:13:00 smithi094 podman[71903]: 2024-09-19 04:13:00.814897894 +0000 UTC m=+0.367205071 container start d640449f79613d7332d8e12aeb8bf9c02b5ce4f8d860aba97640d77e62f16ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-g, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T04:13:01.091 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:13:00 smithi094 ceph-mds[71940]: -- 172.21.15.94:0/3362057264 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7c0a79860 con 0x55f7c0a57400 2024-09-19T04:13:01.091 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:13:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-g[71918]: starting mds.g at 2024-09-19T04:13:01.091 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:13:00 smithi094 bash[71903]: d640449f79613d7332d8e12aeb8bf9c02b5ce4f8d860aba97640d77e62f16ecd 2024-09-19T04:13:01.091 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:13:00 smithi094 systemd[1]: Started Ceph mds.g for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:13:01.366 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:01 smithi094 ceph-mon[30972]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:01.366 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:01.366 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:01.367 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:01.367 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:13:01.367 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:13:01.367 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:01 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 -- 172.21.15.94:0/1493733309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102ef0 msgr2=0x7fe8bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1493733309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102ef0 0x7fe8bc10da40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009900 tx=0x7fe8b402a480 comp rx=0 tx=0).stop 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 -- 172.21.15.94:0/1493733309 shutdown_connections 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1493733309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102ef0 0x7fe8bc10da40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1493733309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8bc102570 0x7fe8bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1493733309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8bc100e20 0x7fe8bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 -- 172.21.15.94:0/1493733309 >> 172.21.15.94:0/1493733309 conn(0x7fe8bc0fc7d0 msgr2=0x7fe8bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 -- 172.21.15.94:0/1493733309 shutdown_connections 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 -- 172.21.15.94:0/1493733309 wait complete. 2024-09-19T04:13:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 Processor -- start 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.448+0000 7fe8c45db640 1 -- start start 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c45db640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8bc100e20 0x7fe8bc10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c45db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102570 0x7fe8bc10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c45db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8bc102ef0 0x7fe8bc10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c45db640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc111f50 con 0x7fe8bc102570 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c45db640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc111dd0 con 0x7fe8bc100e20 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c45db640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1120d0 con 0x7fe8bc102ef0 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c1b76640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102570 0x7fe8bc10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c1b76640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102570 0x7fe8bc10ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36446/0 (socket says 172.21.15.94:36446) 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c1b76640 1 -- 172.21.15.94:0/1378520467 learned_addr learned my addr 172.21.15.94:0/1378520467 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c2377640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8bc100e20 0x7fe8bc10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c2b78640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8bc102ef0 0x7fe8bc10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c1b76640 1 -- 172.21.15.94:0/1378520467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8bc102ef0 msgr2=0x7fe8bc10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c1b76640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8bc102ef0 0x7fe8bc10ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c1b76640 1 -- 172.21.15.94:0/1378520467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8bc100e20 msgr2=0x7fe8bc10e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c1b76640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8bc100e20 0x7fe8bc10e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c1b76640 1 -- 172.21.15.94:0/1378520467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc1a6810 con 0x7fe8bc102570 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c1b76640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102570 0x7fe8bc10ea10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c970 tx=0x7fe8ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8b37fe640 1 -- 172.21.15.94:0/1378520467 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac007bb0 con 0x7fe8bc102570 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc1a6b00 con 0x7fe8bc102570 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8b37fe640 1 -- 172.21.15.94:0/1378520467 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8ac007d50 con 0x7fe8bc102570 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8b37fe640 1 -- 172.21.15.94:0/1378520467 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00f450 con 0x7fe8bc102570 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.449+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc1a7040 con 0x7fe8bc102570 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.450+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8bc101200 con 0x7fe8bc102570 2024-09-19T04:13:01.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.452+0000 7fe8b37fe640 1 -- 172.21.15.94:0/1378520467 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe8ac004f00 con 0x7fe8bc102570 2024-09-19T04:13:01.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.452+0000 7fe8b37fe640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8a80a3240 0x7fe8a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.452+0000 7fe8c2377640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8a80a3240 0x7fe8a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.452+0000 7fe8b37fe640 1 -- 172.21.15.94:0/1378520467 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fe8ac0cf5a0 con 0x7fe8bc102570 2024-09-19T04:13:01.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.453+0000 7fe8c2377640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8a80a3240 0x7fe8a80a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8007350 tx=0x7fe8b80072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:01.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.455+0000 7fe8b37fe640 1 -- 172.21.15.94:0/1378520467 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8ac014030 con 0x7fe8bc102570 2024-09-19T04:13:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:01 smithi112 ceph-mon[35051]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:13:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:13:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:01 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:01 smithi104 ceph-mon[34145]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:13:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:13:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:01 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:01.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.672+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8bc1a72d0 con 0x7fe8bc102570 2024-09-19T04:13:01.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.673+0000 7fe8b37fe640 1 -- 172.21.15.94:0/1378520467 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8878 (secure 0 0 0) 0x7fe8ac09c760 con 0x7fe8bc102570 2024-09-19T04:13:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:01.675 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":11,"btime":"2024-09-19T04:13:01:179605+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T04:13:01.675 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 11 2024-09-19T04:13:01.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8a80a3240 msgr2=0x7fe8a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8a80a3240 0x7fe8a80a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8007350 tx=0x7fe8b80072e0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102570 msgr2=0x7fe8bc10ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102570 0x7fe8bc10ea10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c970 tx=0x7fe8ac00ce30 comp rx=0 tx=0).stop 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 shutdown_connections 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe8a80a3240 0x7fe8a80a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8bc102ef0 0x7fe8bc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8bc102570 0x7fe8bc10ea10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 --2- 172.21.15.94:0/1378520467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8bc100e20 0x7fe8bc10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 >> 172.21.15.94:0/1378520467 conn(0x7fe8bc0fc7d0 msgr2=0x7fe8bc106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 shutdown_connections 2024-09-19T04:13:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:01.675+0000 7fe8c45db640 1 -- 172.21.15.94:0/1378520467 wait complete. 2024-09-19T04:13:02.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:02 smithi094 ceph-mon[30972]: Deploying daemon mds.a on smithi094 2024-09-19T04:13:02.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:02 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.94:6838/4157648434,v1:172.21.15.94:6839/4157648434] up:boot 2024-09-19T04:13:02.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:02 smithi094 ceph-mon[30972]: fsmap 10 up:standby 2024-09-19T04:13:02.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:02 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:13:02.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:02 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1378520467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:02 smithi112 ceph-mon[35051]: Deploying daemon mds.a on smithi094 2024-09-19T04:13:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:02 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.94:6838/4157648434,v1:172.21.15.94:6839/4157648434] up:boot 2024-09-19T04:13:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:02 smithi112 ceph-mon[35051]: fsmap 10 up:standby 2024-09-19T04:13:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:02 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:13:02.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:02 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1378520467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:02.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:02 smithi104 ceph-mon[34145]: Deploying daemon mds.a on smithi094 2024-09-19T04:13:02.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:02 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.94:6838/4157648434,v1:172.21.15.94:6839/4157648434] up:boot 2024-09-19T04:13:02.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:02 smithi104 ceph-mon[34145]: fsmap 10 up:standby 2024-09-19T04:13:02.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:02 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:13:02.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:02 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1378520467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:03.261 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:03 smithi094 ceph-mon[30972]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:03.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:03 smithi112 ceph-mon[35051]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:03 smithi104 ceph-mon[34145]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:03.841 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:13:03 smithi094 systemd[1]: Starting Ceph mds.a for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:13:04.266 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:13:03 smithi094 podman[72317]: 2024-09-19 04:13:03.856590687 +0000 UTC m=+0.016045700 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:13:04.266 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:13:03 smithi094 podman[72317]: 2024-09-19 04:13:03.954183999 +0000 UTC m=+0.113639003 container create f2db2a61c56b3d5ecab580394510e12bdfd6f1acc1816d1a9670ffc1817c68f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T04:13:04.266 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:13:04 smithi094 podman[72317]: 2024-09-19 04:13:04.16477817 +0000 UTC m=+0.324233173 container init f2db2a61c56b3d5ecab580394510e12bdfd6f1acc1816d1a9670ffc1817c68f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:13:04.266 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:13:04 smithi094 podman[72317]: 2024-09-19 04:13:04.167428621 +0000 UTC m=+0.326883625 container start f2db2a61c56b3d5ecab580394510e12bdfd6f1acc1816d1a9670ffc1817c68f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T04:13:04.266 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:13:04 smithi094 ceph-mds[72336]: -- 172.21.15.94:0/4177733545 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b9285d860 con 0x563b9283a800 2024-09-19T04:13:04.266 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:13:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-a[72332]: starting mds.a at 2024-09-19T04:13:04.267 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:13:04 smithi094 bash[72317]: f2db2a61c56b3d5ecab580394510e12bdfd6f1acc1816d1a9670ffc1817c68f2 2024-09-19T04:13:04.353 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:13:04.590 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:13:04 smithi094 systemd[1]: Started Ceph mds.a for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:13:04.613 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:05.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 -- 172.21.15.94:0/1607892951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128102790 msgr2=0x7f61281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:05.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 --2- 172.21.15.94:0/1607892951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128102790 0x7f61281067e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6118009900 tx=0x7f611802a410 comp rx=0 tx=0).stop 2024-09-19T04:13:05.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 -- 172.21.15.94:0/1607892951 shutdown_connections 2024-09-19T04:13:05.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 --2- 172.21.15.94:0/1607892951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128106f10 0x7f612810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:05.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 --2- 172.21.15.94:0/1607892951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128102790 0x7f61281067e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:05.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 --2- 172.21.15.94:0/1607892951 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6128101de0 0x7f61281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:05.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 -- 172.21.15.94:0/1607892951 >> 172.21.15.94:0/1607892951 conn(0x7f61280fd8c0 msgr2=0x7f61280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:05.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 -- 172.21.15.94:0/1607892951 shutdown_connections 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 -- 172.21.15.94:0/1607892951 wait complete. 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 Processor -- start 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.091+0000 7f61304de640 1 -- start start 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f61304de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101de0 0x7f6128198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f61304de640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6128102790 0x7f61281988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f61304de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128106f10 0x7f612819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f61304de640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6128111e20 con 0x7f6128106f10 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f61304de640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6128111ca0 con 0x7f6128102790 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f61304de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6128111fa0 con 0x7f6128101de0 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612ea7b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128106f10 0x7f612819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612ea7b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128106f10 0x7f612819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56834/0 (socket says 172.21.15.94:56834) 2024-09-19T04:13:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612ea7b640 1 -- 172.21.15.94:0/3831684497 learned_addr learned my addr 172.21.15.94:0/3831684497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612da79640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6128102790 0x7f61281988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612e27a640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101de0 0x7f6128198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612ea7b640 1 -- 172.21.15.94:0/3831684497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101de0 msgr2=0x7f6128198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612ea7b640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101de0 0x7f6128198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612ea7b640 1 -- 172.21.15.94:0/3831684497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6128102790 msgr2=0x7f61281988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612ea7b640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6128102790 0x7f61281988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612ea7b640 1 -- 172.21.15.94:0/3831684497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612819d310 con 0x7f6128106f10 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612da79640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6128102790 0x7f61281988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f612ea7b640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128106f10 0x7f612819cc30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f612000cad0 tx=0x7f612000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f611f7fe640 1 -- 172.21.15.94:0/3831684497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612000ae20 con 0x7f6128106f10 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612819d600 con 0x7f6128106f10 2024-09-19T04:13:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f611f7fe640 1 -- 172.21.15.94:0/3831684497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6120005020 con 0x7f6128106f10 2024-09-19T04:13:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f611f7fe640 1 -- 172.21.15.94:0/3831684497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612000f6b0 con 0x7f6128106f10 2024-09-19T04:13:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.092+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61281a4ee0 con 0x7f6128106f10 2024-09-19T04:13:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.094+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61281021c0 con 0x7f6128106f10 2024-09-19T04:13:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.094+0000 7f611f7fe640 1 -- 172.21.15.94:0/3831684497 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f61200051d0 con 0x7f6128106f10 2024-09-19T04:13:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.095+0000 7f611f7fe640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61140a32c0 0x7f61140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.095+0000 7f612e27a640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61140a32c0 0x7f61140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.095+0000 7f611f7fe640 1 -- 172.21.15.94:0/3831684497 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f61200cf460 con 0x7f6128106f10 2024-09-19T04:13:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.096+0000 7f612e27a640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61140a32c0 0x7f61140a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6124005fd0 tx=0x7f6124005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.099+0000 7f611f7fe640 1 -- 172.21.15.94:0/3831684497 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6120014030 con 0x7f6128106f10 2024-09-19T04:13:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.302+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f612810a120 con 0x7f6128106f10 2024-09-19T04:13:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.303+0000 7f611f7fe640 1 -- 172.21.15.94:0/3831684497 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9698 (secure 0 0 0) 0x7f612009c620 con 0x7f6128106f10 2024-09-19T04:13:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:05.305 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":12,"btime":"2024-09-19T04:13:05:192856+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T04:13:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-09-19T04:13:05.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61140a32c0 msgr2=0x7f61140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:05.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61140a32c0 0x7f61140a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6124005fd0 tx=0x7f6124005850 comp rx=0 tx=0).stop 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128106f10 msgr2=0x7f612819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128106f10 0x7f612819cc30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f612000cad0 tx=0x7f612000cf90 comp rx=0 tx=0).stop 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 shutdown_connections 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f61140a32c0 0x7f61140a5780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6128106f10 0x7f612819cc30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6128102790 0x7f61281988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 --2- 172.21.15.94:0/3831684497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101de0 0x7f6128198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 >> 172.21.15.94:0/3831684497 conn(0x7f61280fd8c0 msgr2=0x7f61280fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 shutdown_connections 2024-09-19T04:13:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:05.305+0000 7f61304de640 1 -- 172.21.15.94:0/3831684497 wait complete. 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: Deploying daemon mds.b on smithi104 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.94:6840/44219912,v1:172.21.15.94:6841/44219912] up:boot 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: fsmap 11 up:standby 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:13:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:05 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3831684497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:05.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:05.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:13:05.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:13:05.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:05.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: Deploying daemon mds.b on smithi104 2024-09-19T04:13:05.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.94:6840/44219912,v1:172.21.15.94:6841/44219912] up:boot 2024-09-19T04:13:05.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: fsmap 11 up:standby 2024-09-19T04:13:05.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:13:05.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:05 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3831684497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:13:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:13:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: Deploying daemon mds.b on smithi104 2024-09-19T04:13:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.94:6840/44219912,v1:172.21.15.94:6841/44219912] up:boot 2024-09-19T04:13:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: fsmap 11 up:standby 2024-09-19T04:13:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:13:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:05 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3831684497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:06.546 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:06 smithi104 systemd[1]: Starting Ceph mds.b for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:13:06.802 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:06 smithi104 podman[63943]: 2024-09-19 04:13:06.448459529 +0000 UTC m=+0.015942389 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:13:06.802 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:06 smithi104 podman[63943]: 2024-09-19 04:13:06.5565873 +0000 UTC m=+0.124070150 container create 707f082b4b8faf27c92a614b089cccd85595304e745bdfe1ed697695e887477a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T04:13:06.802 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:06 smithi104 podman[63943]: 2024-09-19 04:13:06.775579248 +0000 UTC m=+0.343062106 container init 707f082b4b8faf27c92a614b089cccd85595304e745bdfe1ed697695e887477a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T04:13:06.802 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:06 smithi104 podman[63943]: 2024-09-19 04:13:06.778338425 +0000 UTC m=+0.345821287 container start 707f082b4b8faf27c92a614b089cccd85595304e745bdfe1ed697695e887477a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:13:07.104 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:06 smithi104 ceph-mds[63962]: -- 172.21.15.104:0/66158381 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b263cf1860 con 0x55b263cce800 2024-09-19T04:13:07.104 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:06 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b[63958]: starting mds.b at 2024-09-19T04:13:07.104 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:06 smithi104 bash[63943]: 707f082b4b8faf27c92a614b089cccd85595304e745bdfe1ed697695e887477a 2024-09-19T04:13:07.104 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:06 smithi104 systemd[1]: Started Ceph mds.b for 503c4526-763c-11ef-bceb-c7b262605968. 2024-09-19T04:13:07.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:07 smithi094 ceph-mon[30972]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:07.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:07 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:07 smithi104 ceph-mon[34145]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:07 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:07 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:07 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:07 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:07 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:07 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:07.652 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:07 smithi112 ceph-mon[35051]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:07.652 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.652 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.652 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.652 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.652 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:07.652 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:07 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:07.903 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:13:08.195 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:08.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:08 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] up:boot 2024-09-19T04:13:08.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:08 smithi094 ceph-mon[30972]: fsmap 12 up:standby 2024-09-19T04:13:08.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:08 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:13:08.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:08 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] up:boot 2024-09-19T04:13:08.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:08 smithi104 ceph-mon[34145]: fsmap 12 up:standby 2024-09-19T04:13:08.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:08 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 -- 172.21.15.94:0/3606699350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 msgr2=0x7f79881067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 --2- 172.21.15.94:0/3606699350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 0x7f79881067f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7978009900 tx=0x7f797802a410 comp rx=0 tx=0).stop 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 -- 172.21.15.94:0/3606699350 shutdown_connections 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 --2- 172.21.15.94:0/3606699350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7988106f20 0x7f798810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 --2- 172.21.15.94:0/3606699350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 0x7f79881067f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 --2- 172.21.15.94:0/3606699350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7988101df0 0x7f79881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 -- 172.21.15.94:0/3606699350 >> 172.21.15.94:0/3606699350 conn(0x7f79880fd8f0 msgr2=0x7f79880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 -- 172.21.15.94:0/3606699350 shutdown_connections 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 -- 172.21.15.94:0/3606699350 wait complete. 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 Processor -- start 2024-09-19T04:13:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.717+0000 7f798fff9640 1 -- start start 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798fff9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7988101df0 0x7f7988198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798fff9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 0x7f7988198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798fff9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7988106f20 0x7f798819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798fff9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7988111e30 con 0x7f79881027a0 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798fff9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7988111cb0 con 0x7f7988101df0 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798fff9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7988111fb0 con 0x7f7988106f20 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798d594640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 0x7f7988198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798d594640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 0x7f7988198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56860/0 (socket says 172.21.15.94:56860) 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798d594640 1 -- 172.21.15.94:0/895554562 learned_addr learned my addr 172.21.15.94:0/895554562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798e596640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7988106f20 0x7f798819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798dd95640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7988101df0 0x7f7988198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:08.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798d594640 1 -- 172.21.15.94:0/895554562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7988106f20 msgr2=0x7f798819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798d594640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7988106f20 0x7f798819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798d594640 1 -- 172.21.15.94:0/895554562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7988101df0 msgr2=0x7f7988198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798d594640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7988101df0 0x7f7988198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798d594640 1 -- 172.21.15.94:0/895554562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798819d2c0 con 0x7f79881027a0 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798d594640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 0x7f7988198850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7978005940 tx=0x7f7978004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f797effd640 1 -- 172.21.15.94:0/895554562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797803f030 con 0x7f79881027a0 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798819d550 con 0x7f79881027a0 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f797effd640 1 -- 172.21.15.94:0/895554562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7978003aa0 con 0x7f79881027a0 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f797effd640 1 -- 172.21.15.94:0/895554562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978035070 con 0x7f79881027a0 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.718+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79881a4e90 con 0x7f79881027a0 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.719+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79881032d0 con 0x7f79881027a0 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.720+0000 7f797effd640 1 -- 172.21.15.94:0/895554562 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7978042030 con 0x7f79881027a0 2024-09-19T04:13:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.721+0000 7f797effd640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79740a3240 0x7f79740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:08.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.721+0000 7f797effd640 1 -- 172.21.15.94:0/895554562 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f79780efc10 con 0x7f79881027a0 2024-09-19T04:13:08.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.722+0000 7f798dd95640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79740a3240 0x7f79740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:08.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.722+0000 7f798dd95640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79740a3240 0x7f79740a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7984005fd0 tx=0x7f7984009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:08.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.726+0000 7f797effd640 1 -- 172.21.15.94:0/895554562 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f79780bc6e0 con 0x7f79881027a0 2024-09-19T04:13:08.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:08 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] up:boot 2024-09-19T04:13:08.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:08 smithi112 ceph-mon[35051]: fsmap 12 up:standby 2024-09-19T04:13:08.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:08 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:13:08.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.948+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f798810a100 con 0x7f79881027a0 2024-09-19T04:13:08.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.949+0000 7f797effd640 1 -- 172.21.15.94:0/895554562 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10527 (secure 0 0 0) 0x7f79780c09e0 con 0x7f79881027a0 2024-09-19T04:13:08.951 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:08.951 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":13,"btime":"2024-09-19T04:13:07:315789+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T04:13:08.951 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 13 2024-09-19T04:13:08.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.951+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79740a3240 msgr2=0x7f79740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:08.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.951+0000 7f798fff9640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79740a3240 0x7f79740a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7984005fd0 tx=0x7f7984009760 comp rx=0 tx=0).stop 2024-09-19T04:13:08.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.951+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 msgr2=0x7f7988198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:08.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.951+0000 7f798fff9640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 0x7f7988198850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7978005940 tx=0x7f7978004270 comp rx=0 tx=0).stop 2024-09-19T04:13:08.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.951+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 shutdown_connections 2024-09-19T04:13:08.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.951+0000 7f798fff9640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79740a3240 0x7f79740a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:08.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.951+0000 7f798fff9640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7988106f20 0x7f798819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:08.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.951+0000 7f798fff9640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79881027a0 0x7f7988198850 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:08.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.951+0000 7f798fff9640 1 --2- 172.21.15.94:0/895554562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7988101df0 0x7f7988198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:08.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.952+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 >> 172.21.15.94:0/895554562 conn(0x7f79880fd8f0 msgr2=0x7f79880fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:08.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.952+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 shutdown_connections 2024-09-19T04:13:08.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:08.952+0000 7f798fff9640 1 -- 172.21.15.94:0/895554562 wait complete. 2024-09-19T04:13:09.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:09 smithi094 ceph-mon[30972]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:09.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:09 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:09 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/895554562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:09.563 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T04:13:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:09 smithi104 ceph-mon[34145]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:09.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:09 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:09 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/895554562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:09.686 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:09 smithi112 ceph-mon[35051]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:09.686 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.686 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.686 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.686 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:09 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:09.687 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:09 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/895554562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.284+0000 7f200365a640 1 -- 172.21.15.94:0/39160121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc106f20 msgr2=0x7f1ffc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.284+0000 7f200365a640 1 --2- 172.21.15.94:0/39160121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc106f20 0x7f1ffc10eaa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009ca0 tx=0x7f1ff402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.284+0000 7f200365a640 1 -- 172.21.15.94:0/39160121 shutdown_connections 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.284+0000 7f200365a640 1 --2- 172.21.15.94:0/39160121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc106f20 0x7f1ffc10eaa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.284+0000 7f200365a640 1 --2- 172.21.15.94:0/39160121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ffc1027a0 0x7f1ffc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.284+0000 7f200365a640 1 --2- 172.21.15.94:0/39160121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ffc101df0 0x7f1ffc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.284+0000 7f200365a640 1 -- 172.21.15.94:0/39160121 >> 172.21.15.94:0/39160121 conn(0x7f1ffc0fd8b0 msgr2=0x7f1ffc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.285+0000 7f200365a640 1 -- 172.21.15.94:0/39160121 shutdown_connections 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.285+0000 7f200365a640 1 -- 172.21.15.94:0/39160121 wait complete. 2024-09-19T04:13:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.285+0000 7f200365a640 1 Processor -- start 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.285+0000 7f200365a640 1 -- start start 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f200365a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc101df0 0x7f1ffc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f200365a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ffc1027a0 0x7f1ffc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f200365a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ffc106f20 0x7f1ffc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f200365a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc111e30 con 0x7f1ffc101df0 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f200365a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc111cb0 con 0x7f1ffc106f20 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f200365a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc111fb0 con 0x7f1ffc1027a0 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f20013f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc101df0 0x7f1ffc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f20013f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc101df0 0x7f1ffc198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56890/0 (socket says 172.21.15.94:56890) 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f20013f6640 1 -- 172.21.15.94:0/2244020755 learned_addr learned my addr 172.21.15.94:0/2244020755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f2000bf5640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ffc1027a0 0x7f1ffc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f2001bf7640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ffc106f20 0x7f1ffc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f20013f6640 1 -- 172.21.15.94:0/2244020755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ffc1027a0 msgr2=0x7f1ffc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f20013f6640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ffc1027a0 0x7f1ffc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f20013f6640 1 -- 172.21.15.94:0/2244020755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ffc106f20 msgr2=0x7f1ffc19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f20013f6640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ffc106f20 0x7f1ffc19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f20013f6640 1 -- 172.21.15.94:0/2244020755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc19d2e0 con 0x7f1ffc101df0 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f20013f6640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc101df0 0x7f1ffc198330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800d950 tx=0x7f1ff800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f1ff27fc640 1 -- 172.21.15.94:0/2244020755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8017070 con 0x7f1ffc101df0 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f1ff27fc640 1 -- 172.21.15.94:0/2244020755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ff8009c30 con 0x7f1ffc101df0 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f1ff27fc640 1 -- 172.21.15.94:0/2244020755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800f620 con 0x7f1ffc101df0 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ffc19d5d0 con 0x7f1ffc101df0 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.286+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ffc1a4eb0 con 0x7f1ffc101df0 2024-09-19T04:13:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.287+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ffc1021d0 con 0x7f1ffc101df0 2024-09-19T04:13:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.288+0000 7f1ff27fc640 1 -- 172.21.15.94:0/2244020755 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1ff80109a0 con 0x7f1ffc101df0 2024-09-19T04:13:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.289+0000 7f1ff27fc640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1fe80a3240 0x7f1fe80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.289+0000 7f2000bf5640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1fe80a3240 0x7f1fe80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.289+0000 7f1ff27fc640 1 -- 172.21.15.94:0/2244020755 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f1ff80cf0a0 con 0x7f1ffc101df0 2024-09-19T04:13:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.289+0000 7f2000bf5640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1fe80a3240 0x7f1fe80a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0fdcd0 tx=0x7f1fec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.293+0000 7f1ff27fc640 1 -- 172.21.15.94:0/2244020755 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ff8009de0 con 0x7f1ffc101df0 2024-09-19T04:13:10.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.479+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1ffc109bd0 con 0x7f1ffc101df0 2024-09-19T04:13:10.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.480+0000 7f1ff27fc640 1 -- 172.21.15.94:0/2244020755 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7f1ff809c260 con 0x7f1ffc101df0 2024-09-19T04:13:10.481 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:10.482 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-19T04:13:10.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:13:10.482 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-19T04:13:10.482 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-19T04:13:10.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:13:10.482 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1fe80a3240 msgr2=0x7f1fe80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1fe80a3240 0x7f1fe80a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0fdcd0 tx=0x7f1fec005f90 comp rx=0 tx=0).stop 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc101df0 msgr2=0x7f1ffc198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc101df0 0x7f1ffc198330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800d950 tx=0x7f1ff800de10 comp rx=0 tx=0).stop 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 shutdown_connections 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1fe80a3240 0x7f1fe80a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ffc106f20 0x7f1ffc19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ffc1027a0 0x7f1ffc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 --2- 172.21.15.94:0/2244020755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc101df0 0x7f1ffc198330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 >> 172.21.15.94:0/2244020755 conn(0x7f1ffc0fd8b0 msgr2=0x7f1ffc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 shutdown_connections 2024-09-19T04:13:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:10.482+0000 7f200365a640 1 -- 172.21.15.94:0/2244020755 wait complete. 2024-09-19T04:13:10.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:10.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:10.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:10.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:10.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:10.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:10.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:10 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:10.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:10 smithi094 ceph-mon[30972]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:10.790 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:10 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2244020755' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:13:10.976 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:10.976 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:10.976 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:10.976 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:10.977 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:10.977 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:10.977 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:10 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:10.977 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:10 smithi112 ceph-mon[35051]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:10.977 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:10 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2244020755' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:13:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:11.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:10 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:11.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:10 smithi104 ceph-mon[34145]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:11.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:10 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2244020755' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:13:11.758 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:13:11.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:13:12.042 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:12.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:12 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:12 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:12 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:12 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:12 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:12 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:12 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:12 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.558+0000 7f816efac640 1 -- 172.21.15.94:0/1883770613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168101ba0 msgr2=0x7f816810d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.558+0000 7f816efac640 1 --2- 172.21.15.94:0/1883770613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168101ba0 0x7f816810d8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8160009ca0 tx=0x7f816002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.558+0000 7f816efac640 1 -- 172.21.15.94:0/1883770613 shutdown_connections 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.558+0000 7f816efac640 1 --2- 172.21.15.94:0/1883770613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168101ba0 0x7f816810d8f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.558+0000 7f816efac640 1 --2- 172.21.15.94:0/1883770613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8168101220 0x7f8168101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.558+0000 7f816efac640 1 --2- 172.21.15.94:0/1883770613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8168105a70 0x7f8168105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.558+0000 7f816efac640 1 -- 172.21.15.94:0/1883770613 >> 172.21.15.94:0/1883770613 conn(0x7f81680fbee0 msgr2=0x7f81680fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.558+0000 7f816efac640 1 -- 172.21.15.94:0/1883770613 shutdown_connections 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.558+0000 7f816efac640 1 -- 172.21.15.94:0/1883770613 wait complete. 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816efac640 1 Processor -- start 2024-09-19T04:13:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816efac640 1 -- start start 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816efac640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8168101220 0x7f8168197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816efac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8168101ba0 0x7f8168197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816efac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168105a70 0x7f816819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816efac640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8168110e00 con 0x7f8168105a70 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816efac640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8168110c80 con 0x7f8168101220 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816efac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8168110f80 con 0x7f8168101ba0 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816d549640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168105a70 0x7f816819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816d549640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168105a70 0x7f816819bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58380/0 (socket says 172.21.15.94:58380) 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816d549640 1 -- 172.21.15.94:0/3052319498 learned_addr learned my addr 172.21.15.94:0/3052319498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f8167fff640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8168101ba0 0x7f8168197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.559+0000 7f816cd48640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8168101220 0x7f8168197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f816d549640 1 -- 172.21.15.94:0/3052319498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8168101ba0 msgr2=0x7f8168197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f816d549640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8168101ba0 0x7f8168197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f816d549640 1 -- 172.21.15.94:0/3052319498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8168101220 msgr2=0x7f8168197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f816d549640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8168101220 0x7f8168197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f816d549640 1 -- 172.21.15.94:0/3052319498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816819c1f0 con 0x7f8168105a70 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f816d549640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168105a70 0x7f816819bb10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8160009dc0 tx=0x7f8160032fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f8165ffb640 1 -- 172.21.15.94:0/3052319498 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160003cf0 con 0x7f8168105a70 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f8165ffb640 1 -- 172.21.15.94:0/3052319498 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81600044f0 con 0x7f8168105a70 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816819c480 con 0x7f8168105a70 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f8165ffb640 1 -- 172.21.15.94:0/3052319498 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160030bf0 con 0x7f8168105a70 2024-09-19T04:13:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.560+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8168100110 con 0x7f8168105a70 2024-09-19T04:13:12.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.561+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8168102230 con 0x7f8168105a70 2024-09-19T04:13:12.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.562+0000 7f8165ffb640 1 -- 172.21.15.94:0/3052319498 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8160004040 con 0x7f8168105a70 2024-09-19T04:13:12.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.563+0000 7f8165ffb640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f81480a5450 0x7f81480a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:12.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.563+0000 7f8165ffb640 1 -- 172.21.15.94:0/3052319498 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f81600f09d0 con 0x7f8168105a70 2024-09-19T04:13:12.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.563+0000 7f816cd48640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f81480a5450 0x7f81480a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:12.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.563+0000 7f816cd48640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f81480a5450 0x7f81480a7910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8158005fd0 tx=0x7f815800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:12.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.567+0000 7f8165ffb640 1 -- 172.21.15.94:0/3052319498 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f81600bdb90 con 0x7f8168105a70 2024-09-19T04:13:12.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:12 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:12 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:12 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:12 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:12.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:12.733+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8168100370 con 0x7f8168105a70 2024-09-19T04:13:13.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.410+0000 7f8165ffb640 1 -- 172.21.15.94:0/3052319498 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7f81600bd490 con 0x7f8168105a70 2024-09-19T04:13:13.411 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f81480a5450 msgr2=0x7f81480a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f81480a5450 0x7f81480a7910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8158005fd0 tx=0x7f815800b390 comp rx=0 tx=0).stop 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168105a70 msgr2=0x7f816819bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168105a70 0x7f816819bb10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8160009dc0 tx=0x7f8160032fe0 comp rx=0 tx=0).stop 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 shutdown_connections 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f81480a5450 0x7f81480a7910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8168105a70 0x7f816819bb10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8168101ba0 0x7f8168197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 --2- 172.21.15.94:0/3052319498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8168101220 0x7f8168197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 >> 172.21.15.94:0/3052319498 conn(0x7f81680fbee0 msgr2=0x7f81680fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:13.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 shutdown_connections 2024-09-19T04:13:13.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:13.412+0000 7f816efac640 1 -- 172.21.15.94:0/3052319498 wait complete. 2024-09-19T04:13:13.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:13 smithi094 ceph-mon[30972]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:13.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:13 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3052319498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:13:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:13 smithi104 ceph-mon[34145]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:13.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:13.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:13 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3052319498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:13:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:13 smithi112 ceph-mon[35051]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:13:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:13 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3052319498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:13:14.066 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:13:14.313 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:14.717 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:14 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3052319498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:13:14.717 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:14 smithi094 ceph-mon[30972]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T04:13:14.717 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:14 smithi094 ceph-mon[30972]: pgmap v267: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:14.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:14 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3052319498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:13:14.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:14 smithi112 ceph-mon[35051]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T04:13:14.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:14 smithi112 ceph-mon[35051]: pgmap v267: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:14.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.802+0000 7f60a5fd9640 1 -- 172.21.15.94:0/2133736476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0106a70 msgr2=0x7f60a0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:14.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.802+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/2133736476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0106a70 0x7f60a0106e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6090009cd0 tx=0x7f609002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:13:14.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.802+0000 7f60a5fd9640 1 -- 172.21.15.94:0/2133736476 shutdown_connections 2024-09-19T04:13:14.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.802+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/2133736476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a0101410 0x7f60a010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:14.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.802+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/2133736476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a0100a70 0x7f60a0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:14.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.802+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/2133736476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0106a70 0x7f60a0106e50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:14.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.802+0000 7f60a5fd9640 1 -- 172.21.15.94:0/2133736476 >> 172.21.15.94:0/2133736476 conn(0x7f60a00fc790 msgr2=0x7f60a00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:14.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.802+0000 7f60a5fd9640 1 -- 172.21.15.94:0/2133736476 shutdown_connections 2024-09-19T04:13:14.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.802+0000 7f60a5fd9640 1 -- 172.21.15.94:0/2133736476 wait complete. 2024-09-19T04:13:14.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.803+0000 7f60a5fd9640 1 Processor -- start 2024-09-19T04:13:14.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.803+0000 7f60a5fd9640 1 -- start start 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.803+0000 7f60a5fd9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a0100a70 0x7f60a0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.803+0000 7f60a5fd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0101410 0x7f60a0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.803+0000 7f60a5fd9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a0106a70 0x7f60a019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.803+0000 7f60a5fd9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0110d70 con 0x7f60a0101410 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.803+0000 7f60a5fd9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0110bf0 con 0x7f60a0100a70 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.803+0000 7f60a5fd9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0110ef0 con 0x7f60a0106a70 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.803+0000 7f609effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0101410 0x7f60a0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0101410 0x7f60a0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58404/0 (socket says 172.21.15.94:58404) 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609effd640 1 -- 172.21.15.94:0/200134645 learned_addr learned my addr 172.21.15.94:0/200134645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:14.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609f7fe640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a0100a70 0x7f60a0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609ffff640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a0106a70 0x7f60a019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609effd640 1 -- 172.21.15.94:0/200134645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a0106a70 msgr2=0x7f60a019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609effd640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a0106a70 0x7f60a019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609effd640 1 -- 172.21.15.94:0/200134645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a0100a70 msgr2=0x7f60a0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609effd640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a0100a70 0x7f60a0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609effd640 1 -- 172.21.15.94:0/200134645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a019d2e0 con 0x7f60a0101410 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609f7fe640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a0100a70 0x7f60a0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609effd640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0101410 0x7f60a0198870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f608800d950 tx=0x7f608800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609cff9640 1 -- 172.21.15.94:0/200134645 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088017070 con 0x7f60a0101410 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a019d5d0 con 0x7f60a0101410 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a01a4eb0 con 0x7f60a0101410 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609cff9640 1 -- 172.21.15.94:0/200134645 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6088009c30 con 0x7f60a0101410 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.804+0000 7f609cff9640 1 -- 172.21.15.94:0/200134645 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608800f620 con 0x7f60a0101410 2024-09-19T04:13:14.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.805+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a0100ed0 con 0x7f60a0101410 2024-09-19T04:13:14.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.806+0000 7f609cff9640 1 -- 172.21.15.94:0/200134645 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f60880109a0 con 0x7f60a0101410 2024-09-19T04:13:14.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.807+0000 7f609cff9640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f608c0a32c0 0x7f608c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:14.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.807+0000 7f609f7fe640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f608c0a32c0 0x7f608c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:14.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.807+0000 7f609cff9640 1 -- 172.21.15.94:0/200134645 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f60880cf090 con 0x7f60a0101410 2024-09-19T04:13:14.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.808+0000 7f609f7fe640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f608c0a32c0 0x7f608c0a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6090009ca0 tx=0x7f6090004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:14.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.811+0000 7f609cff9640 1 -- 172.21.15.94:0/200134645 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6088009de0 con 0x7f60a0101410 2024-09-19T04:13:14.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:14 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3052319498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:13:14.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:14 smithi104 ceph-mon[34145]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T04:13:14.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:14 smithi104 ceph-mon[34145]: pgmap v267: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:14.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:14.977+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f60a01a51d0 con 0x7f60a0101410 2024-09-19T04:13:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.422+0000 7f609cff9640 1 -- 172.21.15.94:0/200134645 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f608809bb10 con 0x7f60a0101410 2024-09-19T04:13:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f608c0a32c0 msgr2=0x7f608c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f608c0a32c0 0x7f608c0a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6090009ca0 tx=0x7f6090004a80 comp rx=0 tx=0).stop 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0101410 msgr2=0x7f60a0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0101410 0x7f60a0198870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f608800d950 tx=0x7f608800de10 comp rx=0 tx=0).stop 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 shutdown_connections 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f608c0a32c0 0x7f608c0a5780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a0106a70 0x7f60a019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0101410 0x7f60a0198870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 --2- 172.21.15.94:0/200134645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a0100a70 0x7f60a0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.424+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 >> 172.21.15.94:0/200134645 conn(0x7f60a00fc790 msgr2=0x7f60a006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:15.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.425+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 shutdown_connections 2024-09-19T04:13:15.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:15.425+0000 7f60a5fd9640 1 -- 172.21.15.94:0/200134645 wait complete. 2024-09-19T04:13:15.779 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:15 smithi094 ceph-mon[30972]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:13:15.779 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:15 smithi094 ceph-mon[30972]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T04:13:15.779 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:15 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/200134645' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:13:15.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:15 smithi112 ceph-mon[35051]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:13:15.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:15 smithi112 ceph-mon[35051]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T04:13:15.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:15 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/200134645' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:13:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:15 smithi104 ceph-mon[34145]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:13:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:15 smithi104 ceph-mon[34145]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T04:13:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:15 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/200134645' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:13:16.126 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:13:16.371 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:16.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:16 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/200134645' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:13:16.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:16 smithi094 ceph-mon[30972]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:13:16.734 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:16 smithi094 ceph-mon[30972]: pgmap v270: 97 pgs: 11 creating+peering, 10 active+clean, 76 unknown; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:16.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:16 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/200134645' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:13:16.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:16 smithi112 ceph-mon[35051]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:13:16.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:16 smithi112 ceph-mon[35051]: pgmap v270: 97 pgs: 11 creating+peering, 10 active+clean, 76 unknown; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.813+0000 7f6ea2f80640 1 -- 172.21.15.94:0/2178243810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c106a80 msgr2=0x7f6e9c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.813+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/2178243810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c106a80 0x7f6e9c106e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009930 tx=0x7f6e9002a420 comp rx=0 tx=0).stop 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.813+0000 7f6ea2f80640 1 -- 172.21.15.94:0/2178243810 shutdown_connections 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.813+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/2178243810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e9c101420 0x7f6e9c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.813+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/2178243810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e9c100a80 0x7f6e9c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.813+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/2178243810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c106a80 0x7f6e9c106e60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.813+0000 7f6ea2f80640 1 -- 172.21.15.94:0/2178243810 >> 172.21.15.94:0/2178243810 conn(0x7f6e9c0fc7a0 msgr2=0x7f6e9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.813+0000 7f6ea2f80640 1 -- 172.21.15.94:0/2178243810 shutdown_connections 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.813+0000 7f6ea2f80640 1 -- 172.21.15.94:0/2178243810 wait complete. 2024-09-19T04:13:16.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea2f80640 1 Processor -- start 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea2f80640 1 -- start start 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea2f80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c100a80 0x7f6e9c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea2f80640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e9c101420 0x7f6e9c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea2f80640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e9c106a80 0x7f6e9c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea2f80640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c111e70 con 0x7f6e9c100a80 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea2f80640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c111cf0 con 0x7f6e9c101420 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea2f80640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c111ff0 con 0x7f6e9c106a80 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea0d1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c100a80 0x7f6e9c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea0d1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c100a80 0x7f6e9c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58410/0 (socket says 172.21.15.94:58410) 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea0d1c640 1 -- 172.21.15.94:0/683616034 learned_addr learned my addr 172.21.15.94:0/683616034 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea151d640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e9c106a80 0x7f6e9c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6e9bfff640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e9c101420 0x7f6e9c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea0d1c640 1 -- 172.21.15.94:0/683616034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e9c106a80 msgr2=0x7f6e9c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea0d1c640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e9c106a80 0x7f6e9c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea0d1c640 1 -- 172.21.15.94:0/683616034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e9c101420 msgr2=0x7f6e9c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea0d1c640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e9c101420 0x7f6e9c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.814+0000 7f6ea0d1c640 1 -- 172.21.15.94:0/683616034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c19d3c0 con 0x7f6e9c100a80 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.815+0000 7f6ea0d1c640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c100a80 0x7f6e9c198410 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e90004770 tx=0x7f6e900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.815+0000 7f6e99ffb640 1 -- 172.21.15.94:0/683616034 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9003f070 con 0x7f6e9c100a80 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.815+0000 7f6e99ffb640 1 -- 172.21.15.94:0/683616034 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e9003b460 con 0x7f6e9c100a80 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.815+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9c19d650 con 0x7f6e9c100a80 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.815+0000 7f6e99ffb640 1 -- 172.21.15.94:0/683616034 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90035070 con 0x7f6e9c100a80 2024-09-19T04:13:16.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.815+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9c1a4f90 con 0x7f6e9c100a80 2024-09-19T04:13:16.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.816+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e9c1021c0 con 0x7f6e9c100a80 2024-09-19T04:13:16.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.817+0000 7f6e99ffb640 1 -- 172.21.15.94:0/683616034 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6e9003b610 con 0x7f6e9c100a80 2024-09-19T04:13:16.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.818+0000 7f6e99ffb640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6e8c0a3240 0x7f6e8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:16.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.818+0000 7f6e9bfff640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6e8c0a3240 0x7f6e8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:16.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.818+0000 7f6e99ffb640 1 -- 172.21.15.94:0/683616034 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f6e900efa70 con 0x7f6e9c100a80 2024-09-19T04:13:16.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.818+0000 7f6e9bfff640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6e8c0a3240 0x7f6e8c0a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0fee00 tx=0x7f6e8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:16.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:16.822+0000 7f6e99ffb640 1 -- 172.21.15.94:0/683616034 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6e900bc2f0 con 0x7f6e9c100a80 2024-09-19T04:13:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:16 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/200134645' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:13:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:16 smithi104 ceph-mon[34145]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:13:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:16 smithi104 ceph-mon[34145]: pgmap v270: 97 pgs: 11 creating+peering, 10 active+clean, 76 unknown; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:13:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.022+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6e9c108a60 con 0x7f6e9c100a80 2024-09-19T04:13:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.444+0000 7f6e99ffb640 1 -- 172.21.15.94:0/683616034 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f6e900c05f0 con 0x7f6e9c100a80 2024-09-19T04:13:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T04:13:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6e8c0a3240 msgr2=0x7f6e8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6e8c0a3240 0x7f6e8c0a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0fee00 tx=0x7f6e8800b390 comp rx=0 tx=0).stop 2024-09-19T04:13:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c100a80 msgr2=0x7f6e9c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c100a80 0x7f6e9c198410 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e90004770 tx=0x7f6e900042f0 comp rx=0 tx=0).stop 2024-09-19T04:13:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 shutdown_connections 2024-09-19T04:13:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6e8c0a3240 0x7f6e8c0a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e9c106a80 0x7f6e9c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e9c101420 0x7f6e9c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 --2- 172.21.15.94:0/683616034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9c100a80 0x7f6e9c198410 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 >> 172.21.15.94:0/683616034 conn(0x7f6e9c0fc7a0 msgr2=0x7f6e9c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 shutdown_connections 2024-09-19T04:13:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:17.447+0000 7f6ea2f80640 1 -- 172.21.15.94:0/683616034 wait complete. 2024-09-19T04:13:17.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:17 smithi094 ceph-mon[30972]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:13:17.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:17 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/683616034' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:13:17.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:17 smithi094 ceph-mon[30972]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:13:17.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:17 smithi112 ceph-mon[35051]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:13:17.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:17 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/683616034' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:13:17.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:17 smithi112 ceph-mon[35051]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:13:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:17 smithi104 ceph-mon[34145]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:13:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:17 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/683616034' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:13:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:17 smithi104 ceph-mon[34145]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:13:18.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:13:18.319 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:18.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:18 smithi094 ceph-mon[30972]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:13:18.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:18 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/683616034' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:13:18.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:18 smithi094 ceph-mon[30972]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:13:18.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:18 smithi094 ceph-mon[30972]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:13:18.734 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:18 smithi094 ceph-mon[30972]: pgmap v273: 97 pgs: 11 creating+peering, 27 active+clean, 59 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-09-19T04:13:18.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:18 smithi112 ceph-mon[35051]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:13:18.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:18 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/683616034' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:13:18.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:18 smithi112 ceph-mon[35051]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:13:18.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:18 smithi112 ceph-mon[35051]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:13:18.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:18 smithi112 ceph-mon[35051]: pgmap v273: 97 pgs: 11 creating+peering, 27 active+clean, 59 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-09-19T04:13:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 -- 172.21.15.94:0/1546548191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120101de0 msgr2=0x7f01201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 --2- 172.21.15.94:0/1546548191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120101de0 0x7f01201021c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0110009900 tx=0x7f011002a410 comp rx=0 tx=0).stop 2024-09-19T04:13:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 -- 172.21.15.94:0/1546548191 shutdown_connections 2024-09-19T04:13:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 --2- 172.21.15.94:0/1546548191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0120106f10 0x7f012010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 --2- 172.21.15.94:0/1546548191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0120102790 0x7f01201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 --2- 172.21.15.94:0/1546548191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120101de0 0x7f01201021c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 -- 172.21.15.94:0/1546548191 >> 172.21.15.94:0/1546548191 conn(0x7f01200fd8c0 msgr2=0x7f01200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 -- 172.21.15.94:0/1546548191 shutdown_connections 2024-09-19T04:13:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 -- 172.21.15.94:0/1546548191 wait complete. 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 Processor -- start 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.809+0000 7f01262b1640 1 -- start start 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f01262b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0120101de0 0x7f01201984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f01262b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120102790 0x7f0120198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f01262b1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0120106f10 0x7f012019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f01262b1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0120111f10 con 0x7f0120102790 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f01262b1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0120111d90 con 0x7f0120106f10 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f01262b1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0120112090 con 0x7f0120101de0 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f0117fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120102790 0x7f0120198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f0117fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120102790 0x7f0120198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58426/0 (socket says 172.21.15.94:58426) 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f0117fff640 1 -- 172.21.15.94:0/3472628923 learned_addr learned my addr 172.21.15.94:0/3472628923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f012484e640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0120106f10 0x7f012019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f011ffff640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0120101de0 0x7f01201984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f0117fff640 1 -- 172.21.15.94:0/3472628923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0120101de0 msgr2=0x7f01201984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f0117fff640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0120101de0 0x7f01201984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f0117fff640 1 -- 172.21.15.94:0/3472628923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0120106f10 msgr2=0x7f012019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f0117fff640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0120106f10 0x7f012019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f0117fff640 1 -- 172.21.15.94:0/3472628923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012019d480 con 0x7f0120102790 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f012484e640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0120106f10 0x7f012019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f011ffff640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0120101de0 0x7f01201984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f0117fff640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120102790 0x7f0120198a10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f010800b4a0 tx=0x7f010800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f011dffb640 1 -- 172.21.15.94:0/3472628923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108014070 con 0x7f0120102790 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012019d770 con 0x7f0120102790 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f011dffb640 1 -- 172.21.15.94:0/3472628923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0108010ac0 con 0x7f0120102790 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f011dffb640 1 -- 172.21.15.94:0/3472628923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108017c30 con 0x7f0120102790 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.810+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01201a5050 con 0x7f0120102790 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.811+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0120102240 con 0x7f0120102790 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.813+0000 7f011dffb640 1 -- 172.21.15.94:0/3472628923 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0108017070 con 0x7f0120102790 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.813+0000 7f011dffb640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f010c0a3240 0x7f010c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.813+0000 7f011ffff640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f010c0a3240 0x7f010c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.813+0000 7f011dffb640 1 -- 172.21.15.94:0/3472628923 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f01080cefa0 con 0x7f0120102790 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.814+0000 7f011ffff640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f010c0a3240 0x7f010c0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0110000c00 tx=0x7f0110031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.817+0000 7f011dffb640 1 -- 172.21.15.94:0/3472628923 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f010809be10 con 0x7f0120102790 2024-09-19T04:13:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:18 smithi104 ceph-mon[34145]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:13:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:18 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/683616034' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:13:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:18 smithi104 ceph-mon[34145]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:13:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:18 smithi104 ceph-mon[34145]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:13:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:18 smithi104 ceph-mon[34145]: pgmap v273: 97 pgs: 11 creating+peering, 27 active+clean, 59 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-09-19T04:13:18.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.984+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0120199830 con 0x7f0120102790 2024-09-19T04:13:18.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.985+0000 7f011dffb640 1 -- 172.21.15.94:0/3472628923 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22304 (secure 0 0 0) 0x7f0120199830 con 0x7f0120102790 2024-09-19T04:13:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:18.988 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":79,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:13:17.025174+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f010c0a3240 msgr2=0x7f010c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f010c0a3240 0x7f010c0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0110000c00 tx=0x7f0110031400 comp rx=0 tx=0).stop 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120102790 msgr2=0x7f0120198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120102790 0x7f0120198a10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f010800b4a0 tx=0x7f010800b960 comp rx=0 tx=0).stop 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 shutdown_connections 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f010c0a3240 0x7f010c0a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0120106f10 0x7f012019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0120102790 0x7f0120198a10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 --2- 172.21.15.94:0/3472628923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0120101de0 0x7f01201984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.987+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 >> 172.21.15.94:0/3472628923 conn(0x7f01200fd8c0 msgr2=0x7f01200fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:18.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.988+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 shutdown_connections 2024-09-19T04:13:18.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:18.988+0000 7f01262b1640 1 -- 172.21.15.94:0/3472628923 wait complete. 2024-09-19T04:13:19.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:19 smithi094 ceph-mon[30972]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:13:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:19 smithi094 ceph-mon[30972]: Cluster is now healthy 2024-09-19T04:13:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:19 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] up:active 2024-09-19T04:13:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:19 smithi094 ceph-mon[30972]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:19 smithi094 ceph-mon[30972]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:19.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:19 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3472628923' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:19.632 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:13:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:19 smithi112 ceph-mon[35051]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:13:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:19 smithi112 ceph-mon[35051]: Cluster is now healthy 2024-09-19T04:13:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:19 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] up:active 2024-09-19T04:13:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:19 smithi112 ceph-mon[35051]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:19 smithi112 ceph-mon[35051]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:19 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3472628923' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:19.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:19 smithi104 ceph-mon[34145]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:13:19.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:19 smithi104 ceph-mon[34145]: Cluster is now healthy 2024-09-19T04:13:19.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:19 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] up:active 2024-09-19T04:13:19.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:19 smithi104 ceph-mon[34145]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:19.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:19 smithi104 ceph-mon[34145]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:19.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:19 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3472628923' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:19.962 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:20.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.359+0000 7fc27034d640 1 -- 172.21.15.94:0/4284075841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2681037a0 msgr2=0x7fc26810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:20.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.359+0000 7fc27034d640 1 --2- 172.21.15.94:0/4284075841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2681037a0 0x7fc26810bc00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009900 tx=0x7fc25c02a410 comp rx=0 tx=0).stop 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.359+0000 7fc27034d640 1 -- 172.21.15.94:0/4284075841 shutdown_connections 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.359+0000 7fc27034d640 1 --2- 172.21.15.94:0/4284075841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc26810c2f0 0x7fc26810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.359+0000 7fc27034d640 1 --2- 172.21.15.94:0/4284075841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2681037a0 0x7fc26810bc00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.359+0000 7fc27034d640 1 --2- 172.21.15.94:0/4284075841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc268102df0 0x7fc2681031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.359+0000 7fc27034d640 1 -- 172.21.15.94:0/4284075841 >> 172.21.15.94:0/4284075841 conn(0x7fc2680fc7d0 msgr2=0x7fc2680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 -- 172.21.15.94:0/4284075841 shutdown_connections 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 -- 172.21.15.94:0/4284075841 wait complete. 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 Processor -- start 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 -- start start 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc268102df0 0x7fc268198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2681037a0 0x7fc268198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc26810c2f0 0x7fc26819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:20.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc268111fe0 con 0x7fc268102df0 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc268111e60 con 0x7fc2681037a0 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc27034d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc268112160 con 0x7fc26810c2f0 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc26e0e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc268102df0 0x7fc268198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc26e0e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc268102df0 0x7fc268198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58444/0 (socket says 172.21.15.94:58444) 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.360+0000 7fc26e0e9640 1 -- 172.21.15.94:0/4175599116 learned_addr learned my addr 172.21.15.94:0/4175599116 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc26d8e8640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2681037a0 0x7fc268198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc26e8ea640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc26810c2f0 0x7fc26819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc26e0e9640 1 -- 172.21.15.94:0/4175599116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc26810c2f0 msgr2=0x7fc26819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc26e0e9640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc26810c2f0 0x7fc26819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc26e0e9640 1 -- 172.21.15.94:0/4175599116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2681037a0 msgr2=0x7fc268198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc26e0e9640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2681037a0 0x7fc268198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc26e0e9640 1 -- 172.21.15.94:0/4175599116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26819d5d0 con 0x7fc268102df0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc26e8ea640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc26810c2f0 0x7fc26819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc26e0e9640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc268102df0 0x7fc268198620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc25800cce0 tx=0x7fc25800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc2677fe640 1 -- 172.21.15.94:0/4175599116 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25800f650 con 0x7fc268102df0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26819d8c0 con 0x7fc268102df0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc2677fe640 1 -- 172.21.15.94:0/4175599116 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc25800fca0 con 0x7fc268102df0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc2677fe640 1 -- 172.21.15.94:0/4175599116 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258018b90 con 0x7fc268102df0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.361+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2681a51a0 con 0x7fc268102df0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.362+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2681031d0 con 0x7fc268102df0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.363+0000 7fc2677fe640 1 -- 172.21.15.94:0/4175599116 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc25800f7f0 con 0x7fc268102df0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.364+0000 7fc2677fe640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc2500a3240 0x7fc2500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.364+0000 7fc2677fe640 1 -- 172.21.15.94:0/4175599116 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fc2580cfd50 con 0x7fc268102df0 2024-09-19T04:13:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.364+0000 7fc26d8e8640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc2500a3240 0x7fc2500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:20.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.365+0000 7fc26d8e8640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc2500a3240 0x7fc2500a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc25c006010 tx=0x7fc25c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:20.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.368+0000 7fc2677fe640 1 -- 172.21.15.94:0/4175599116 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc2580d3030 con 0x7fc268102df0 2024-09-19T04:13:20.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.538+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2681072c0 con 0x7fc268102df0 2024-09-19T04:13:20.540 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:20 smithi094 ceph-mon[30972]: pgmap v274: 97 pgs: 77 active+clean, 20 unknown; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-19T04:13:20.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.539+0000 7fc2677fe640 1 -- 172.21.15.94:0/4175599116 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22304 (secure 0 0 0) 0x7fc258014030 con 0x7fc268102df0 2024-09-19T04:13:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:20.542 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":79,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:13:17.025174+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:20.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc2500a3240 msgr2=0x7fc2500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:20.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc2500a3240 0x7fc2500a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc25c006010 tx=0x7fc25c000bc0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc268102df0 msgr2=0x7fc268198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc268102df0 0x7fc268198620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc25800cce0 tx=0x7fc25800a620 comp rx=0 tx=0).stop 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 shutdown_connections 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc2500a3240 0x7fc2500a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc26810c2f0 0x7fc26819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2681037a0 0x7fc268198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 --2- 172.21.15.94:0/4175599116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc268102df0 0x7fc268198620 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 >> 172.21.15.94:0/4175599116 conn(0x7fc2680fc7d0 msgr2=0x7fc2680fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 shutdown_connections 2024-09-19T04:13:20.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:20.542+0000 7fc27034d640 1 -- 172.21.15.94:0/4175599116 wait complete. 2024-09-19T04:13:20.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:20 smithi112 ceph-mon[35051]: pgmap v274: 97 pgs: 77 active+clean, 20 unknown; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-19T04:13:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:20 smithi104 ceph-mon[34145]: pgmap v274: 97 pgs: 77 active+clean, 20 unknown; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-19T04:13:21.202 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:13:21.456 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:21.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:21 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4175599116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:21.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:21 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4175599116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:21.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:21 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4175599116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:21.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.972+0000 7f06eaaac640 1 -- 172.21.15.94:0/2395062500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 msgr2=0x7f06e4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.972+0000 7f06eaaac640 1 --2- 172.21.15.94:0/2395062500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 0x7f06e4101200 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009900 tx=0x7f06d802a480 comp rx=0 tx=0).stop 2024-09-19T04:13:21.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.972+0000 7f06eaaac640 1 -- 172.21.15.94:0/2395062500 shutdown_connections 2024-09-19T04:13:21.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.972+0000 7f06eaaac640 1 --2- 172.21.15.94:0/2395062500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e4102ef0 0x7f06e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.972+0000 7f06eaaac640 1 --2- 172.21.15.94:0/2395062500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e4102570 0x7f06e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.972+0000 7f06eaaac640 1 --2- 172.21.15.94:0/2395062500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 0x7f06e4101200 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.972+0000 7f06eaaac640 1 -- 172.21.15.94:0/2395062500 >> 172.21.15.94:0/2395062500 conn(0x7f06e40fc7d0 msgr2=0x7f06e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.972+0000 7f06eaaac640 1 -- 172.21.15.94:0/2395062500 shutdown_connections 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.972+0000 7f06eaaac640 1 -- 172.21.15.94:0/2395062500 wait complete. 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.973+0000 7f06eaaac640 1 Processor -- start 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.973+0000 7f06eaaac640 1 -- start start 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.973+0000 7f06eaaac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 0x7f06e4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.973+0000 7f06eaaac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e4102570 0x7f06e4199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06eaaac640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e4102ef0 0x7f06e41a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06eaaac640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4110ed0 con 0x7f06e4100e20 2024-09-19T04:13:21.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06eaaac640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4110d50 con 0x7f06e4102ef0 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06eaaac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4111050 con 0x7f06e4102570 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e8848640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 0x7f06e4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e8848640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 0x7f06e4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58456/0 (socket says 172.21.15.94:58456) 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e8848640 1 -- 172.21.15.94:0/3651805649 learned_addr learned my addr 172.21.15.94:0/3651805649 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e3fff640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e4102570 0x7f06e4199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e9049640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e4102ef0 0x7f06e41a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e8848640 1 -- 172.21.15.94:0/3651805649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e4102570 msgr2=0x7f06e4199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e8848640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e4102570 0x7f06e4199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e8848640 1 -- 172.21.15.94:0/3651805649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e4102ef0 msgr2=0x7f06e41a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e8848640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e4102ef0 0x7f06e41a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e8848640 1 -- 172.21.15.94:0/3651805649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e41a4900 con 0x7f06e4100e20 2024-09-19T04:13:21.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e8848640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 0x7f06e4199340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f06d8006010 tx=0x7f06d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:21.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e1ffb640 1 -- 172.21.15.94:0/3651805649 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d803f070 con 0x7f06e4100e20 2024-09-19T04:13:21.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e41a4ad0 con 0x7f06e4100e20 2024-09-19T04:13:21.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e1ffb640 1 -- 172.21.15.94:0/3651805649 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d8003aa0 con 0x7f06e4100e20 2024-09-19T04:13:21.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.974+0000 7f06e1ffb640 1 -- 172.21.15.94:0/3651805649 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8035030 con 0x7f06e4100e20 2024-09-19T04:13:21.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.975+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e41a4f20 con 0x7f06e4100e20 2024-09-19T04:13:21.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.977+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e4101200 con 0x7f06e4100e20 2024-09-19T04:13:21.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.977+0000 7f06e1ffb640 1 -- 172.21.15.94:0/3651805649 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f06d8042070 con 0x7f06e4100e20 2024-09-19T04:13:21.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.977+0000 7f06e1ffb640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06d40a3240 0x7f06d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.978+0000 7f06e3fff640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06d40a3240 0x7f06d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.978+0000 7f06e1ffb640 1 -- 172.21.15.94:0/3651805649 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f06d80f0790 con 0x7f06e4100e20 2024-09-19T04:13:21.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.978+0000 7f06e3fff640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06d40a3240 0x7f06d40a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f06e406aac0 tx=0x7f06d0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:21.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:21.986+0000 7f06e1ffb640 1 -- 172.21.15.94:0/3651805649 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f06d80bd600 con 0x7f06e4100e20 2024-09-19T04:13:22.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.192+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f06e4109090 con 0x7f06e4100e20 2024-09-19T04:13:22.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.451+0000 7f06e1ffb640 1 -- 172.21.15.94:0/3651805649 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f06d80bcf00 con 0x7f06e4100e20 2024-09-19T04:13:22.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06d40a3240 msgr2=0x7f06d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:22.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06d40a3240 0x7f06d40a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f06e406aac0 tx=0x7f06d0009110 comp rx=0 tx=0).stop 2024-09-19T04:13:22.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 msgr2=0x7f06e4199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:22.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 0x7f06e4199340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f06d8006010 tx=0x7f06d80042f0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 shutdown_connections 2024-09-19T04:13:22.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06d40a3240 0x7f06d40a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e4102ef0 0x7f06e41a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e4102570 0x7f06e4199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 --2- 172.21.15.94:0/3651805649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06e4100e20 0x7f06e4199340 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 >> 172.21.15.94:0/3651805649 conn(0x7f06e40fc7d0 msgr2=0x7f06e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:22.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 shutdown_connections 2024-09-19T04:13:22.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:22.454+0000 7f06eaaac640 1 -- 172.21.15.94:0/3651805649 wait complete. 2024-09-19T04:13:22.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:22 smithi104 ceph-mon[34145]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-19T04:13:22.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:22 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3651805649' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:13:22.604 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:22 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b[63958]: 2024-09-19T04:13:22.331+0000 7f634f081640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:13:22.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:22 smithi112 ceph-mon[35051]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-19T04:13:22.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:22 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3651805649' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:13:22.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:22 smithi094 ceph-mon[30972]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-19T04:13:22.802 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:22 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3651805649' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:13:23.107 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-19T04:13:23.107 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-19T04:13:23.358 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:23 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3651805649' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:13:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:23 smithi104 ceph-mon[34145]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:23.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:23 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3651805649' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:13:23.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:23 smithi112 ceph-mon[35051]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:23.798 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:23 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3651805649' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:13:23.798 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:23 smithi094 ceph-mon[30972]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.881+0000 7fbded341640 1 -- 172.21.15.94:0/228147115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 msgr2=0x7fbde8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.881+0000 7fbded341640 1 --2- 172.21.15.94:0/228147115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 0x7fbde8100ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009930 tx=0x7fbdd002a420 comp rx=0 tx=0).stop 2024-09-19T04:13:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.881+0000 7fbded341640 1 -- 172.21.15.94:0/228147115 shutdown_connections 2024-09-19T04:13:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.881+0000 7fbded341640 1 --2- 172.21.15.94:0/228147115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbde8101420 0x7fbde810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.881+0000 7fbded341640 1 --2- 172.21.15.94:0/228147115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 0x7fbde8100ee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.881+0000 7fbded341640 1 --2- 172.21.15.94:0/228147115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbde8106a80 0x7fbde8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.881+0000 7fbded341640 1 -- 172.21.15.94:0/228147115 >> 172.21.15.94:0/228147115 conn(0x7fbde80fc7a0 msgr2=0x7fbde80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.881+0000 7fbded341640 1 -- 172.21.15.94:0/228147115 shutdown_connections 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.881+0000 7fbded341640 1 -- 172.21.15.94:0/228147115 wait complete. 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbded341640 1 Processor -- start 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbded341640 1 -- start start 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbded341640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 0x7fbde8196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbded341640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbde8101420 0x7fbde8197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbded341640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbde8106a80 0x7fbde8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbded341640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8110cf0 con 0x7fbde8100a80 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbded341640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8110b70 con 0x7fbde8101420 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbded341640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8110e70 con 0x7fbde8106a80 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 0x7fbde8196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 0x7fbde8196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33802/0 (socket says 172.21.15.94:33802) 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbde6ffd640 1 -- 172.21.15.94:0/3348819695 learned_addr learned my addr 172.21.15.94:0/3348819695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.882+0000 7fbde67fc640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbde8101420 0x7fbde8197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbde6ffd640 1 -- 172.21.15.94:0/3348819695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbde8106a80 msgr2=0x7fbde8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbde77fe640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbde8106a80 0x7fbde8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbde6ffd640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbde8106a80 0x7fbde8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbde6ffd640 1 -- 172.21.15.94:0/3348819695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbde8101420 msgr2=0x7fbde8197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbde6ffd640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbde8101420 0x7fbde8197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbde6ffd640 1 -- 172.21.15.94:0/3348819695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde81a6780 con 0x7fbde8100a80 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbde67fc640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbde8101420 0x7fbde8197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbde77fe640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbde8106a80 0x7fbde8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbde6ffd640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 0x7fbde8196f10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbde000cce0 tx=0x7fbde000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbddffff640 1 -- 172.21.15.94:0/3348819695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde000f650 con 0x7fbde8100a80 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbddffff640 1 -- 172.21.15.94:0/3348819695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbde000fca0 con 0x7fbde8100a80 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde81a6a70 con 0x7fbde8100a80 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbddffff640 1 -- 172.21.15.94:0/3348819695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0018b90 con 0x7fbde8100a80 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.883+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde81a6f80 con 0x7fbde8100a80 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.884+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde8100ee0 con 0x7fbde8100a80 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.885+0000 7fbddffff640 1 -- 172.21.15.94:0/3348819695 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbde000f7f0 con 0x7fbde8100a80 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.886+0000 7fbddffff640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbdd40a3240 0x7fbdd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.886+0000 7fbde67fc640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbdd40a3240 0x7fbdd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.886+0000 7fbddffff640 1 -- 172.21.15.94:0/3348819695 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fbde00cfc90 con 0x7fbde8100a80 2024-09-19T04:13:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.886+0000 7fbde67fc640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbdd40a3240 0x7fbdd40a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0000c00 tx=0x7fbdd0031cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:23.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:23.890+0000 7fbddffff640 1 -- 172.21.15.94:0/3348819695 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbde0014030 con 0x7fbde8100a80 2024-09-19T04:13:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.091+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fbde802d7c0 con 0x7fbde8100a80 2024-09-19T04:13:24.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.458+0000 7fbddffff640 1 -- 172.21.15.94:0/3348819695 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fbde009cb00 con 0x7fbde8100a80 2024-09-19T04:13:24.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.460+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbdd40a3240 msgr2=0x7fbdd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:24.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.460+0000 7fbded341640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbdd40a3240 0x7fbdd40a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0000c00 tx=0x7fbdd0031cf0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.460+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 msgr2=0x7fbde8196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:24.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.460+0000 7fbded341640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 0x7fbde8196f10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbde000cce0 tx=0x7fbde000a620 comp rx=0 tx=0).stop 2024-09-19T04:13:24.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.461+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 shutdown_connections 2024-09-19T04:13:24.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.461+0000 7fbded341640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbdd40a3240 0x7fbdd40a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.461+0000 7fbded341640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbde8106a80 0x7fbde8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.461+0000 7fbded341640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbde8101420 0x7fbde8197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.461+0000 7fbded341640 1 --2- 172.21.15.94:0/3348819695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbde8100a80 0x7fbde8196f10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.461+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 >> 172.21.15.94:0/3348819695 conn(0x7fbde80fc7a0 msgr2=0x7fbde80fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:24.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.461+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 shutdown_connections 2024-09-19T04:13:24.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:24.461+0000 7fbded341640 1 -- 172.21.15.94:0/3348819695 wait complete. 2024-09-19T04:13:24.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:24 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3348819695' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T04:13:24.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:24 smithi094 ceph-mon[30972]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T04:13:24.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:24 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3348819695' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T04:13:24.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:24 smithi112 ceph-mon[35051]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T04:13:24.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:24 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3348819695' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T04:13:24.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:24 smithi104 ceph-mon[34145]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T04:13:25.081 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T04:13:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:25.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:25 smithi094 ceph-mon[30972]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:13:25.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:25 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3348819695' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T04:13:25.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:25 smithi094 ceph-mon[30972]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:13:25.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:25 smithi094 ceph-mon[30972]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:25.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:25 smithi094 ceph-mon[30972]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:13:25.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:25 smithi094 ceph-mon[30972]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:13:25.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:25 smithi112 ceph-mon[35051]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:13:25.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:25 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3348819695' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T04:13:25.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:25 smithi112 ceph-mon[35051]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:13:25.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:25 smithi112 ceph-mon[35051]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:25.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:25 smithi112 ceph-mon[35051]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:13:25.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:25 smithi112 ceph-mon[35051]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:13:25.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.793+0000 7f6ceb149640 1 -- 172.21.15.94:0/3019353394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 msgr2=0x7f6ce4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:25.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.793+0000 7f6ceb149640 1 --2- 172.21.15.94:0/3019353394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 0x7f6ce4101200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009900 tx=0x7f6cd802a480 comp rx=0 tx=0).stop 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.793+0000 7f6ceb149640 1 -- 172.21.15.94:0/3019353394 shutdown_connections 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.793+0000 7f6ceb149640 1 --2- 172.21.15.94:0/3019353394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce4102ef0 0x7f6ce410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.793+0000 7f6ceb149640 1 --2- 172.21.15.94:0/3019353394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ce4102570 0x7f6ce41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.793+0000 7f6ceb149640 1 --2- 172.21.15.94:0/3019353394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 0x7f6ce4101200 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.793+0000 7f6ceb149640 1 -- 172.21.15.94:0/3019353394 >> 172.21.15.94:0/3019353394 conn(0x7f6ce40fc7d0 msgr2=0x7f6ce40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 -- 172.21.15.94:0/3019353394 shutdown_connections 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 -- 172.21.15.94:0/3019353394 wait complete. 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 Processor -- start 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 -- start start 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 0x7f6ce4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce4102570 0x7f6ce4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ce4102ef0 0x7f6ce419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce4110e50 con 0x7f6ce4100e20 2024-09-19T04:13:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce4110cd0 con 0x7f6ce4102ef0 2024-09-19T04:13:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ceb149640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce4110fd0 con 0x7f6ce4102570 2024-09-19T04:13:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ce8ee5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 0x7f6ce4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ce8ee5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 0x7f6ce4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33820/0 (socket says 172.21.15.94:33820) 2024-09-19T04:13:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.794+0000 7f6ce8ee5640 1 -- 172.21.15.94:0/353829159 learned_addr learned my addr 172.21.15.94:0/353829159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce8ee5640 1 -- 172.21.15.94:0/353829159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce4102570 msgr2=0x7f6ce4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce96e6640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ce4102ef0 0x7f6ce419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce8ee5640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce4102570 0x7f6ce4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce8ee5640 1 -- 172.21.15.94:0/353829159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ce4102ef0 msgr2=0x7f6ce419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce8ee5640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ce4102ef0 0x7f6ce419bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce8ee5640 1 -- 172.21.15.94:0/353829159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce419c2e0 con 0x7f6ce4100e20 2024-09-19T04:13:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce8ee5640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 0x7f6ce4197330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8006010 tx=0x7f6cd80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce1ffb640 1 -- 172.21.15.94:0/353829159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd803f030 con 0x7f6ce4100e20 2024-09-19T04:13:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce1ffb640 1 -- 172.21.15.94:0/353829159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cd8003cb0 con 0x7f6ce4100e20 2024-09-19T04:13:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce419c570 con 0x7f6ce4100e20 2024-09-19T04:13:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ce1ffb640 1 -- 172.21.15.94:0/353829159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8035070 con 0x7f6ce4100e20 2024-09-19T04:13:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.795+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce41a3eb0 con 0x7f6ce4100e20 2024-09-19T04:13:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.796+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce4101200 con 0x7f6ce4100e20 2024-09-19T04:13:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.798+0000 7f6ce1ffb640 1 -- 172.21.15.94:0/353829159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6cd8042030 con 0x7f6ce4100e20 2024-09-19T04:13:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.799+0000 7f6ce1ffb640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6cd40a32c0 0x7f6cd40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.799+0000 7f6ce3fff640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6cd40a32c0 0x7f6cd40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.799+0000 7f6ce1ffb640 1 -- 172.21.15.94:0/353829159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f6cd80eff60 con 0x7f6ce4100e20 2024-09-19T04:13:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.799+0000 7f6ce3fff640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6cd40a32c0 0x7f6cd40a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40ffe30 tx=0x7f6cd0005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:25.802+0000 7f6ce1ffb640 1 -- 172.21.15.94:0/353829159 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6cd80bcdd0 con 0x7f6ce4100e20 2024-09-19T04:13:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:25 smithi104 ceph-mon[34145]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:13:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:25 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3348819695' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T04:13:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:25 smithi104 ceph-mon[34145]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:13:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:25 smithi104 ceph-mon[34145]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:13:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:25 smithi104 ceph-mon[34145]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:13:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:25 smithi104 ceph-mon[34145]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:13:26.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.006+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6ce4109090 con 0x7f6ce4100e20 2024-09-19T04:13:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.478+0000 7f6ce1ffb640 1 -- 172.21.15.94:0/353829159 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f6cd80bc6d0 con 0x7f6ce4100e20 2024-09-19T04:13:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6cd40a32c0 msgr2=0x7f6cd40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6cd40a32c0 0x7f6cd40a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40ffe30 tx=0x7f6cd0005ed0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 msgr2=0x7f6ce4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 0x7f6ce4197330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8006010 tx=0x7f6cd80042f0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 shutdown_connections 2024-09-19T04:13:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6cd40a32c0 0x7f6cd40a5780 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ce4102ef0 0x7f6ce419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce4102570 0x7f6ce4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 --2- 172.21.15.94:0/353829159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ce4100e20 0x7f6ce4197330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.480+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 >> 172.21.15.94:0/353829159 conn(0x7f6ce40fc7d0 msgr2=0x7f6ce40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.481+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 shutdown_connections 2024-09-19T04:13:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:26.481+0000 7f6ceb149640 1 -- 172.21.15.94:0/353829159 wait complete. 2024-09-19T04:13:26.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:26 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] up:active 2024-09-19T04:13:26.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:26 smithi112 ceph-mon[35051]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:13:26.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:26 smithi112 ceph-mon[35051]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:13:26.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:26 smithi112 ceph-mon[35051]: Cluster is now healthy 2024-09-19T04:13:26.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:26 smithi112 ceph-mon[35051]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:13:26.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:26 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T04:13:26.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:26 smithi112 ceph-mon[35051]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T04:13:26.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:26 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/353829159' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:13:26.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:26 smithi112 ceph-mon[35051]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-19T04:13:26.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:26 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] up:active 2024-09-19T04:13:26.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:26 smithi094 ceph-mon[30972]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:13:26.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:26 smithi094 ceph-mon[30972]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:13:26.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:26 smithi094 ceph-mon[30972]: Cluster is now healthy 2024-09-19T04:13:26.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:26 smithi094 ceph-mon[30972]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:13:26.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:26 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T04:13:26.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:26 smithi094 ceph-mon[30972]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T04:13:26.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:26 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/353829159' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:13:26.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:26 smithi094 ceph-mon[30972]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-19T04:13:26.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:26 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] up:active 2024-09-19T04:13:26.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:26 smithi104 ceph-mon[34145]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:13:26.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:26 smithi104 ceph-mon[34145]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:13:26.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:26 smithi104 ceph-mon[34145]: Cluster is now healthy 2024-09-19T04:13:26.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:26 smithi104 ceph-mon[34145]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:13:26.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:26 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T04:13:26.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:26 smithi104 ceph-mon[34145]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T04:13:26.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:26 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/353829159' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:13:26.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:26 smithi104 ceph-mon[34145]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-19T04:13:27.187 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T04:13:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:27 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/353829159' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:13:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:27 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] up:active 2024-09-19T04:13:27.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:27 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T04:13:27.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:27 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:13:27.605 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:27 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b[63958]: 2024-09-19T04:13:27.331+0000 7f634f081640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T04:13:27.605 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:13:27 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b[63958]: 2024-09-19T04:13:27.331+0000 7f634f081640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T04:13:27.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:27 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/353829159' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:13:27.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:27 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] up:active 2024-09-19T04:13:27.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:27 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T04:13:27.733 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:27 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:13:27.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:27 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/353829159' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:13:27.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:27 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] up:active 2024-09-19T04:13:27.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:27 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T04:13:27.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:27 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:13:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.953+0000 7f19874a3640 1 -- 172.21.15.94:0/3540279169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f198010c170 msgr2=0x7f198010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.953+0000 7f19874a3640 1 --2- 172.21.15.94:0/3540279169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f198010c170 0x7f198010e610 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1978009ca0 tx=0x7f197802a950 comp rx=0 tx=0).stop 2024-09-19T04:13:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.953+0000 7f19874a3640 1 -- 172.21.15.94:0/3540279169 shutdown_connections 2024-09-19T04:13:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.953+0000 7f19874a3640 1 --2- 172.21.15.94:0/3540279169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f198010c170 0x7f198010e610 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.953+0000 7f19874a3640 1 --2- 172.21.15.94:0/3540279169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1980103660 0x7f198010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.953+0000 7f19874a3640 1 --2- 172.21.15.94:0/3540279169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1980102d40 0x7f1980103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.953+0000 7f19874a3640 1 -- 172.21.15.94:0/3540279169 >> 172.21.15.94:0/3540279169 conn(0x7f19800fc7a0 msgr2=0x7f19800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.953+0000 7f19874a3640 1 -- 172.21.15.94:0/3540279169 shutdown_connections 2024-09-19T04:13:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.953+0000 7f19874a3640 1 -- 172.21.15.94:0/3540279169 wait complete. 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f19874a3640 1 Processor -- start 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f19874a3640 1 -- start start 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f19874a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1980102d40 0x7f1980198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f19874a3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1980103660 0x7f19801989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f19874a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f198010c170 0x7f198019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f19874a3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1980111f30 con 0x7f1980102d40 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f19874a3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1980111db0 con 0x7f1980103660 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f19874a3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19801120b0 con 0x7f198010c170 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f198523f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1980102d40 0x7f1980198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f198523f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1980102d40 0x7f1980198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33834/0 (socket says 172.21.15.94:33834) 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f198523f640 1 -- 172.21.15.94:0/1805928409 learned_addr learned my addr 172.21.15.94:0/1805928409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f1984a3e640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1980103660 0x7f19801989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f198523f640 1 -- 172.21.15.94:0/1805928409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f198010c170 msgr2=0x7f198019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.954+0000 7f1985a40640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f198010c170 0x7f198019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f198523f640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f198010c170 0x7f198019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f198523f640 1 -- 172.21.15.94:0/1805928409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1980103660 msgr2=0x7f19801989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f198523f640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1980103660 0x7f19801989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f198523f640 1 -- 172.21.15.94:0/1805928409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198019d430 con 0x7f1980102d40 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f1984a3e640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1980103660 0x7f19801989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f198523f640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1980102d40 0x7f1980198480 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f197c00b4a0 tx=0x7f197c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f19767fc640 1 -- 172.21.15.94:0/1805928409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c014070 con 0x7f1980102d40 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f19767fc640 1 -- 172.21.15.94:0/1805928409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f197c010ac0 con 0x7f1980102d40 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198019d720 con 0x7f1980102d40 2024-09-19T04:13:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f19767fc640 1 -- 172.21.15.94:0/1805928409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c017c30 con 0x7f1980102d40 2024-09-19T04:13:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.955+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801a5000 con 0x7f1980102d40 2024-09-19T04:13:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.956+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19801031a0 con 0x7f1980102d40 2024-09-19T04:13:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.957+0000 7f19767fc640 1 -- 172.21.15.94:0/1805928409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f197c017070 con 0x7f1980102d40 2024-09-19T04:13:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.957+0000 7f19767fc640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f196c0a32c0 0x7f196c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.958+0000 7f19767fc640 1 -- 172.21.15.94:0/1805928409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f197c0ceeb0 con 0x7f1980102d40 2024-09-19T04:13:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.958+0000 7f1984a3e640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f196c0a32c0 0x7f196c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.958+0000 7f1984a3e640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f196c0a32c0 0x7f196c0a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1970006fd0 tx=0x7f1970008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:27.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:27.962+0000 7f19767fc640 1 -- 172.21.15.94:0/1805928409 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f197c09bd20 con 0x7f1980102d40 2024-09-19T04:13:28.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.165+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f19801071e0 con 0x7f1980102d40 2024-09-19T04:13:28.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.484+0000 7f19767fc640 1 -- 172.21.15.94:0/1805928409 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f197c09b620 con 0x7f1980102d40 2024-09-19T04:13:28.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.486+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f196c0a32c0 msgr2=0x7f196c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:28.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.486+0000 7f19874a3640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f196c0a32c0 0x7f196c0a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1970006fd0 tx=0x7f1970008040 comp rx=0 tx=0).stop 2024-09-19T04:13:28.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.486+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1980102d40 msgr2=0x7f1980198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:28.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.486+0000 7f19874a3640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1980102d40 0x7f1980198480 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f197c00b4a0 tx=0x7f197c00b960 comp rx=0 tx=0).stop 2024-09-19T04:13:28.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.487+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 shutdown_connections 2024-09-19T04:13:28.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.487+0000 7f19874a3640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f196c0a32c0 0x7f196c0a5780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.487+0000 7f19874a3640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f198010c170 0x7f198019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.487+0000 7f19874a3640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1980103660 0x7f19801989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.487+0000 7f19874a3640 1 --2- 172.21.15.94:0/1805928409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1980102d40 0x7f1980198480 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.487+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 >> 172.21.15.94:0/1805928409 conn(0x7f19800fc7a0 msgr2=0x7f19800fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:28.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.487+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 shutdown_connections 2024-09-19T04:13:28.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:28.487+0000 7f19874a3640 1 -- 172.21.15.94:0/1805928409 wait complete. 2024-09-19T04:13:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:28 smithi112 ceph-mon[35051]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 478 B/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-19T04:13:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:28 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1805928409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:13:28.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:28 smithi094 ceph-mon[30972]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 478 B/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-19T04:13:28.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:28 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1805928409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:13:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:28 smithi104 ceph-mon[34145]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 478 B/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-19T04:13:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:28 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1805928409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:13:29.228 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-19T04:13:29.228 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T04:13:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:29.739 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:29 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1805928409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:13:29.739 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:29 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:13:29.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:29 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1805928409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:13:29.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:29 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:13:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:29 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1805928409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:13:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:29 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:13:29.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.943+0000 7f1f0261e640 1 -- 172.21.15.94:0/1662564149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101240 msgr2=0x7f1efc101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:29.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.943+0000 7f1f0261e640 1 --2- 172.21.15.94:0/1662564149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101240 0x7f1efc101680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef002a410 comp rx=0 tx=0).stop 2024-09-19T04:13:29.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 -- 172.21.15.94:0/1662564149 shutdown_connections 2024-09-19T04:13:29.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 --2- 172.21.15.94:0/1662564149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1efc101bc0 0x7f1efc10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 --2- 172.21.15.94:0/1662564149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101240 0x7f1efc101680 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 --2- 172.21.15.94:0/1662564149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1efc105a90 0x7f1efc105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 -- 172.21.15.94:0/1662564149 >> 172.21.15.94:0/1662564149 conn(0x7f1efc077210 msgr2=0x7f1efc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 -- 172.21.15.94:0/1662564149 shutdown_connections 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 -- 172.21.15.94:0/1662564149 wait complete. 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 Processor -- start 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 -- start start 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1efc101240 0x7f1efc10e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.944+0000 7f1f0261e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101bc0 0x7f1efc10e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1f0261e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1efc105a90 0x7f1efc10eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1f0261e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc111f60 con 0x7f1efc101bc0 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1f0261e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc111de0 con 0x7f1efc101240 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1f0261e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1120e0 con 0x7f1efc105a90 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101bc0 0x7f1efc10e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101bc0 0x7f1efc10e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33844/0 (socket says 172.21.15.94:33844) 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efb7fe640 1 -- 172.21.15.94:0/2329896009 learned_addr learned my addr 172.21.15.94:0/2329896009 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1f00bbb640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1efc105a90 0x7f1efc10eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efbfff640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1efc101240 0x7f1efc10e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efb7fe640 1 -- 172.21.15.94:0/2329896009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1efc105a90 msgr2=0x7f1efc10eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efb7fe640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1efc105a90 0x7f1efc10eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efb7fe640 1 -- 172.21.15.94:0/2329896009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1efc101240 msgr2=0x7f1efc10e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efb7fe640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1efc101240 0x7f1efc10e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efb7fe640 1 -- 172.21.15.94:0/2329896009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc1a6930 con 0x7f1efc101bc0 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efbfff640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1efc101240 0x7f1efc10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1efb7fe640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101bc0 0x7f1efc10e960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0006010 tx=0x7f1ef00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1ef97fa640 1 -- 172.21.15.94:0/2329896009 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef003f030 con 0x7f1efc101bc0 2024-09-19T04:13:29.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1efc1a6bc0 con 0x7f1efc101bc0 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1ef97fa640 1 -- 172.21.15.94:0/2329896009 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef0003cb0 con 0x7f1efc101bc0 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1ef97fa640 1 -- 172.21.15.94:0/2329896009 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0035070 con 0x7f1efc101bc0 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.945+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1efc1a70f0 con 0x7f1efc101bc0 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.946+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1efc101680 con 0x7f1efc101bc0 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.948+0000 7f1ef97fa640 1 -- 172.21.15.94:0/2329896009 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1ef0042030 con 0x7f1efc101bc0 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.949+0000 7f1ef97fa640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1ee40a3240 0x7f1ee40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.949+0000 7f1efbfff640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1ee40a3240 0x7f1ee40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.949+0000 7f1ef97fa640 1 -- 172.21.15.94:0/2329896009 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f1ef00eff20 con 0x7f1efc101bc0 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.949+0000 7f1efbfff640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1ee40a3240 0x7f1ee40a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0058c0 tx=0x7f1eec005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:29.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:29.952+0000 7f1ef97fa640 1 -- 172.21.15.94:0/2329896009 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ef00bcd70 con 0x7f1efc101bc0 2024-09-19T04:13:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.143+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1efc062630 con 0x7f1ee40a3240 2024-09-19T04:13:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.171+0000 7f1ef97fa640 1 -- 172.21.15.94:0/2329896009 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1efc062630 con 0x7f1ee40a3240 2024-09-19T04:13:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1ee40a3240 msgr2=0x7f1ee40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1ee40a3240 0x7f1ee40a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0058c0 tx=0x7f1eec005850 comp rx=0 tx=0).stop 2024-09-19T04:13:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101bc0 msgr2=0x7f1efc10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101bc0 0x7f1efc10e960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0006010 tx=0x7f1ef00042f0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 shutdown_connections 2024-09-19T04:13:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f1ee40a3240 0x7f1ee40a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1efc105a90 0x7f1efc10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1efc101bc0 0x7f1efc10e960 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 --2- 172.21.15.94:0/2329896009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1efc101240 0x7f1efc10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 >> 172.21.15.94:0/2329896009 conn(0x7f1efc077210 msgr2=0x7f1efc104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 shutdown_connections 2024-09-19T04:13:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:30.174+0000 7f1f0261e640 1 -- 172.21.15.94:0/2329896009 wait complete. 2024-09-19T04:13:30.475 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:13:30.148+0000 7f74c19c7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:13:30.476 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:13:30.148+0000 7f74c19c7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:13:30.476 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:13:30.148+0000 7f74c19c7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:13:30.476 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:13:30.148+0000 7f74c19c7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:13:30.476 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:13:30.148+0000 7f74c19c7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:13:30.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T04:13:30.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-19T04:13:30.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-19T04:13:30.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-19T04:13:30.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-19T04:13:30.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-09-19T04:13:30.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-09-19T04:13:30.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-19T04:13:30.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-19T04:13:30.752 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: pgmap v279: 97 pgs: 97 active+clean; 580 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-19T04:13:30.752 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:30 smithi094 ceph-mon[30972]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:30.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-19T04:13:30.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: pgmap v279: 97 pgs: 97 active+clean; 580 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-19T04:13:30.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:30 smithi112 ceph-mon[35051]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:30.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T04:13:30.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-19T04:13:30.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-19T04:13:30.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-19T04:13:30.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-19T04:13:30.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-09-19T04:13:30.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-09-19T04:13:30.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-19T04:13:30.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-19T04:13:30.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: pgmap v279: 97 pgs: 97 active+clean; 580 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-19T04:13:30.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:30 smithi104 ceph-mon[34145]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:31.022 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:31.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.486+0000 7fc49e962640 1 -- 172.21.15.94:0/3192697740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 msgr2=0x7fc4981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.486+0000 7fc49e962640 1 --2- 172.21.15.94:0/3192697740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 0x7fc4981067e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009900 tx=0x7fc48c02a410 comp rx=0 tx=0).stop 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.486+0000 7fc49e962640 1 -- 172.21.15.94:0/3192697740 shutdown_connections 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.486+0000 7fc49e962640 1 --2- 172.21.15.94:0/3192697740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc498106f10 0x7fc49810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.486+0000 7fc49e962640 1 --2- 172.21.15.94:0/3192697740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 0x7fc4981067e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.486+0000 7fc49e962640 1 --2- 172.21.15.94:0/3192697740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc498101de0 0x7fc4981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.486+0000 7fc49e962640 1 -- 172.21.15.94:0/3192697740 >> 172.21.15.94:0/3192697740 conn(0x7fc4980fd8c0 msgr2=0x7fc4980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.486+0000 7fc49e962640 1 -- 172.21.15.94:0/3192697740 shutdown_connections 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.486+0000 7fc49e962640 1 -- 172.21.15.94:0/3192697740 wait complete. 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc49e962640 1 Processor -- start 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc49e962640 1 -- start start 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc49e962640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc498101de0 0x7fc498198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc49e962640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 0x7fc498198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc49e962640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc498106f10 0x7fc49819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc49e962640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc498111e40 con 0x7fc498102790 2024-09-19T04:13:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc49e962640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc498111cc0 con 0x7fc498106f10 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc49e962640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc498111fc0 con 0x7fc498101de0 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc4977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 0x7fc498198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc4977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 0x7fc498198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33876/0 (socket says 172.21.15.94:33876) 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc4977fe640 1 -- 172.21.15.94:0/2656949360 learned_addr learned my addr 172.21.15.94:0/2656949360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc49ceff640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc498106f10 0x7fc49819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.487+0000 7fc497fff640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc498101de0 0x7fc498198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc4977fe640 1 -- 172.21.15.94:0/2656949360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc498101de0 msgr2=0x7fc498198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc4977fe640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc498101de0 0x7fc498198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc4977fe640 1 -- 172.21.15.94:0/2656949360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc498106f10 msgr2=0x7fc49819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc4977fe640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc498106f10 0x7fc49819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc4977fe640 1 -- 172.21.15.94:0/2656949360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49819d3d0 con 0x7fc498102790 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc4977fe640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 0x7fc498198960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc48c005940 tx=0x7fc48c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc4957fa640 1 -- 172.21.15.94:0/2656949360 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c03f030 con 0x7fc498102790 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc4957fa640 1 -- 172.21.15.94:0/2656949360 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc48c003cb0 con 0x7fc498102790 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49819d660 con 0x7fc498102790 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc4957fa640 1 -- 172.21.15.94:0/2656949360 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c035070 con 0x7fc498102790 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.488+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4981a4fa0 con 0x7fc498102790 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.489+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4981032c0 con 0x7fc498102790 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.490+0000 7fc4957fa640 1 -- 172.21.15.94:0/2656949360 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc48c042030 con 0x7fc498102790 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.491+0000 7fc4957fa640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc4800a32d0 0x7fc4800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.491+0000 7fc497fff640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc4800a32d0 0x7fc4800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.491+0000 7fc4957fa640 1 -- 172.21.15.94:0/2656949360 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc48c0f0200 con 0x7fc498102790 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.491+0000 7fc497fff640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc4800a32d0 0x7fc4800a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc488009990 tx=0x7fc488009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.495+0000 7fc4957fa640 1 -- 172.21.15.94:0/2656949360 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc48c0bcdc0 con 0x7fc498102790 2024-09-19T04:13:31.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.690+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc498062630 con 0x7fc4800a32d0 2024-09-19T04:13:31.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T04:13:31.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-19T04:13:31.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-19T04:13:31.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-19T04:13:31.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-19T04:13:31.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-09-19T04:13:31.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-09-19T04:13:31.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-19T04:13:31.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-19T04:13:31.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:13:31.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:31 smithi094 ceph-mon[30972]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:13:31.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.717+0000 7fc4957fa640 1 -- 172.21.15.94:0/2656949360 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc498062630 con 0x7fc4800a32d0 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc4800a32d0 msgr2=0x7fc4800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc4800a32d0 0x7fc4800a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc488009990 tx=0x7fc488009110 comp rx=0 tx=0).stop 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 msgr2=0x7fc498198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 0x7fc498198960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc48c005940 tx=0x7fc48c0042f0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 shutdown_connections 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc4800a32d0 0x7fc4800a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc498106f10 0x7fc49819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc498102790 0x7fc498198960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 --2- 172.21.15.94:0/2656949360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc498101de0 0x7fc498198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.719+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 >> 172.21.15.94:0/2656949360 conn(0x7fc4980fd8c0 msgr2=0x7fc4980fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.720+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 shutdown_connections 2024-09-19T04:13:31.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:31.720+0000 7fc49e962640 1 -- 172.21.15.94:0/2656949360 wait complete. 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:13:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:31 smithi112 ceph-mon[35051]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:13:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T04:13:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-19T04:13:31.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-19T04:13:31.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-19T04:13:31.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-19T04:13:31.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-09-19T04:13:31.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-09-19T04:13:31.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-19T04:13:31.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-19T04:13:31.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:13:31.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:31 smithi104 ceph-mon[34145]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:13:32.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T04:13:32.627 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:32.748 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:32 smithi094 ceph-mon[30972]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:13:32.748 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:32 smithi094 ceph-mon[30972]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:32.748 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:32 smithi094 ceph-mon[30972]: pgmap v282: 97 pgs: 97 active+clean; 582 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-09-19T04:13:32.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:32 smithi112 ceph-mon[35051]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:13:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:32 smithi112 ceph-mon[35051]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:32.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:32 smithi112 ceph-mon[35051]: pgmap v282: 97 pgs: 97 active+clean; 582 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-09-19T04:13:32.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:32 smithi104 ceph-mon[34145]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:13:32.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:32 smithi104 ceph-mon[34145]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:32.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:32 smithi104 ceph-mon[34145]: pgmap v282: 97 pgs: 97 active+clean; 582 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-09-19T04:13:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.103+0000 7f79f6bae640 1 -- 172.21.15.94:0/2819643579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0102790 msgr2=0x7f79f01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.103+0000 7f79f6bae640 1 --2- 172.21.15.94:0/2819643579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0102790 0x7f79f01067e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e002a410 comp rx=0 tx=0).stop 2024-09-19T04:13:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.103+0000 7f79f6bae640 1 -- 172.21.15.94:0/2819643579 shutdown_connections 2024-09-19T04:13:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.103+0000 7f79f6bae640 1 --2- 172.21.15.94:0/2819643579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79f0106f10 0x7f79f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.103+0000 7f79f6bae640 1 --2- 172.21.15.94:0/2819643579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0102790 0x7f79f01067e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.103+0000 7f79f6bae640 1 --2- 172.21.15.94:0/2819643579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f0101de0 0x7f79f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.103+0000 7f79f6bae640 1 -- 172.21.15.94:0/2819643579 >> 172.21.15.94:0/2819643579 conn(0x7f79f00fd8c0 msgr2=0x7f79f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.103+0000 7f79f6bae640 1 -- 172.21.15.94:0/2819643579 shutdown_connections 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.103+0000 7f79f6bae640 1 -- 172.21.15.94:0/2819643579 wait complete. 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f6bae640 1 Processor -- start 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f6bae640 1 -- start start 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f6bae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0101de0 0x7f79f0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f6bae640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79f0102790 0x7f79f0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f6bae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f0106f10 0x7f79f019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f6bae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111e40 con 0x7f79f0101de0 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f6bae640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111cc0 con 0x7f79f0102790 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f6bae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111fc0 con 0x7f79f0106f10 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f494a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0101de0 0x7f79f0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:33.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f494a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0101de0 0x7f79f0198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53148/0 (socket says 172.21.15.94:53148) 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f494a640 1 -- 172.21.15.94:0/1543258338 learned_addr learned my addr 172.21.15.94:0/1543258338 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.104+0000 7f79f514b640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f0106f10 0x7f79f019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79f494a640 1 -- 172.21.15.94:0/1543258338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f0106f10 msgr2=0x7f79f019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79f494a640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f0106f10 0x7f79f019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79f494a640 1 -- 172.21.15.94:0/1543258338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79f0102790 msgr2=0x7f79f0198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79f494a640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79f0102790 0x7f79f0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79f494a640 1 -- 172.21.15.94:0/1543258338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f019d4d0 con 0x7f79f0101de0 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79f494a640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0101de0 0x7f79f0198520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00b4a0 tx=0x7f79ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79e5ffb640 1 -- 172.21.15.94:0/1543258338 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec014070 con 0x7f79f0101de0 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79e5ffb640 1 -- 172.21.15.94:0/1543258338 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79ec010ac0 con 0x7f79f0101de0 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79e5ffb640 1 -- 172.21.15.94:0/1543258338 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec017c30 con 0x7f79f0101de0 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f019d7c0 con 0x7f79f0101de0 2024-09-19T04:13:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.105+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01a50a0 con 0x7f79f0101de0 2024-09-19T04:13:33.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.106+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f01021c0 con 0x7f79f0101de0 2024-09-19T04:13:33.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.107+0000 7f79e5ffb640 1 -- 172.21.15.94:0/1543258338 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f79ec017070 con 0x7f79f0101de0 2024-09-19T04:13:33.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.108+0000 7f79e5ffb640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79d80a3350 0x7f79d80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:33.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.108+0000 7f79e7fff640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79d80a3350 0x7f79d80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:33.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.108+0000 7f79e5ffb640 1 -- 172.21.15.94:0/1543258338 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f79ec0ced90 con 0x7f79f0101de0 2024-09-19T04:13:33.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.108+0000 7f79e7fff640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79d80a3350 0x7f79d80a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79e00096c0 tx=0x7f79e0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:33.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.112+0000 7f79e5ffb640 1 -- 172.21.15.94:0/1543258338 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f79ec0d2030 con 0x7f79f0101de0 2024-09-19T04:13:33.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.305+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f79f010a160 con 0x7f79d80a3350 2024-09-19T04:13:33.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.309+0000 7f79e5ffb640 1 -- 172.21.15.94:0/1543258338 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f79e0032d30 con 0x7f79d80a3350 2024-09-19T04:13:33.310 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_0/e3926d97-f545-4f3f-8efc-f496b72b2ed6 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79d80a3350 msgr2=0x7f79d80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79d80a3350 0x7f79d80a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79e00096c0 tx=0x7f79e0031400 comp rx=0 tx=0).stop 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0101de0 msgr2=0x7f79f0198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0101de0 0x7f79f0198520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00b4a0 tx=0x7f79ec00b960 comp rx=0 tx=0).stop 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 shutdown_connections 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f79d80a3350 0x7f79d80a5810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f0106f10 0x7f79f019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79f0102790 0x7f79f0198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 --2- 172.21.15.94:0/1543258338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0101de0 0x7f79f0198520 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 >> 172.21.15.94:0/1543258338 conn(0x7f79f00fd8c0 msgr2=0x7f79f00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 shutdown_connections 2024-09-19T04:13:33.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:33.311+0000 7f79f6bae640 1 -- 172.21.15.94:0/1543258338 wait complete. 2024-09-19T04:13:33.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:33 smithi094 ceph-mon[30972]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:33.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:33 smithi112 ceph-mon[35051]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:33.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:33 smithi104 ceph-mon[34145]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:34.010 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T04:13:34.255 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:34.637 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:34 smithi094 ceph-mon[30972]: pgmap v283: 97 pgs: 97 active+clean; 607 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 16 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:34.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.720+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2336263229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc10e4d0 msgr2=0x7ff3cc1108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:34.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.720+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2336263229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc10e4d0 0x7ff3cc1108e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4009900 tx=0x7ff3c402a410 comp rx=0 tx=0).stop 2024-09-19T04:13:34.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.720+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2336263229 shutdown_connections 2024-09-19T04:13:34.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.720+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2336263229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc10e4d0 0x7ff3cc1108e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:34.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.720+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2336263229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc101360 0x7ff3cc10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:34.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.720+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2336263229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3cc1009b0 0x7ff3cc100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:34.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.720+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2336263229 >> 172.21.15.94:0/2336263229 conn(0x7ff3cc0fc7d0 msgr2=0x7ff3cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:34.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.720+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2336263229 shutdown_connections 2024-09-19T04:13:34.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.720+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2336263229 wait complete. 2024-09-19T04:13:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3d2d35640 1 Processor -- start 2024-09-19T04:13:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3d2d35640 1 -- start start 2024-09-19T04:13:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3d2d35640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1009b0 0x7ff3cc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3d2d35640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc101360 0x7ff3cc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3d2d35640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3cc10e4d0 0x7ff3cc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3d2d35640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1140d0 con 0x7ff3cc101360 2024-09-19T04:13:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3d2d35640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc113f50 con 0x7ff3cc10e4d0 2024-09-19T04:13:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3d2d35640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc114250 con 0x7ff3cc1009b0 2024-09-19T04:13:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc101360 0x7ff3cc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc101360 0x7ff3cc19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53160/0 (socket says 172.21.15.94:53160) 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.721+0000 7ff3cbfff640 1 -- 172.21.15.94:0/2827011025 learned_addr learned my addr 172.21.15.94:0/2827011025 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3d12d2640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3cc10e4d0 0x7ff3cc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3d0ad1640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1009b0 0x7ff3cc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3cbfff640 1 -- 172.21.15.94:0/2827011025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1009b0 msgr2=0x7ff3cc19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3cbfff640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1009b0 0x7ff3cc19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3cbfff640 1 -- 172.21.15.94:0/2827011025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3cc10e4d0 msgr2=0x7ff3cc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3cbfff640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3cc10e4d0 0x7ff3cc19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3cbfff640 1 -- 172.21.15.94:0/2827011025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc19f640 con 0x7ff3cc101360 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3cbfff640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc101360 0x7ff3cc19abd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c970 tx=0x7ff3b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:34.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3c9ffb640 1 -- 172.21.15.94:0/2827011025 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4007bb0 con 0x7ff3cc101360 2024-09-19T04:13:34.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc19f930 con 0x7ff3cc101360 2024-09-19T04:13:34.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3c9ffb640 1 -- 172.21.15.94:0/2827011025 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3b4007d50 con 0x7ff3cc101360 2024-09-19T04:13:34.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3c9ffb640 1 -- 172.21.15.94:0/2827011025 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400f450 con 0x7ff3cc101360 2024-09-19T04:13:34.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.722+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc1a7210 con 0x7ff3cc101360 2024-09-19T04:13:34.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.723+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3cc100d90 con 0x7ff3cc101360 2024-09-19T04:13:34.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.724+0000 7ff3c9ffb640 1 -- 172.21.15.94:0/2827011025 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff3b4004f20 con 0x7ff3cc101360 2024-09-19T04:13:34.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.725+0000 7ff3c9ffb640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3b80a32d0 0x7ff3b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:34.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.725+0000 7ff3d0ad1640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3b80a32d0 0x7ff3b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:34.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.725+0000 7ff3c9ffb640 1 -- 172.21.15.94:0/2827011025 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff3b40cf6e0 con 0x7ff3cc101360 2024-09-19T04:13:34.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.725+0000 7ff3d0ad1640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3b80a32d0 0x7ff3b80a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc005c40 tx=0x7ff3bc00a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:34.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.729+0000 7ff3c9ffb640 1 -- 172.21.15.94:0/2827011025 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff3b4014030 con 0x7ff3cc101360 2024-09-19T04:13:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:34 smithi112 ceph-mon[35051]: pgmap v283: 97 pgs: 97 active+clean; 607 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 16 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:34 smithi104 ceph-mon[34145]: pgmap v283: 97 pgs: 97 active+clean; 607 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 16 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:34.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.920+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff3cc062630 con 0x7ff3b80a32d0 2024-09-19T04:13:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.944+0000 7ff3c9ffb640 1 -- 172.21.15.94:0/2827011025 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff3cc062630 con 0x7ff3b80a32d0 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3b80a32d0 msgr2=0x7ff3b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3b80a32d0 0x7ff3b80a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc005c40 tx=0x7ff3bc00a590 comp rx=0 tx=0).stop 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc101360 msgr2=0x7ff3cc19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc101360 0x7ff3cc19abd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c970 tx=0x7ff3b400ce30 comp rx=0 tx=0).stop 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 shutdown_connections 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff3b80a32d0 0x7ff3b80a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3cc10e4d0 0x7ff3cc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc101360 0x7ff3cc19abd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 --2- 172.21.15.94:0/2827011025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1009b0 0x7ff3cc19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 >> 172.21.15.94:0/2827011025 conn(0x7ff3cc0fc7d0 msgr2=0x7ff3cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 shutdown_connections 2024-09-19T04:13:34.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:34.946+0000 7ff3d2d35640 1 -- 172.21.15.94:0/2827011025 wait complete. 2024-09-19T04:13:35.593 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T04:13:35.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:35 smithi112 ceph-mon[35051]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:35.837 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:35.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:35 smithi094 ceph-mon[30972]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:35.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:35 smithi104 ceph-mon[34145]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:36.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 -- 172.21.15.94:0/101398750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8102790 msgr2=0x7fdec81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 --2- 172.21.15.94:0/101398750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8102790 0x7fdec81067e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009900 tx=0x7fdeb802a410 comp rx=0 tx=0).stop 2024-09-19T04:13:36.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 -- 172.21.15.94:0/101398750 shutdown_connections 2024-09-19T04:13:36.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 --2- 172.21.15.94:0/101398750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdec8106f10 0x7fdec810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 --2- 172.21.15.94:0/101398750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8102790 0x7fdec81067e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 --2- 172.21.15.94:0/101398750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec8101de0 0x7fdec81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 -- 172.21.15.94:0/101398750 >> 172.21.15.94:0/101398750 conn(0x7fdec80fd8c0 msgr2=0x7fdec80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 -- 172.21.15.94:0/101398750 shutdown_connections 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 -- 172.21.15.94:0/101398750 wait complete. 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.310+0000 7fdecfe65640 1 Processor -- start 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecfe65640 1 -- start start 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecfe65640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec8101de0 0x7fdec8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecfe65640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdec8102790 0x7fdec8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecfe65640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8106f10 0x7fdec819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecfe65640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8111e20 con 0x7fdec8106f10 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecfe65640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8111ca0 con 0x7fdec8101de0 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecfe65640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8111fa0 con 0x7fdec8102790 2024-09-19T04:13:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdece402640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8106f10 0x7fdec819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdece402640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8106f10 0x7fdec819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53186/0 (socket says 172.21.15.94:53186) 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdece402640 1 -- 172.21.15.94:0/828707538 learned_addr learned my addr 172.21.15.94:0/828707538 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecdc01640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec8101de0 0x7fdec8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecd400640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdec8102790 0x7fdec8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdece402640 1 -- 172.21.15.94:0/828707538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdec8102790 msgr2=0x7fdec8198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdece402640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdec8102790 0x7fdec8198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdece402640 1 -- 172.21.15.94:0/828707538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec8101de0 msgr2=0x7fdec8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdece402640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec8101de0 0x7fdec8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdece402640 1 -- 172.21.15.94:0/828707538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec819d2f0 con 0x7fdec8106f10 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecdc01640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec8101de0 0x7fdec8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecd400640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdec8102790 0x7fdec8198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdece402640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8106f10 0x7fdec819cc10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdec000eaa0 tx=0x7fdec000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdebeffd640 1 -- 172.21.15.94:0/828707538 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec000cdb0 con 0x7fdec8106f10 2024-09-19T04:13:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdebeffd640 1 -- 172.21.15.94:0/828707538 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdec0005020 con 0x7fdec8106f10 2024-09-19T04:13:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.311+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec819d5e0 con 0x7fdec8106f10 2024-09-19T04:13:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.312+0000 7fdebeffd640 1 -- 172.21.15.94:0/828707538 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00116d0 con 0x7fdec8106f10 2024-09-19T04:13:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.312+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec81a4ec0 con 0x7fdec8106f10 2024-09-19T04:13:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.312+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec81021c0 con 0x7fdec8106f10 2024-09-19T04:13:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.314+0000 7fdebeffd640 1 -- 172.21.15.94:0/828707538 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdec00051d0 con 0x7fdec8106f10 2024-09-19T04:13:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.314+0000 7fdebeffd640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdeb40a3350 0x7fdeb40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.314+0000 7fdecdc01640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdeb40a3350 0x7fdeb40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.314+0000 7fdebeffd640 1 -- 172.21.15.94:0/828707538 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdec00cfc10 con 0x7fdec8106f10 2024-09-19T04:13:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.315+0000 7fdecdc01640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdeb40a3350 0x7fdeb40a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdec400abe0 tx=0x7fdec400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.319+0000 7fdebeffd640 1 -- 172.21.15.94:0/828707538 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdec0016030 con 0x7fdec8106f10 2024-09-19T04:13:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.511+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fdec8062630 con 0x7fdeb40a3350 2024-09-19T04:13:36.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.516+0000 7fdebeffd640 1 -- 172.21.15.94:0/828707538 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdec8062630 con 0x7fdeb40a3350 2024-09-19T04:13:36.518 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_1/eac5cf9b-0fe8-450f-91d7-9b3e51a8e59e 2024-09-19T04:13:36.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdeb40a3350 msgr2=0x7fdeb40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdeb40a3350 0x7fdeb40a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdec400abe0 tx=0x7fdec400a5c0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8106f10 msgr2=0x7fdec819cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8106f10 0x7fdec819cc10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdec000eaa0 tx=0x7fdec000ef60 comp rx=0 tx=0).stop 2024-09-19T04:13:36.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 shutdown_connections 2024-09-19T04:13:36.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fdeb40a3350 0x7fdeb40a5810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec8106f10 0x7fdec819cc10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdec8102790 0x7fdec8198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 --2- 172.21.15.94:0/828707538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec8101de0 0x7fdec8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 >> 172.21.15.94:0/828707538 conn(0x7fdec80fd8c0 msgr2=0x7fdec80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 shutdown_connections 2024-09-19T04:13:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:36.519+0000 7fdecfe65640 1 -- 172.21.15.94:0/828707538 wait complete. 2024-09-19T04:13:36.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:36 smithi112 ceph-mon[35051]: pgmap v284: 97 pgs: 97 active+clean; 607 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 15 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:36.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:36 smithi094 ceph-mon[30972]: pgmap v284: 97 pgs: 97 active+clean; 607 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 15 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:36.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:36 smithi104 ceph-mon[34145]: pgmap v284: 97 pgs: 97 active+clean; 607 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 15 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:37.169 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:13:37.415 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:37.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:37 smithi112 ceph-mon[35051]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:37.820 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:37 smithi094 ceph-mon[30972]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:37 smithi104 ceph-mon[34145]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.897+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1621619793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc074eb0 msgr2=0x7f6adc112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.897+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1621619793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc074eb0 0x7f6adc112650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009900 tx=0x7f6ac402a480 comp rx=0 tx=0).stop 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.897+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1621619793 shutdown_connections 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.897+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1621619793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6adc112b90 0x7f6adc114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.897+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1621619793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc074eb0 0x7f6adc112650 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.897+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1621619793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6adc074590 0x7f6adc074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.897+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1621619793 >> 172.21.15.94:0/1621619793 conn(0x7f6adc0fd8c0 msgr2=0x7f6adc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.897+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1621619793 shutdown_connections 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.897+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1621619793 wait complete. 2024-09-19T04:13:37.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6ae1cab640 1 Processor -- start 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6ae1cab640 1 -- start start 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6ae1cab640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6adc074590 0x7f6adc1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6ae1cab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6adc074eb0 0x7f6adc1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6ae1cab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc112b90 0x7f6adc19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6ae1cab640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc118470 con 0x7f6adc112b90 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6ae1cab640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1182f0 con 0x7f6adc074590 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6ae1cab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1185f0 con 0x7f6adc074eb0 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6adbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc112b90 0x7f6adc19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6adbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc112b90 0x7f6adc19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53214/0 (socket says 172.21.15.94:53214) 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6adbfff640 1 -- 172.21.15.94:0/1934436364 learned_addr learned my addr 172.21.15.94:0/1934436364 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6adb7fe640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6adc074590 0x7f6adc1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6ad3fff640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6adc074eb0 0x7f6adc1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6adbfff640 1 -- 172.21.15.94:0/1934436364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6adc074eb0 msgr2=0x7f6adc1a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6adbfff640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6adc074eb0 0x7f6adc1a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6adbfff640 1 -- 172.21.15.94:0/1934436364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6adc074590 msgr2=0x7f6adc1a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6adbfff640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6adc074590 0x7f6adc1a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.898+0000 7f6adbfff640 1 -- 172.21.15.94:0/1934436364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc19ef90 con 0x7f6adc112b90 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.899+0000 7f6adbfff640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc112b90 0x7f6adc19e860 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400e9d0 tx=0x7f6ad400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.899+0000 7f6ad97fa640 1 -- 172.21.15.94:0/1934436364 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4004d20 con 0x7f6adc112b90 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.899+0000 7f6ad97fa640 1 -- 172.21.15.94:0/1934436364 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad4005020 con 0x7f6adc112b90 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.899+0000 7f6ad97fa640 1 -- 172.21.15.94:0/1934436364 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40116b0 con 0x7f6adc112b90 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.899+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc19f280 con 0x7f6adc112b90 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.899+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc1ab570 con 0x7f6adc112b90 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.900+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6adc074970 con 0x7f6adc112b90 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.901+0000 7f6ad97fa640 1 -- 172.21.15.94:0/1934436364 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6ad40051d0 con 0x7f6adc112b90 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.901+0000 7f6ad97fa640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6ac80a32d0 0x7f6ac80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.901+0000 7f6adb7fe640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6ac80a32d0 0x7f6ac80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.902+0000 7f6ad97fa640 1 -- 172.21.15.94:0/1934436364 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6ad40cfc70 con 0x7f6adc112b90 2024-09-19T04:13:37.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.902+0000 7f6adb7fe640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6ac80a32d0 0x7f6ac80a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6adc19f9a0 tx=0x7f6acc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:37.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:37.906+0000 7f6ad97fa640 1 -- 172.21.15.94:0/1934436364 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ad4016030 con 0x7f6adc112b90 2024-09-19T04:13:38.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.105+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6adc02d7c0 con 0x7f6adc112b90 2024-09-19T04:13:38.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.106+0000 7f6ad97fa640 1 -- 172.21.15.94:0/1934436364 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11915 (secure 0 0 0) 0x7f6ad409c7f0 con 0x7f6adc112b90 2024-09-19T04:13:38.107 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:38.108 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"btime":"2024-09-19T04:13:28:480286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:13:28.166880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:13:38.108 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-09-19T04:13:38.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6ac80a32d0 msgr2=0x7f6ac80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6ac80a32d0 0x7f6ac80a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6adc19f9a0 tx=0x7f6acc009110 comp rx=0 tx=0).stop 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc112b90 msgr2=0x7f6adc19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc112b90 0x7f6adc19e860 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400e9d0 tx=0x7f6ad400edd0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 shutdown_connections 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6ac80a32d0 0x7f6ac80a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6adc112b90 0x7f6adc19e860 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6adc074eb0 0x7f6adc1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 --2- 172.21.15.94:0/1934436364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6adc074590 0x7f6adc1a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 >> 172.21.15.94:0/1934436364 conn(0x7f6adc0fd8c0 msgr2=0x7f6adc0ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 shutdown_connections 2024-09-19T04:13:38.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:38.108+0000 7f6ae1cab640 1 -- 172.21.15.94:0/1934436364 wait complete. 2024-09-19T04:13:38.740 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:38 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1934436364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:38.740 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:38 smithi094 ceph-mon[30972]: pgmap v285: 97 pgs: 97 active+clean; 607 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.4 KiB/s wr, 8 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:38.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:13:38.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:38 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1934436364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:38.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:38 smithi112 ceph-mon[35051]: pgmap v285: 97 pgs: 97 active+clean; 607 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.4 KiB/s wr, 8 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:38.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:38 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1934436364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:13:38.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:38 smithi104 ceph-mon[34145]: pgmap v285: 97 pgs: 97 active+clean; 607 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.4 KiB/s wr, 8 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:39.017 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.539+0000 7fb911338640 1 -- 172.21.15.94:0/2151948200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c101420 msgr2=0x7fb90c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.539+0000 7fb911338640 1 --2- 172.21.15.94:0/2151948200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c101420 0x7fb90c10d950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb904009cd0 tx=0x7fb90402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.539+0000 7fb911338640 1 -- 172.21.15.94:0/2151948200 shutdown_connections 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.539+0000 7fb911338640 1 --2- 172.21.15.94:0/2151948200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c101420 0x7fb90c10d950 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.539+0000 7fb911338640 1 --2- 172.21.15.94:0/2151948200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb90c100a80 0x7fb90c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.539+0000 7fb911338640 1 --2- 172.21.15.94:0/2151948200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb90c106a80 0x7fb90c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.539+0000 7fb911338640 1 -- 172.21.15.94:0/2151948200 >> 172.21.15.94:0/2151948200 conn(0x7fb90c0fc7a0 msgr2=0x7fb90c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.539+0000 7fb911338640 1 -- 172.21.15.94:0/2151948200 shutdown_connections 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.539+0000 7fb911338640 1 -- 172.21.15.94:0/2151948200 wait complete. 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb911338640 1 Processor -- start 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb911338640 1 -- start start 2024-09-19T04:13:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb911338640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb90c100a80 0x7fb90c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb911338640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb90c101420 0x7fb90c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb911338640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c106a80 0x7fb90c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb911338640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c111e50 con 0x7fb90c106a80 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb911338640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c111cd0 con 0x7fb90c101420 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb911338640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c111fd0 con 0x7fb90c100a80 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb90b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c106a80 0x7fb90c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb90b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c106a80 0x7fb90c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53238/0 (socket says 172.21.15.94:53238) 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb90b7fe640 1 -- 172.21.15.94:0/1066893846 learned_addr learned my addr 172.21.15.94:0/1066893846 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.540+0000 7fb90a7fc640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb90c101420 0x7fb90c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb90affd640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb90c100a80 0x7fb90c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb90b7fe640 1 -- 172.21.15.94:0/1066893846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb90c100a80 msgr2=0x7fb90c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb90b7fe640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb90c100a80 0x7fb90c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb90b7fe640 1 -- 172.21.15.94:0/1066893846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb90c101420 msgr2=0x7fb90c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb90b7fe640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb90c101420 0x7fb90c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb90b7fe640 1 -- 172.21.15.94:0/1066893846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c19d340 con 0x7fb90c106a80 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb90affd640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb90c100a80 0x7fb90c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb90a7fc640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb90c101420 0x7fb90c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb90b7fe640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c106a80 0x7fb90c19cc60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb904005cc0 tx=0x7fb904003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb8f3fff640 1 -- 172.21.15.94:0/1066893846 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904004450 con 0x7fb90c106a80 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90c19d5d0 con 0x7fb90c106a80 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb8f3fff640 1 -- 172.21.15.94:0/1066893846 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb904031c90 con 0x7fb90c106a80 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb8f3fff640 1 -- 172.21.15.94:0/1066893846 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90403b400 con 0x7fb90c106a80 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.541+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90c1a4f10 con 0x7fb90c106a80 2024-09-19T04:13:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.542+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90c100ee0 con 0x7fb90c106a80 2024-09-19T04:13:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.543+0000 7fb8f3fff640 1 -- 172.21.15.94:0/1066893846 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb904004680 con 0x7fb90c106a80 2024-09-19T04:13:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.544+0000 7fb8f3fff640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8f40a32d0 0x7fb8f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.544+0000 7fb90affd640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8f40a32d0 0x7fb8f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.544+0000 7fb8f3fff640 1 -- 172.21.15.94:0/1066893846 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb90402e030 con 0x7fb90c106a80 2024-09-19T04:13:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.544+0000 7fb90affd640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8f40a32d0 0x7fb8f40a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006fd0 tx=0x7fb8fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.548+0000 7fb8f3fff640 1 -- 172.21.15.94:0/1066893846 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb9040bd8c0 con 0x7fb90c106a80 2024-09-19T04:13:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.715+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb90c109010 con 0x7fb90c106a80 2024-09-19T04:13:39.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.717+0000 7fb8f3fff640 1 -- 172.21.15.94:0/1066893846 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22842 (secure 0 0 0) 0x7fb9040098b0 con 0x7fb90c106a80 2024-09-19T04:13:39.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:39.719 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:13:31.495964+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:39.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8f40a32d0 msgr2=0x7fb8f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8f40a32d0 0x7fb8f40a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006fd0 tx=0x7fb8fc008040 comp rx=0 tx=0).stop 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c106a80 msgr2=0x7fb90c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c106a80 0x7fb90c19cc60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb904005cc0 tx=0x7fb904003c20 comp rx=0 tx=0).stop 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 shutdown_connections 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb8f40a32d0 0x7fb8f40a5790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c106a80 0x7fb90c19cc60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb90c101420 0x7fb90c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 --2- 172.21.15.94:0/1066893846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb90c100a80 0x7fb90c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 >> 172.21.15.94:0/1066893846 conn(0x7fb90c0fc7a0 msgr2=0x7fb90c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 shutdown_connections 2024-09-19T04:13:39.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:39.720+0000 7fb911338640 1 -- 172.21.15.94:0/1066893846 wait complete. 2024-09-19T04:13:40.024 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:39 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1066893846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:40.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:39 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1066893846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:40.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:39 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1066893846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:40.421 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T04:13:40.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:13:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:41.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:40 smithi112 ceph-mon[35051]: pgmap v286: 97 pgs: 97 active+clean; 630 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 KiB/s wr, 9 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:41.053 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:40 smithi094 ceph-mon[30972]: pgmap v286: 97 pgs: 97 active+clean; 630 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 KiB/s wr, 9 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:41.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:40 smithi104 ceph-mon[34145]: pgmap v286: 97 pgs: 97 active+clean; 630 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 KiB/s wr, 9 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:13:41.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.131+0000 7f84dcc78640 1 -- 172.21.15.94:0/1705520404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8101e10 msgr2=0x7f84d81021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.131+0000 7f84dcc78640 1 --2- 172.21.15.94:0/1705520404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8101e10 0x7f84d81021f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009900 tx=0x7f84c802a410 comp rx=0 tx=0).stop 2024-09-19T04:13:41.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.131+0000 7f84dcc78640 1 -- 172.21.15.94:0/1705520404 shutdown_connections 2024-09-19T04:13:41.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.131+0000 7f84dcc78640 1 --2- 172.21.15.94:0/1705520404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8106f40 0x7f84d810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.131+0000 7f84dcc78640 1 --2- 172.21.15.94:0/1705520404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d81027c0 0x7f84d8106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.131+0000 7f84dcc78640 1 --2- 172.21.15.94:0/1705520404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8101e10 0x7f84d81021f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.131+0000 7f84dcc78640 1 -- 172.21.15.94:0/1705520404 >> 172.21.15.94:0/1705520404 conn(0x7f84d80fd8d0 msgr2=0x7f84d80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.131+0000 7f84dcc78640 1 -- 172.21.15.94:0/1705520404 shutdown_connections 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.131+0000 7f84dcc78640 1 -- 172.21.15.94:0/1705520404 wait complete. 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84dcc78640 1 Processor -- start 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84dcc78640 1 -- start start 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84dcc78640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8101e10 0x7f84d8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84dcc78640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d81027c0 0x7f84d81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84dcc78640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8106f40 0x7f84d819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84dcc78640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8111e50 con 0x7f84d8106f40 2024-09-19T04:13:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84dcc78640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8111cd0 con 0x7f84d8101e10 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84dcc78640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8111fd0 con 0x7f84d81027c0 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84d6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8106f40 0x7f84d819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84d6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8106f40 0x7f84d819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53248/0 (socket says 172.21.15.94:53248) 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84d6d9d640 1 -- 172.21.15.94:0/2846125742 learned_addr learned my addr 172.21.15.94:0/2846125742 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84cffff640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d81027c0 0x7f84d81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84d659c640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8101e10 0x7f84d8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84d6d9d640 1 -- 172.21.15.94:0/2846125742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d81027c0 msgr2=0x7f84d81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84d6d9d640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d81027c0 0x7f84d81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84d6d9d640 1 -- 172.21.15.94:0/2846125742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8101e10 msgr2=0x7f84d8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84d6d9d640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8101e10 0x7f84d8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.132+0000 7f84d6d9d640 1 -- 172.21.15.94:0/2846125742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d819d320 con 0x7f84d8106f40 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.133+0000 7f84d6d9d640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8106f40 0x7f84d819cc40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f84d000cad0 tx=0x7f84d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.133+0000 7f84cf7fe640 1 -- 172.21.15.94:0/2846125742 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d000ae20 con 0x7f84d8106f40 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.133+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d819d610 con 0x7f84d8106f40 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.133+0000 7f84cf7fe640 1 -- 172.21.15.94:0/2846125742 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84d0005020 con 0x7f84d8106f40 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.133+0000 7f84cf7fe640 1 -- 172.21.15.94:0/2846125742 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d000f6b0 con 0x7f84d8106f40 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.133+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d81a4ef0 con 0x7f84d8106f40 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.134+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d81021f0 con 0x7f84d8106f40 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.135+0000 7f84cf7fe640 1 -- 172.21.15.94:0/2846125742 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f84d00051d0 con 0x7f84d8106f40 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.136+0000 7f84cf7fe640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f84c40a3350 0x7f84c40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.136+0000 7f84d659c640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f84c40a3350 0x7f84c40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.136+0000 7f84cf7fe640 1 -- 172.21.15.94:0/2846125742 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f84d00cfce0 con 0x7f84d8106f40 2024-09-19T04:13:41.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.136+0000 7f84d659c640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f84c40a3350 0x7f84c40a5810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84c8031440 tx=0x7f84c8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.139+0000 7f84cf7fe640 1 -- 172.21.15.94:0/2846125742 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84d0014030 con 0x7f84d8106f40 2024-09-19T04:13:41.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.304+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84d8062630 con 0x7f84c40a3350 2024-09-19T04:13:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.311+0000 7f84cf7fe640 1 -- 172.21.15.94:0/2846125742 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258715 (secure 0 0 0) 0x7f84d8062630 con 0x7f84c40a3350 2024-09-19T04:13:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:41.316 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-19T04:13:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.313+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f84c40a3350 msgr2=0x7f84c40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.313+0000 7f84dcc78640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f84c40a3350 0x7f84c40a5810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84c8031440 tx=0x7f84c8005b50 comp rx=0 tx=0).stop 2024-09-19T04:13:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.313+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8106f40 msgr2=0x7f84d819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.313+0000 7f84dcc78640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8106f40 0x7f84d819cc40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f84d000cad0 tx=0x7f84d000cf90 comp rx=0 tx=0).stop 2024-09-19T04:13:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.314+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 shutdown_connections 2024-09-19T04:13:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.314+0000 7f84dcc78640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f84c40a3350 0x7f84c40a5810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.314+0000 7f84dcc78640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84d8106f40 0x7f84d819cc40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.314+0000 7f84dcc78640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d81027c0 0x7f84d81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.314+0000 7f84dcc78640 1 --2- 172.21.15.94:0/2846125742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8101e10 0x7f84d8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.314+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 >> 172.21.15.94:0/2846125742 conn(0x7f84d80fd8d0 msgr2=0x7f84d80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.314+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 shutdown_connections 2024-09-19T04:13:41.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:41.314+0000 7f84dcc78640 1 -- 172.21.15.94:0/2846125742 wait complete. 2024-09-19T04:13:42.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:41 smithi112 ceph-mon[35051]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:13:42.068 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:41 smithi094 ceph-mon[30972]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:13:42.094 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-19T04:13:40.135881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645110,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":232,"num_read_kb":226,"num_write":336,"num_write_kb":3037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399196,"kb_used_data":12464,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542628,"statfs":{"total":1151940427776,"available":1151531651072,"internally_reserved":0,"allocated":12763136,"data_stored":7058135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49127,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":80,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.642453"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501952+0000","last_change":"2024-09-19T04:13:16.451031+0000","last_active":"2024-09-19T04:13:31.501952+0000","last_peered":"2024-09-19T04:13:31.501952+0000","last_clean":"2024-09-19T04:13:31.501952+0000","last_became_active":"2024-09-19T04:13:16.450713+0000","last_became_peered":"2024-09-19T04:13:16.450713+0000","last_unstale":"2024-09-19T04:13:31.501952+0000","last_undegraded":"2024-09-19T04:13:31.501952+0000","last_fullsized":"2024-09-19T04:13:31.501952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:59.485415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735600+0000","last_change":"2024-09-19T04:13:16.455689+0000","last_active":"2024-09-19T04:13:31.735600+0000","last_peered":"2024-09-19T04:13:31.735600+0000","last_clean":"2024-09-19T04:13:31.735600+0000","last_became_active":"2024-09-19T04:13:16.455154+0000","last_became_peered":"2024-09-19T04:13:16.455154+0000","last_unstale":"2024-09-19T04:13:31.735600+0000","last_undegraded":"2024-09-19T04:13:31.735600+0000","last_fullsized":"2024-09-19T04:13:31.735600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:57.610619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735526+0000","last_change":"2024-09-19T04:13:16.447392+0000","last_active":"2024-09-19T04:13:31.735526+0000","last_peered":"2024-09-19T04:13:31.735526+0000","last_clean":"2024-09-19T04:13:31.735526+0000","last_became_active":"2024-09-19T04:13:16.447088+0000","last_became_peered":"2024-09-19T04:13:16.447088+0000","last_unstale":"2024-09-19T04:13:31.735526+0000","last_undegraded":"2024-09-19T04:13:31.735526+0000","last_fullsized":"2024-09-19T04:13:31.735526+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:16.591420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504467+0000","last_change":"2024-09-19T04:13:16.452642+0000","last_active":"2024-09-19T04:13:31.504467+0000","last_peered":"2024-09-19T04:13:31.504467+0000","last_clean":"2024-09-19T04:13:31.504467+0000","last_became_active":"2024-09-19T04:13:16.452329+0000","last_became_peered":"2024-09-19T04:13:16.452329+0000","last_unstale":"2024-09-19T04:13:31.504467+0000","last_undegraded":"2024-09-19T04:13:31.504467+0000","last_fullsized":"2024-09-19T04:13:31.504467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:54.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505930+0000","last_change":"2024-09-19T04:13:16.447171+0000","last_active":"2024-09-19T04:13:31.505930+0000","last_peered":"2024-09-19T04:13:31.505930+0000","last_clean":"2024-09-19T04:13:31.505930+0000","last_became_active":"2024-09-19T04:13:16.446835+0000","last_became_peered":"2024-09-19T04:13:16.446835+0000","last_unstale":"2024-09-19T04:13:31.505930+0000","last_undegraded":"2024-09-19T04:13:31.505930+0000","last_fullsized":"2024-09-19T04:13:31.505930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:41.892118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.507665+0000","last_change":"2024-09-19T04:13:31.507665+0000","last_active":"2024-09-19T04:13:31.507665+0000","last_peered":"2024-09-19T04:13:31.507665+0000","last_clean":"2024-09-19T04:13:31.507665+0000","last_became_active":"2024-09-19T04:13:31.507329+0000","last_became_peered":"2024-09-19T04:13:31.507329+0000","last_unstale":"2024-09-19T04:13:31.507665+0000","last_undegraded":"2024-09-19T04:13:31.507665+0000","last_fullsized":"2024-09-19T04:13:31.507665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:35.832776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500742+0000","last_change":"2024-09-19T04:13:16.454568+0000","last_active":"2024-09-19T04:13:31.500742+0000","last_peered":"2024-09-19T04:13:31.500742+0000","last_clean":"2024-09-19T04:13:31.500742+0000","last_became_active":"2024-09-19T04:13:16.454230+0000","last_became_peered":"2024-09-19T04:13:16.454230+0000","last_unstale":"2024-09-19T04:13:31.500742+0000","last_undegraded":"2024-09-19T04:13:31.500742+0000","last_fullsized":"2024-09-19T04:13:31.500742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:23.401316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501235+0000","last_change":"2024-09-19T04:13:16.449223+0000","last_active":"2024-09-19T04:13:31.501235+0000","last_peered":"2024-09-19T04:13:31.501235+0000","last_clean":"2024-09-19T04:13:31.501235+0000","last_became_active":"2024-09-19T04:13:16.448665+0000","last_became_peered":"2024-09-19T04:13:16.448665+0000","last_unstale":"2024-09-19T04:13:31.501235+0000","last_undegraded":"2024-09-19T04:13:31.501235+0000","last_fullsized":"2024-09-19T04:13:31.501235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:29.619847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.701069+0000","last_change":"2024-09-19T04:13:16.458203+0000","last_active":"2024-09-19T04:13:31.701069+0000","last_peered":"2024-09-19T04:13:31.701069+0000","last_clean":"2024-09-19T04:13:31.701069+0000","last_became_active":"2024-09-19T04:13:16.457544+0000","last_became_peered":"2024-09-19T04:13:16.457544+0000","last_unstale":"2024-09-19T04:13:31.701069+0000","last_undegraded":"2024-09-19T04:13:31.701069+0000","last_fullsized":"2024-09-19T04:13:31.701069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:47.611695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500145+0000","last_change":"2024-09-19T04:13:16.454213+0000","last_active":"2024-09-19T04:13:31.500145+0000","last_peered":"2024-09-19T04:13:31.500145+0000","last_clean":"2024-09-19T04:13:31.500145+0000","last_became_active":"2024-09-19T04:13:16.453047+0000","last_became_peered":"2024-09-19T04:13:16.453047+0000","last_unstale":"2024-09-19T04:13:31.500145+0000","last_undegraded":"2024-09-19T04:13:31.500145+0000","last_fullsized":"2024-09-19T04:13:31.500145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:27.425259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501213+0000","last_change":"2024-09-19T04:13:16.458195+0000","last_active":"2024-09-19T04:13:31.501213+0000","last_peered":"2024-09-19T04:13:31.501213+0000","last_clean":"2024-09-19T04:13:31.501213+0000","last_became_active":"2024-09-19T04:13:16.457891+0000","last_became_peered":"2024-09-19T04:13:16.457891+0000","last_unstale":"2024-09-19T04:13:31.501213+0000","last_undegraded":"2024-09-19T04:13:31.501213+0000","last_fullsized":"2024-09-19T04:13:31.501213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:53.833249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.508474+0000","last_change":"2024-09-19T04:13:31.508117+0000","last_active":"2024-09-19T04:13:31.508474+0000","last_peered":"2024-09-19T04:13:31.508474+0000","last_clean":"2024-09-19T04:13:31.508474+0000","last_became_active":"2024-09-19T04:13:31.507693+0000","last_became_peered":"2024-09-19T04:13:31.507693+0000","last_unstale":"2024-09-19T04:13:31.508474+0000","last_undegraded":"2024-09-19T04:13:31.508474+0000","last_fullsized":"2024-09-19T04:13:31.508474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:58.447965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499011+0000","last_change":"2024-09-19T04:13:16.440079+0000","last_active":"2024-09-19T04:13:31.499011+0000","last_peered":"2024-09-19T04:13:31.499011+0000","last_clean":"2024-09-19T04:13:31.499011+0000","last_became_active":"2024-09-19T04:13:16.439474+0000","last_became_peered":"2024-09-19T04:13:16.439474+0000","last_unstale":"2024-09-19T04:13:31.499011+0000","last_undegraded":"2024-09-19T04:13:31.499011+0000","last_fullsized":"2024-09-19T04:13:31.499011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:11.151412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735816+0000","last_change":"2024-09-19T04:13:16.449958+0000","last_active":"2024-09-19T04:13:31.735816+0000","last_peered":"2024-09-19T04:13:31.735816+0000","last_clean":"2024-09-19T04:13:31.735816+0000","last_became_active":"2024-09-19T04:13:16.449665+0000","last_became_peered":"2024-09-19T04:13:16.449665+0000","last_unstale":"2024-09-19T04:13:31.735816+0000","last_undegraded":"2024-09-19T04:13:31.735816+0000","last_fullsized":"2024-09-19T04:13:31.735816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:24.023779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735453+0000","last_change":"2024-09-19T04:13:16.454333+0000","last_active":"2024-09-19T04:13:31.735453+0000","last_peered":"2024-09-19T04:13:31.735453+0000","last_clean":"2024-09-19T04:13:31.735453+0000","last_became_active":"2024-09-19T04:13:16.454066+0000","last_became_peered":"2024-09-19T04:13:16.454066+0000","last_unstale":"2024-09-19T04:13:31.735453+0000","last_undegraded":"2024-09-19T04:13:31.735453+0000","last_fullsized":"2024-09-19T04:13:31.735453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:16.815173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700446+0000","last_change":"2024-09-19T04:13:16.451864+0000","last_active":"2024-09-19T04:13:31.700446+0000","last_peered":"2024-09-19T04:13:31.700446+0000","last_clean":"2024-09-19T04:13:31.700446+0000","last_became_active":"2024-09-19T04:13:16.450995+0000","last_became_peered":"2024-09-19T04:13:16.450995+0000","last_unstale":"2024-09-19T04:13:31.700446+0000","last_undegraded":"2024-09-19T04:13:31.700446+0000","last_fullsized":"2024-09-19T04:13:31.700446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:40.750221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502172+0000","last_change":"2024-09-19T04:13:16.453176+0000","last_active":"2024-09-19T04:13:31.502172+0000","last_peered":"2024-09-19T04:13:31.502172+0000","last_clean":"2024-09-19T04:13:31.502172+0000","last_became_active":"2024-09-19T04:13:16.452788+0000","last_became_peered":"2024-09-19T04:13:16.452788+0000","last_unstale":"2024-09-19T04:13:31.502172+0000","last_undegraded":"2024-09-19T04:13:31.502172+0000","last_fullsized":"2024-09-19T04:13:31.502172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:47.099180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501446+0000","last_change":"2024-09-19T04:13:16.449167+0000","last_active":"2024-09-19T04:13:31.501446+0000","last_peered":"2024-09-19T04:13:31.501446+0000","last_clean":"2024-09-19T04:13:31.501446+0000","last_became_active":"2024-09-19T04:13:16.448579+0000","last_became_peered":"2024-09-19T04:13:16.448579+0000","last_unstale":"2024-09-19T04:13:31.501446+0000","last_undegraded":"2024-09-19T04:13:31.501446+0000","last_fullsized":"2024-09-19T04:13:31.501446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:22:40.023000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505138+0000","last_change":"2024-09-19T04:13:16.445449+0000","last_active":"2024-09-19T04:13:31.505138+0000","last_peered":"2024-09-19T04:13:31.505138+0000","last_clean":"2024-09-19T04:13:31.505138+0000","last_became_active":"2024-09-19T04:13:16.445028+0000","last_became_peered":"2024-09-19T04:13:16.445028+0000","last_unstale":"2024-09-19T04:13:31.505138+0000","last_undegraded":"2024-09-19T04:13:31.505138+0000","last_fullsized":"2024-09-19T04:13:31.505138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:13.095348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501714+0000","last_change":"2024-09-19T04:13:16.454033+0000","last_active":"2024-09-19T04:13:31.501714+0000","last_peered":"2024-09-19T04:13:31.501714+0000","last_clean":"2024-09-19T04:13:31.501714+0000","last_became_active":"2024-09-19T04:13:16.453592+0000","last_became_peered":"2024-09-19T04:13:16.453592+0000","last_unstale":"2024-09-19T04:13:31.501714+0000","last_undegraded":"2024-09-19T04:13:31.501714+0000","last_fullsized":"2024-09-19T04:13:31.501714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.304478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499027+0000","last_change":"2024-09-19T04:13:16.448763+0000","last_active":"2024-09-19T04:13:31.499027+0000","last_peered":"2024-09-19T04:13:31.499027+0000","last_clean":"2024-09-19T04:13:31.499027+0000","last_became_active":"2024-09-19T04:13:16.448459+0000","last_became_peered":"2024-09-19T04:13:16.448459+0000","last_unstale":"2024-09-19T04:13:31.499027+0000","last_undegraded":"2024-09-19T04:13:31.499027+0000","last_fullsized":"2024-09-19T04:13:31.499027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700507+0000","last_change":"2024-09-19T04:13:16.451489+0000","last_active":"2024-09-19T04:13:31.700507+0000","last_peered":"2024-09-19T04:13:31.700507+0000","last_clean":"2024-09-19T04:13:31.700507+0000","last_became_active":"2024-09-19T04:13:16.451176+0000","last_became_peered":"2024-09-19T04:13:16.451176+0000","last_unstale":"2024-09-19T04:13:31.700507+0000","last_undegraded":"2024-09-19T04:13:31.700507+0000","last_fullsized":"2024-09-19T04:13:31.700507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.921102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501314+0000","last_change":"2024-09-19T04:13:16.454551+0000","last_active":"2024-09-19T04:13:31.501314+0000","last_peered":"2024-09-19T04:13:31.501314+0000","last_clean":"2024-09-19T04:13:31.501314+0000","last_became_active":"2024-09-19T04:13:16.453976+0000","last_became_peered":"2024-09-19T04:13:16.453976+0000","last_unstale":"2024-09-19T04:13:31.501314+0000","last_undegraded":"2024-09-19T04:13:31.501314+0000","last_fullsized":"2024-09-19T04:13:31.501314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:06.504615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504331+0000","last_change":"2024-09-19T04:13:31.503222+0000","last_active":"2024-09-19T04:13:31.504331+0000","last_peered":"2024-09-19T04:13:31.504331+0000","last_clean":"2024-09-19T04:13:31.504331+0000","last_became_active":"2024-09-19T04:13:31.502873+0000","last_became_peered":"2024-09-19T04:13:31.502873+0000","last_unstale":"2024-09-19T04:13:31.504331+0000","last_undegraded":"2024-09-19T04:13:31.504331+0000","last_fullsized":"2024-09-19T04:13:31.504331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:54.559592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505834+0000","last_change":"2024-09-19T04:13:31.505834+0000","last_active":"2024-09-19T04:13:31.505834+0000","last_peered":"2024-09-19T04:13:31.505834+0000","last_clean":"2024-09-19T04:13:31.505834+0000","last_became_active":"2024-09-19T04:13:31.505501+0000","last_became_peered":"2024-09-19T04:13:31.505501+0000","last_unstale":"2024-09-19T04:13:31.505834+0000","last_undegraded":"2024-09-19T04:13:31.505834+0000","last_fullsized":"2024-09-19T04:13:31.505834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:45.136128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504841+0000","last_change":"2024-09-19T04:13:16.449237+0000","last_active":"2024-09-19T04:13:31.504841+0000","last_peered":"2024-09-19T04:13:31.504841+0000","last_clean":"2024-09-19T04:13:31.504841+0000","last_became_active":"2024-09-19T04:13:16.448896+0000","last_became_peered":"2024-09-19T04:13:16.448896+0000","last_unstale":"2024-09-19T04:13:31.504841+0000","last_undegraded":"2024-09-19T04:13:31.504841+0000","last_fullsized":"2024-09-19T04:13:31.504841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:38.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501835+0000","last_change":"2024-09-19T04:13:16.456053+0000","last_active":"2024-09-19T04:13:31.501835+0000","last_peered":"2024-09-19T04:13:31.501835+0000","last_clean":"2024-09-19T04:13:31.501835+0000","last_became_active":"2024-09-19T04:13:16.454775+0000","last_became_peered":"2024-09-19T04:13:16.454775+0000","last_unstale":"2024-09-19T04:13:31.501835+0000","last_undegraded":"2024-09-19T04:13:31.501835+0000","last_fullsized":"2024-09-19T04:13:31.501835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.902036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499121+0000","last_change":"2024-09-19T04:13:16.451473+0000","last_active":"2024-09-19T04:13:31.499121+0000","last_peered":"2024-09-19T04:13:31.499121+0000","last_clean":"2024-09-19T04:13:31.499121+0000","last_became_active":"2024-09-19T04:13:16.451150+0000","last_became_peered":"2024-09-19T04:13:16.451150+0000","last_unstale":"2024-09-19T04:13:31.499121+0000","last_undegraded":"2024-09-19T04:13:31.499121+0000","last_fullsized":"2024-09-19T04:13:31.499121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:59.332745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735656+0000","last_change":"2024-09-19T04:13:16.452017+0000","last_active":"2024-09-19T04:13:31.735656+0000","last_peered":"2024-09-19T04:13:31.735656+0000","last_clean":"2024-09-19T04:13:31.735656+0000","last_became_active":"2024-09-19T04:13:16.450715+0000","last_became_peered":"2024-09-19T04:13:16.450715+0000","last_unstale":"2024-09-19T04:13:31.735656+0000","last_undegraded":"2024-09-19T04:13:31.735656+0000","last_fullsized":"2024-09-19T04:13:31.735656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:43.548770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.736597+0000","last_change":"2024-09-19T04:13:31.736597+0000","last_active":"2024-09-19T04:13:31.736597+0000","last_peered":"2024-09-19T04:13:31.736597+0000","last_clean":"2024-09-19T04:13:31.736597+0000","last_became_active":"2024-09-19T04:13:31.736301+0000","last_became_peered":"2024-09-19T04:13:31.736301+0000","last_unstale":"2024-09-19T04:13:31.736597+0000","last_undegraded":"2024-09-19T04:13:31.736597+0000","last_fullsized":"2024-09-19T04:13:31.736597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:06.614117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.509742+0000","last_change":"2024-09-19T04:13:31.509742+0000","last_active":"2024-09-19T04:13:31.509742+0000","last_peered":"2024-09-19T04:13:31.509742+0000","last_clean":"2024-09-19T04:13:31.509742+0000","last_became_active":"2024-09-19T04:13:31.509363+0000","last_became_peered":"2024-09-19T04:13:31.509363+0000","last_unstale":"2024-09-19T04:13:31.509742+0000","last_undegraded":"2024-09-19T04:13:31.509742+0000","last_fullsized":"2024-09-19T04:13:31.509742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.094930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501525+0000","last_change":"2024-09-19T04:13:16.449459+0000","last_active":"2024-09-19T04:13:31.501525+0000","last_peered":"2024-09-19T04:13:31.501525+0000","last_clean":"2024-09-19T04:13:31.501525+0000","last_became_active":"2024-09-19T04:13:16.449119+0000","last_became_peered":"2024-09-19T04:13:16.449119+0000","last_unstale":"2024-09-19T04:13:31.501525+0000","last_undegraded":"2024-09-19T04:13:31.501525+0000","last_fullsized":"2024-09-19T04:13:31.501525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:26:26.941609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504930+0000","last_change":"2024-09-19T04:13:16.448939+0000","last_active":"2024-09-19T04:13:31.504930+0000","last_peered":"2024-09-19T04:13:31.504930+0000","last_clean":"2024-09-19T04:13:31.504930+0000","last_became_active":"2024-09-19T04:13:16.448619+0000","last_became_peered":"2024-09-19T04:13:16.448619+0000","last_unstale":"2024-09-19T04:13:31.504930+0000","last_undegraded":"2024-09-19T04:13:31.504930+0000","last_fullsized":"2024-09-19T04:13:31.504930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:41.017404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:37.517120+0000","last_change":"2024-09-19T04:13:14.430467+0000","last_active":"2024-09-19T04:13:37.517120+0000","last_peered":"2024-09-19T04:13:37.517120+0000","last_clean":"2024-09-19T04:13:37.517120+0000","last_became_active":"2024-09-19T04:13:14.430095+0000","last_became_peered":"2024-09-19T04:13:14.430095+0000","last_unstale":"2024-09-19T04:13:37.517120+0000","last_undegraded":"2024-09-19T04:13:37.517120+0000","last_fullsized":"2024-09-19T04:13:37.517120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:50.471745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506178+0000","last_change":"2024-09-19T04:13:31.505798+0000","last_active":"2024-09-19T04:13:31.506178+0000","last_peered":"2024-09-19T04:13:31.506178+0000","last_clean":"2024-09-19T04:13:31.506178+0000","last_became_active":"2024-09-19T04:13:31.505469+0000","last_became_peered":"2024-09-19T04:13:31.505469+0000","last_unstale":"2024-09-19T04:13:31.506178+0000","last_undegraded":"2024-09-19T04:13:31.506178+0000","last_fullsized":"2024-09-19T04:13:31.506178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:13:42.405075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499636+0000","last_change":"2024-09-19T04:13:14.426443+0000","last_active":"2024-09-19T04:13:31.499636+0000","last_peered":"2024-09-19T04:13:31.499636+0000","last_clean":"2024-09-19T04:13:31.499636+0000","last_became_active":"2024-09-19T04:13:14.426103+0000","last_became_peered":"2024-09-19T04:13:14.426103+0000","last_unstale":"2024-09-19T04:13:31.499636+0000","last_undegraded":"2024-09-19T04:13:31.499636+0000","last_fullsized":"2024-09-19T04:13:31.499636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:28.461962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498768+0000","last_change":"2024-09-19T04:13:14.424439+0000","last_active":"2024-09-19T04:13:31.498768+0000","last_peered":"2024-09-19T04:13:31.498768+0000","last_clean":"2024-09-19T04:13:31.498768+0000","last_became_active":"2024-09-19T04:13:14.423975+0000","last_became_peered":"2024-09-19T04:13:14.423975+0000","last_unstale":"2024-09-19T04:13:31.498768+0000","last_undegraded":"2024-09-19T04:13:31.498768+0000","last_fullsized":"2024-09-19T04:13:31.498768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:12.032125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501447+0000","last_change":"2024-09-19T04:13:16.444865+0000","last_active":"2024-09-19T04:13:31.501447+0000","last_peered":"2024-09-19T04:13:31.501447+0000","last_clean":"2024-09-19T04:13:31.501447+0000","last_became_active":"2024-09-19T04:13:16.444556+0000","last_became_peered":"2024-09-19T04:13:16.444556+0000","last_unstale":"2024-09-19T04:13:31.501447+0000","last_undegraded":"2024-09-19T04:13:31.501447+0000","last_fullsized":"2024-09-19T04:13:31.501447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:38.025604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505570+0000","last_change":"2024-09-19T04:13:14.429018+0000","last_active":"2024-09-19T04:13:31.505570+0000","last_peered":"2024-09-19T04:13:31.505570+0000","last_clean":"2024-09-19T04:13:31.505570+0000","last_became_active":"2024-09-19T04:13:14.428383+0000","last_became_peered":"2024-09-19T04:13:14.428383+0000","last_unstale":"2024-09-19T04:13:31.505570+0000","last_undegraded":"2024-09-19T04:13:31.505570+0000","last_fullsized":"2024-09-19T04:13:31.505570+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:46.190421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501335+0000","last_change":"2024-09-19T04:13:16.449013+0000","last_active":"2024-09-19T04:13:31.501335+0000","last_peered":"2024-09-19T04:13:31.501335+0000","last_clean":"2024-09-19T04:13:31.501335+0000","last_became_active":"2024-09-19T04:13:16.448703+0000","last_became_peered":"2024-09-19T04:13:16.448703+0000","last_unstale":"2024-09-19T04:13:31.501335+0000","last_undegraded":"2024-09-19T04:13:31.501335+0000","last_fullsized":"2024-09-19T04:13:31.501335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:41.231759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:36.515081+0000","last_change":"2024-09-19T04:13:14.430050+0000","last_active":"2024-09-19T04:13:36.515081+0000","last_peered":"2024-09-19T04:13:36.515081+0000","last_clean":"2024-09-19T04:13:36.515081+0000","last_became_active":"2024-09-19T04:13:14.429710+0000","last_became_peered":"2024-09-19T04:13:14.429710+0000","last_unstale":"2024-09-19T04:13:36.515081+0000","last_undegraded":"2024-09-19T04:13:36.515081+0000","last_fullsized":"2024-09-19T04:13:36.515081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:55.796080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700932+0000","last_change":"2024-09-19T04:13:16.454431+0000","last_active":"2024-09-19T04:13:31.700932+0000","last_peered":"2024-09-19T04:13:31.700932+0000","last_clean":"2024-09-19T04:13:31.700932+0000","last_became_active":"2024-09-19T04:13:16.453033+0000","last_became_peered":"2024-09-19T04:13:16.453033+0000","last_unstale":"2024-09-19T04:13:31.700932+0000","last_undegraded":"2024-09-19T04:13:31.700932+0000","last_fullsized":"2024-09-19T04:13:31.700932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:52.817740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501175+0000","last_change":"2024-09-19T04:13:14.428949+0000","last_active":"2024-09-19T04:13:31.501175+0000","last_peered":"2024-09-19T04:13:31.501175+0000","last_clean":"2024-09-19T04:13:31.501175+0000","last_became_active":"2024-09-19T04:13:14.428525+0000","last_became_peered":"2024-09-19T04:13:14.428525+0000","last_unstale":"2024-09-19T04:13:31.501175+0000","last_undegraded":"2024-09-19T04:13:31.501175+0000","last_fullsized":"2024-09-19T04:13:31.501175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:30.296730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735848+0000","last_change":"2024-09-19T04:13:16.454077+0000","last_active":"2024-09-19T04:13:31.735848+0000","last_peered":"2024-09-19T04:13:31.735848+0000","last_clean":"2024-09-19T04:13:31.735848+0000","last_became_active":"2024-09-19T04:13:16.453791+0000","last_became_peered":"2024-09-19T04:13:16.453791+0000","last_unstale":"2024-09-19T04:13:31.735848+0000","last_undegraded":"2024-09-19T04:13:31.735848+0000","last_fullsized":"2024-09-19T04:13:31.735848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:29.426475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505069+0000","last_change":"2024-09-19T04:13:14.620179+0000","last_active":"2024-09-19T04:13:31.505069+0000","last_peered":"2024-09-19T04:13:31.505069+0000","last_clean":"2024-09-19T04:13:31.505069+0000","last_became_active":"2024-09-19T04:13:14.619758+0000","last_became_peered":"2024-09-19T04:13:14.619758+0000","last_unstale":"2024-09-19T04:13:31.505069+0000","last_undegraded":"2024-09-19T04:13:31.505069+0000","last_fullsized":"2024-09-19T04:13:31.505069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:11.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700712+0000","last_change":"2024-09-19T04:13:16.453036+0000","last_active":"2024-09-19T04:13:31.700712+0000","last_peered":"2024-09-19T04:13:31.700712+0000","last_clean":"2024-09-19T04:13:31.700712+0000","last_became_active":"2024-09-19T04:13:16.452736+0000","last_became_peered":"2024-09-19T04:13:16.452736+0000","last_unstale":"2024-09-19T04:13:31.700712+0000","last_undegraded":"2024-09-19T04:13:31.700712+0000","last_fullsized":"2024-09-19T04:13:31.700712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:30.003652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'25","reported_seq":77,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:37.520459+0000","last_change":"2024-09-19T04:13:14.428715+0000","last_active":"2024-09-19T04:13:37.520459+0000","last_peered":"2024-09-19T04:13:37.520459+0000","last_clean":"2024-09-19T04:13:37.520459+0000","last_became_active":"2024-09-19T04:13:14.428409+0000","last_became_peered":"2024-09-19T04:13:14.428409+0000","last_unstale":"2024-09-19T04:13:37.520459+0000","last_undegraded":"2024-09-19T04:13:37.520459+0000","last_fullsized":"2024-09-19T04:13:37.520459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:23.130857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":63,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504291+0000","last_change":"2024-09-19T04:13:16.454512+0000","last_active":"2024-09-19T04:13:31.504291+0000","last_peered":"2024-09-19T04:13:31.504291+0000","last_clean":"2024-09-19T04:13:31.504291+0000","last_became_active":"2024-09-19T04:13:16.454193+0000","last_became_peered":"2024-09-19T04:13:16.454193+0000","last_unstale":"2024-09-19T04:13:31.504291+0000","last_undegraded":"2024-09-19T04:13:31.504291+0000","last_fullsized":"2024-09-19T04:13:31.504291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:11.825938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.515863+0000","last_change":"2024-09-19T04:13:31.515448+0000","last_active":"2024-09-19T04:13:31.515863+0000","last_peered":"2024-09-19T04:13:31.515863+0000","last_clean":"2024-09-19T04:13:31.515863+0000","last_became_active":"2024-09-19T04:13:31.506099+0000","last_became_peered":"2024-09-19T04:13:31.506099+0000","last_unstale":"2024-09-19T04:13:31.515863+0000","last_undegraded":"2024-09-19T04:13:31.515863+0000","last_fullsized":"2024-09-19T04:13:31.515863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:21.462488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498861+0000","last_change":"2024-09-19T04:13:16.447607+0000","last_active":"2024-09-19T04:13:31.498861+0000","last_peered":"2024-09-19T04:13:31.498861+0000","last_clean":"2024-09-19T04:13:31.498861+0000","last_became_active":"2024-09-19T04:13:16.447029+0000","last_became_peered":"2024-09-19T04:13:16.447029+0000","last_unstale":"2024-09-19T04:13:31.498861+0000","last_undegraded":"2024-09-19T04:13:31.498861+0000","last_fullsized":"2024-09-19T04:13:31.498861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:52.418104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505867+0000","last_change":"2024-09-19T04:13:14.428856+0000","last_active":"2024-09-19T04:13:31.505867+0000","last_peered":"2024-09-19T04:13:31.505867+0000","last_clean":"2024-09-19T04:13:31.505867+0000","last_became_active":"2024-09-19T04:13:14.428510+0000","last_became_peered":"2024-09-19T04:13:14.428510+0000","last_unstale":"2024-09-19T04:13:31.505867+0000","last_undegraded":"2024-09-19T04:13:31.505867+0000","last_fullsized":"2024-09-19T04:13:31.505867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:49.633325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501063+0000","last_change":"2024-09-19T04:13:16.455628+0000","last_active":"2024-09-19T04:13:31.501063+0000","last_peered":"2024-09-19T04:13:31.501063+0000","last_clean":"2024-09-19T04:13:31.501063+0000","last_became_active":"2024-09-19T04:13:16.453817+0000","last_became_peered":"2024-09-19T04:13:16.453817+0000","last_unstale":"2024-09-19T04:13:31.501063+0000","last_undegraded":"2024-09-19T04:13:31.501063+0000","last_fullsized":"2024-09-19T04:13:31.501063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.832806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:33.507675+0000","last_change":"2024-09-19T04:13:14.424924+0000","last_active":"2024-09-19T04:13:33.507675+0000","last_peered":"2024-09-19T04:13:33.507675+0000","last_clean":"2024-09-19T04:13:33.507675+0000","last_became_active":"2024-09-19T04:13:14.424627+0000","last_became_peered":"2024-09-19T04:13:14.424627+0000","last_unstale":"2024-09-19T04:13:33.507675+0000","last_undegraded":"2024-09-19T04:13:33.507675+0000","last_fullsized":"2024-09-19T04:13:33.507675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:49.830205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500550+0000","last_change":"2024-09-19T04:13:16.455365+0000","last_active":"2024-09-19T04:13:31.500550+0000","last_peered":"2024-09-19T04:13:31.500550+0000","last_clean":"2024-09-19T04:13:31.500550+0000","last_became_active":"2024-09-19T04:13:16.455029+0000","last_became_peered":"2024-09-19T04:13:16.455029+0000","last_unstale":"2024-09-19T04:13:31.500550+0000","last_undegraded":"2024-09-19T04:13:31.500550+0000","last_fullsized":"2024-09-19T04:13:31.500550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:20.947391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504041+0000","last_change":"2024-09-19T04:13:14.620462+0000","last_active":"2024-09-19T04:13:31.504041+0000","last_peered":"2024-09-19T04:13:31.504041+0000","last_clean":"2024-09-19T04:13:31.504041+0000","last_became_active":"2024-09-19T04:13:14.620103+0000","last_became_peered":"2024-09-19T04:13:14.620103+0000","last_unstale":"2024-09-19T04:13:31.504041+0000","last_undegraded":"2024-09-19T04:13:31.504041+0000","last_fullsized":"2024-09-19T04:13:31.504041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:14.496075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506534+0000","last_change":"2024-09-19T04:13:16.450222+0000","last_active":"2024-09-19T04:13:31.506534+0000","last_peered":"2024-09-19T04:13:31.506534+0000","last_clean":"2024-09-19T04:13:31.506534+0000","last_became_active":"2024-09-19T04:13:16.449916+0000","last_became_peered":"2024-09-19T04:13:16.449916+0000","last_unstale":"2024-09-19T04:13:31.506534+0000","last_undegraded":"2024-09-19T04:13:31.506534+0000","last_fullsized":"2024-09-19T04:13:31.506534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:31.886342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.519612+0000","last_change":"2024-09-19T04:13:31.519190+0000","last_active":"2024-09-19T04:13:31.519612+0000","last_peered":"2024-09-19T04:13:31.519612+0000","last_clean":"2024-09-19T04:13:31.519612+0000","last_became_active":"2024-09-19T04:13:31.508605+0000","last_became_peered":"2024-09-19T04:13:31.508605+0000","last_unstale":"2024-09-19T04:13:31.519612+0000","last_undegraded":"2024-09-19T04:13:31.519612+0000","last_fullsized":"2024-09-19T04:13:31.519612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:51.437135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502356+0000","last_change":"2024-09-19T04:13:16.450848+0000","last_active":"2024-09-19T04:13:31.502356+0000","last_peered":"2024-09-19T04:13:31.502356+0000","last_clean":"2024-09-19T04:13:31.502356+0000","last_became_active":"2024-09-19T04:13:16.450496+0000","last_became_peered":"2024-09-19T04:13:16.450496+0000","last_unstale":"2024-09-19T04:13:31.502356+0000","last_undegraded":"2024-09-19T04:13:31.502356+0000","last_fullsized":"2024-09-19T04:13:31.502356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:49:00.346998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499054+0000","last_change":"2024-09-19T04:13:14.428654+0000","last_active":"2024-09-19T04:13:31.499054+0000","last_peered":"2024-09-19T04:13:31.499054+0000","last_clean":"2024-09-19T04:13:31.499054+0000","last_became_active":"2024-09-19T04:13:14.428297+0000","last_became_peered":"2024-09-19T04:13:14.428297+0000","last_unstale":"2024-09-19T04:13:31.499054+0000","last_undegraded":"2024-09-19T04:13:31.499054+0000","last_fullsized":"2024-09-19T04:13:31.499054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:30.834262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501214+0000","last_change":"2024-09-19T04:13:16.456002+0000","last_active":"2024-09-19T04:13:31.501214+0000","last_peered":"2024-09-19T04:13:31.501214+0000","last_clean":"2024-09-19T04:13:31.501214+0000","last_became_active":"2024-09-19T04:13:16.455463+0000","last_became_peered":"2024-09-19T04:13:16.455463+0000","last_unstale":"2024-09-19T04:13:31.501214+0000","last_undegraded":"2024-09-19T04:13:31.501214+0000","last_fullsized":"2024-09-19T04:13:31.501214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:47.266364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505707+0000","last_change":"2024-09-19T04:13:14.429555+0000","last_active":"2024-09-19T04:13:31.505707+0000","last_peered":"2024-09-19T04:13:31.505707+0000","last_clean":"2024-09-19T04:13:31.505707+0000","last_became_active":"2024-09-19T04:13:14.429224+0000","last_became_peered":"2024-09-19T04:13:14.429224+0000","last_unstale":"2024-09-19T04:13:31.505707+0000","last_undegraded":"2024-09-19T04:13:31.505707+0000","last_fullsized":"2024-09-19T04:13:31.505707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:35.304750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501182+0000","last_change":"2024-09-19T04:13:16.454337+0000","last_active":"2024-09-19T04:13:31.501182+0000","last_peered":"2024-09-19T04:13:31.501182+0000","last_clean":"2024-09-19T04:13:31.501182+0000","last_became_active":"2024-09-19T04:13:16.453831+0000","last_became_peered":"2024-09-19T04:13:16.453831+0000","last_unstale":"2024-09-19T04:13:31.501182+0000","last_undegraded":"2024-09-19T04:13:31.501182+0000","last_fullsized":"2024-09-19T04:13:31.501182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:41.751933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506399+0000","last_change":"2024-09-19T04:13:14.620406+0000","last_active":"2024-09-19T04:13:31.506399+0000","last_peered":"2024-09-19T04:13:31.506399+0000","last_clean":"2024-09-19T04:13:31.506399+0000","last_became_active":"2024-09-19T04:13:14.620050+0000","last_became_peered":"2024-09-19T04:13:14.620050+0000","last_unstale":"2024-09-19T04:13:31.506399+0000","last_undegraded":"2024-09-19T04:13:31.506399+0000","last_fullsized":"2024-09-19T04:13:31.506399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:35.828760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501221+0000","last_change":"2024-09-19T04:13:16.459016+0000","last_active":"2024-09-19T04:13:31.501221+0000","last_peered":"2024-09-19T04:13:31.501221+0000","last_clean":"2024-09-19T04:13:31.501221+0000","last_became_active":"2024-09-19T04:13:16.458674+0000","last_became_peered":"2024-09-19T04:13:16.458674+0000","last_unstale":"2024-09-19T04:13:31.501221+0000","last_undegraded":"2024-09-19T04:13:31.501221+0000","last_fullsized":"2024-09-19T04:13:31.501221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:55.894456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":346,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500892+0000","last_change":"2024-09-19T04:12:05.913704+0000","last_active":"2024-09-19T04:13:31.500892+0000","last_peered":"2024-09-19T04:13:31.500892+0000","last_clean":"2024-09-19T04:13:31.500892+0000","last_became_active":"2024-09-19T04:12:05.913299+0000","last_became_peered":"2024-09-19T04:12:05.913299+0000","last_unstale":"2024-09-19T04:13:31.500892+0000","last_undegraded":"2024-09-19T04:13:31.500892+0000","last_fullsized":"2024-09-19T04:13:31.500892+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_clean_scrub_stamp":"2024-09-19T04:08:46.646000+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.638160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504518+0000","last_change":"2024-09-19T04:13:14.423962+0000","last_active":"2024-09-19T04:13:31.504518+0000","last_peered":"2024-09-19T04:13:31.504518+0000","last_clean":"2024-09-19T04:13:31.504518+0000","last_became_active":"2024-09-19T04:13:14.423591+0000","last_became_peered":"2024-09-19T04:13:14.423591+0000","last_unstale":"2024-09-19T04:13:31.504518+0000","last_undegraded":"2024-09-19T04:13:31.504518+0000","last_fullsized":"2024-09-19T04:13:31.504518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:02.239975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501743+0000","last_change":"2024-09-19T04:13:16.447113+0000","last_active":"2024-09-19T04:13:31.501743+0000","last_peered":"2024-09-19T04:13:31.501743+0000","last_clean":"2024-09-19T04:13:31.501743+0000","last_became_active":"2024-09-19T04:13:16.446792+0000","last_became_peered":"2024-09-19T04:13:16.446792+0000","last_unstale":"2024-09-19T04:13:31.501743+0000","last_undegraded":"2024-09-19T04:13:31.501743+0000","last_fullsized":"2024-09-19T04:13:31.501743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:28:21.527010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500749+0000","last_change":"2024-09-19T04:13:14.432663+0000","last_active":"2024-09-19T04:13:31.500749+0000","last_peered":"2024-09-19T04:13:31.500749+0000","last_clean":"2024-09-19T04:13:31.500749+0000","last_became_active":"2024-09-19T04:13:14.432318+0000","last_became_peered":"2024-09-19T04:13:14.432318+0000","last_unstale":"2024-09-19T04:13:31.500749+0000","last_undegraded":"2024-09-19T04:13:31.500749+0000","last_fullsized":"2024-09-19T04:13:31.500749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:35.787596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498761+0000","last_change":"2024-09-19T04:13:16.451010+0000","last_active":"2024-09-19T04:13:31.498761+0000","last_peered":"2024-09-19T04:13:31.498761+0000","last_clean":"2024-09-19T04:13:31.498761+0000","last_became_active":"2024-09-19T04:13:16.450708+0000","last_became_peered":"2024-09-19T04:13:16.450708+0000","last_unstale":"2024-09-19T04:13:31.498761+0000","last_undegraded":"2024-09-19T04:13:31.498761+0000","last_fullsized":"2024-09-19T04:13:31.498761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:30.108541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:37.513030+0000","last_change":"2024-09-19T04:13:14.620037+0000","last_active":"2024-09-19T04:13:37.513030+0000","last_peered":"2024-09-19T04:13:37.513030+0000","last_clean":"2024-09-19T04:13:37.513030+0000","last_became_active":"2024-09-19T04:13:14.619686+0000","last_became_peered":"2024-09-19T04:13:14.619686+0000","last_unstale":"2024-09-19T04:13:37.513030+0000","last_undegraded":"2024-09-19T04:13:37.513030+0000","last_fullsized":"2024-09-19T04:13:37.513030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:32.490708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504838+0000","last_change":"2024-09-19T04:13:16.446429+0000","last_active":"2024-09-19T04:13:31.504838+0000","last_peered":"2024-09-19T04:13:31.504838+0000","last_clean":"2024-09-19T04:13:31.504838+0000","last_became_active":"2024-09-19T04:13:16.446075+0000","last_became_peered":"2024-09-19T04:13:16.446075+0000","last_unstale":"2024-09-19T04:13:31.504838+0000","last_undegraded":"2024-09-19T04:13:31.504838+0000","last_fullsized":"2024-09-19T04:13:31.504838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:03.613926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501031+0000","last_change":"2024-09-19T04:13:14.429899+0000","last_active":"2024-09-19T04:13:31.501031+0000","last_peered":"2024-09-19T04:13:31.501031+0000","last_clean":"2024-09-19T04:13:31.501031+0000","last_became_active":"2024-09-19T04:13:14.429571+0000","last_became_peered":"2024-09-19T04:13:14.429571+0000","last_unstale":"2024-09-19T04:13:31.501031+0000","last_undegraded":"2024-09-19T04:13:31.501031+0000","last_fullsized":"2024-09-19T04:13:31.501031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:37.017431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499733+0000","last_change":"2024-09-19T04:13:16.458267+0000","last_active":"2024-09-19T04:13:31.499733+0000","last_peered":"2024-09-19T04:13:31.499733+0000","last_clean":"2024-09-19T04:13:31.499733+0000","last_became_active":"2024-09-19T04:13:16.457911+0000","last_became_peered":"2024-09-19T04:13:16.457911+0000","last_unstale":"2024-09-19T04:13:31.499733+0000","last_undegraded":"2024-09-19T04:13:31.499733+0000","last_fullsized":"2024-09-19T04:13:31.499733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:32.465923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501677+0000","last_change":"2024-09-19T04:13:14.621552+0000","last_active":"2024-09-19T04:13:31.501677+0000","last_peered":"2024-09-19T04:13:31.501677+0000","last_clean":"2024-09-19T04:13:31.501677+0000","last_became_active":"2024-09-19T04:13:14.620592+0000","last_became_peered":"2024-09-19T04:13:14.620592+0000","last_unstale":"2024-09-19T04:13:31.501677+0000","last_undegraded":"2024-09-19T04:13:31.501677+0000","last_fullsized":"2024-09-19T04:13:31.501677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:10.055202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500150+0000","last_change":"2024-09-19T04:13:16.443117+0000","last_active":"2024-09-19T04:13:31.500150+0000","last_peered":"2024-09-19T04:13:31.500150+0000","last_clean":"2024-09-19T04:13:31.500150+0000","last_became_active":"2024-09-19T04:13:16.442808+0000","last_became_peered":"2024-09-19T04:13:16.442808+0000","last_unstale":"2024-09-19T04:13:31.500150+0000","last_undegraded":"2024-09-19T04:13:31.500150+0000","last_fullsized":"2024-09-19T04:13:31.500150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:40.274320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504531+0000","last_change":"2024-09-19T04:13:14.620618+0000","last_active":"2024-09-19T04:13:31.504531+0000","last_peered":"2024-09-19T04:13:31.504531+0000","last_clean":"2024-09-19T04:13:31.504531+0000","last_became_active":"2024-09-19T04:13:14.620290+0000","last_became_peered":"2024-09-19T04:13:14.620290+0000","last_unstale":"2024-09-19T04:13:31.504531+0000","last_undegraded":"2024-09-19T04:13:31.504531+0000","last_fullsized":"2024-09-19T04:13:31.504531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:51.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499218+0000","last_change":"2024-09-19T04:13:16.443498+0000","last_active":"2024-09-19T04:13:31.499218+0000","last_peered":"2024-09-19T04:13:31.499218+0000","last_clean":"2024-09-19T04:13:31.499218+0000","last_became_active":"2024-09-19T04:13:16.443104+0000","last_became_peered":"2024-09-19T04:13:16.443104+0000","last_unstale":"2024-09-19T04:13:31.499218+0000","last_undegraded":"2024-09-19T04:13:31.499218+0000","last_fullsized":"2024-09-19T04:13:31.499218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:10.733570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:35.455482+0000","last_change":"2024-09-19T04:13:14.425881+0000","last_active":"2024-09-19T04:13:35.455482+0000","last_peered":"2024-09-19T04:13:35.455482+0000","last_clean":"2024-09-19T04:13:35.455482+0000","last_became_active":"2024-09-19T04:13:14.425542+0000","last_became_peered":"2024-09-19T04:13:14.425542+0000","last_unstale":"2024-09-19T04:13:35.455482+0000","last_undegraded":"2024-09-19T04:13:35.455482+0000","last_fullsized":"2024-09-19T04:13:35.455482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:44.250890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502301+0000","last_change":"2024-09-19T04:13:16.454672+0000","last_active":"2024-09-19T04:13:31.502301+0000","last_peered":"2024-09-19T04:13:31.502301+0000","last_clean":"2024-09-19T04:13:31.502301+0000","last_became_active":"2024-09-19T04:13:16.454003+0000","last_became_peered":"2024-09-19T04:13:16.454003+0000","last_unstale":"2024-09-19T04:13:31.502301+0000","last_undegraded":"2024-09-19T04:13:31.502301+0000","last_fullsized":"2024-09-19T04:13:31.502301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:26.849617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504465+0000","last_change":"2024-09-19T04:13:14.432312+0000","last_active":"2024-09-19T04:13:31.504465+0000","last_peered":"2024-09-19T04:13:31.504465+0000","last_clean":"2024-09-19T04:13:31.504465+0000","last_became_active":"2024-09-19T04:13:14.431994+0000","last_became_peered":"2024-09-19T04:13:14.431994+0000","last_unstale":"2024-09-19T04:13:31.504465+0000","last_undegraded":"2024-09-19T04:13:31.504465+0000","last_fullsized":"2024-09-19T04:13:31.504465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:00.410400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735768+0000","last_change":"2024-09-19T04:13:16.455312+0000","last_active":"2024-09-19T04:13:31.735768+0000","last_peered":"2024-09-19T04:13:31.735768+0000","last_clean":"2024-09-19T04:13:31.735768+0000","last_became_active":"2024-09-19T04:13:16.454990+0000","last_became_peered":"2024-09-19T04:13:16.454990+0000","last_unstale":"2024-09-19T04:13:31.735768+0000","last_undegraded":"2024-09-19T04:13:31.735768+0000","last_fullsized":"2024-09-19T04:13:31.735768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:09.207395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700505+0000","last_change":"2024-09-19T04:13:14.620316+0000","last_active":"2024-09-19T04:13:31.700505+0000","last_peered":"2024-09-19T04:13:31.700505+0000","last_clean":"2024-09-19T04:13:31.700505+0000","last_became_active":"2024-09-19T04:13:14.619979+0000","last_became_peered":"2024-09-19T04:13:14.619979+0000","last_unstale":"2024-09-19T04:13:31.700505+0000","last_undegraded":"2024-09-19T04:13:31.700505+0000","last_fullsized":"2024-09-19T04:13:31.700505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:24.382144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505808+0000","last_change":"2024-09-19T04:13:16.455684+0000","last_active":"2024-09-19T04:13:31.505808+0000","last_peered":"2024-09-19T04:13:31.505808+0000","last_clean":"2024-09-19T04:13:31.505808+0000","last_became_active":"2024-09-19T04:13:16.455355+0000","last_became_peered":"2024-09-19T04:13:16.455355+0000","last_unstale":"2024-09-19T04:13:31.505808+0000","last_undegraded":"2024-09-19T04:13:31.505808+0000","last_fullsized":"2024-09-19T04:13:31.505808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:09.657447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501550+0000","last_change":"2024-09-19T04:13:14.423599+0000","last_active":"2024-09-19T04:13:31.501550+0000","last_peered":"2024-09-19T04:13:31.501550+0000","last_clean":"2024-09-19T04:13:31.501550+0000","last_became_active":"2024-09-19T04:13:14.423282+0000","last_became_peered":"2024-09-19T04:13:14.423282+0000","last_unstale":"2024-09-19T04:13:31.501550+0000","last_undegraded":"2024-09-19T04:13:31.501550+0000","last_fullsized":"2024-09-19T04:13:31.501550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.975368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501564+0000","last_change":"2024-09-19T04:13:16.450435+0000","last_active":"2024-09-19T04:13:31.501564+0000","last_peered":"2024-09-19T04:13:31.501564+0000","last_clean":"2024-09-19T04:13:31.501564+0000","last_became_active":"2024-09-19T04:13:16.450087+0000","last_became_peered":"2024-09-19T04:13:16.450087+0000","last_unstale":"2024-09-19T04:13:31.501564+0000","last_undegraded":"2024-09-19T04:13:31.501564+0000","last_fullsized":"2024-09-19T04:13:31.501564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:34:51.764776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504466+0000","last_change":"2024-09-19T04:13:14.432868+0000","last_active":"2024-09-19T04:13:31.504466+0000","last_peered":"2024-09-19T04:13:31.504466+0000","last_clean":"2024-09-19T04:13:31.504466+0000","last_became_active":"2024-09-19T04:13:14.432507+0000","last_became_peered":"2024-09-19T04:13:14.432507+0000","last_unstale":"2024-09-19T04:13:31.504466+0000","last_undegraded":"2024-09-19T04:13:31.504466+0000","last_fullsized":"2024-09-19T04:13:31.504466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:14.773711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501610+0000","last_change":"2024-09-19T04:13:16.450061+0000","last_active":"2024-09-19T04:13:31.501610+0000","last_peered":"2024-09-19T04:13:31.501610+0000","last_clean":"2024-09-19T04:13:31.501610+0000","last_became_active":"2024-09-19T04:13:16.448829+0000","last_became_peered":"2024-09-19T04:13:16.448829+0000","last_unstale":"2024-09-19T04:13:31.501610+0000","last_undegraded":"2024-09-19T04:13:31.501610+0000","last_fullsized":"2024-09-19T04:13:31.501610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:31.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506029+0000","last_change":"2024-09-19T04:13:14.431574+0000","last_active":"2024-09-19T04:13:31.506029+0000","last_peered":"2024-09-19T04:13:31.506029+0000","last_clean":"2024-09-19T04:13:31.506029+0000","last_became_active":"2024-09-19T04:13:14.431202+0000","last_became_peered":"2024-09-19T04:13:14.431202+0000","last_unstale":"2024-09-19T04:13:31.506029+0000","last_undegraded":"2024-09-19T04:13:31.506029+0000","last_fullsized":"2024-09-19T04:13:31.506029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:58.167201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500887+0000","last_change":"2024-09-19T04:13:16.452688+0000","last_active":"2024-09-19T04:13:31.500887+0000","last_peered":"2024-09-19T04:13:31.500887+0000","last_clean":"2024-09-19T04:13:31.500887+0000","last_became_active":"2024-09-19T04:13:16.452342+0000","last_became_peered":"2024-09-19T04:13:16.452342+0000","last_unstale":"2024-09-19T04:13:31.500887+0000","last_undegraded":"2024-09-19T04:13:31.500887+0000","last_fullsized":"2024-09-19T04:13:31.500887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:03.162688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504711+0000","last_change":"2024-09-19T04:13:16.449402+0000","last_active":"2024-09-19T04:13:31.504711+0000","last_peered":"2024-09-19T04:13:31.504711+0000","last_clean":"2024-09-19T04:13:31.504711+0000","last_became_active":"2024-09-19T04:13:16.449108+0000","last_became_peered":"2024-09-19T04:13:16.449108+0000","last_unstale":"2024-09-19T04:13:31.504711+0000","last_undegraded":"2024-09-19T04:13:31.504711+0000","last_fullsized":"2024-09-19T04:13:31.504711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:58.874638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505684+0000","last_change":"2024-09-19T04:13:14.430684+0000","last_active":"2024-09-19T04:13:31.505684+0000","last_peered":"2024-09-19T04:13:31.505684+0000","last_clean":"2024-09-19T04:13:31.505684+0000","last_became_active":"2024-09-19T04:13:14.430137+0000","last_became_peered":"2024-09-19T04:13:14.430137+0000","last_unstale":"2024-09-19T04:13:31.505684+0000","last_undegraded":"2024-09-19T04:13:31.505684+0000","last_fullsized":"2024-09-19T04:13:31.505684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:39.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700648+0000","last_change":"2024-09-19T04:13:16.452491+0000","last_active":"2024-09-19T04:13:31.700648+0000","last_peered":"2024-09-19T04:13:31.700648+0000","last_clean":"2024-09-19T04:13:31.700648+0000","last_became_active":"2024-09-19T04:13:16.452204+0000","last_became_peered":"2024-09-19T04:13:16.452204+0000","last_unstale":"2024-09-19T04:13:31.700648+0000","last_undegraded":"2024-09-19T04:13:31.700648+0000","last_fullsized":"2024-09-19T04:13:31.700648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:10.113510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504401+0000","last_change":"2024-09-19T04:13:14.431608+0000","last_active":"2024-09-19T04:13:31.504401+0000","last_peered":"2024-09-19T04:13:31.504401+0000","last_clean":"2024-09-19T04:13:31.504401+0000","last_became_active":"2024-09-19T04:13:14.431111+0000","last_became_peered":"2024-09-19T04:13:14.431111+0000","last_unstale":"2024-09-19T04:13:31.504401+0000","last_undegraded":"2024-09-19T04:13:31.504401+0000","last_fullsized":"2024-09-19T04:13:31.504401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:03:41.125315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735802+0000","last_change":"2024-09-19T04:13:16.457867+0000","last_active":"2024-09-19T04:13:31.735802+0000","last_peered":"2024-09-19T04:13:31.735802+0000","last_clean":"2024-09-19T04:13:31.735802+0000","last_became_active":"2024-09-19T04:13:16.457582+0000","last_became_peered":"2024-09-19T04:13:16.457582+0000","last_unstale":"2024-09-19T04:13:31.735802+0000","last_undegraded":"2024-09-19T04:13:31.735802+0000","last_fullsized":"2024-09-19T04:13:31.735802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:58.076250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501886+0000","last_change":"2024-09-19T04:13:14.620070+0000","last_active":"2024-09-19T04:13:31.501886+0000","last_peered":"2024-09-19T04:13:31.501886+0000","last_clean":"2024-09-19T04:13:31.501886+0000","last_became_active":"2024-09-19T04:13:14.619734+0000","last_became_peered":"2024-09-19T04:13:14.619734+0000","last_unstale":"2024-09-19T04:13:31.501886+0000","last_undegraded":"2024-09-19T04:13:31.501886+0000","last_fullsized":"2024-09-19T04:13:31.501886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:44:02.818859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501786+0000","last_change":"2024-09-19T04:13:16.443832+0000","last_active":"2024-09-19T04:13:31.501786+0000","last_peered":"2024-09-19T04:13:31.501786+0000","last_clean":"2024-09-19T04:13:31.501786+0000","last_became_active":"2024-09-19T04:13:16.442690+0000","last_became_peered":"2024-09-19T04:13:16.442690+0000","last_unstale":"2024-09-19T04:13:31.501786+0000","last_undegraded":"2024-09-19T04:13:31.501786+0000","last_fullsized":"2024-09-19T04:13:31.501786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:39.535628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501657+0000","last_change":"2024-09-19T04:13:14.427186+0000","last_active":"2024-09-19T04:13:31.501657+0000","last_peered":"2024-09-19T04:13:31.501657+0000","last_clean":"2024-09-19T04:13:31.501657+0000","last_became_active":"2024-09-19T04:13:14.426949+0000","last_became_peered":"2024-09-19T04:13:14.426949+0000","last_unstale":"2024-09-19T04:13:31.501657+0000","last_undegraded":"2024-09-19T04:13:31.501657+0000","last_fullsized":"2024-09-19T04:13:31.501657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:26.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54508,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":96,"num_read_kb":107,"num_write":153,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":145430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":431409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1073339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005083,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40388,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704764,"statfs":{"total":95995035648,"available":95953678336,"internally_reserved":0,"allocated":905216,"data_stored":431589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":431487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1023834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":892928,"data_stored":430978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1511424,"data_stored":1023125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":431808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":431172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476797,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":459427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":457718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":913408,"data_stored":432249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":41065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":42774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:13:42.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:13:42.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:41 smithi104 ceph-mon[34145]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:13:42.344 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:42.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 -- 172.21.15.94:0/2384168760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 msgr2=0x7f53f0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 --2- 172.21.15.94:0/2384168760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 0x7f53f0103170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009930 tx=0x7f53e002a420 comp rx=0 tx=0).stop 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 -- 172.21.15.94:0/2384168760 shutdown_connections 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 --2- 172.21.15.94:0/2384168760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0108290 0x7f53f010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 --2- 172.21.15.94:0/2384168760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f0103740 0x7f53f0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 --2- 172.21.15.94:0/2384168760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 0x7f53f0103170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 -- 172.21.15.94:0/2384168760 >> 172.21.15.94:0/2384168760 conn(0x7f53f00fc7d0 msgr2=0x7f53f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 -- 172.21.15.94:0/2384168760 shutdown_connections 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 -- 172.21.15.94:0/2384168760 wait complete. 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 Processor -- start 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.818+0000 7f53f67bc640 1 -- start start 2024-09-19T04:13:42.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53f67bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 0x7f53f0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53f67bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0103740 0x7f53f0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53f67bc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f0108290 0x7f53f019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53f67bc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0111f80 con 0x7f53f0102d90 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53f67bc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0111e00 con 0x7f53f0108290 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53f67bc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0112100 con 0x7f53f0103740 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 0x7f53f0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 0x7f53f0198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42078/0 (socket says 172.21.15.94:42078) 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53effff640 1 -- 172.21.15.94:0/1554912649 learned_addr learned my addr 172.21.15.94:0/1554912649 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53ef7fe640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0103740 0x7f53f0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53effff640 1 -- 172.21.15.94:0/1554912649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0103740 msgr2=0x7f53f0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53effff640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0103740 0x7f53f0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53effff640 1 -- 172.21.15.94:0/1554912649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f0108290 msgr2=0x7f53f019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53f4d59640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f0108290 0x7f53f019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53effff640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f0108290 0x7f53f019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53effff640 1 -- 172.21.15.94:0/1554912649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f019d4f0 con 0x7f53f0102d90 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53f4d59640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f0108290 0x7f53f019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53effff640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 0x7f53f0198540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f53e0005940 tx=0x7f53e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.819+0000 7f53ed7fa640 1 -- 172.21.15.94:0/1554912649 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e003f030 con 0x7f53f0102d90 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.820+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f019d780 con 0x7f53f0102d90 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.820+0000 7f53ed7fa640 1 -- 172.21.15.94:0/1554912649 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e0003cb0 con 0x7f53f0102d90 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.820+0000 7f53ed7fa640 1 -- 172.21.15.94:0/1554912649 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0035070 con 0x7f53f0102d90 2024-09-19T04:13:42.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.820+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f01a50c0 con 0x7f53f0102d90 2024-09-19T04:13:42.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.821+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f006a7f0 con 0x7f53f0102d90 2024-09-19T04:13:42.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.822+0000 7f53ed7fa640 1 -- 172.21.15.94:0/1554912649 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f53e0042030 con 0x7f53f0102d90 2024-09-19T04:13:42.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.822+0000 7f53ed7fa640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f53d80a32d0 0x7f53d80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.822+0000 7f53ef7fe640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f53d80a32d0 0x7f53d80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.823+0000 7f53ed7fa640 1 -- 172.21.15.94:0/1554912649 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f53e00f00c0 con 0x7f53f0102d90 2024-09-19T04:13:42.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.823+0000 7f53ef7fe640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f53d80a32d0 0x7f53d80a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f53f00fee80 tx=0x7f53e4005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:42.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.827+0000 7f53ed7fa640 1 -- 172.21.15.94:0/1554912649 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f53e00bcc80 con 0x7f53f0102d90 2024-09-19T04:13:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.990+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53f0062630 con 0x7f53d80a32d0 2024-09-19T04:13:42.992 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:42 smithi094 ceph-mon[30972]: pgmap v287: 97 pgs: 97 active+clean; 630 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 8 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:13:42.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.996+0000 7f53ed7fa640 1 -- 172.21.15.94:0/1554912649 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258717 (secure 0 0 0) 0x7f53f0062630 con 0x7f53d80a32d0 2024-09-19T04:13:43.000 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:43.001 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-19T04:13:43.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.999+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f53d80a32d0 msgr2=0x7f53d80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:42.999+0000 7f53f67bc640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f53d80a32d0 0x7f53d80a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f53f00fee80 tx=0x7f53e4005c60 comp rx=0 tx=0).stop 2024-09-19T04:13:43.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 msgr2=0x7f53f0198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 0x7f53f0198540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f53e0005940 tx=0x7f53e00042f0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 shutdown_connections 2024-09-19T04:13:43.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f53d80a32d0 0x7f53d80a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f0108290 0x7f53f019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0103740 0x7f53f0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 --2- 172.21.15.94:0/1554912649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f0102d90 0x7f53f0198540 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 >> 172.21.15.94:0/1554912649 conn(0x7f53f00fc7d0 msgr2=0x7f53f0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:43.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 shutdown_connections 2024-09-19T04:13:43.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:43.000+0000 7f53f67bc640 1 -- 172.21.15.94:0/1554912649 wait complete. 2024-09-19T04:13:43.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:42 smithi112 ceph-mon[35051]: pgmap v287: 97 pgs: 97 active+clean; 630 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 8 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:13:43.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:42 smithi104 ceph-mon[34145]: pgmap v287: 97 pgs: 97 active+clean; 630 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 8 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:13:43.695 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-19T04:13:42.136454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645110,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":237,"num_read_kb":226,"num_write":336,"num_write_kb":3037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399232,"kb_used_data":12516,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542592,"statfs":{"total":1151940427776,"available":1151531614208,"internally_reserved":0,"allocated":12816384,"data_stored":7104582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49127,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":80,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.638932"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501952+0000","last_change":"2024-09-19T04:13:16.451031+0000","last_active":"2024-09-19T04:13:31.501952+0000","last_peered":"2024-09-19T04:13:31.501952+0000","last_clean":"2024-09-19T04:13:31.501952+0000","last_became_active":"2024-09-19T04:13:16.450713+0000","last_became_peered":"2024-09-19T04:13:16.450713+0000","last_unstale":"2024-09-19T04:13:31.501952+0000","last_undegraded":"2024-09-19T04:13:31.501952+0000","last_fullsized":"2024-09-19T04:13:31.501952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:59.485415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735600+0000","last_change":"2024-09-19T04:13:16.455689+0000","last_active":"2024-09-19T04:13:31.735600+0000","last_peered":"2024-09-19T04:13:31.735600+0000","last_clean":"2024-09-19T04:13:31.735600+0000","last_became_active":"2024-09-19T04:13:16.455154+0000","last_became_peered":"2024-09-19T04:13:16.455154+0000","last_unstale":"2024-09-19T04:13:31.735600+0000","last_undegraded":"2024-09-19T04:13:31.735600+0000","last_fullsized":"2024-09-19T04:13:31.735600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:57.610619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735526+0000","last_change":"2024-09-19T04:13:16.447392+0000","last_active":"2024-09-19T04:13:31.735526+0000","last_peered":"2024-09-19T04:13:31.735526+0000","last_clean":"2024-09-19T04:13:31.735526+0000","last_became_active":"2024-09-19T04:13:16.447088+0000","last_became_peered":"2024-09-19T04:13:16.447088+0000","last_unstale":"2024-09-19T04:13:31.735526+0000","last_undegraded":"2024-09-19T04:13:31.735526+0000","last_fullsized":"2024-09-19T04:13:31.735526+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:16.591420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504467+0000","last_change":"2024-09-19T04:13:16.452642+0000","last_active":"2024-09-19T04:13:31.504467+0000","last_peered":"2024-09-19T04:13:31.504467+0000","last_clean":"2024-09-19T04:13:31.504467+0000","last_became_active":"2024-09-19T04:13:16.452329+0000","last_became_peered":"2024-09-19T04:13:16.452329+0000","last_unstale":"2024-09-19T04:13:31.504467+0000","last_undegraded":"2024-09-19T04:13:31.504467+0000","last_fullsized":"2024-09-19T04:13:31.504467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:54.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505930+0000","last_change":"2024-09-19T04:13:16.447171+0000","last_active":"2024-09-19T04:13:31.505930+0000","last_peered":"2024-09-19T04:13:31.505930+0000","last_clean":"2024-09-19T04:13:31.505930+0000","last_became_active":"2024-09-19T04:13:16.446835+0000","last_became_peered":"2024-09-19T04:13:16.446835+0000","last_unstale":"2024-09-19T04:13:31.505930+0000","last_undegraded":"2024-09-19T04:13:31.505930+0000","last_fullsized":"2024-09-19T04:13:31.505930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:41.892118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.507665+0000","last_change":"2024-09-19T04:13:31.507665+0000","last_active":"2024-09-19T04:13:31.507665+0000","last_peered":"2024-09-19T04:13:31.507665+0000","last_clean":"2024-09-19T04:13:31.507665+0000","last_became_active":"2024-09-19T04:13:31.507329+0000","last_became_peered":"2024-09-19T04:13:31.507329+0000","last_unstale":"2024-09-19T04:13:31.507665+0000","last_undegraded":"2024-09-19T04:13:31.507665+0000","last_fullsized":"2024-09-19T04:13:31.507665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:35.832776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500742+0000","last_change":"2024-09-19T04:13:16.454568+0000","last_active":"2024-09-19T04:13:31.500742+0000","last_peered":"2024-09-19T04:13:31.500742+0000","last_clean":"2024-09-19T04:13:31.500742+0000","last_became_active":"2024-09-19T04:13:16.454230+0000","last_became_peered":"2024-09-19T04:13:16.454230+0000","last_unstale":"2024-09-19T04:13:31.500742+0000","last_undegraded":"2024-09-19T04:13:31.500742+0000","last_fullsized":"2024-09-19T04:13:31.500742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:23.401316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501235+0000","last_change":"2024-09-19T04:13:16.449223+0000","last_active":"2024-09-19T04:13:31.501235+0000","last_peered":"2024-09-19T04:13:31.501235+0000","last_clean":"2024-09-19T04:13:31.501235+0000","last_became_active":"2024-09-19T04:13:16.448665+0000","last_became_peered":"2024-09-19T04:13:16.448665+0000","last_unstale":"2024-09-19T04:13:31.501235+0000","last_undegraded":"2024-09-19T04:13:31.501235+0000","last_fullsized":"2024-09-19T04:13:31.501235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:29.619847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.701069+0000","last_change":"2024-09-19T04:13:16.458203+0000","last_active":"2024-09-19T04:13:31.701069+0000","last_peered":"2024-09-19T04:13:31.701069+0000","last_clean":"2024-09-19T04:13:31.701069+0000","last_became_active":"2024-09-19T04:13:16.457544+0000","last_became_peered":"2024-09-19T04:13:16.457544+0000","last_unstale":"2024-09-19T04:13:31.701069+0000","last_undegraded":"2024-09-19T04:13:31.701069+0000","last_fullsized":"2024-09-19T04:13:31.701069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:47.611695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500145+0000","last_change":"2024-09-19T04:13:16.454213+0000","last_active":"2024-09-19T04:13:31.500145+0000","last_peered":"2024-09-19T04:13:31.500145+0000","last_clean":"2024-09-19T04:13:31.500145+0000","last_became_active":"2024-09-19T04:13:16.453047+0000","last_became_peered":"2024-09-19T04:13:16.453047+0000","last_unstale":"2024-09-19T04:13:31.500145+0000","last_undegraded":"2024-09-19T04:13:31.500145+0000","last_fullsized":"2024-09-19T04:13:31.500145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:27.425259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501213+0000","last_change":"2024-09-19T04:13:16.458195+0000","last_active":"2024-09-19T04:13:31.501213+0000","last_peered":"2024-09-19T04:13:31.501213+0000","last_clean":"2024-09-19T04:13:31.501213+0000","last_became_active":"2024-09-19T04:13:16.457891+0000","last_became_peered":"2024-09-19T04:13:16.457891+0000","last_unstale":"2024-09-19T04:13:31.501213+0000","last_undegraded":"2024-09-19T04:13:31.501213+0000","last_fullsized":"2024-09-19T04:13:31.501213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:53.833249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.508474+0000","last_change":"2024-09-19T04:13:31.508117+0000","last_active":"2024-09-19T04:13:31.508474+0000","last_peered":"2024-09-19T04:13:31.508474+0000","last_clean":"2024-09-19T04:13:31.508474+0000","last_became_active":"2024-09-19T04:13:31.507693+0000","last_became_peered":"2024-09-19T04:13:31.507693+0000","last_unstale":"2024-09-19T04:13:31.508474+0000","last_undegraded":"2024-09-19T04:13:31.508474+0000","last_fullsized":"2024-09-19T04:13:31.508474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:58.447965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499011+0000","last_change":"2024-09-19T04:13:16.440079+0000","last_active":"2024-09-19T04:13:31.499011+0000","last_peered":"2024-09-19T04:13:31.499011+0000","last_clean":"2024-09-19T04:13:31.499011+0000","last_became_active":"2024-09-19T04:13:16.439474+0000","last_became_peered":"2024-09-19T04:13:16.439474+0000","last_unstale":"2024-09-19T04:13:31.499011+0000","last_undegraded":"2024-09-19T04:13:31.499011+0000","last_fullsized":"2024-09-19T04:13:31.499011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:11.151412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735816+0000","last_change":"2024-09-19T04:13:16.449958+0000","last_active":"2024-09-19T04:13:31.735816+0000","last_peered":"2024-09-19T04:13:31.735816+0000","last_clean":"2024-09-19T04:13:31.735816+0000","last_became_active":"2024-09-19T04:13:16.449665+0000","last_became_peered":"2024-09-19T04:13:16.449665+0000","last_unstale":"2024-09-19T04:13:31.735816+0000","last_undegraded":"2024-09-19T04:13:31.735816+0000","last_fullsized":"2024-09-19T04:13:31.735816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:24.023779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735453+0000","last_change":"2024-09-19T04:13:16.454333+0000","last_active":"2024-09-19T04:13:31.735453+0000","last_peered":"2024-09-19T04:13:31.735453+0000","last_clean":"2024-09-19T04:13:31.735453+0000","last_became_active":"2024-09-19T04:13:16.454066+0000","last_became_peered":"2024-09-19T04:13:16.454066+0000","last_unstale":"2024-09-19T04:13:31.735453+0000","last_undegraded":"2024-09-19T04:13:31.735453+0000","last_fullsized":"2024-09-19T04:13:31.735453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:16.815173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700446+0000","last_change":"2024-09-19T04:13:16.451864+0000","last_active":"2024-09-19T04:13:31.700446+0000","last_peered":"2024-09-19T04:13:31.700446+0000","last_clean":"2024-09-19T04:13:31.700446+0000","last_became_active":"2024-09-19T04:13:16.450995+0000","last_became_peered":"2024-09-19T04:13:16.450995+0000","last_unstale":"2024-09-19T04:13:31.700446+0000","last_undegraded":"2024-09-19T04:13:31.700446+0000","last_fullsized":"2024-09-19T04:13:31.700446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:40.750221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502172+0000","last_change":"2024-09-19T04:13:16.453176+0000","last_active":"2024-09-19T04:13:31.502172+0000","last_peered":"2024-09-19T04:13:31.502172+0000","last_clean":"2024-09-19T04:13:31.502172+0000","last_became_active":"2024-09-19T04:13:16.452788+0000","last_became_peered":"2024-09-19T04:13:16.452788+0000","last_unstale":"2024-09-19T04:13:31.502172+0000","last_undegraded":"2024-09-19T04:13:31.502172+0000","last_fullsized":"2024-09-19T04:13:31.502172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:47.099180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501446+0000","last_change":"2024-09-19T04:13:16.449167+0000","last_active":"2024-09-19T04:13:31.501446+0000","last_peered":"2024-09-19T04:13:31.501446+0000","last_clean":"2024-09-19T04:13:31.501446+0000","last_became_active":"2024-09-19T04:13:16.448579+0000","last_became_peered":"2024-09-19T04:13:16.448579+0000","last_unstale":"2024-09-19T04:13:31.501446+0000","last_undegraded":"2024-09-19T04:13:31.501446+0000","last_fullsized":"2024-09-19T04:13:31.501446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:22:40.023000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505138+0000","last_change":"2024-09-19T04:13:16.445449+0000","last_active":"2024-09-19T04:13:31.505138+0000","last_peered":"2024-09-19T04:13:31.505138+0000","last_clean":"2024-09-19T04:13:31.505138+0000","last_became_active":"2024-09-19T04:13:16.445028+0000","last_became_peered":"2024-09-19T04:13:16.445028+0000","last_unstale":"2024-09-19T04:13:31.505138+0000","last_undegraded":"2024-09-19T04:13:31.505138+0000","last_fullsized":"2024-09-19T04:13:31.505138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:13.095348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501714+0000","last_change":"2024-09-19T04:13:16.454033+0000","last_active":"2024-09-19T04:13:31.501714+0000","last_peered":"2024-09-19T04:13:31.501714+0000","last_clean":"2024-09-19T04:13:31.501714+0000","last_became_active":"2024-09-19T04:13:16.453592+0000","last_became_peered":"2024-09-19T04:13:16.453592+0000","last_unstale":"2024-09-19T04:13:31.501714+0000","last_undegraded":"2024-09-19T04:13:31.501714+0000","last_fullsized":"2024-09-19T04:13:31.501714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.304478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499027+0000","last_change":"2024-09-19T04:13:16.448763+0000","last_active":"2024-09-19T04:13:31.499027+0000","last_peered":"2024-09-19T04:13:31.499027+0000","last_clean":"2024-09-19T04:13:31.499027+0000","last_became_active":"2024-09-19T04:13:16.448459+0000","last_became_peered":"2024-09-19T04:13:16.448459+0000","last_unstale":"2024-09-19T04:13:31.499027+0000","last_undegraded":"2024-09-19T04:13:31.499027+0000","last_fullsized":"2024-09-19T04:13:31.499027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700507+0000","last_change":"2024-09-19T04:13:16.451489+0000","last_active":"2024-09-19T04:13:31.700507+0000","last_peered":"2024-09-19T04:13:31.700507+0000","last_clean":"2024-09-19T04:13:31.700507+0000","last_became_active":"2024-09-19T04:13:16.451176+0000","last_became_peered":"2024-09-19T04:13:16.451176+0000","last_unstale":"2024-09-19T04:13:31.700507+0000","last_undegraded":"2024-09-19T04:13:31.700507+0000","last_fullsized":"2024-09-19T04:13:31.700507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.921102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501314+0000","last_change":"2024-09-19T04:13:16.454551+0000","last_active":"2024-09-19T04:13:31.501314+0000","last_peered":"2024-09-19T04:13:31.501314+0000","last_clean":"2024-09-19T04:13:31.501314+0000","last_became_active":"2024-09-19T04:13:16.453976+0000","last_became_peered":"2024-09-19T04:13:16.453976+0000","last_unstale":"2024-09-19T04:13:31.501314+0000","last_undegraded":"2024-09-19T04:13:31.501314+0000","last_fullsized":"2024-09-19T04:13:31.501314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:06.504615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504331+0000","last_change":"2024-09-19T04:13:31.503222+0000","last_active":"2024-09-19T04:13:31.504331+0000","last_peered":"2024-09-19T04:13:31.504331+0000","last_clean":"2024-09-19T04:13:31.504331+0000","last_became_active":"2024-09-19T04:13:31.502873+0000","last_became_peered":"2024-09-19T04:13:31.502873+0000","last_unstale":"2024-09-19T04:13:31.504331+0000","last_undegraded":"2024-09-19T04:13:31.504331+0000","last_fullsized":"2024-09-19T04:13:31.504331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:54.559592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505834+0000","last_change":"2024-09-19T04:13:31.505834+0000","last_active":"2024-09-19T04:13:31.505834+0000","last_peered":"2024-09-19T04:13:31.505834+0000","last_clean":"2024-09-19T04:13:31.505834+0000","last_became_active":"2024-09-19T04:13:31.505501+0000","last_became_peered":"2024-09-19T04:13:31.505501+0000","last_unstale":"2024-09-19T04:13:31.505834+0000","last_undegraded":"2024-09-19T04:13:31.505834+0000","last_fullsized":"2024-09-19T04:13:31.505834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:45.136128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504841+0000","last_change":"2024-09-19T04:13:16.449237+0000","last_active":"2024-09-19T04:13:31.504841+0000","last_peered":"2024-09-19T04:13:31.504841+0000","last_clean":"2024-09-19T04:13:31.504841+0000","last_became_active":"2024-09-19T04:13:16.448896+0000","last_became_peered":"2024-09-19T04:13:16.448896+0000","last_unstale":"2024-09-19T04:13:31.504841+0000","last_undegraded":"2024-09-19T04:13:31.504841+0000","last_fullsized":"2024-09-19T04:13:31.504841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:38.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501835+0000","last_change":"2024-09-19T04:13:16.456053+0000","last_active":"2024-09-19T04:13:31.501835+0000","last_peered":"2024-09-19T04:13:31.501835+0000","last_clean":"2024-09-19T04:13:31.501835+0000","last_became_active":"2024-09-19T04:13:16.454775+0000","last_became_peered":"2024-09-19T04:13:16.454775+0000","last_unstale":"2024-09-19T04:13:31.501835+0000","last_undegraded":"2024-09-19T04:13:31.501835+0000","last_fullsized":"2024-09-19T04:13:31.501835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.902036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499121+0000","last_change":"2024-09-19T04:13:16.451473+0000","last_active":"2024-09-19T04:13:31.499121+0000","last_peered":"2024-09-19T04:13:31.499121+0000","last_clean":"2024-09-19T04:13:31.499121+0000","last_became_active":"2024-09-19T04:13:16.451150+0000","last_became_peered":"2024-09-19T04:13:16.451150+0000","last_unstale":"2024-09-19T04:13:31.499121+0000","last_undegraded":"2024-09-19T04:13:31.499121+0000","last_fullsized":"2024-09-19T04:13:31.499121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:59.332745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735656+0000","last_change":"2024-09-19T04:13:16.452017+0000","last_active":"2024-09-19T04:13:31.735656+0000","last_peered":"2024-09-19T04:13:31.735656+0000","last_clean":"2024-09-19T04:13:31.735656+0000","last_became_active":"2024-09-19T04:13:16.450715+0000","last_became_peered":"2024-09-19T04:13:16.450715+0000","last_unstale":"2024-09-19T04:13:31.735656+0000","last_undegraded":"2024-09-19T04:13:31.735656+0000","last_fullsized":"2024-09-19T04:13:31.735656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:43.548770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.736597+0000","last_change":"2024-09-19T04:13:31.736597+0000","last_active":"2024-09-19T04:13:31.736597+0000","last_peered":"2024-09-19T04:13:31.736597+0000","last_clean":"2024-09-19T04:13:31.736597+0000","last_became_active":"2024-09-19T04:13:31.736301+0000","last_became_peered":"2024-09-19T04:13:31.736301+0000","last_unstale":"2024-09-19T04:13:31.736597+0000","last_undegraded":"2024-09-19T04:13:31.736597+0000","last_fullsized":"2024-09-19T04:13:31.736597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:06.614117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.509742+0000","last_change":"2024-09-19T04:13:31.509742+0000","last_active":"2024-09-19T04:13:31.509742+0000","last_peered":"2024-09-19T04:13:31.509742+0000","last_clean":"2024-09-19T04:13:31.509742+0000","last_became_active":"2024-09-19T04:13:31.509363+0000","last_became_peered":"2024-09-19T04:13:31.509363+0000","last_unstale":"2024-09-19T04:13:31.509742+0000","last_undegraded":"2024-09-19T04:13:31.509742+0000","last_fullsized":"2024-09-19T04:13:31.509742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.094930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501525+0000","last_change":"2024-09-19T04:13:16.449459+0000","last_active":"2024-09-19T04:13:31.501525+0000","last_peered":"2024-09-19T04:13:31.501525+0000","last_clean":"2024-09-19T04:13:31.501525+0000","last_became_active":"2024-09-19T04:13:16.449119+0000","last_became_peered":"2024-09-19T04:13:16.449119+0000","last_unstale":"2024-09-19T04:13:31.501525+0000","last_undegraded":"2024-09-19T04:13:31.501525+0000","last_fullsized":"2024-09-19T04:13:31.501525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:26:26.941609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504930+0000","last_change":"2024-09-19T04:13:16.448939+0000","last_active":"2024-09-19T04:13:31.504930+0000","last_peered":"2024-09-19T04:13:31.504930+0000","last_clean":"2024-09-19T04:13:31.504930+0000","last_became_active":"2024-09-19T04:13:16.448619+0000","last_became_peered":"2024-09-19T04:13:16.448619+0000","last_unstale":"2024-09-19T04:13:31.504930+0000","last_undegraded":"2024-09-19T04:13:31.504930+0000","last_fullsized":"2024-09-19T04:13:31.504930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:41.017404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:37.517120+0000","last_change":"2024-09-19T04:13:14.430467+0000","last_active":"2024-09-19T04:13:37.517120+0000","last_peered":"2024-09-19T04:13:37.517120+0000","last_clean":"2024-09-19T04:13:37.517120+0000","last_became_active":"2024-09-19T04:13:14.430095+0000","last_became_peered":"2024-09-19T04:13:14.430095+0000","last_unstale":"2024-09-19T04:13:37.517120+0000","last_undegraded":"2024-09-19T04:13:37.517120+0000","last_fullsized":"2024-09-19T04:13:37.517120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:50.471745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506178+0000","last_change":"2024-09-19T04:13:31.505798+0000","last_active":"2024-09-19T04:13:31.506178+0000","last_peered":"2024-09-19T04:13:31.506178+0000","last_clean":"2024-09-19T04:13:31.506178+0000","last_became_active":"2024-09-19T04:13:31.505469+0000","last_became_peered":"2024-09-19T04:13:31.505469+0000","last_unstale":"2024-09-19T04:13:31.506178+0000","last_undegraded":"2024-09-19T04:13:31.506178+0000","last_fullsized":"2024-09-19T04:13:31.506178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:13:42.405075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499636+0000","last_change":"2024-09-19T04:13:14.426443+0000","last_active":"2024-09-19T04:13:31.499636+0000","last_peered":"2024-09-19T04:13:31.499636+0000","last_clean":"2024-09-19T04:13:31.499636+0000","last_became_active":"2024-09-19T04:13:14.426103+0000","last_became_peered":"2024-09-19T04:13:14.426103+0000","last_unstale":"2024-09-19T04:13:31.499636+0000","last_undegraded":"2024-09-19T04:13:31.499636+0000","last_fullsized":"2024-09-19T04:13:31.499636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:28.461962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498768+0000","last_change":"2024-09-19T04:13:14.424439+0000","last_active":"2024-09-19T04:13:31.498768+0000","last_peered":"2024-09-19T04:13:31.498768+0000","last_clean":"2024-09-19T04:13:31.498768+0000","last_became_active":"2024-09-19T04:13:14.423975+0000","last_became_peered":"2024-09-19T04:13:14.423975+0000","last_unstale":"2024-09-19T04:13:31.498768+0000","last_undegraded":"2024-09-19T04:13:31.498768+0000","last_fullsized":"2024-09-19T04:13:31.498768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:12.032125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501447+0000","last_change":"2024-09-19T04:13:16.444865+0000","last_active":"2024-09-19T04:13:31.501447+0000","last_peered":"2024-09-19T04:13:31.501447+0000","last_clean":"2024-09-19T04:13:31.501447+0000","last_became_active":"2024-09-19T04:13:16.444556+0000","last_became_peered":"2024-09-19T04:13:16.444556+0000","last_unstale":"2024-09-19T04:13:31.501447+0000","last_undegraded":"2024-09-19T04:13:31.501447+0000","last_fullsized":"2024-09-19T04:13:31.501447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:38.025604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505570+0000","last_change":"2024-09-19T04:13:14.429018+0000","last_active":"2024-09-19T04:13:31.505570+0000","last_peered":"2024-09-19T04:13:31.505570+0000","last_clean":"2024-09-19T04:13:31.505570+0000","last_became_active":"2024-09-19T04:13:14.428383+0000","last_became_peered":"2024-09-19T04:13:14.428383+0000","last_unstale":"2024-09-19T04:13:31.505570+0000","last_undegraded":"2024-09-19T04:13:31.505570+0000","last_fullsized":"2024-09-19T04:13:31.505570+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:46.190421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501335+0000","last_change":"2024-09-19T04:13:16.449013+0000","last_active":"2024-09-19T04:13:31.501335+0000","last_peered":"2024-09-19T04:13:31.501335+0000","last_clean":"2024-09-19T04:13:31.501335+0000","last_became_active":"2024-09-19T04:13:16.448703+0000","last_became_peered":"2024-09-19T04:13:16.448703+0000","last_unstale":"2024-09-19T04:13:31.501335+0000","last_undegraded":"2024-09-19T04:13:31.501335+0000","last_fullsized":"2024-09-19T04:13:31.501335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:41.231759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:36.515081+0000","last_change":"2024-09-19T04:13:14.430050+0000","last_active":"2024-09-19T04:13:36.515081+0000","last_peered":"2024-09-19T04:13:36.515081+0000","last_clean":"2024-09-19T04:13:36.515081+0000","last_became_active":"2024-09-19T04:13:14.429710+0000","last_became_peered":"2024-09-19T04:13:14.429710+0000","last_unstale":"2024-09-19T04:13:36.515081+0000","last_undegraded":"2024-09-19T04:13:36.515081+0000","last_fullsized":"2024-09-19T04:13:36.515081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:55.796080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700932+0000","last_change":"2024-09-19T04:13:16.454431+0000","last_active":"2024-09-19T04:13:31.700932+0000","last_peered":"2024-09-19T04:13:31.700932+0000","last_clean":"2024-09-19T04:13:31.700932+0000","last_became_active":"2024-09-19T04:13:16.453033+0000","last_became_peered":"2024-09-19T04:13:16.453033+0000","last_unstale":"2024-09-19T04:13:31.700932+0000","last_undegraded":"2024-09-19T04:13:31.700932+0000","last_fullsized":"2024-09-19T04:13:31.700932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:52.817740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501175+0000","last_change":"2024-09-19T04:13:14.428949+0000","last_active":"2024-09-19T04:13:31.501175+0000","last_peered":"2024-09-19T04:13:31.501175+0000","last_clean":"2024-09-19T04:13:31.501175+0000","last_became_active":"2024-09-19T04:13:14.428525+0000","last_became_peered":"2024-09-19T04:13:14.428525+0000","last_unstale":"2024-09-19T04:13:31.501175+0000","last_undegraded":"2024-09-19T04:13:31.501175+0000","last_fullsized":"2024-09-19T04:13:31.501175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:30.296730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735848+0000","last_change":"2024-09-19T04:13:16.454077+0000","last_active":"2024-09-19T04:13:31.735848+0000","last_peered":"2024-09-19T04:13:31.735848+0000","last_clean":"2024-09-19T04:13:31.735848+0000","last_became_active":"2024-09-19T04:13:16.453791+0000","last_became_peered":"2024-09-19T04:13:16.453791+0000","last_unstale":"2024-09-19T04:13:31.735848+0000","last_undegraded":"2024-09-19T04:13:31.735848+0000","last_fullsized":"2024-09-19T04:13:31.735848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:29.426475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505069+0000","last_change":"2024-09-19T04:13:14.620179+0000","last_active":"2024-09-19T04:13:31.505069+0000","last_peered":"2024-09-19T04:13:31.505069+0000","last_clean":"2024-09-19T04:13:31.505069+0000","last_became_active":"2024-09-19T04:13:14.619758+0000","last_became_peered":"2024-09-19T04:13:14.619758+0000","last_unstale":"2024-09-19T04:13:31.505069+0000","last_undegraded":"2024-09-19T04:13:31.505069+0000","last_fullsized":"2024-09-19T04:13:31.505069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:11.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700712+0000","last_change":"2024-09-19T04:13:16.453036+0000","last_active":"2024-09-19T04:13:31.700712+0000","last_peered":"2024-09-19T04:13:31.700712+0000","last_clean":"2024-09-19T04:13:31.700712+0000","last_became_active":"2024-09-19T04:13:16.452736+0000","last_became_peered":"2024-09-19T04:13:16.452736+0000","last_unstale":"2024-09-19T04:13:31.700712+0000","last_undegraded":"2024-09-19T04:13:31.700712+0000","last_fullsized":"2024-09-19T04:13:31.700712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:30.003652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'25","reported_seq":77,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:37.520459+0000","last_change":"2024-09-19T04:13:14.428715+0000","last_active":"2024-09-19T04:13:37.520459+0000","last_peered":"2024-09-19T04:13:37.520459+0000","last_clean":"2024-09-19T04:13:37.520459+0000","last_became_active":"2024-09-19T04:13:14.428409+0000","last_became_peered":"2024-09-19T04:13:14.428409+0000","last_unstale":"2024-09-19T04:13:37.520459+0000","last_undegraded":"2024-09-19T04:13:37.520459+0000","last_fullsized":"2024-09-19T04:13:37.520459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:23.130857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":63,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504291+0000","last_change":"2024-09-19T04:13:16.454512+0000","last_active":"2024-09-19T04:13:31.504291+0000","last_peered":"2024-09-19T04:13:31.504291+0000","last_clean":"2024-09-19T04:13:31.504291+0000","last_became_active":"2024-09-19T04:13:16.454193+0000","last_became_peered":"2024-09-19T04:13:16.454193+0000","last_unstale":"2024-09-19T04:13:31.504291+0000","last_undegraded":"2024-09-19T04:13:31.504291+0000","last_fullsized":"2024-09-19T04:13:31.504291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:11.825938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.515863+0000","last_change":"2024-09-19T04:13:31.515448+0000","last_active":"2024-09-19T04:13:31.515863+0000","last_peered":"2024-09-19T04:13:31.515863+0000","last_clean":"2024-09-19T04:13:31.515863+0000","last_became_active":"2024-09-19T04:13:31.506099+0000","last_became_peered":"2024-09-19T04:13:31.506099+0000","last_unstale":"2024-09-19T04:13:31.515863+0000","last_undegraded":"2024-09-19T04:13:31.515863+0000","last_fullsized":"2024-09-19T04:13:31.515863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:21.462488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498861+0000","last_change":"2024-09-19T04:13:16.447607+0000","last_active":"2024-09-19T04:13:31.498861+0000","last_peered":"2024-09-19T04:13:31.498861+0000","last_clean":"2024-09-19T04:13:31.498861+0000","last_became_active":"2024-09-19T04:13:16.447029+0000","last_became_peered":"2024-09-19T04:13:16.447029+0000","last_unstale":"2024-09-19T04:13:31.498861+0000","last_undegraded":"2024-09-19T04:13:31.498861+0000","last_fullsized":"2024-09-19T04:13:31.498861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:52.418104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505867+0000","last_change":"2024-09-19T04:13:14.428856+0000","last_active":"2024-09-19T04:13:31.505867+0000","last_peered":"2024-09-19T04:13:31.505867+0000","last_clean":"2024-09-19T04:13:31.505867+0000","last_became_active":"2024-09-19T04:13:14.428510+0000","last_became_peered":"2024-09-19T04:13:14.428510+0000","last_unstale":"2024-09-19T04:13:31.505867+0000","last_undegraded":"2024-09-19T04:13:31.505867+0000","last_fullsized":"2024-09-19T04:13:31.505867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:49.633325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501063+0000","last_change":"2024-09-19T04:13:16.455628+0000","last_active":"2024-09-19T04:13:31.501063+0000","last_peered":"2024-09-19T04:13:31.501063+0000","last_clean":"2024-09-19T04:13:31.501063+0000","last_became_active":"2024-09-19T04:13:16.453817+0000","last_became_peered":"2024-09-19T04:13:16.453817+0000","last_unstale":"2024-09-19T04:13:31.501063+0000","last_undegraded":"2024-09-19T04:13:31.501063+0000","last_fullsized":"2024-09-19T04:13:31.501063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.832806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'4","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:38.515074+0000","last_change":"2024-09-19T04:13:14.424924+0000","last_active":"2024-09-19T04:13:38.515074+0000","last_peered":"2024-09-19T04:13:38.515074+0000","last_clean":"2024-09-19T04:13:38.515074+0000","last_became_active":"2024-09-19T04:13:14.424627+0000","last_became_peered":"2024-09-19T04:13:14.424627+0000","last_unstale":"2024-09-19T04:13:38.515074+0000","last_undegraded":"2024-09-19T04:13:38.515074+0000","last_fullsized":"2024-09-19T04:13:38.515074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:49.830205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500550+0000","last_change":"2024-09-19T04:13:16.455365+0000","last_active":"2024-09-19T04:13:31.500550+0000","last_peered":"2024-09-19T04:13:31.500550+0000","last_clean":"2024-09-19T04:13:31.500550+0000","last_became_active":"2024-09-19T04:13:16.455029+0000","last_became_peered":"2024-09-19T04:13:16.455029+0000","last_unstale":"2024-09-19T04:13:31.500550+0000","last_undegraded":"2024-09-19T04:13:31.500550+0000","last_fullsized":"2024-09-19T04:13:31.500550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:20.947391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504041+0000","last_change":"2024-09-19T04:13:14.620462+0000","last_active":"2024-09-19T04:13:31.504041+0000","last_peered":"2024-09-19T04:13:31.504041+0000","last_clean":"2024-09-19T04:13:31.504041+0000","last_became_active":"2024-09-19T04:13:14.620103+0000","last_became_peered":"2024-09-19T04:13:14.620103+0000","last_unstale":"2024-09-19T04:13:31.504041+0000","last_undegraded":"2024-09-19T04:13:31.504041+0000","last_fullsized":"2024-09-19T04:13:31.504041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:14.496075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506534+0000","last_change":"2024-09-19T04:13:16.450222+0000","last_active":"2024-09-19T04:13:31.506534+0000","last_peered":"2024-09-19T04:13:31.506534+0000","last_clean":"2024-09-19T04:13:31.506534+0000","last_became_active":"2024-09-19T04:13:16.449916+0000","last_became_peered":"2024-09-19T04:13:16.449916+0000","last_unstale":"2024-09-19T04:13:31.506534+0000","last_undegraded":"2024-09-19T04:13:31.506534+0000","last_fullsized":"2024-09-19T04:13:31.506534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:31.886342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.519612+0000","last_change":"2024-09-19T04:13:31.519190+0000","last_active":"2024-09-19T04:13:31.519612+0000","last_peered":"2024-09-19T04:13:31.519612+0000","last_clean":"2024-09-19T04:13:31.519612+0000","last_became_active":"2024-09-19T04:13:31.508605+0000","last_became_peered":"2024-09-19T04:13:31.508605+0000","last_unstale":"2024-09-19T04:13:31.519612+0000","last_undegraded":"2024-09-19T04:13:31.519612+0000","last_fullsized":"2024-09-19T04:13:31.519612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:51.437135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502356+0000","last_change":"2024-09-19T04:13:16.450848+0000","last_active":"2024-09-19T04:13:31.502356+0000","last_peered":"2024-09-19T04:13:31.502356+0000","last_clean":"2024-09-19T04:13:31.502356+0000","last_became_active":"2024-09-19T04:13:16.450496+0000","last_became_peered":"2024-09-19T04:13:16.450496+0000","last_unstale":"2024-09-19T04:13:31.502356+0000","last_undegraded":"2024-09-19T04:13:31.502356+0000","last_fullsized":"2024-09-19T04:13:31.502356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:49:00.346998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499054+0000","last_change":"2024-09-19T04:13:14.428654+0000","last_active":"2024-09-19T04:13:31.499054+0000","last_peered":"2024-09-19T04:13:31.499054+0000","last_clean":"2024-09-19T04:13:31.499054+0000","last_became_active":"2024-09-19T04:13:14.428297+0000","last_became_peered":"2024-09-19T04:13:14.428297+0000","last_unstale":"2024-09-19T04:13:31.499054+0000","last_undegraded":"2024-09-19T04:13:31.499054+0000","last_fullsized":"2024-09-19T04:13:31.499054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:30.834262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501214+0000","last_change":"2024-09-19T04:13:16.456002+0000","last_active":"2024-09-19T04:13:31.501214+0000","last_peered":"2024-09-19T04:13:31.501214+0000","last_clean":"2024-09-19T04:13:31.501214+0000","last_became_active":"2024-09-19T04:13:16.455463+0000","last_became_peered":"2024-09-19T04:13:16.455463+0000","last_unstale":"2024-09-19T04:13:31.501214+0000","last_undegraded":"2024-09-19T04:13:31.501214+0000","last_fullsized":"2024-09-19T04:13:31.501214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:47.266364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505707+0000","last_change":"2024-09-19T04:13:14.429555+0000","last_active":"2024-09-19T04:13:31.505707+0000","last_peered":"2024-09-19T04:13:31.505707+0000","last_clean":"2024-09-19T04:13:31.505707+0000","last_became_active":"2024-09-19T04:13:14.429224+0000","last_became_peered":"2024-09-19T04:13:14.429224+0000","last_unstale":"2024-09-19T04:13:31.505707+0000","last_undegraded":"2024-09-19T04:13:31.505707+0000","last_fullsized":"2024-09-19T04:13:31.505707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:35.304750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501182+0000","last_change":"2024-09-19T04:13:16.454337+0000","last_active":"2024-09-19T04:13:31.501182+0000","last_peered":"2024-09-19T04:13:31.501182+0000","last_clean":"2024-09-19T04:13:31.501182+0000","last_became_active":"2024-09-19T04:13:16.453831+0000","last_became_peered":"2024-09-19T04:13:16.453831+0000","last_unstale":"2024-09-19T04:13:31.501182+0000","last_undegraded":"2024-09-19T04:13:31.501182+0000","last_fullsized":"2024-09-19T04:13:31.501182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:41.751933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506399+0000","last_change":"2024-09-19T04:13:14.620406+0000","last_active":"2024-09-19T04:13:31.506399+0000","last_peered":"2024-09-19T04:13:31.506399+0000","last_clean":"2024-09-19T04:13:31.506399+0000","last_became_active":"2024-09-19T04:13:14.620050+0000","last_became_peered":"2024-09-19T04:13:14.620050+0000","last_unstale":"2024-09-19T04:13:31.506399+0000","last_undegraded":"2024-09-19T04:13:31.506399+0000","last_fullsized":"2024-09-19T04:13:31.506399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:35.828760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501221+0000","last_change":"2024-09-19T04:13:16.459016+0000","last_active":"2024-09-19T04:13:31.501221+0000","last_peered":"2024-09-19T04:13:31.501221+0000","last_clean":"2024-09-19T04:13:31.501221+0000","last_became_active":"2024-09-19T04:13:16.458674+0000","last_became_peered":"2024-09-19T04:13:16.458674+0000","last_unstale":"2024-09-19T04:13:31.501221+0000","last_undegraded":"2024-09-19T04:13:31.501221+0000","last_fullsized":"2024-09-19T04:13:31.501221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:55.894456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":346,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500892+0000","last_change":"2024-09-19T04:12:05.913704+0000","last_active":"2024-09-19T04:13:31.500892+0000","last_peered":"2024-09-19T04:13:31.500892+0000","last_clean":"2024-09-19T04:13:31.500892+0000","last_became_active":"2024-09-19T04:12:05.913299+0000","last_became_peered":"2024-09-19T04:12:05.913299+0000","last_unstale":"2024-09-19T04:13:31.500892+0000","last_undegraded":"2024-09-19T04:13:31.500892+0000","last_fullsized":"2024-09-19T04:13:31.500892+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_clean_scrub_stamp":"2024-09-19T04:08:46.646000+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.638160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504518+0000","last_change":"2024-09-19T04:13:14.423962+0000","last_active":"2024-09-19T04:13:31.504518+0000","last_peered":"2024-09-19T04:13:31.504518+0000","last_clean":"2024-09-19T04:13:31.504518+0000","last_became_active":"2024-09-19T04:13:14.423591+0000","last_became_peered":"2024-09-19T04:13:14.423591+0000","last_unstale":"2024-09-19T04:13:31.504518+0000","last_undegraded":"2024-09-19T04:13:31.504518+0000","last_fullsized":"2024-09-19T04:13:31.504518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:02.239975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501743+0000","last_change":"2024-09-19T04:13:16.447113+0000","last_active":"2024-09-19T04:13:31.501743+0000","last_peered":"2024-09-19T04:13:31.501743+0000","last_clean":"2024-09-19T04:13:31.501743+0000","last_became_active":"2024-09-19T04:13:16.446792+0000","last_became_peered":"2024-09-19T04:13:16.446792+0000","last_unstale":"2024-09-19T04:13:31.501743+0000","last_undegraded":"2024-09-19T04:13:31.501743+0000","last_fullsized":"2024-09-19T04:13:31.501743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:28:21.527010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500749+0000","last_change":"2024-09-19T04:13:14.432663+0000","last_active":"2024-09-19T04:13:31.500749+0000","last_peered":"2024-09-19T04:13:31.500749+0000","last_clean":"2024-09-19T04:13:31.500749+0000","last_became_active":"2024-09-19T04:13:14.432318+0000","last_became_peered":"2024-09-19T04:13:14.432318+0000","last_unstale":"2024-09-19T04:13:31.500749+0000","last_undegraded":"2024-09-19T04:13:31.500749+0000","last_fullsized":"2024-09-19T04:13:31.500749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:35.787596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498761+0000","last_change":"2024-09-19T04:13:16.451010+0000","last_active":"2024-09-19T04:13:31.498761+0000","last_peered":"2024-09-19T04:13:31.498761+0000","last_clean":"2024-09-19T04:13:31.498761+0000","last_became_active":"2024-09-19T04:13:16.450708+0000","last_became_peered":"2024-09-19T04:13:16.450708+0000","last_unstale":"2024-09-19T04:13:31.498761+0000","last_undegraded":"2024-09-19T04:13:31.498761+0000","last_fullsized":"2024-09-19T04:13:31.498761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:30.108541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:37.513030+0000","last_change":"2024-09-19T04:13:14.620037+0000","last_active":"2024-09-19T04:13:37.513030+0000","last_peered":"2024-09-19T04:13:37.513030+0000","last_clean":"2024-09-19T04:13:37.513030+0000","last_became_active":"2024-09-19T04:13:14.619686+0000","last_became_peered":"2024-09-19T04:13:14.619686+0000","last_unstale":"2024-09-19T04:13:37.513030+0000","last_undegraded":"2024-09-19T04:13:37.513030+0000","last_fullsized":"2024-09-19T04:13:37.513030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:32.490708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504838+0000","last_change":"2024-09-19T04:13:16.446429+0000","last_active":"2024-09-19T04:13:31.504838+0000","last_peered":"2024-09-19T04:13:31.504838+0000","last_clean":"2024-09-19T04:13:31.504838+0000","last_became_active":"2024-09-19T04:13:16.446075+0000","last_became_peered":"2024-09-19T04:13:16.446075+0000","last_unstale":"2024-09-19T04:13:31.504838+0000","last_undegraded":"2024-09-19T04:13:31.504838+0000","last_fullsized":"2024-09-19T04:13:31.504838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:03.613926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501031+0000","last_change":"2024-09-19T04:13:14.429899+0000","last_active":"2024-09-19T04:13:31.501031+0000","last_peered":"2024-09-19T04:13:31.501031+0000","last_clean":"2024-09-19T04:13:31.501031+0000","last_became_active":"2024-09-19T04:13:14.429571+0000","last_became_peered":"2024-09-19T04:13:14.429571+0000","last_unstale":"2024-09-19T04:13:31.501031+0000","last_undegraded":"2024-09-19T04:13:31.501031+0000","last_fullsized":"2024-09-19T04:13:31.501031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:37.017431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499733+0000","last_change":"2024-09-19T04:13:16.458267+0000","last_active":"2024-09-19T04:13:31.499733+0000","last_peered":"2024-09-19T04:13:31.499733+0000","last_clean":"2024-09-19T04:13:31.499733+0000","last_became_active":"2024-09-19T04:13:16.457911+0000","last_became_peered":"2024-09-19T04:13:16.457911+0000","last_unstale":"2024-09-19T04:13:31.499733+0000","last_undegraded":"2024-09-19T04:13:31.499733+0000","last_fullsized":"2024-09-19T04:13:31.499733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:32.465923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501677+0000","last_change":"2024-09-19T04:13:14.621552+0000","last_active":"2024-09-19T04:13:31.501677+0000","last_peered":"2024-09-19T04:13:31.501677+0000","last_clean":"2024-09-19T04:13:31.501677+0000","last_became_active":"2024-09-19T04:13:14.620592+0000","last_became_peered":"2024-09-19T04:13:14.620592+0000","last_unstale":"2024-09-19T04:13:31.501677+0000","last_undegraded":"2024-09-19T04:13:31.501677+0000","last_fullsized":"2024-09-19T04:13:31.501677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:10.055202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500150+0000","last_change":"2024-09-19T04:13:16.443117+0000","last_active":"2024-09-19T04:13:31.500150+0000","last_peered":"2024-09-19T04:13:31.500150+0000","last_clean":"2024-09-19T04:13:31.500150+0000","last_became_active":"2024-09-19T04:13:16.442808+0000","last_became_peered":"2024-09-19T04:13:16.442808+0000","last_unstale":"2024-09-19T04:13:31.500150+0000","last_undegraded":"2024-09-19T04:13:31.500150+0000","last_fullsized":"2024-09-19T04:13:31.500150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:40.274320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504531+0000","last_change":"2024-09-19T04:13:14.620618+0000","last_active":"2024-09-19T04:13:31.504531+0000","last_peered":"2024-09-19T04:13:31.504531+0000","last_clean":"2024-09-19T04:13:31.504531+0000","last_became_active":"2024-09-19T04:13:14.620290+0000","last_became_peered":"2024-09-19T04:13:14.620290+0000","last_unstale":"2024-09-19T04:13:31.504531+0000","last_undegraded":"2024-09-19T04:13:31.504531+0000","last_fullsized":"2024-09-19T04:13:31.504531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:51.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499218+0000","last_change":"2024-09-19T04:13:16.443498+0000","last_active":"2024-09-19T04:13:31.499218+0000","last_peered":"2024-09-19T04:13:31.499218+0000","last_clean":"2024-09-19T04:13:31.499218+0000","last_became_active":"2024-09-19T04:13:16.443104+0000","last_became_peered":"2024-09-19T04:13:16.443104+0000","last_unstale":"2024-09-19T04:13:31.499218+0000","last_undegraded":"2024-09-19T04:13:31.499218+0000","last_fullsized":"2024-09-19T04:13:31.499218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:10.733570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:35.455482+0000","last_change":"2024-09-19T04:13:14.425881+0000","last_active":"2024-09-19T04:13:35.455482+0000","last_peered":"2024-09-19T04:13:35.455482+0000","last_clean":"2024-09-19T04:13:35.455482+0000","last_became_active":"2024-09-19T04:13:14.425542+0000","last_became_peered":"2024-09-19T04:13:14.425542+0000","last_unstale":"2024-09-19T04:13:35.455482+0000","last_undegraded":"2024-09-19T04:13:35.455482+0000","last_fullsized":"2024-09-19T04:13:35.455482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:44.250890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502301+0000","last_change":"2024-09-19T04:13:16.454672+0000","last_active":"2024-09-19T04:13:31.502301+0000","last_peered":"2024-09-19T04:13:31.502301+0000","last_clean":"2024-09-19T04:13:31.502301+0000","last_became_active":"2024-09-19T04:13:16.454003+0000","last_became_peered":"2024-09-19T04:13:16.454003+0000","last_unstale":"2024-09-19T04:13:31.502301+0000","last_undegraded":"2024-09-19T04:13:31.502301+0000","last_fullsized":"2024-09-19T04:13:31.502301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:26.849617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504465+0000","last_change":"2024-09-19T04:13:14.432312+0000","last_active":"2024-09-19T04:13:31.504465+0000","last_peered":"2024-09-19T04:13:31.504465+0000","last_clean":"2024-09-19T04:13:31.504465+0000","last_became_active":"2024-09-19T04:13:14.431994+0000","last_became_peered":"2024-09-19T04:13:14.431994+0000","last_unstale":"2024-09-19T04:13:31.504465+0000","last_undegraded":"2024-09-19T04:13:31.504465+0000","last_fullsized":"2024-09-19T04:13:31.504465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:00.410400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735768+0000","last_change":"2024-09-19T04:13:16.455312+0000","last_active":"2024-09-19T04:13:31.735768+0000","last_peered":"2024-09-19T04:13:31.735768+0000","last_clean":"2024-09-19T04:13:31.735768+0000","last_became_active":"2024-09-19T04:13:16.454990+0000","last_became_peered":"2024-09-19T04:13:16.454990+0000","last_unstale":"2024-09-19T04:13:31.735768+0000","last_undegraded":"2024-09-19T04:13:31.735768+0000","last_fullsized":"2024-09-19T04:13:31.735768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:09.207395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700505+0000","last_change":"2024-09-19T04:13:14.620316+0000","last_active":"2024-09-19T04:13:31.700505+0000","last_peered":"2024-09-19T04:13:31.700505+0000","last_clean":"2024-09-19T04:13:31.700505+0000","last_became_active":"2024-09-19T04:13:14.619979+0000","last_became_peered":"2024-09-19T04:13:14.619979+0000","last_unstale":"2024-09-19T04:13:31.700505+0000","last_undegraded":"2024-09-19T04:13:31.700505+0000","last_fullsized":"2024-09-19T04:13:31.700505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:24.382144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505808+0000","last_change":"2024-09-19T04:13:16.455684+0000","last_active":"2024-09-19T04:13:31.505808+0000","last_peered":"2024-09-19T04:13:31.505808+0000","last_clean":"2024-09-19T04:13:31.505808+0000","last_became_active":"2024-09-19T04:13:16.455355+0000","last_became_peered":"2024-09-19T04:13:16.455355+0000","last_unstale":"2024-09-19T04:13:31.505808+0000","last_undegraded":"2024-09-19T04:13:31.505808+0000","last_fullsized":"2024-09-19T04:13:31.505808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:09.657447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501550+0000","last_change":"2024-09-19T04:13:14.423599+0000","last_active":"2024-09-19T04:13:31.501550+0000","last_peered":"2024-09-19T04:13:31.501550+0000","last_clean":"2024-09-19T04:13:31.501550+0000","last_became_active":"2024-09-19T04:13:14.423282+0000","last_became_peered":"2024-09-19T04:13:14.423282+0000","last_unstale":"2024-09-19T04:13:31.501550+0000","last_undegraded":"2024-09-19T04:13:31.501550+0000","last_fullsized":"2024-09-19T04:13:31.501550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.975368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501564+0000","last_change":"2024-09-19T04:13:16.450435+0000","last_active":"2024-09-19T04:13:31.501564+0000","last_peered":"2024-09-19T04:13:31.501564+0000","last_clean":"2024-09-19T04:13:31.501564+0000","last_became_active":"2024-09-19T04:13:16.450087+0000","last_became_peered":"2024-09-19T04:13:16.450087+0000","last_unstale":"2024-09-19T04:13:31.501564+0000","last_undegraded":"2024-09-19T04:13:31.501564+0000","last_fullsized":"2024-09-19T04:13:31.501564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:34:51.764776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504466+0000","last_change":"2024-09-19T04:13:14.432868+0000","last_active":"2024-09-19T04:13:31.504466+0000","last_peered":"2024-09-19T04:13:31.504466+0000","last_clean":"2024-09-19T04:13:31.504466+0000","last_became_active":"2024-09-19T04:13:14.432507+0000","last_became_peered":"2024-09-19T04:13:14.432507+0000","last_unstale":"2024-09-19T04:13:31.504466+0000","last_undegraded":"2024-09-19T04:13:31.504466+0000","last_fullsized":"2024-09-19T04:13:31.504466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:14.773711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501610+0000","last_change":"2024-09-19T04:13:16.450061+0000","last_active":"2024-09-19T04:13:31.501610+0000","last_peered":"2024-09-19T04:13:31.501610+0000","last_clean":"2024-09-19T04:13:31.501610+0000","last_became_active":"2024-09-19T04:13:16.448829+0000","last_became_peered":"2024-09-19T04:13:16.448829+0000","last_unstale":"2024-09-19T04:13:31.501610+0000","last_undegraded":"2024-09-19T04:13:31.501610+0000","last_fullsized":"2024-09-19T04:13:31.501610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:31.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506029+0000","last_change":"2024-09-19T04:13:14.431574+0000","last_active":"2024-09-19T04:13:31.506029+0000","last_peered":"2024-09-19T04:13:31.506029+0000","last_clean":"2024-09-19T04:13:31.506029+0000","last_became_active":"2024-09-19T04:13:14.431202+0000","last_became_peered":"2024-09-19T04:13:14.431202+0000","last_unstale":"2024-09-19T04:13:31.506029+0000","last_undegraded":"2024-09-19T04:13:31.506029+0000","last_fullsized":"2024-09-19T04:13:31.506029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:58.167201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500887+0000","last_change":"2024-09-19T04:13:16.452688+0000","last_active":"2024-09-19T04:13:31.500887+0000","last_peered":"2024-09-19T04:13:31.500887+0000","last_clean":"2024-09-19T04:13:31.500887+0000","last_became_active":"2024-09-19T04:13:16.452342+0000","last_became_peered":"2024-09-19T04:13:16.452342+0000","last_unstale":"2024-09-19T04:13:31.500887+0000","last_undegraded":"2024-09-19T04:13:31.500887+0000","last_fullsized":"2024-09-19T04:13:31.500887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:03.162688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504711+0000","last_change":"2024-09-19T04:13:16.449402+0000","last_active":"2024-09-19T04:13:31.504711+0000","last_peered":"2024-09-19T04:13:31.504711+0000","last_clean":"2024-09-19T04:13:31.504711+0000","last_became_active":"2024-09-19T04:13:16.449108+0000","last_became_peered":"2024-09-19T04:13:16.449108+0000","last_unstale":"2024-09-19T04:13:31.504711+0000","last_undegraded":"2024-09-19T04:13:31.504711+0000","last_fullsized":"2024-09-19T04:13:31.504711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:58.874638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505684+0000","last_change":"2024-09-19T04:13:14.430684+0000","last_active":"2024-09-19T04:13:31.505684+0000","last_peered":"2024-09-19T04:13:31.505684+0000","last_clean":"2024-09-19T04:13:31.505684+0000","last_became_active":"2024-09-19T04:13:14.430137+0000","last_became_peered":"2024-09-19T04:13:14.430137+0000","last_unstale":"2024-09-19T04:13:31.505684+0000","last_undegraded":"2024-09-19T04:13:31.505684+0000","last_fullsized":"2024-09-19T04:13:31.505684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:39.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700648+0000","last_change":"2024-09-19T04:13:16.452491+0000","last_active":"2024-09-19T04:13:31.700648+0000","last_peered":"2024-09-19T04:13:31.700648+0000","last_clean":"2024-09-19T04:13:31.700648+0000","last_became_active":"2024-09-19T04:13:16.452204+0000","last_became_peered":"2024-09-19T04:13:16.452204+0000","last_unstale":"2024-09-19T04:13:31.700648+0000","last_undegraded":"2024-09-19T04:13:31.700648+0000","last_fullsized":"2024-09-19T04:13:31.700648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:10.113510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504401+0000","last_change":"2024-09-19T04:13:14.431608+0000","last_active":"2024-09-19T04:13:31.504401+0000","last_peered":"2024-09-19T04:13:31.504401+0000","last_clean":"2024-09-19T04:13:31.504401+0000","last_became_active":"2024-09-19T04:13:14.431111+0000","last_became_peered":"2024-09-19T04:13:14.431111+0000","last_unstale":"2024-09-19T04:13:31.504401+0000","last_undegraded":"2024-09-19T04:13:31.504401+0000","last_fullsized":"2024-09-19T04:13:31.504401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:03:41.125315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735802+0000","last_change":"2024-09-19T04:13:16.457867+0000","last_active":"2024-09-19T04:13:31.735802+0000","last_peered":"2024-09-19T04:13:31.735802+0000","last_clean":"2024-09-19T04:13:31.735802+0000","last_became_active":"2024-09-19T04:13:16.457582+0000","last_became_peered":"2024-09-19T04:13:16.457582+0000","last_unstale":"2024-09-19T04:13:31.735802+0000","last_undegraded":"2024-09-19T04:13:31.735802+0000","last_fullsized":"2024-09-19T04:13:31.735802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:58.076250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501886+0000","last_change":"2024-09-19T04:13:14.620070+0000","last_active":"2024-09-19T04:13:31.501886+0000","last_peered":"2024-09-19T04:13:31.501886+0000","last_clean":"2024-09-19T04:13:31.501886+0000","last_became_active":"2024-09-19T04:13:14.619734+0000","last_became_peered":"2024-09-19T04:13:14.619734+0000","last_unstale":"2024-09-19T04:13:31.501886+0000","last_undegraded":"2024-09-19T04:13:31.501886+0000","last_fullsized":"2024-09-19T04:13:31.501886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:44:02.818859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501786+0000","last_change":"2024-09-19T04:13:16.443832+0000","last_active":"2024-09-19T04:13:31.501786+0000","last_peered":"2024-09-19T04:13:31.501786+0000","last_clean":"2024-09-19T04:13:31.501786+0000","last_became_active":"2024-09-19T04:13:16.442690+0000","last_became_peered":"2024-09-19T04:13:16.442690+0000","last_unstale":"2024-09-19T04:13:31.501786+0000","last_undegraded":"2024-09-19T04:13:31.501786+0000","last_fullsized":"2024-09-19T04:13:31.501786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:39.535628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501657+0000","last_change":"2024-09-19T04:13:14.427186+0000","last_active":"2024-09-19T04:13:31.501657+0000","last_peered":"2024-09-19T04:13:31.501657+0000","last_clean":"2024-09-19T04:13:31.501657+0000","last_became_active":"2024-09-19T04:13:14.426949+0000","last_became_peered":"2024-09-19T04:13:14.426949+0000","last_unstale":"2024-09-19T04:13:31.501657+0000","last_undegraded":"2024-09-19T04:13:31.501657+0000","last_fullsized":"2024-09-19T04:13:31.501657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:26.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54508,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":101,"num_read_kb":107,"num_write":153,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":163524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":431409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1073339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005083,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40388,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704764,"statfs":{"total":95995035648,"available":95953678336,"internally_reserved":0,"allocated":905216,"data_stored":431589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":431487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1023834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":892928,"data_stored":430978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1511424,"data_stored":1023125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986228,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":431808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215161,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":431289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476798,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":482592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607619,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":480883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":913408,"data_stored":432249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":50112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":51821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:13:43.699 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T04:13:43.700 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T04:13:43.700 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T04:13:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:44.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:43 smithi112 ceph-mon[35051]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:13:44.068 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:43 smithi094 ceph-mon[30972]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:13:44.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:43 smithi104 ceph-mon[34145]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:13:44.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 -- 172.21.15.94:0/911513025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18107960 msgr2=0x7f5c18109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/911513025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18107960 0x7f5c18109d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009ca0 tx=0x7f5c1002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 -- 172.21.15.94:0/911513025 shutdown_connections 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/911513025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18107960 0x7f5c18109d50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/911513025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c18105030 0x7f5c18107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/911513025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c18102700 0x7f5c18104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 -- 172.21.15.94:0/911513025 >> 172.21.15.94:0/911513025 conn(0x7f5c180fc2a0 msgr2=0x7f5c180fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 -- 172.21.15.94:0/911513025 shutdown_connections 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 -- 172.21.15.94:0/911513025 wait complete. 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 Processor -- start 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.450+0000 7f5c1e48a640 1 -- start start 2024-09-19T04:13:44.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c1e48a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18102700 0x7f5c18197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c1e48a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c18105030 0x7f5c181983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c1e48a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c18107960 0x7f5c1819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c1e48a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1810d7d0 con 0x7f5c18102700 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c1e48a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1810d650 con 0x7f5c18105030 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c1e48a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1810d950 con 0x7f5c18107960 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18102700 0x7f5c18197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18102700 0x7f5c18197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42084/0 (socket says 172.21.15.94:42084) 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c17fff640 1 -- 172.21.15.94:0/3068504451 learned_addr learned my addr 172.21.15.94:0/3068504451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c177fe640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c18105030 0x7f5c181983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c1ca27640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c18107960 0x7f5c1819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c17fff640 1 -- 172.21.15.94:0/3068504451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c18107960 msgr2=0x7f5c1819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c17fff640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c18107960 0x7f5c1819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c17fff640 1 -- 172.21.15.94:0/3068504451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c18105030 msgr2=0x7f5c181983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c17fff640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c18105030 0x7f5c181983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c17fff640 1 -- 172.21.15.94:0/3068504451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1819fa90 con 0x7f5c18102700 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c1ca27640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c18107960 0x7f5c1819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c177fe640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c18105030 0x7f5c181983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c17fff640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18102700 0x7f5c18197e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800b4a0 tx=0x7f5c0800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.451+0000 7f5c157fa640 1 -- 172.21.15.94:0/3068504451 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08014070 con 0x7f5c18102700 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.452+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1819fd80 con 0x7f5c18102700 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.452+0000 7f5c157fa640 1 -- 172.21.15.94:0/3068504451 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c08010ac0 con 0x7f5c18102700 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.452+0000 7f5c157fa640 1 -- 172.21.15.94:0/3068504451 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08017c30 con 0x7f5c18102700 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.452+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c181a0200 con 0x7f5c18102700 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.453+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c180fdd90 con 0x7f5c18102700 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.454+0000 7f5c157fa640 1 -- 172.21.15.94:0/3068504451 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5c08017070 con 0x7f5c18102700 2024-09-19T04:13:44.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.454+0000 7f5c157fa640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5c000a3300 0x7f5c000a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.454+0000 7f5c177fe640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5c000a3300 0x7f5c000a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.454+0000 7f5c157fa640 1 -- 172.21.15.94:0/3068504451 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5c080cee00 con 0x7f5c18102700 2024-09-19T04:13:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.455+0000 7f5c177fe640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5c000a3300 0x7f5c000a57c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0058c0 tx=0x7f5c0c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.458+0000 7f5c157fa640 1 -- 172.21.15.94:0/3068504451 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5c0809bad0 con 0x7f5c18102700 2024-09-19T04:13:44.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.687+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f5c18062df0 con 0x7f5c18102700 2024-09-19T04:13:44.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.689+0000 7f5c157fa640 1 -- 172.21.15.94:0/3068504451 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f5c0809b3d0 con 0x7f5c18102700 2024-09-19T04:13:44.690 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-19T04:13:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQD4pOtmwD0MKRAADG+7mjTzFFVAP/KBrGI+3g== 2024-09-19T04:13:44.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5c000a3300 msgr2=0x7f5c000a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5c000a3300 0x7f5c000a57c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0058c0 tx=0x7f5c0c005850 comp rx=0 tx=0).stop 2024-09-19T04:13:44.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18102700 msgr2=0x7f5c18197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18102700 0x7f5c18197e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800b4a0 tx=0x7f5c0800b960 comp rx=0 tx=0).stop 2024-09-19T04:13:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 shutdown_connections 2024-09-19T04:13:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5c000a3300 0x7f5c000a57c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c18107960 0x7f5c1819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c18105030 0x7f5c181983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 --2- 172.21.15.94:0/3068504451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c18102700 0x7f5c18197e60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 >> 172.21.15.94:0/3068504451 conn(0x7f5c180fc2a0 msgr2=0x7f5c18102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 shutdown_connections 2024-09-19T04:13:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:44.692+0000 7f5c1e48a640 1 -- 172.21.15.94:0/3068504451 wait complete. 2024-09-19T04:13:45.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:44 smithi094 ceph-mon[30972]: pgmap v288: 97 pgs: 97 active+clean; 634 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:13:45.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:44 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3068504451' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:13:45.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:44 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3068504451' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:13:45.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:44 smithi104 ceph-mon[34145]: pgmap v288: 97 pgs: 97 active+clean; 634 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:13:45.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:44 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3068504451' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:13:45.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:44 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3068504451' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:13:45.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:44 smithi112 ceph-mon[35051]: pgmap v288: 97 pgs: 97 active+clean; 634 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:13:45.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:44 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3068504451' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:13:45.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:44 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3068504451' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:13:45.420 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:13:45.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T04:13:45.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T04:13:45.456 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T04:13:45.456 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T04:13:45.456 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T04:13:45.739 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:46.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 -- 172.21.15.94:0/1464957687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102ef0 msgr2=0x7f245c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 --2- 172.21.15.94:0/1464957687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102ef0 0x7f245c10da40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2454009ca0 tx=0x7f245402a950 comp rx=0 tx=0).stop 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 -- 172.21.15.94:0/1464957687 shutdown_connections 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 --2- 172.21.15.94:0/1464957687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102ef0 0x7f245c10da40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 --2- 172.21.15.94:0/1464957687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102570 0x7f245c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 --2- 172.21.15.94:0/1464957687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c100e20 0x7f245c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 -- 172.21.15.94:0/1464957687 >> 172.21.15.94:0/1464957687 conn(0x7f245c0fc7d0 msgr2=0x7f245c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 -- 172.21.15.94:0/1464957687 shutdown_connections 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 -- 172.21.15.94:0/1464957687 wait complete. 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 Processor -- start 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.209+0000 7f2461d6c640 1 -- start start 2024-09-19T04:13:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2461d6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c100e20 0x7f245c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2461d6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102570 0x7f245c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2461d6c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c102ef0 0x7f245c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2461d6c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f245c110e50 con 0x7f245c102570 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2461d6c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f245c110cd0 con 0x7f245c102ef0 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2461d6c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f245c110fd0 con 0x7f245c100e20 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102570 0x7f245c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102570 0x7f245c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42092/0 (socket says 172.21.15.94:42092) 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245affd640 1 -- 172.21.15.94:0/3765925707 learned_addr learned my addr 172.21.15.94:0/3765925707 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245b7fe640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c100e20 0x7f245c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245bfff640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c102ef0 0x7f245c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245affd640 1 -- 172.21.15.94:0/3765925707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c100e20 msgr2=0x7f245c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245affd640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c100e20 0x7f245c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245affd640 1 -- 172.21.15.94:0/3765925707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c102ef0 msgr2=0x7f245c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245affd640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c102ef0 0x7f245c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245affd640 1 -- 172.21.15.94:0/3765925707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c19d3c0 con 0x7f245c102570 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245b7fe640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c100e20 0x7f245c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f245affd640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102570 0x7f245c198950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f245000c890 tx=0x7f245000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2458ff9640 1 -- 172.21.15.94:0/3765925707 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450004220 con 0x7f245c102570 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c19d6b0 con 0x7f245c102570 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2458ff9640 1 -- 172.21.15.94:0/3765925707 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24500043c0 con 0x7f245c102570 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2458ff9640 1 -- 172.21.15.94:0/3765925707 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450004a00 con 0x7f245c102570 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.210+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c1a4f90 con 0x7f245c102570 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.211+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c101200 con 0x7f245c102570 2024-09-19T04:13:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.212+0000 7f2458ff9640 1 -- 172.21.15.94:0/3765925707 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2450004ba0 con 0x7f245c102570 2024-09-19T04:13:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.213+0000 7f2458ff9640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24440a32d0 0x7f24440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.213+0000 7f245b7fe640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24440a32d0 0x7f24440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.213+0000 7f2458ff9640 1 -- 172.21.15.94:0/3765925707 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f24500cf810 con 0x7f245c102570 2024-09-19T04:13:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.214+0000 7f245b7fe640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24440a32d0 0x7f24440a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f244c009930 tx=0x7f244c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.217+0000 7f2458ff9640 1 -- 172.21.15.94:0/3765925707 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2450014030 con 0x7f245c102570 2024-09-19T04:13:46.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.429+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f245c10a0d0 con 0x7f245c102570 2024-09-19T04:13:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.434+0000 7f2458ff9640 1 -- 172.21.15.94:0/3765925707 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264049 (secure 0 0 0) 0x7f245009c3d0 con 0x7f245c102570 2024-09-19T04:13:46.439 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24440a32d0 msgr2=0x7f24440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24440a32d0 0x7f24440a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f244c009930 tx=0x7f244c008040 comp rx=0 tx=0).stop 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102570 msgr2=0x7f245c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102570 0x7f245c198950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f245000c890 tx=0x7f245000cd50 comp rx=0 tx=0).stop 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 shutdown_connections 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24440a32d0 0x7f24440a5790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c102ef0 0x7f245c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c102570 0x7f245c198950 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 --2- 172.21.15.94:0/3765925707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c100e20 0x7f245c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 >> 172.21.15.94:0/3765925707 conn(0x7f245c0fc7d0 msgr2=0x7f245c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 shutdown_connections 2024-09-19T04:13:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:46.437+0000 7f2461d6c640 1 -- 172.21.15.94:0/3765925707 wait complete. 2024-09-19T04:13:47.191 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2308569937},{"type":"v1","addr":"172.21.15.94:6801","nonce":2308569937}]},"active_addr":"172.21.15.94:6801/2308569937","active_change":"2024-09-19T04:06:30.053656+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.94:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3284064221}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3123077951}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3663117113}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1265443753}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":860358556}]}]} 2024-09-19T04:13:47.197 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T04:13:47.197 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T04:13:47.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:13:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:47 smithi112 ceph-mon[35051]: pgmap v289: 97 pgs: 97 active+clean; 634 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-19T04:13:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:47 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3765925707' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:13:47.565 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:47 smithi094 ceph-mon[30972]: pgmap v289: 97 pgs: 97 active+clean; 634 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-19T04:13:47.565 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:47 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3765925707' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:13:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:47 smithi104 ceph-mon[34145]: pgmap v289: 97 pgs: 97 active+clean; 634 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-19T04:13:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:47 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3765925707' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:13:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 -- 172.21.15.94:0/1930540880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40100a80 msgr2=0x7fee40100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 --2- 172.21.15.94:0/1930540880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40100a80 0x7fee40100ee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fee34009930 tx=0x7fee3402a420 comp rx=0 tx=0).stop 2024-09-19T04:13:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 -- 172.21.15.94:0/1930540880 shutdown_connections 2024-09-19T04:13:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 --2- 172.21.15.94:0/1930540880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee40101420 0x7fee4010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 --2- 172.21.15.94:0/1930540880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40100a80 0x7fee40100ee0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 --2- 172.21.15.94:0/1930540880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee40106a80 0x7fee40106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 -- 172.21.15.94:0/1930540880 >> 172.21.15.94:0/1930540880 conn(0x7fee400fc7a0 msgr2=0x7fee400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 -- 172.21.15.94:0/1930540880 shutdown_connections 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 -- 172.21.15.94:0/1930540880 wait complete. 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 Processor -- start 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.907+0000 7fee46511640 1 -- start start 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee46511640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee40100a80 0x7fee401972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee46511640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40101420 0x7fee401977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee46511640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee40106a80 0x7fee4019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee46511640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee40110d50 con 0x7fee40101420 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee46511640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee40110bd0 con 0x7fee40100a80 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee46511640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee40110ed0 con 0x7fee40106a80 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40101420 0x7fee401977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40101420 0x7fee401977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42118/0 (socket says 172.21.15.94:42118) 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3f7fe640 1 -- 172.21.15.94:0/3196354353 learned_addr learned my addr 172.21.15.94:0/3196354353 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee44aae640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee40106a80 0x7fee4019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3ffff640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee40100a80 0x7fee401972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3f7fe640 1 -- 172.21.15.94:0/3196354353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee40106a80 msgr2=0x7fee4019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3f7fe640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee40106a80 0x7fee4019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3f7fe640 1 -- 172.21.15.94:0/3196354353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee40100a80 msgr2=0x7fee401972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3f7fe640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee40100a80 0x7fee401972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3f7fe640 1 -- 172.21.15.94:0/3196354353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4019c260 con 0x7fee40101420 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee44aae640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee40106a80 0x7fee4019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.908+0000 7fee3f7fe640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40101420 0x7fee401977f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fee34005940 tx=0x7fee340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.909+0000 7fee3d7fa640 1 -- 172.21.15.94:0/3196354353 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3403f030 con 0x7fee40101420 2024-09-19T04:13:47.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.909+0000 7fee3d7fa640 1 -- 172.21.15.94:0/3196354353 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee34003cb0 con 0x7fee40101420 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.909+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4019c4f0 con 0x7fee40101420 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.909+0000 7fee3d7fa640 1 -- 172.21.15.94:0/3196354353 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34035070 con 0x7fee40101420 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.909+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee401a3e30 con 0x7fee40101420 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.910+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee40100ee0 con 0x7fee40101420 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.911+0000 7fee3d7fa640 1 -- 172.21.15.94:0/3196354353 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fee34042030 con 0x7fee40101420 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.911+0000 7fee3d7fa640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fee280a32d0 0x7fee280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.911+0000 7fee3ffff640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fee280a32d0 0x7fee280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.912+0000 7fee3d7fa640 1 -- 172.21.15.94:0/3196354353 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fee340efff0 con 0x7fee40101420 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.912+0000 7fee3ffff640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fee280a32d0 0x7fee280a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee30009990 tx=0x7fee30009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:47.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:47.915+0000 7fee3d7fa640 1 -- 172.21.15.94:0/3196354353 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fee340bcbb0 con 0x7fee40101420 2024-09-19T04:13:48.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.083+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee40109070 con 0x7fee40101420 2024-09-19T04:13:48.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.085+0000 7fee3d7fa640 1 -- 172.21.15.94:0/3196354353 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22842 (secure 0 0 0) 0x7fee340bc4b0 con 0x7fee40101420 2024-09-19T04:13:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:48.086 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:13:31.495964+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:48.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fee280a32d0 msgr2=0x7fee280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:48.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fee280a32d0 0x7fee280a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee30009990 tx=0x7fee30009110 comp rx=0 tx=0).stop 2024-09-19T04:13:48.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40101420 msgr2=0x7fee401977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:48.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40101420 0x7fee401977f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fee34005940 tx=0x7fee340042f0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 shutdown_connections 2024-09-19T04:13:48.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fee280a32d0 0x7fee280a5790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee40106a80 0x7fee4019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee40101420 0x7fee401977f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 --2- 172.21.15.94:0/3196354353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee40100a80 0x7fee401972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 >> 172.21.15.94:0/3196354353 conn(0x7fee400fc7a0 msgr2=0x7fee400fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:48.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 shutdown_connections 2024-09-19T04:13:48.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:48.087+0000 7fee46511640 1 -- 172.21.15.94:0/3196354353 wait complete. 2024-09-19T04:13:48.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:48 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3196354353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:48 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3196354353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:48 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3196354353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:48.821 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T04:13:48.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:13:49.068 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:49.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:49 smithi094 ceph-mon[30972]: pgmap v290: 97 pgs: 97 active+clean; 634 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-19T04:13:49.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:49 smithi112 ceph-mon[35051]: pgmap v290: 97 pgs: 97 active+clean; 634 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-19T04:13:49.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.524+0000 7f994aba5640 1 -- 172.21.15.94:0/3660448642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944106250 msgr2=0x7f9944106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:49.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.524+0000 7f994aba5640 1 --2- 172.21.15.94:0/3660448642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944106250 0x7f9944106630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f994000aa10 tx=0x7f994002b720 comp rx=0 tx=0).stop 2024-09-19T04:13:49.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 -- 172.21.15.94:0/3660448642 shutdown_connections 2024-09-19T04:13:49.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 --2- 172.21.15.94:0/3660448642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9944100e10 0x7f994410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 --2- 172.21.15.94:0/3660448642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9944100490 0x7f99441008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 --2- 172.21.15.94:0/3660448642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944106250 0x7f9944106630 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 -- 172.21.15.94:0/3660448642 >> 172.21.15.94:0/3660448642 conn(0x7f99440fba90 msgr2=0x7f99440fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 -- 172.21.15.94:0/3660448642 shutdown_connections 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 -- 172.21.15.94:0/3660448642 wait complete. 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 Processor -- start 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 -- start start 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9944100490 0x7f9944198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944100e10 0x7f9944198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9944106250 0x7f994419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f994410fbe0 con 0x7f9944100e10 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f994410fa60 con 0x7f9944100490 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.525+0000 7f994aba5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f994410fd60 con 0x7f9944106250 2024-09-19T04:13:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f993bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944100e10 0x7f9944198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f993bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944100e10 0x7f9944198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42136/0 (socket says 172.21.15.94:42136) 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f993bfff640 1 -- 172.21.15.94:0/2063149002 learned_addr learned my addr 172.21.15.94:0/2063149002 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f9949142640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9944106250 0x7f994419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f9948941640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9944100490 0x7f9944198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f993bfff640 1 -- 172.21.15.94:0/2063149002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9944106250 msgr2=0x7f994419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f993bfff640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9944106250 0x7f994419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f993bfff640 1 -- 172.21.15.94:0/2063149002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9944100490 msgr2=0x7f9944198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f993bfff640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9944100490 0x7f9944198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f993bfff640 1 -- 172.21.15.94:0/2063149002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994419d2f0 con 0x7f9944100e10 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f9949142640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9944106250 0x7f994419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:49.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f9948941640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9944100490 0x7f9944198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f993bfff640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944100e10 0x7f9944198880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9f0 tx=0x7f993400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f9939ffb640 1 -- 172.21.15.94:0/2063149002 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993400cd80 con 0x7f9944100e10 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994419d5e0 con 0x7f9944100e10 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f9939ffb640 1 -- 172.21.15.94:0/2063149002 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99340054d0 con 0x7f9944100e10 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f9939ffb640 1 -- 172.21.15.94:0/2063149002 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934010490 con 0x7f9944100e10 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.526+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99441a4ec0 con 0x7f9944100e10 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.527+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99441008d0 con 0x7f9944100e10 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.528+0000 7f9939ffb640 1 -- 172.21.15.94:0/2063149002 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9934005020 con 0x7f9944100e10 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.529+0000 7f9939ffb640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f992c0a32d0 0x7f992c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.529+0000 7f9948941640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f992c0a32d0 0x7f992c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.529+0000 7f9939ffb640 1 -- 172.21.15.94:0/2063149002 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f99340cff50 con 0x7f9944100e10 2024-09-19T04:13:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.529+0000 7f9948941640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f992c0a32d0 0x7f992c0a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9940000c00 tx=0x7f9940030510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.533+0000 7f9939ffb640 1 -- 172.21.15.94:0/2063149002 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f993409cb10 con 0x7f9944100e10 2024-09-19T04:13:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:49 smithi104 ceph-mon[34145]: pgmap v290: 97 pgs: 97 active+clean; 634 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-19T04:13:49.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.701+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f994410a140 con 0x7f9944100e10 2024-09-19T04:13:49.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.703+0000 7f9939ffb640 1 -- 172.21.15.94:0/2063149002 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22842 (secure 0 0 0) 0x7f993409c410 con 0x7f9944100e10 2024-09-19T04:13:49.705 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:13:49.705 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:13:31.495964+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:49.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f992c0a32d0 msgr2=0x7f992c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:49.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f992c0a32d0 0x7f992c0a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9940000c00 tx=0x7f9940030510 comp rx=0 tx=0).stop 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944100e10 msgr2=0x7f9944198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944100e10 0x7f9944198880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9f0 tx=0x7f993400edf0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 shutdown_connections 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f992c0a32d0 0x7f992c0a5790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9944106250 0x7f994419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9944100e10 0x7f9944198880 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 --2- 172.21.15.94:0/2063149002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9944100490 0x7f9944198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 >> 172.21.15.94:0/2063149002 conn(0x7f99440fba90 msgr2=0x7f9944104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 shutdown_connections 2024-09-19T04:13:49.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:49.705+0000 7f994aba5640 1 -- 172.21.15.94:0/2063149002 wait complete. 2024-09-19T04:13:50.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:50 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2063149002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:50.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T04:13:50.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T04:13:50.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T04:13:50.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T04:13:50.485 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T04:13:50.485 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T04:13:50.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T04:13:50.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T04:13:50.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T04:13:50.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T04:13:50.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T04:13:50.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T04:13:50.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:50 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2063149002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:50 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2063149002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:13:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:51.236 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:51.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:51 smithi112 ceph-mon[35051]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-19T04:13:51.553 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:51 smithi094 ceph-mon[30972]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-19T04:13:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:51 smithi104 ceph-mon[34145]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-19T04:13:51.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.832+0000 7f7456df0640 1 -- 172.21.15.94:0/2876592881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7450102570 msgr2=0x7f74501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.832+0000 7f7456df0640 1 --2- 172.21.15.94:0/2876592881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7450102570 0x7f74501029b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f743c009900 tx=0x7f743c02a480 comp rx=0 tx=0).stop 2024-09-19T04:13:51.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.832+0000 7f7456df0640 1 -- 172.21.15.94:0/2876592881 shutdown_connections 2024-09-19T04:13:51.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.832+0000 7f7456df0640 1 --2- 172.21.15.94:0/2876592881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7450102ef0 0x7f745010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.832+0000 7f7456df0640 1 --2- 172.21.15.94:0/2876592881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7450102570 0x7f74501029b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.832+0000 7f7456df0640 1 --2- 172.21.15.94:0/2876592881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450100e20 0x7f7450101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.832+0000 7f7456df0640 1 -- 172.21.15.94:0/2876592881 >> 172.21.15.94:0/2876592881 conn(0x7f74500fc7d0 msgr2=0x7f74500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:51.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.832+0000 7f7456df0640 1 -- 172.21.15.94:0/2876592881 shutdown_connections 2024-09-19T04:13:51.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.832+0000 7f7456df0640 1 -- 172.21.15.94:0/2876592881 wait complete. 2024-09-19T04:13:51.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.833+0000 7f7456df0640 1 Processor -- start 2024-09-19T04:13:51.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.833+0000 7f7456df0640 1 -- start start 2024-09-19T04:13:51.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.833+0000 7f7456df0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7450100e20 0x7f74501992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.833+0000 7f7456df0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102570 0x7f7450199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.833+0000 7f7456df0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7450102ef0 0x7f74501a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.833+0000 7f7456df0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7450110e50 con 0x7f7450102ef0 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.833+0000 7f7456df0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7450110cd0 con 0x7f7450102570 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.833+0000 7f7456df0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7450110fd0 con 0x7f7450100e20 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f745538d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7450102ef0 0x7f74501a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f745538d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7450102ef0 0x7f74501a4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42144/0 (socket says 172.21.15.94:42144) 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f745538d640 1 -- 172.21.15.94:0/231380506 learned_addr learned my addr 172.21.15.94:0/231380506 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f7454b8c640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7450100e20 0x7f74501992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f744ffff640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102570 0x7f7450199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f745538d640 1 -- 172.21.15.94:0/231380506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7450100e20 msgr2=0x7f74501992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f745538d640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7450100e20 0x7f74501992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f745538d640 1 -- 172.21.15.94:0/231380506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102570 msgr2=0x7f7450199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f745538d640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102570 0x7f7450199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f745538d640 1 -- 172.21.15.94:0/231380506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74501a4880 con 0x7f7450102ef0 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f745538d640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7450102ef0 0x7f74501a4340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f744800b970 tx=0x7f744800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f744dffb640 1 -- 172.21.15.94:0/231380506 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448019070 con 0x7f7450102ef0 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74501a4a50 con 0x7f7450102ef0 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f744dffb640 1 -- 172.21.15.94:0/231380506 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7448014440 con 0x7f7450102ef0 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.834+0000 7f744dffb640 1 -- 172.21.15.94:0/231380506 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74480075d0 con 0x7f7450102ef0 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.835+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501a4f10 con 0x7f7450102ef0 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.835+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7450062980 con 0x7f7450102ef0 2024-09-19T04:13:51.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.837+0000 7f744dffb640 1 -- 172.21.15.94:0/231380506 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f744801c070 con 0x7f7450102ef0 2024-09-19T04:13:51.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.837+0000 7f744dffb640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f74400a32d0 0x7f74400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.837+0000 7f7454b8c640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f74400a32d0 0x7f74400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.837+0000 7f744dffb640 1 -- 172.21.15.94:0/231380506 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f74480cf710 con 0x7f7450102ef0 2024-09-19T04:13:51.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.838+0000 7f744dffb640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] conn(0x7f74400acf00 0x7f74400af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.838+0000 7f744dffb640 1 -- 172.21.15.94:0/231380506 --> [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74480047d0 con 0x7f74400acf00 2024-09-19T04:13:51.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.838+0000 7f744dffb640 1 -- 172.21.15.94:0/231380506 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f74480cfb40 con 0x7f7450102ef0 2024-09-19T04:13:51.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.838+0000 7f744ffff640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] conn(0x7f74400acf00 0x7f74400af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.838+0000 7f7454b8c640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f74400a32d0 0x7f74400a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74440058c0 tx=0x7f744400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:51.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.838+0000 7f744ffff640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] conn(0x7f74400acf00 0x7f74400af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.839+0000 7f744dffb640 1 -- 172.21.15.94:0/231380506 <== osd.6 v2:172.21.15.94:6818/1178759966 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74480047d0 con 0x7f74400acf00 2024-09-19T04:13:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.852+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 --> [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7450062e00 con 0x7f74400acf00 2024-09-19T04:13:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.852+0000 7f744dffb640 1 -- 172.21.15.94:0/231380506 <== osd.6 v2:172.21.15.94:6818/1178759966 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7450062e00 con 0x7f74400acf00 2024-09-19T04:13:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] conn(0x7f74400acf00 msgr2=0x7f74400af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] conn(0x7f74400acf00 0x7f74400af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f74400a32d0 msgr2=0x7f74400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f74400a32d0 0x7f74400a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74440058c0 tx=0x7f744400a2d0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7450102ef0 msgr2=0x7f74501a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 --2- 172.21.15.94:0/231380506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7450102ef0 0x7f74501a4340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f744800b970 tx=0x7f744800be30 comp rx=0 tx=0).stop 2024-09-19T04:13:51.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f745538d640 1 -- 172.21.15.94:0/231380506 reap_dead start 2024-09-19T04:13:51.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 shutdown_connections 2024-09-19T04:13:51.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 >> 172.21.15.94:0/231380506 conn(0x7f74500fc7d0 msgr2=0x7f74500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:51.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 shutdown_connections 2024-09-19T04:13:51.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:51.853+0000 7f7456df0640 1 -- 172.21.15.94:0/231380506 wait complete. 2024-09-19T04:13:52.312 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:52.313 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:52.313 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:52.314 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:52.566 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout:184683593773 2024-09-19T04:13:53.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T04:13:53.330 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:53 smithi094 ceph-mon[30972]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-19T04:13:53.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:53 smithi112 ceph-mon[35051]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-19T04:13:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:53 smithi104 ceph-mon[34145]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-19T04:13:53.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.981+0000 7f20a66f8640 1 -- 172.21.15.94:0/3594064784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a0106f10 msgr2=0x7f20a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:53.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.981+0000 7f20a66f8640 1 --2- 172.21.15.94:0/3594064784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a0106f10 0x7f20a010ea90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2098009ca0 tx=0x7f209802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:13:53.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.981+0000 7f20a66f8640 1 -- 172.21.15.94:0/3594064784 shutdown_connections 2024-09-19T04:13:53.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.981+0000 7f20a66f8640 1 --2- 172.21.15.94:0/3594064784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a0106f10 0x7f20a010ea90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:53.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.981+0000 7f20a66f8640 1 --2- 172.21.15.94:0/3594064784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a0102790 0x7f20a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:53.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.981+0000 7f20a66f8640 1 --2- 172.21.15.94:0/3594064784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a0101de0 0x7f20a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:53.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.981+0000 7f20a66f8640 1 -- 172.21.15.94:0/3594064784 >> 172.21.15.94:0/3594064784 conn(0x7f20a00fd8c0 msgr2=0x7f20a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:53.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.981+0000 7f20a66f8640 1 -- 172.21.15.94:0/3594064784 shutdown_connections 2024-09-19T04:13:53.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.981+0000 7f20a66f8640 1 -- 172.21.15.94:0/3594064784 wait complete. 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a66f8640 1 Processor -- start 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a66f8640 1 -- start start 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a66f8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a0101de0 0x7f20a00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a66f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a0102790 0x7f20a0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a66f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a0106f10 0x7f20a0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a66f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20a0111e40 con 0x7f20a0106f10 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a66f8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20a0111cc0 con 0x7f20a0101de0 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a66f8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20a0111fc0 con 0x7f20a0102790 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a4c95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a0106f10 0x7f20a0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a4c95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a0106f10 0x7f20a0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56008/0 (socket says 172.21.15.94:56008) 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a4c95640 1 -- 172.21.15.94:0/1273311469 learned_addr learned my addr 172.21.15.94:0/1273311469 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:53.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f209ffff640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a0101de0 0x7f20a00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f209f7fe640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a0102790 0x7f20a0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a4c95640 1 -- 172.21.15.94:0/1273311469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a0102790 msgr2=0x7f20a0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a4c95640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a0102790 0x7f20a0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a4c95640 1 -- 172.21.15.94:0/1273311469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a0101de0 msgr2=0x7f20a00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a4c95640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a0101de0 0x7f20a00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f20a4c95640 1 -- 172.21.15.94:0/1273311469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a01a67e0 con 0x7f20a0106f10 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.982+0000 7f209f7fe640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a0102790 0x7f20a0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.983+0000 7f20a4c95640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a0106f10 0x7f20a0074f70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2098009dc0 tx=0x7f2098003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.983+0000 7f209d7fa640 1 -- 172.21.15.94:0/1273311469 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098004040 con 0x7f20a0106f10 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.983+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a01a6a70 con 0x7f20a0106f10 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.983+0000 7f209d7fa640 1 -- 172.21.15.94:0/1273311469 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2098004690 con 0x7f20a0106f10 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.983+0000 7f209d7fa640 1 -- 172.21.15.94:0/1273311469 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098030c90 con 0x7f20a0106f10 2024-09-19T04:13:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.983+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a01a7000 con 0x7f20a0106f10 2024-09-19T04:13:53.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.984+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20a0062880 con 0x7f20a0106f10 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.985+0000 7f209d7fa640 1 -- 172.21.15.94:0/1273311469 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f20980041e0 con 0x7f20a0106f10 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.985+0000 7f209d7fa640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f20880a32d0 0x7f20880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.986+0000 7f209ffff640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f20880a32d0 0x7f20880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.986+0000 7f209d7fa640 1 -- 172.21.15.94:0/1273311469 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f20980f13f0 con 0x7f20a0106f10 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.986+0000 7f209d7fa640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] conn(0x7f20880acdd0 0x7f20880af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.986+0000 7f209d7fa640 1 -- 172.21.15.94:0/1273311469 --> [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2098050ea0 con 0x7f20880acdd0 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.986+0000 7f209d7fa640 1 -- 172.21.15.94:0/1273311469 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f20980f1820 con 0x7f20a0106f10 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.986+0000 7f209ffff640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f20880a32d0 0x7f20880a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2090009990 tx=0x7f2090009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.986+0000 7f209f7fe640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] conn(0x7f20880acdd0 0x7f20880af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.986+0000 7f209f7fe640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] conn(0x7f20880acdd0 0x7f20880af270 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:53.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:53.987+0000 7f209d7fa640 1 -- 172.21.15.94:0/1273311469 <== osd.8 v2:172.21.15.112:6816/3390158844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2098050ea0 con 0x7f20880acdd0 2024-09-19T04:13:54.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.002+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 --> [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20a0062cb0 con 0x7f20880acdd0 2024-09-19T04:13:54.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.002+0000 7f209d7fa640 1 -- 172.21.15.94:0/1273311469 <== osd.8 v2:172.21.15.112:6816/3390158844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20a0062cb0 con 0x7f20880acdd0 2024-09-19T04:13:54.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 >> [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] conn(0x7f20880acdd0 msgr2=0x7f20880af270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] conn(0x7f20880acdd0 0x7f20880af270 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f20880a32d0 msgr2=0x7f20880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f20880a32d0 0x7f20880a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2090009990 tx=0x7f2090009110 comp rx=0 tx=0).stop 2024-09-19T04:13:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a0106f10 msgr2=0x7f20a0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 --2- 172.21.15.94:0/1273311469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a0106f10 0x7f20a0074f70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2098009dc0 tx=0x7f2098003ba0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a4c95640 1 -- 172.21.15.94:0/1273311469 reap_dead start 2024-09-19T04:13:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 shutdown_connections 2024-09-19T04:13:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 >> 172.21.15.94:0/1273311469 conn(0x7f20a00fd8c0 msgr2=0x7f20a00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 shutdown_connections 2024-09-19T04:13:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.003+0000 7f20a66f8640 1 -- 172.21.15.94:0/1273311469 wait complete. 2024-09-19T04:13:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.216+0000 7fda466ce640 1 -- 172.21.15.94:0/3338751880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda40102ef0 msgr2=0x7fda4010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.216+0000 7fda466ce640 1 --2- 172.21.15.94:0/3338751880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda40102ef0 0x7fda4010da40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fda38009ca0 tx=0x7fda3802a950 comp rx=0 tx=0).stop 2024-09-19T04:13:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.216+0000 7fda466ce640 1 -- 172.21.15.94:0/3338751880 shutdown_connections 2024-09-19T04:13:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.216+0000 7fda466ce640 1 --2- 172.21.15.94:0/3338751880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda40102ef0 0x7fda4010da40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.216+0000 7fda466ce640 1 --2- 172.21.15.94:0/3338751880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102570 0x7fda401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.216+0000 7fda466ce640 1 --2- 172.21.15.94:0/3338751880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda40100e20 0x7fda40101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.216+0000 7fda466ce640 1 -- 172.21.15.94:0/3338751880 >> 172.21.15.94:0/3338751880 conn(0x7fda400fc7d0 msgr2=0x7fda400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.216+0000 7fda466ce640 1 -- 172.21.15.94:0/3338751880 shutdown_connections 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.216+0000 7fda466ce640 1 -- 172.21.15.94:0/3338751880 wait complete. 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda466ce640 1 Processor -- start 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda466ce640 1 -- start start 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda466ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda40100e20 0x7fda40199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda466ce640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda40102570 0x7fda40199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda466ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda401a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda466ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda40110e80 con 0x7fda40100e20 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda466ce640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fda40110d00 con 0x7fda40102570 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda466ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda40111000 con 0x7fda40102ef0 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda3ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda40100e20 0x7fda40199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda3ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda40100e20 0x7fda40199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56036/0 (socket says 172.21.15.94:56036) 2024-09-19T04:13:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.217+0000 7fda3ffff640 1 -- 172.21.15.94:0/3661490484 learned_addr learned my addr 172.21.15.94:0/3661490484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:54.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3f7fe640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda40102570 0x7fda40199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda44c6b640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda401a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3ffff640 1 -- 172.21.15.94:0/3661490484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 msgr2=0x7fda401a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3ffff640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda401a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3ffff640 1 -- 172.21.15.94:0/3661490484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda40102570 msgr2=0x7fda40199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3ffff640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda40102570 0x7fda40199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3ffff640 1 -- 172.21.15.94:0/3661490484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda401a47f0 con 0x7fda40100e20 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda44c6b640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda401a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3ffff640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda40100e20 0x7fda40199230 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fda3000d950 tx=0x7fda3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3d7fa640 1 -- 172.21.15.94:0/3661490484 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30017030 con 0x7fda40100e20 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3d7fa640 1 -- 172.21.15.94:0/3661490484 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda30004020 con 0x7fda40100e20 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda3d7fa640 1 -- 172.21.15.94:0/3661490484 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300047a0 con 0x7fda40100e20 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda401a49c0 con 0x7fda40100e20 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.218+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda401a4f00 con 0x7fda40100e20 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.219+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda40062980 con 0x7fda40100e20 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.220+0000 7fda3d7fa640 1 -- 172.21.15.94:0/3661490484 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fda300041d0 con 0x7fda40100e20 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.221+0000 7fda3d7fa640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fda280a3350 0x7fda280a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.221+0000 7fda3f7fe640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fda280a3350 0x7fda280a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.221+0000 7fda3d7fa640 1 -- 172.21.15.94:0/3661490484 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fda300cf510 con 0x7fda40100e20 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.221+0000 7fda3d7fa640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] conn(0x7fda280acf80 0x7fda280af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.221+0000 7fda3d7fa640 1 -- 172.21.15.94:0/3661490484 --> [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda30021e50 con 0x7fda280acf80 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.221+0000 7fda3d7fa640 1 -- 172.21.15.94:0/3661490484 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fda300d6050 con 0x7fda40100e20 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.221+0000 7fda3f7fe640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fda280a3350 0x7fda280a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fda34005d70 tx=0x7fda34005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.221+0000 7fda44c6b640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] conn(0x7fda280acf80 0x7fda280af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.222+0000 7fda44c6b640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] conn(0x7fda280acf80 0x7fda280af420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.223+0000 7fda3d7fa640 1 -- 172.21.15.94:0/3661490484 <== osd.10 v2:172.21.15.104:6824/1626292599 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda30021e50 con 0x7fda280acf80 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.236+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 --> [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda40062e00 con 0x7fda280acf80 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda3d7fa640 1 -- 172.21.15.94:0/3661490484 <== osd.10 v2:172.21.15.104:6824/1626292599 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fda40062e00 con 0x7fda280acf80 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 >> [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] conn(0x7fda280acf80 msgr2=0x7fda280af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] conn(0x7fda280acf80 0x7fda280af420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fda280a3350 msgr2=0x7fda280a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fda280a3350 0x7fda280a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fda34005d70 tx=0x7fda34005d00 comp rx=0 tx=0).stop 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda40100e20 msgr2=0x7fda40199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 --2- 172.21.15.94:0/3661490484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda40100e20 0x7fda40199230 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fda3000d950 tx=0x7fda3000de10 comp rx=0 tx=0).stop 2024-09-19T04:13:54.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda44c6b640 1 -- 172.21.15.94:0/3661490484 reap_dead start 2024-09-19T04:13:54.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 shutdown_connections 2024-09-19T04:13:54.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 >> 172.21.15.94:0/3661490484 conn(0x7fda400fc7d0 msgr2=0x7fda400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:54.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 shutdown_connections 2024-09-19T04:13:54.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:54.237+0000 7fda466ce640 1 -- 172.21.15.94:0/3661490484 wait complete. 2024-09-19T04:13:55.406 INFO:teuthology.orchestra.run.smithi094.stdout:283467841562 2024-09-19T04:13:55.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T04:13:55.497 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:55 smithi094 ceph-mon[30972]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-19T04:13:55.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:55 smithi112 ceph-mon[35051]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-19T04:13:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:55 smithi104 ceph-mon[34145]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-19T04:13:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 -- 172.21.15.94:0/2376586370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0100a70 msgr2=0x7f2ca0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 --2- 172.21.15.94:0/2376586370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0100a70 0x7f2ca0100ed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009930 tx=0x7f2c8802a420 comp rx=0 tx=0).stop 2024-09-19T04:13:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 -- 172.21.15.94:0/2376586370 shutdown_connections 2024-09-19T04:13:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 --2- 172.21.15.94:0/2376586370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ca0101410 0x7f2ca010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 --2- 172.21.15.94:0/2376586370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0100a70 0x7f2ca0100ed0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 --2- 172.21.15.94:0/2376586370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ca0106a70 0x7f2ca0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 -- 172.21.15.94:0/2376586370 >> 172.21.15.94:0/2376586370 conn(0x7f2ca00fc790 msgr2=0x7f2ca00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 -- 172.21.15.94:0/2376586370 shutdown_connections 2024-09-19T04:13:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 -- 172.21.15.94:0/2376586370 wait complete. 2024-09-19T04:13:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.693+0000 7f2ca5742640 1 Processor -- start 2024-09-19T04:13:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2ca5742640 1 -- start start 2024-09-19T04:13:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2ca5742640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ca0100a70 0x7f2ca00fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2ca5742640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0101410 0x7f2ca00fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2ca5742640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ca0106a70 0x7f2ca00fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2ca5742640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0110d20 con 0x7f2ca0101410 2024-09-19T04:13:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2ca5742640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0110ba0 con 0x7f2ca0106a70 2024-09-19T04:13:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2ca5742640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0110ea0 con 0x7f2ca0100a70 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0101410 0x7f2ca00fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0101410 0x7f2ca00fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56052/0 (socket says 172.21.15.94:56052) 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c97fff640 1 -- 172.21.15.94:0/2780008890 learned_addr learned my addr 172.21.15.94:0/2780008890 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c9effd640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ca0100a70 0x7f2ca00fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c9f7fe640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ca0106a70 0x7f2ca00fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c97fff640 1 -- 172.21.15.94:0/2780008890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ca0100a70 msgr2=0x7f2ca00fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c97fff640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ca0100a70 0x7f2ca00fcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c97fff640 1 -- 172.21.15.94:0/2780008890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ca0106a70 msgr2=0x7f2ca00fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c97fff640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ca0106a70 0x7f2ca00fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c97fff640 1 -- 172.21.15.94:0/2780008890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca01a6890 con 0x7f2ca0101410 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.694+0000 7f2c97fff640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0101410 0x7f2ca00fd450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c88004770 tx=0x7f2c880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.695+0000 7f2c9cff9640 1 -- 172.21.15.94:0/2780008890 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8803f070 con 0x7f2ca0101410 2024-09-19T04:13:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.695+0000 7f2c9cff9640 1 -- 172.21.15.94:0/2780008890 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c8803b460 con 0x7f2ca0101410 2024-09-19T04:13:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.695+0000 7f2c9cff9640 1 -- 172.21.15.94:0/2780008890 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88035070 con 0x7f2ca0101410 2024-09-19T04:13:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.695+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca01a6b20 con 0x7f2ca0101410 2024-09-19T04:13:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.695+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca01a7050 con 0x7f2ca0101410 2024-09-19T04:13:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.696+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ca00628c0 con 0x7f2ca0101410 2024-09-19T04:13:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.697+0000 7f2c9cff9640 1 -- 172.21.15.94:0/2780008890 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2c8803b610 con 0x7f2ca0101410 2024-09-19T04:13:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.697+0000 7f2c9cff9640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2c8c0a3350 0x7f2c8c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.698+0000 7f2c9effd640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2c8c0a3350 0x7f2c8c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.698+0000 7f2c9cff9640 1 -- 172.21.15.94:0/2780008890 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2c880efdb0 con 0x7f2ca0101410 2024-09-19T04:13:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.698+0000 7f2c9cff9640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] conn(0x7f2c8c0acf80 0x7f2c8c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.698+0000 7f2c9cff9640 1 -- 172.21.15.94:0/2780008890 --> [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c8c0afaf0 con 0x7f2c8c0acf80 2024-09-19T04:13:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.698+0000 7f2c9cff9640 1 -- 172.21.15.94:0/2780008890 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f2c880f2450 con 0x7f2ca0101410 2024-09-19T04:13:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.698+0000 7f2c9effd640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2c8c0a3350 0x7f2c8c0a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c90005fd0 tx=0x7f2c9000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.698+0000 7f2c9f7fe640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] conn(0x7f2c8c0acf80 0x7f2c8c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.698+0000 7f2c9f7fe640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] conn(0x7f2c8c0acf80 0x7f2c8c0af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.699+0000 7f2c9cff9640 1 -- 172.21.15.94:0/2780008890 <== osd.5 v2:172.21.15.112:6808/3131983830 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c8c0afaf0 con 0x7f2c8c0acf80 2024-09-19T04:13:55.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.714+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 --> [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ca0062cf0 con 0x7f2c8c0acf80 2024-09-19T04:13:55.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.715+0000 7f2c9cff9640 1 -- 172.21.15.94:0/2780008890 <== osd.5 v2:172.21.15.112:6808/3131983830 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2ca0062cf0 con 0x7f2c8c0acf80 2024-09-19T04:13:55.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 >> [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] conn(0x7f2c8c0acf80 msgr2=0x7f2c8c0af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] conn(0x7f2c8c0acf80 0x7f2c8c0af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2c8c0a3350 msgr2=0x7f2c8c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2c8c0a3350 0x7f2c8c0a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c90005fd0 tx=0x7f2c9000b390 comp rx=0 tx=0).stop 2024-09-19T04:13:55.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0101410 msgr2=0x7f2ca00fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 --2- 172.21.15.94:0/2780008890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0101410 0x7f2ca00fd450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c88004770 tx=0x7f2c880042f0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2c9f7fe640 1 -- 172.21.15.94:0/2780008890 reap_dead start 2024-09-19T04:13:55.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 shutdown_connections 2024-09-19T04:13:55.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 >> 172.21.15.94:0/2780008890 conn(0x7f2ca00fc790 msgr2=0x7f2ca006ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:55.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 shutdown_connections 2024-09-19T04:13:55.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.716+0000 7f2ca5742640 1 -- 172.21.15.94:0/2780008890 wait complete. 2024-09-19T04:13:55.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 -- 172.21.15.94:0/2043155257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4c101420 msgr2=0x7f9a4c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 --2- 172.21.15.94:0/2043155257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4c101420 0x7f9a4c10d950 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009cd0 tx=0x7f9a4402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 -- 172.21.15.94:0/2043155257 shutdown_connections 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 --2- 172.21.15.94:0/2043155257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4c101420 0x7f9a4c10d950 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 --2- 172.21.15.94:0/2043155257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4c100a80 0x7f9a4c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 --2- 172.21.15.94:0/2043155257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a4c106a80 0x7f9a4c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 -- 172.21.15.94:0/2043155257 >> 172.21.15.94:0/2043155257 conn(0x7f9a4c0fc7a0 msgr2=0x7f9a4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 -- 172.21.15.94:0/2043155257 shutdown_connections 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 -- 172.21.15.94:0/2043155257 wait complete. 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 Processor -- start 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 -- start start 2024-09-19T04:13:55.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.908+0000 7f9a52a2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4c100a80 0x7f9a4c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a52a2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4c101420 0x7f9a4c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a52a2e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a4c106a80 0x7f9a4c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a52a2e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c111e50 con 0x7f9a4c101420 2024-09-19T04:13:55.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a52a2e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c111cd0 con 0x7f9a4c106a80 2024-09-19T04:13:55.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a52a2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c111fd0 con 0x7f9a4c100a80 2024-09-19T04:13:55.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a43fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4c101420 0x7f9a4c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a43fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4c101420 0x7f9a4c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56068/0 (socket says 172.21.15.94:56068) 2024-09-19T04:13:55.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a43fff640 1 -- 172.21.15.94:0/3755412625 learned_addr learned my addr 172.21.15.94:0/3755412625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:55.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a4bfff640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4c100a80 0x7f9a4c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a50fcb640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a4c106a80 0x7f9a4c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a43fff640 1 -- 172.21.15.94:0/3755412625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4c100a80 msgr2=0x7f9a4c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a43fff640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4c100a80 0x7f9a4c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a43fff640 1 -- 172.21.15.94:0/3755412625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a4c106a80 msgr2=0x7f9a4c19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a43fff640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a4c106a80 0x7f9a4c19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a43fff640 1 -- 172.21.15.94:0/3755412625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c19d210 con 0x7f9a4c101420 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a43fff640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4c101420 0x7f9a4c1987a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400c890 tx=0x7f9a3400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a49ffb640 1 -- 172.21.15.94:0/3755412625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34004220 con 0x7f9a4c101420 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a49ffb640 1 -- 172.21.15.94:0/3755412625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a340043c0 con 0x7f9a4c101420 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c19d500 con 0x7f9a4c101420 2024-09-19T04:13:55.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a49ffb640 1 -- 172.21.15.94:0/3755412625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34004a00 con 0x7f9a4c101420 2024-09-19T04:13:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.909+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c1a4de0 con 0x7f9a4c101420 2024-09-19T04:13:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.910+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a4c0fe940 con 0x7f9a4c101420 2024-09-19T04:13:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.911+0000 7f9a49ffb640 1 -- 172.21.15.94:0/3755412625 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9a340079b0 con 0x7f9a4c101420 2024-09-19T04:13:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.912+0000 7f9a49ffb640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9a380a32d0 0x7f9a380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.912+0000 7f9a4bfff640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9a380a32d0 0x7f9a380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.912+0000 7f9a49ffb640 1 -- 172.21.15.94:0/3755412625 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9a340cfaa0 con 0x7f9a4c101420 2024-09-19T04:13:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.912+0000 7f9a49ffb640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] conn(0x7f9a380acf00 0x7f9a380af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.912+0000 7f9a49ffb640 1 -- 172.21.15.94:0/3755412625 --> [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a380afa70 con 0x7f9a380acf00 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.912+0000 7f9a49ffb640 1 -- 172.21.15.94:0/3755412625 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f9a340cfed0 con 0x7f9a4c101420 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.912+0000 7f9a4bfff640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9a380a32d0 0x7f9a380a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ab60 tx=0x7f9a3c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.913+0000 7f9a50fcb640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] conn(0x7f9a380acf00 0x7f9a380af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.913+0000 7f9a50fcb640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] conn(0x7f9a380acf00 0x7f9a380af3a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.914+0000 7f9a49ffb640 1 -- 172.21.15.94:0/3755412625 <== osd.4 v2:172.21.15.104:6808/571438521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a380afa70 con 0x7f9a380acf00 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.927+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 --> [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a4c1021c0 con 0x7f9a380acf00 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a49ffb640 1 -- 172.21.15.94:0/3755412625 <== osd.4 v2:172.21.15.104:6808/571438521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9a4c1021c0 con 0x7f9a380acf00 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 >> [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] conn(0x7f9a380acf00 msgr2=0x7f9a380af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] conn(0x7f9a380acf00 0x7f9a380af3a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9a380a32d0 msgr2=0x7f9a380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f9a380a32d0 0x7f9a380a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ab60 tx=0x7f9a3c00a2d0 comp rx=0 tx=0).stop 2024-09-19T04:13:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4c101420 msgr2=0x7f9a4c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:55.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 --2- 172.21.15.94:0/3755412625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4c101420 0x7f9a4c1987a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400c890 tx=0x7f9a3400cc90 comp rx=0 tx=0).stop 2024-09-19T04:13:55.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a50fcb640 1 -- 172.21.15.94:0/3755412625 reap_dead start 2024-09-19T04:13:55.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 shutdown_connections 2024-09-19T04:13:55.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 >> 172.21.15.94:0/3755412625 conn(0x7f9a4c0fc7a0 msgr2=0x7f9a4c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:55.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 shutdown_connections 2024-09-19T04:13:55.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:55.928+0000 7f9a52a2e640 1 -- 172.21.15.94:0/3755412625 wait complete. 2024-09-19T04:13:56.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.324+0000 7ff149230640 1 -- 172.21.15.94:0/3605617486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff144100a80 msgr2=0x7ff144100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:56.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.324+0000 7ff149230640 1 --2- 172.21.15.94:0/3605617486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff144100a80 0x7ff144100ee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff138009930 tx=0x7ff13802a420 comp rx=0 tx=0).stop 2024-09-19T04:13:56.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.324+0000 7ff149230640 1 -- 172.21.15.94:0/3605617486 shutdown_connections 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.324+0000 7ff149230640 1 --2- 172.21.15.94:0/3605617486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144101420 0x7ff14410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.324+0000 7ff149230640 1 --2- 172.21.15.94:0/3605617486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff144100a80 0x7ff144100ee0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.324+0000 7ff149230640 1 --2- 172.21.15.94:0/3605617486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff144106a80 0x7ff144106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.324+0000 7ff149230640 1 -- 172.21.15.94:0/3605617486 >> 172.21.15.94:0/3605617486 conn(0x7ff1440fc7a0 msgr2=0x7ff1440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.324+0000 7ff149230640 1 -- 172.21.15.94:0/3605617486 shutdown_connections 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.324+0000 7ff149230640 1 -- 172.21.15.94:0/3605617486 wait complete. 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff149230640 1 Processor -- start 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff149230640 1 -- start start 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff149230640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff144100a80 0x7ff144198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff149230640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144101420 0x7ff144198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff149230640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff144106a80 0x7ff14419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:56.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff149230640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff144111e50 con 0x7ff144100a80 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff149230640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff144111cd0 con 0x7ff144106a80 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff149230640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff144111fd0 con 0x7ff144101420 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff142d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff144100a80 0x7ff144198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff142d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff144100a80 0x7ff144198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56080/0 (socket says 172.21.15.94:56080) 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.325+0000 7ff142d9d640 1 -- 172.21.15.94:0/4148591485 learned_addr learned my addr 172.21.15.94:0/4148591485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff14259c640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144101420 0x7ff144198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff142d9d640 1 -- 172.21.15.94:0/4148591485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144101420 msgr2=0x7ff144198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff14359e640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff144106a80 0x7ff14419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff142d9d640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144101420 0x7ff144198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff142d9d640 1 -- 172.21.15.94:0/4148591485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff144106a80 msgr2=0x7ff14419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff142d9d640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff144106a80 0x7ff14419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff142d9d640 1 -- 172.21.15.94:0/4148591485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14419d3c0 con 0x7ff144100a80 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff14259c640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144101420 0x7ff144198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff142d9d640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff144100a80 0x7ff144198410 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff13400d950 tx=0x7ff13400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff12bfff640 1 -- 172.21.15.94:0/4148591485 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134017070 con 0x7ff144100a80 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14419d6b0 con 0x7ff144100a80 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff12bfff640 1 -- 172.21.15.94:0/4148591485 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff134009c30 con 0x7ff144100a80 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff12bfff640 1 -- 172.21.15.94:0/4148591485 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13400f620 con 0x7ff144100a80 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.326+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1441a4f90 con 0x7ff144100a80 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.327+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff144062980 con 0x7ff144100a80 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.328+0000 7ff12bfff640 1 -- 172.21.15.94:0/4148591485 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff1340109a0 con 0x7ff144100a80 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.329+0000 7ff12bfff640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff12c0a32d0 0x7ff12c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:56.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.329+0000 7ff14259c640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff12c0a32d0 0x7ff12c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.329+0000 7ff12bfff640 1 -- 172.21.15.94:0/4148591485 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff1340cfb20 con 0x7ff144100a80 2024-09-19T04:13:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.329+0000 7ff12bfff640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] conn(0x7ff12c0acf00 0x7ff12c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.329+0000 7ff12bfff640 1 -- 172.21.15.94:0/4148591485 --> [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff12c0afa70 con 0x7ff12c0acf00 2024-09-19T04:13:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.329+0000 7ff12bfff640 1 -- 172.21.15.94:0/4148591485 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ff1340cff50 con 0x7ff144100a80 2024-09-19T04:13:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.329+0000 7ff14259c640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff12c0a32d0 0x7ff12c0a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff138009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.329+0000 7ff14359e640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] conn(0x7ff12c0acf00 0x7ff12c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.330+0000 7ff14359e640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] conn(0x7ff12c0acf00 0x7ff12c0af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.331+0000 7ff12bfff640 1 -- 172.21.15.94:0/4148591485 <== osd.1 v2:172.21.15.104:6800/1485019338 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff12c0afa70 con 0x7ff12c0acf00 2024-09-19T04:13:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.344+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 --> [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff144062e00 con 0x7ff12c0acf00 2024-09-19T04:13:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.344+0000 7ff12bfff640 1 -- 172.21.15.94:0/4148591485 <== osd.1 v2:172.21.15.104:6800/1485019338 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff144062e00 con 0x7ff12c0acf00 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 >> [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] conn(0x7ff12c0acf00 msgr2=0x7ff12c0af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] conn(0x7ff12c0acf00 0x7ff12c0af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff12c0a32d0 msgr2=0x7ff12c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ff12c0a32d0 0x7ff12c0a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff138009610 comp rx=0 tx=0).stop 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff144100a80 msgr2=0x7ff144198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 --2- 172.21.15.94:0/4148591485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff144100a80 0x7ff144198410 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff13400d950 tx=0x7ff13400de10 comp rx=0 tx=0).stop 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff14359e640 1 -- 172.21.15.94:0/4148591485 reap_dead start 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 shutdown_connections 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 >> 172.21.15.94:0/4148591485 conn(0x7ff1440fc7a0 msgr2=0x7ff1440fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 shutdown_connections 2024-09-19T04:13:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:56.345+0000 7ff149230640 1 -- 172.21.15.94:0/4148591485 wait complete. 2024-09-19T04:13:57.237 INFO:teuthology.orchestra.run.smithi094.stdout:133143986232 2024-09-19T04:13:57.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T04:13:57.467 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:57 smithi094 ceph-mon[30972]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T04:13:57.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:57 smithi112 ceph-mon[35051]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T04:13:57.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.548+0000 7fa482126640 1 -- 172.21.15.94:0/2134604456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa47c102570 msgr2=0x7fa47c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.548+0000 7fa482126640 1 --2- 172.21.15.94:0/2134604456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa47c102570 0x7fa47c1029b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa470009900 tx=0x7fa47002a480 comp rx=0 tx=0).stop 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.548+0000 7fa482126640 1 -- 172.21.15.94:0/2134604456 shutdown_connections 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.548+0000 7fa482126640 1 --2- 172.21.15.94:0/2134604456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47c102ef0 0x7fa47c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.548+0000 7fa482126640 1 --2- 172.21.15.94:0/2134604456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa47c102570 0x7fa47c1029b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.548+0000 7fa482126640 1 --2- 172.21.15.94:0/2134604456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa47c100e20 0x7fa47c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.548+0000 7fa482126640 1 -- 172.21.15.94:0/2134604456 >> 172.21.15.94:0/2134604456 conn(0x7fa47c0fc7d0 msgr2=0x7fa47c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.548+0000 7fa482126640 1 -- 172.21.15.94:0/2134604456 shutdown_connections 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.548+0000 7fa482126640 1 -- 172.21.15.94:0/2134604456 wait complete. 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa482126640 1 Processor -- start 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa482126640 1 -- start start 2024-09-19T04:13:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa482126640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47c100e20 0x7fa47c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa482126640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa47c102570 0x7fa47c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa482126640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa47c102ef0 0x7fa47c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa482126640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c110e30 con 0x7fa47c102ef0 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa482126640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c110cb0 con 0x7fa47c102570 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa482126640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c110fb0 con 0x7fa47c100e20 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa47c102ef0 0x7fa47c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa47c102ef0 0x7fa47c19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56092/0 (socket says 172.21.15.94:56092) 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47bfff640 1 -- 172.21.15.94:0/480226715 learned_addr learned my addr 172.21.15.94:0/480226715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47b7fe640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47c100e20 0x7fa47c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47affd640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa47c102570 0x7fa47c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47bfff640 1 -- 172.21.15.94:0/480226715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47c100e20 msgr2=0x7fa47c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47bfff640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47c100e20 0x7fa47c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47bfff640 1 -- 172.21.15.94:0/480226715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa47c102570 msgr2=0x7fa47c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47bfff640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa47c102570 0x7fa47c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.549+0000 7fa47bfff640 1 -- 172.21.15.94:0/480226715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c19d3d0 con 0x7fa47c102ef0 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.550+0000 7fa47bfff640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa47c102ef0 0x7fa47c19ccf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa47400eaa0 tx=0x7fa47400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.550+0000 7fa478ff9640 1 -- 172.21.15.94:0/480226715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47400cdb0 con 0x7fa47c102ef0 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.550+0000 7fa478ff9640 1 -- 172.21.15.94:0/480226715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa474005020 con 0x7fa47c102ef0 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.550+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47c19d6c0 con 0x7fa47c102ef0 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.550+0000 7fa478ff9640 1 -- 172.21.15.94:0/480226715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4740116d0 con 0x7fa47c102ef0 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.550+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47c1a4fa0 con 0x7fa47c102ef0 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.551+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa47c062980 con 0x7fa47c102ef0 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.552+0000 7fa478ff9640 1 -- 172.21.15.94:0/480226715 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa4740051d0 con 0x7fa47c102ef0 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.553+0000 7fa478ff9640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa4640a3350 0x7fa4640a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.553+0000 7fa47b7fe640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa4640a3350 0x7fa4640a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:57.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.553+0000 7fa478ff9640 1 -- 172.21.15.94:0/480226715 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa4740cff20 con 0x7fa47c102ef0 2024-09-19T04:13:57.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.553+0000 7fa478ff9640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] conn(0x7fa4640acf80 0x7fa4640af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:57.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.553+0000 7fa478ff9640 1 -- 172.21.15.94:0/480226715 --> [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa474014ec0 con 0x7fa4640acf80 2024-09-19T04:13:57.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.553+0000 7fa478ff9640 1 -- 172.21.15.94:0/480226715 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa4740d0350 con 0x7fa47c102ef0 2024-09-19T04:13:57.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.553+0000 7fa47affd640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] conn(0x7fa4640acf80 0x7fa4640af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:57.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.553+0000 7fa47b7fe640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa4640a3350 0x7fa4640a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0058c0 tx=0x7fa46c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:57.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.553+0000 7fa47affd640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] conn(0x7fa4640acf80 0x7fa4640af420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:57.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.554+0000 7fa478ff9640 1 -- 172.21.15.94:0/480226715 <== osd.3 v2:172.21.15.94:6810/1707310526 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa474014ec0 con 0x7fa4640acf80 2024-09-19T04:13:57.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.567+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 --> [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa47c062e00 con 0x7fa4640acf80 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.567+0000 7fa478ff9640 1 -- 172.21.15.94:0/480226715 <== osd.3 v2:172.21.15.94:6810/1707310526 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa47c062e00 con 0x7fa4640acf80 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] conn(0x7fa4640acf80 msgr2=0x7fa4640af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] conn(0x7fa4640acf80 0x7fa4640af420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa4640a3350 msgr2=0x7fa4640a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa4640a3350 0x7fa4640a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0058c0 tx=0x7fa46c00a2d0 comp rx=0 tx=0).stop 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa47c102ef0 msgr2=0x7fa47c19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 --2- 172.21.15.94:0/480226715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa47c102ef0 0x7fa47c19ccf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa47400eaa0 tx=0x7fa47400ef60 comp rx=0 tx=0).stop 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa47bfff640 1 -- 172.21.15.94:0/480226715 reap_dead start 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 shutdown_connections 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 >> 172.21.15.94:0/480226715 conn(0x7fa47c0fc7d0 msgr2=0x7fa47c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 shutdown_connections 2024-09-19T04:13:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:57.568+0000 7fa482126640 1 -- 172.21.15.94:0/480226715 wait complete. 2024-09-19T04:13:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:57 smithi104 ceph-mon[34145]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T04:13:58.506 INFO:teuthology.orchestra.run.smithi094.stdout:51539607623 2024-09-19T04:13:58.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.688+0000 7f610095f640 1 -- 172.21.15.94:0/3819681962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc100a80 msgr2=0x7f60fc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.688+0000 7f610095f640 1 --2- 172.21.15.94:0/3819681962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc100a80 0x7f60fc100ee0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009930 tx=0x7f60e402a420 comp rx=0 tx=0).stop 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.688+0000 7f610095f640 1 -- 172.21.15.94:0/3819681962 shutdown_connections 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.688+0000 7f610095f640 1 --2- 172.21.15.94:0/3819681962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60fc101420 0x7f60fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.688+0000 7f610095f640 1 --2- 172.21.15.94:0/3819681962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc100a80 0x7f60fc100ee0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.688+0000 7f610095f640 1 --2- 172.21.15.94:0/3819681962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc106a80 0x7f60fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.688+0000 7f610095f640 1 -- 172.21.15.94:0/3819681962 >> 172.21.15.94:0/3819681962 conn(0x7f60fc0fc7a0 msgr2=0x7f60fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.688+0000 7f610095f640 1 -- 172.21.15.94:0/3819681962 shutdown_connections 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.689+0000 7f610095f640 1 -- 172.21.15.94:0/3819681962 wait complete. 2024-09-19T04:13:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.689+0000 7f610095f640 1 Processor -- start 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.689+0000 7f610095f640 1 -- start start 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.689+0000 7f610095f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc100a80 0x7f60fc1114b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.689+0000 7f610095f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc101420 0x7f60fc1119f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.689+0000 7f610095f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60fc106a80 0x7f60fc111f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.689+0000 7f610095f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc110d30 con 0x7f60fc100a80 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.689+0000 7f610095f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc110bb0 con 0x7f60fc106a80 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.689+0000 7f610095f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc110eb0 con 0x7f60fc101420 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc100a80 0x7f60fc1114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc100a80 0x7f60fc1114b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56104/0 (socket says 172.21.15.94:56104) 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fa59c640 1 -- 172.21.15.94:0/527178965 learned_addr learned my addr 172.21.15.94:0/527178965 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fad9d640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60fc106a80 0x7f60fc111f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fa59c640 1 -- 172.21.15.94:0/527178965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc101420 msgr2=0x7f60fc1119f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fa59c640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc101420 0x7f60fc1119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fa59c640 1 -- 172.21.15.94:0/527178965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60fc106a80 msgr2=0x7f60fc111f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fa59c640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60fc106a80 0x7f60fc111f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fa59c640 1 -- 172.21.15.94:0/527178965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc1a6850 con 0x7f60fc100a80 2024-09-19T04:13:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60fa59c640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc100a80 0x7f60fc1114b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00d770 tx=0x7f60ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60f37fe640 1 -- 172.21.15.94:0/527178965 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec017070 con 0x7f60fc100a80 2024-09-19T04:13:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.690+0000 7f60f37fe640 1 -- 172.21.15.94:0/527178965 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60ec0054d0 con 0x7f60fc100a80 2024-09-19T04:13:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.691+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc1a6b40 con 0x7f60fc100a80 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.691+0000 7f60f37fe640 1 -- 172.21.15.94:0/527178965 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec004cf0 con 0x7f60fc100a80 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.691+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1a7030 con 0x7f60fc100a80 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.691+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60fc104820 con 0x7f60fc100a80 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.694+0000 7f60f37fe640 1 -- 172.21.15.94:0/527178965 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f60ec004e90 con 0x7f60fc100a80 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.694+0000 7f60f37fe640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f60e80a32d0 0x7f60e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.694+0000 7f60f3fff640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f60e80a32d0 0x7f60e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.694+0000 7f60f37fe640 1 -- 172.21.15.94:0/527178965 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f60ec0cf990 con 0x7f60fc100a80 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.694+0000 7f60f37fe640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] conn(0x7f60e80acf00 0x7f60e80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.694+0000 7f60f37fe640 1 -- 172.21.15.94:0/527178965 --> [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60ec005320 con 0x7f60e80acf00 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.694+0000 7f60f37fe640 1 -- 172.21.15.94:0/527178965 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f60ec0cfd40 con 0x7f60fc100a80 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.695+0000 7f60fad9d640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] conn(0x7f60e80acf00 0x7f60e80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.695+0000 7f60f3fff640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f60e80a32d0 0x7f60e80a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60e40096c0 tx=0x7f60e4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.695+0000 7f60fad9d640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] conn(0x7f60e80acf00 0x7f60e80af3a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.696+0000 7f60f37fe640 1 -- 172.21.15.94:0/527178965 <== osd.9 v2:172.21.15.94:6826/704568533 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60ec005320 con 0x7f60e80acf00 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.709+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 --> [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60fc1021c0 con 0x7f60e80acf00 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.709+0000 7f60f37fe640 1 -- 172.21.15.94:0/527178965 <== osd.9 v2:172.21.15.94:6826/704568533 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f60fc1021c0 con 0x7f60e80acf00 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] conn(0x7f60e80acf00 msgr2=0x7f60e80af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] conn(0x7f60e80acf00 0x7f60e80af3a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f60e80a32d0 msgr2=0x7f60e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f60e80a32d0 0x7f60e80a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60e40096c0 tx=0x7f60e4031400 comp rx=0 tx=0).stop 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc100a80 msgr2=0x7f60fc1114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 --2- 172.21.15.94:0/527178965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc100a80 0x7f60fc1114b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00d770 tx=0x7f60ec00dc30 comp rx=0 tx=0).stop 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f60fad9d640 1 -- 172.21.15.94:0/527178965 reap_dead start 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 shutdown_connections 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 >> 172.21.15.94:0/527178965 conn(0x7f60fc0fc7a0 msgr2=0x7f60fc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 shutdown_connections 2024-09-19T04:13:58.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:58.710+0000 7f610095f640 1 -- 172.21.15.94:0/527178965 wait complete. 2024-09-19T04:13:59.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.102+0000 7f0cec4eb640 1 -- 172.21.15.94:0/3689511647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ce4106f10 msgr2=0x7f0ce410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.102+0000 7f0cec4eb640 1 --2- 172.21.15.94:0/3689511647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ce4106f10 0x7f0ce410ea90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009ca0 tx=0x7f0cdc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.102+0000 7f0cec4eb640 1 -- 172.21.15.94:0/3689511647 shutdown_connections 2024-09-19T04:13:59.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.102+0000 7f0cec4eb640 1 --2- 172.21.15.94:0/3689511647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ce4106f10 0x7f0ce410ea90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.102+0000 7f0cec4eb640 1 --2- 172.21.15.94:0/3689511647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce4102790 0x7f0ce41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.102+0000 7f0cec4eb640 1 --2- 172.21.15.94:0/3689511647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ce4101de0 0x7f0ce41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.102+0000 7f0cec4eb640 1 -- 172.21.15.94:0/3689511647 >> 172.21.15.94:0/3689511647 conn(0x7f0ce40fd8c0 msgr2=0x7f0ce40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:59.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.102+0000 7f0cec4eb640 1 -- 172.21.15.94:0/3689511647 shutdown_connections 2024-09-19T04:13:59.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.102+0000 7f0cec4eb640 1 -- 172.21.15.94:0/3689511647 wait complete. 2024-09-19T04:13:59.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0cec4eb640 1 Processor -- start 2024-09-19T04:13:59.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0cec4eb640 1 -- start start 2024-09-19T04:13:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0cec4eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce4101de0 0x7f0ce4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0cec4eb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ce4102790 0x7f0ce41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0cec4eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ce4106f10 0x7f0ce419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0cec4eb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4111e20 con 0x7f0ce4106f10 2024-09-19T04:13:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0cec4eb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4111ca0 con 0x7f0ce4102790 2024-09-19T04:13:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0cec4eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4111fa0 con 0x7f0ce4101de0 2024-09-19T04:13:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0ceaa88640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ce4106f10 0x7f0ce419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0ceaa88640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ce4106f10 0x7f0ce419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56124/0 (socket says 172.21.15.94:56124) 2024-09-19T04:13:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.103+0000 7f0ceaa88640 1 -- 172.21.15.94:0/1809989034 learned_addr learned my addr 172.21.15.94:0/1809989034 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0cea287640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce4101de0 0x7f0ce4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0ce9a86640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ce4102790 0x7f0ce41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0ceaa88640 1 -- 172.21.15.94:0/1809989034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce4101de0 msgr2=0x7f0ce4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0ceaa88640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce4101de0 0x7f0ce4198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0ceaa88640 1 -- 172.21.15.94:0/1809989034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ce4102790 msgr2=0x7f0ce41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0ceaa88640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ce4102790 0x7f0ce41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0ceaa88640 1 -- 172.21.15.94:0/1809989034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce419d330 con 0x7f0ce4106f10 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0ce9a86640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ce4102790 0x7f0ce41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0ceaa88640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ce4106f10 0x7f0ce419cc50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009dc0 tx=0x7f0cdc032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0cdb7fe640 1 -- 172.21.15.94:0/1809989034 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc003cf0 con 0x7f0ce4106f10 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0cdb7fe640 1 -- 172.21.15.94:0/1809989034 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cdc0044f0 con 0x7f0ce4106f10 2024-09-19T04:13:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce419d5c0 con 0x7f0ce4106f10 2024-09-19T04:13:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce41a4f00 con 0x7f0ce4106f10 2024-09-19T04:13:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.104+0000 7f0cdb7fe640 1 -- 172.21.15.94:0/1809989034 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc030bb0 con 0x7f0ce4106f10 2024-09-19T04:13:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.105+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ce4062900 con 0x7f0ce4106f10 2024-09-19T04:13:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.106+0000 7f0cdb7fe640 1 -- 172.21.15.94:0/1809989034 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0cdc004040 con 0x7f0ce4106f10 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.107+0000 7f0cdb7fe640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0cd00a32d0 0x7f0cd00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.107+0000 7f0cea287640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0cd00a32d0 0x7f0cd00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.107+0000 7f0cdb7fe640 1 -- 172.21.15.94:0/1809989034 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0cdc0f1470 con 0x7f0ce4106f10 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.107+0000 7f0cdb7fe640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] conn(0x7f0cd00acf00 0x7f0cd00af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.107+0000 7f0cdb7fe640 1 -- 172.21.15.94:0/1809989034 --> [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cdc035e50 con 0x7f0cd00acf00 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.107+0000 7f0cdb7fe640 1 -- 172.21.15.94:0/1809989034 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f0cdc0f18a0 con 0x7f0ce4106f10 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.107+0000 7f0ce9a86640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] conn(0x7f0cd00acf00 0x7f0cd00af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.108+0000 7f0cea287640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0cd00a32d0 0x7f0cd00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005fd0 tx=0x7f0ce0009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.108+0000 7f0ce9a86640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] conn(0x7f0cd00acf00 0x7f0cd00af3a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.108+0000 7f0cdb7fe640 1 -- 172.21.15.94:0/1809989034 <== osd.0 v2:172.21.15.94:6802/2563797567 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0cdc035e50 con 0x7f0cd00acf00 2024-09-19T04:13:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 --> [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ce4062d00 con 0x7f0cd00acf00 2024-09-19T04:13:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cdb7fe640 1 -- 172.21.15.94:0/1809989034 <== osd.0 v2:172.21.15.94:6802/2563797567 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0ce4062d00 con 0x7f0cd00acf00 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] conn(0x7f0cd00acf00 msgr2=0x7f0cd00af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] conn(0x7f0cd00acf00 0x7f0cd00af3a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0cd00a32d0 msgr2=0x7f0cd00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f0cd00a32d0 0x7f0cd00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005fd0 tx=0x7f0ce0009470 comp rx=0 tx=0).stop 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ce4106f10 msgr2=0x7f0ce419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 --2- 172.21.15.94:0/1809989034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ce4106f10 0x7f0ce419cc50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009dc0 tx=0x7f0cdc032ff0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0ceaa88640 1 -- 172.21.15.94:0/1809989034 reap_dead start 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 shutdown_connections 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 >> 172.21.15.94:0/1809989034 conn(0x7f0ce40fd8c0 msgr2=0x7f0ce40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 shutdown_connections 2024-09-19T04:13:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:13:59.122+0000 7f0cec4eb640 1 -- 172.21.15.94:0/1809989034 wait complete. 2024-09-19T04:13:59.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:13:59 smithi094 ceph-mon[30972]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T04:13:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:13:59 smithi112 ceph-mon[35051]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T04:13:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:13:59 smithi104 ceph-mon[34145]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T04:13:59.942 INFO:teuthology.orchestra.run.smithi094.stdout:154618822707 2024-09-19T04:13:59.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T04:14:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 -- 172.21.15.94:0/2912983534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb614102570 msgr2=0x7fb6141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 --2- 172.21.15.94:0/2912983534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb614102570 0x7fb6141029b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb608009900 tx=0x7fb60802a480 comp rx=0 tx=0).stop 2024-09-19T04:14:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 -- 172.21.15.94:0/2912983534 shutdown_connections 2024-09-19T04:14:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 --2- 172.21.15.94:0/2912983534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb614102ef0 0x7fb61410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 --2- 172.21.15.94:0/2912983534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb614102570 0x7fb6141029b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 --2- 172.21.15.94:0/2912983534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb614100e20 0x7fb614101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 -- 172.21.15.94:0/2912983534 >> 172.21.15.94:0/2912983534 conn(0x7fb6140fc7d0 msgr2=0x7fb6140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 -- 172.21.15.94:0/2912983534 shutdown_connections 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 -- 172.21.15.94:0/2912983534 wait complete. 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.137+0000 7fb6190de640 1 Processor -- start 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb6190de640 1 -- start start 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb6190de640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb614100e20 0x7fb6140fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb6190de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb614102570 0x7fb6140fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb6190de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb614102ef0 0x7fb6140fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb6190de640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb614110e50 con 0x7fb614102ef0 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb6190de640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb614110cd0 con 0x7fb614100e20 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb6190de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb614110fd0 con 0x7fb614102570 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb61359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb614102ef0 0x7fb6140fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb61359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb614102ef0 0x7fb6140fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56148/0 (socket says 172.21.15.94:56148) 2024-09-19T04:14:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb61359e640 1 -- 172.21.15.94:0/377695343 learned_addr learned my addr 172.21.15.94:0/377695343 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb612d9d640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb614100e20 0x7fb6140fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb61259c640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb614102570 0x7fb6140fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb61359e640 1 -- 172.21.15.94:0/377695343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb614102570 msgr2=0x7fb6140fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb61359e640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb614102570 0x7fb6140fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb61359e640 1 -- 172.21.15.94:0/377695343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb614100e20 msgr2=0x7fb6140fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb61359e640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb614100e20 0x7fb6140fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.138+0000 7fb61359e640 1 -- 172.21.15.94:0/377695343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6141aae60 con 0x7fb614102ef0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.139+0000 7fb61359e640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb614102ef0 0x7fb6140fd700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00b970 tx=0x7fb60c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.140+0000 7fb5fbfff640 1 -- 172.21.15.94:0/377695343 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c019070 con 0x7fb614102ef0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.140+0000 7fb5fbfff640 1 -- 172.21.15.94:0/377695343 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb60c014440 con 0x7fb614102ef0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.140+0000 7fb5fbfff640 1 -- 172.21.15.94:0/377695343 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c007590 con 0x7fb614102ef0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.140+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6141ab090 con 0x7fb614102ef0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.140+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6141ab550 con 0x7fb614102ef0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.141+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb614062900 con 0x7fb614102ef0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.142+0000 7fb5fbfff640 1 -- 172.21.15.94:0/377695343 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb60c01c070 con 0x7fb614102ef0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.143+0000 7fb5fbfff640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb5fc0a32d0 0x7fb5fc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.143+0000 7fb612d9d640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb5fc0a32d0 0x7fb5fc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.143+0000 7fb5fbfff640 1 -- 172.21.15.94:0/377695343 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb60c0cf720 con 0x7fb614102ef0 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.143+0000 7fb5fbfff640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] conn(0x7fb5fc0acf00 0x7fb5fc0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.143+0000 7fb5fbfff640 1 -- 172.21.15.94:0/377695343 --> [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb60c0047d0 con 0x7fb5fc0acf00 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.143+0000 7fb5fbfff640 1 -- 172.21.15.94:0/377695343 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fb60c0cfb50 con 0x7fb614102ef0 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.144+0000 7fb61259c640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] conn(0x7fb5fc0acf00 0x7fb5fc0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.144+0000 7fb612d9d640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb5fc0a32d0 0x7fb5fc0a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb60400bbe0 tx=0x7fb60400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.144+0000 7fb61259c640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] conn(0x7fb5fc0acf00 0x7fb5fc0af3a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.145+0000 7fb5fbfff640 1 -- 172.21.15.94:0/377695343 <== osd.11 v2:172.21.15.112:6824/3391428889 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb60c0047d0 con 0x7fb5fc0acf00 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.159+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 --> [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb614062d80 con 0x7fb5fc0acf00 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb5fbfff640 1 -- 172.21.15.94:0/377695343 <== osd.11 v2:172.21.15.112:6824/3391428889 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb614062d80 con 0x7fb5fc0acf00 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 >> [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] conn(0x7fb5fc0acf00 msgr2=0x7fb5fc0af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb6190de640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] conn(0x7fb5fc0acf00 0x7fb5fc0af3a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb5fc0a32d0 msgr2=0x7fb5fc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb6190de640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb5fc0a32d0 0x7fb5fc0a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb60400bbe0 tx=0x7fb60400b390 comp rx=0 tx=0).stop 2024-09-19T04:14:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb614102ef0 msgr2=0x7fb6140fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb6190de640 1 --2- 172.21.15.94:0/377695343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb614102ef0 0x7fb6140fd700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00b970 tx=0x7fb60c00be30 comp rx=0 tx=0).stop 2024-09-19T04:14:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb61359e640 1 -- 172.21.15.94:0/377695343 reap_dead start 2024-09-19T04:14:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 shutdown_connections 2024-09-19T04:14:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.160+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 >> 172.21.15.94:0/377695343 conn(0x7fb6140fc7d0 msgr2=0x7fb61406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.161+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 shutdown_connections 2024-09-19T04:14:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:00.161+0000 7fb6190de640 1 -- 172.21.15.94:0/377695343 wait complete. 2024-09-19T04:14:01.152 INFO:teuthology.orchestra.run.smithi094.stdout:313532612630 2024-09-19T04:14:01.152 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T04:14:01.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.301+0000 7fe29602e640 1 -- 172.21.15.94:0/113879747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe290102570 msgr2=0x7fe2901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.301+0000 7fe29602e640 1 --2- 172.21.15.94:0/113879747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe290102570 0x7fe2901029b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe284009ca0 tx=0x7fe28402a950 comp rx=0 tx=0).stop 2024-09-19T04:14:01.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.301+0000 7fe29602e640 1 -- 172.21.15.94:0/113879747 shutdown_connections 2024-09-19T04:14:01.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.301+0000 7fe29602e640 1 --2- 172.21.15.94:0/113879747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe290102ef0 0x7fe29010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.301+0000 7fe29602e640 1 --2- 172.21.15.94:0/113879747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe290102570 0x7fe2901029b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.301+0000 7fe29602e640 1 --2- 172.21.15.94:0/113879747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe290100e20 0x7fe290101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.301+0000 7fe29602e640 1 -- 172.21.15.94:0/113879747 >> 172.21.15.94:0/113879747 conn(0x7fe2900fc7d0 msgr2=0x7fe2900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:01.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.302+0000 7fe29602e640 1 -- 172.21.15.94:0/113879747 shutdown_connections 2024-09-19T04:14:01.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.302+0000 7fe29602e640 1 -- 172.21.15.94:0/113879747 wait complete. 2024-09-19T04:14:01.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.302+0000 7fe29602e640 1 Processor -- start 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe29602e640 1 -- start start 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe29602e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe290100e20 0x7fe290198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe29602e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe290102570 0x7fe2901989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe29602e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe290102ef0 0x7fe29019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe29602e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe290110e50 con 0x7fe290102ef0 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe29602e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe290110cd0 con 0x7fe290100e20 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe29602e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe290110fd0 con 0x7fe290102570 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe290102ef0 0x7fe29019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe290102ef0 0x7fe29019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56174/0 (socket says 172.21.15.94:56174) 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28ffff640 1 -- 172.21.15.94:0/2168493227 learned_addr learned my addr 172.21.15.94:0/2168493227 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28effd640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe290102570 0x7fe2901989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28f7fe640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe290100e20 0x7fe290198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28ffff640 1 -- 172.21.15.94:0/2168493227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe290102570 msgr2=0x7fe2901989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28ffff640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe290102570 0x7fe2901989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28ffff640 1 -- 172.21.15.94:0/2168493227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe290100e20 msgr2=0x7fe290198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28ffff640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe290100e20 0x7fe290198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28ffff640 1 -- 172.21.15.94:0/2168493227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29019d420 con 0x7fe290102ef0 2024-09-19T04:14:01.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28f7fe640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe290100e20 0x7fe290198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:01.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.303+0000 7fe28ffff640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe290102ef0 0x7fe29019cd40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c970 tx=0x7fe28800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:01.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.304+0000 7fe28cff9640 1 -- 172.21.15.94:0/2168493227 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288007bb0 con 0x7fe290102ef0 2024-09-19T04:14:01.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.304+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29019d710 con 0x7fe290102ef0 2024-09-19T04:14:01.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.304+0000 7fe28cff9640 1 -- 172.21.15.94:0/2168493227 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe288007d50 con 0x7fe290102ef0 2024-09-19T04:14:01.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.304+0000 7fe28cff9640 1 -- 172.21.15.94:0/2168493227 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288018700 con 0x7fe290102ef0 2024-09-19T04:14:01.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.304+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2900fcbb0 con 0x7fe290102ef0 2024-09-19T04:14:01.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.305+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe290062980 con 0x7fe290102ef0 2024-09-19T04:14:01.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.307+0000 7fe28cff9640 1 -- 172.21.15.94:0/2168493227 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe28800fa20 con 0x7fe290102ef0 2024-09-19T04:14:01.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.307+0000 7fe28cff9640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe2780a32d0 0x7fe2780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.307+0000 7fe28f7fe640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe2780a32d0 0x7fe2780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.307+0000 7fe28cff9640 1 -- 172.21.15.94:0/2168493227 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe2880cfe50 con 0x7fe290102ef0 2024-09-19T04:14:01.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.308+0000 7fe28cff9640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] conn(0x7fe2780acf00 0x7fe2780af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.308+0000 7fe28cff9640 1 -- 172.21.15.94:0/2168493227 --> [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe28801ee50 con 0x7fe2780acf00 2024-09-19T04:14:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.308+0000 7fe28cff9640 1 -- 172.21.15.94:0/2168493227 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fe2880d0280 con 0x7fe290102ef0 2024-09-19T04:14:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.308+0000 7fe28effd640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] conn(0x7fe2780acf00 0x7fe2780af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.308+0000 7fe28f7fe640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe2780a32d0 0x7fe2780a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe280009930 tx=0x7fe280008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.308+0000 7fe28effd640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] conn(0x7fe2780acf00 0x7fe2780af3a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.309+0000 7fe28cff9640 1 -- 172.21.15.94:0/2168493227 <== osd.7 v2:172.21.15.104:6816/2625303953 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe28801ee50 con 0x7fe2780acf00 2024-09-19T04:14:01.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.323+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 --> [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe290062e00 con 0x7fe2780acf00 2024-09-19T04:14:01.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe28cff9640 1 -- 172.21.15.94:0/2168493227 <== osd.7 v2:172.21.15.104:6816/2625303953 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe290062e00 con 0x7fe2780acf00 2024-09-19T04:14:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 >> [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] conn(0x7fe2780acf00 msgr2=0x7fe2780af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] conn(0x7fe2780acf00 0x7fe2780af3a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe2780a32d0 msgr2=0x7fe2780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe2780a32d0 0x7fe2780a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe280009930 tx=0x7fe280008040 comp rx=0 tx=0).stop 2024-09-19T04:14:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe290102ef0 msgr2=0x7fe29019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 --2- 172.21.15.94:0/2168493227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe290102ef0 0x7fe29019cd40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c970 tx=0x7fe28800ce30 comp rx=0 tx=0).stop 2024-09-19T04:14:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe28ffff640 1 -- 172.21.15.94:0/2168493227 reap_dead start 2024-09-19T04:14:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 shutdown_connections 2024-09-19T04:14:01.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 >> 172.21.15.94:0/2168493227 conn(0x7fe2900fc7d0 msgr2=0x7fe29006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:01.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 shutdown_connections 2024-09-19T04:14:01.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:01.324+0000 7fe29602e640 1 -- 172.21.15.94:0/2168493227 wait complete. 2024-09-19T04:14:01.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:01 smithi112 ceph-mon[35051]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T04:14:01.538 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:01 smithi094 ceph-mon[30972]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T04:14:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:01 smithi104 ceph-mon[34145]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T04:14:01.804 INFO:teuthology.orchestra.run.smithi094.stdout:103079215165 2024-09-19T04:14:01.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T04:14:02.684 INFO:teuthology.orchestra.run.smithi094.stdout:231928234020 2024-09-19T04:14:02.685 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T04:14:03.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:03 smithi112 ceph-mon[35051]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:03.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:03 smithi094 ceph-mon[30972]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:03 smithi104 ceph-mon[34145]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:03.671 INFO:teuthology.orchestra.run.smithi094.stdout:34359738443 2024-09-19T04:14:03.672 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T04:14:03.840 INFO:teuthology.orchestra.run.smithi094.stdout:201863462953 2024-09-19T04:14:03.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi094.stdout:261993005088 2024-09-19T04:14:04.351 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T04:14:04.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.611+0000 7faacd1e9640 1 -- 172.21.15.94:0/1882192922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8100e20 msgr2=0x7faac8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.611+0000 7faacd1e9640 1 --2- 172.21.15.94:0/1882192922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8100e20 0x7faac8101200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faab8009900 tx=0x7faab802a480 comp rx=0 tx=0).stop 2024-09-19T04:14:04.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.611+0000 7faacd1e9640 1 -- 172.21.15.94:0/1882192922 shutdown_connections 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.611+0000 7faacd1e9640 1 --2- 172.21.15.94:0/1882192922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faac8102ef0 0x7faac810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.611+0000 7faacd1e9640 1 --2- 172.21.15.94:0/1882192922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac8102570 0x7faac81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.611+0000 7faacd1e9640 1 --2- 172.21.15.94:0/1882192922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8100e20 0x7faac8101200 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.611+0000 7faacd1e9640 1 -- 172.21.15.94:0/1882192922 >> 172.21.15.94:0/1882192922 conn(0x7faac80fc7d0 msgr2=0x7faac80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.611+0000 7faacd1e9640 1 -- 172.21.15.94:0/1882192922 shutdown_connections 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.611+0000 7faacd1e9640 1 -- 172.21.15.94:0/1882192922 wait complete. 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.612+0000 7faacd1e9640 1 Processor -- start 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.612+0000 7faacd1e9640 1 -- start start 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.612+0000 7faacd1e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac8100e20 0x7faac8197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.612+0000 7faacd1e9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faac8102570 0x7faac81978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.612+0000 7faacd1e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8102ef0 0x7faac819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.612+0000 7faacd1e9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faac8110e50 con 0x7faac8102ef0 2024-09-19T04:14:04.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.612+0000 7faacd1e9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faac8110cd0 con 0x7faac8102570 2024-09-19T04:14:04.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.612+0000 7faacd1e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faac8110fd0 con 0x7faac8100e20 2024-09-19T04:14:04.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8102ef0 0x7faac819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8102ef0 0x7faac819bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50578/0 (socket says 172.21.15.94:50578) 2024-09-19T04:14:04.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac759e640 1 -- 172.21.15.94:0/4186224093 learned_addr learned my addr 172.21.15.94:0/4186224093 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:04.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac6d9d640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac8100e20 0x7faac8197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac659c640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faac8102570 0x7faac81978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac759e640 1 -- 172.21.15.94:0/4186224093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac8100e20 msgr2=0x7faac8197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac759e640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac8100e20 0x7faac8197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac759e640 1 -- 172.21.15.94:0/4186224093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faac8102570 msgr2=0x7faac81978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac759e640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faac8102570 0x7faac81978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac759e640 1 -- 172.21.15.94:0/4186224093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac819c340 con 0x7faac8102ef0 2024-09-19T04:14:04.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac6d9d640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac8100e20 0x7faac8197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:04.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faac759e640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8102ef0 0x7faac819bc60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faac000eaa0 tx=0x7faac000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faaaffff640 1 -- 172.21.15.94:0/4186224093 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac000cdb0 con 0x7faac8102ef0 2024-09-19T04:14:04.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac819c630 con 0x7faac8102ef0 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faaaffff640 1 -- 172.21.15.94:0/4186224093 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faac0005020 con 0x7faac8102ef0 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faaaffff640 1 -- 172.21.15.94:0/4186224093 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac00116d0 con 0x7faac8102ef0 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.613+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac81a3f10 con 0x7faac8102ef0 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.614+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faac8062940 con 0x7faac8102ef0 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.615+0000 7faaaffff640 1 -- 172.21.15.94:0/4186224093 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7faac00051d0 con 0x7faac8102ef0 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.616+0000 7faaaffff640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faab00a32d0 0x7faab00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.616+0000 7faac6d9d640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faab00a32d0 0x7faab00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.616+0000 7faaaffff640 1 -- 172.21.15.94:0/4186224093 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faac00d0120 con 0x7faac8102ef0 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.616+0000 7faaaffff640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] conn(0x7faab00acf00 0x7faab00af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.616+0000 7faaaffff640 1 -- 172.21.15.94:0/4186224093 --> [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faac0014ec0 con 0x7faab00acf00 2024-09-19T04:14:04.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.616+0000 7faaaffff640 1 -- 172.21.15.94:0/4186224093 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7faac00d0550 con 0x7faac8102ef0 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.616+0000 7faac6d9d640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faab00a32d0 0x7faab00a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faab8005ab0 tx=0x7faab8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.616+0000 7faac659c640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] conn(0x7faab00acf00 0x7faab00af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.617+0000 7faac659c640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] conn(0x7faab00acf00 0x7faab00af3a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.618+0000 7faaaffff640 1 -- 172.21.15.94:0/4186224093 <== osd.2 v2:172.21.15.112:6800/1893936170 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faac0014ec0 con 0x7faab00acf00 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.631+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 --> [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faac8062dc0 con 0x7faab00acf00 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.632+0000 7faaaffff640 1 -- 172.21.15.94:0/4186224093 <== osd.2 v2:172.21.15.112:6800/1893936170 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faac8062dc0 con 0x7faab00acf00 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.632+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 >> [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] conn(0x7faab00acf00 msgr2=0x7faab00af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.632+0000 7faacd1e9640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] conn(0x7faab00acf00 0x7faab00af3a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.632+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faab00a32d0 msgr2=0x7faab00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.632+0000 7faacd1e9640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7faab00a32d0 0x7faab00a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faab8005ab0 tx=0x7faab8005a20 comp rx=0 tx=0).stop 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.632+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8102ef0 msgr2=0x7faac819bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.632+0000 7faacd1e9640 1 --2- 172.21.15.94:0/4186224093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8102ef0 0x7faac819bc60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faac000eaa0 tx=0x7faac000ef60 comp rx=0 tx=0).stop 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.632+0000 7faac759e640 1 -- 172.21.15.94:0/4186224093 reap_dead start 2024-09-19T04:14:04.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.633+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 shutdown_connections 2024-09-19T04:14:04.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.633+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 >> 172.21.15.94:0/4186224093 conn(0x7faac80fc7d0 msgr2=0x7faac80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:04.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.633+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 shutdown_connections 2024-09-19T04:14:04.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:04.633+0000 7faacd1e9640 1 -- 172.21.15.94:0/4186224093 wait complete. 2024-09-19T04:14:04.648 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:04.862 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:04.879 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:04.880 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:04.888 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:05.316 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:05 smithi094 ceph-mon[30972]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-19T04:14:05.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:05 smithi112 ceph-mon[35051]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-19T04:14:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:05 smithi104 ceph-mon[34145]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-19T04:14:05.709 INFO:teuthology.orchestra.run.smithi094.stdout:68719476803 2024-09-19T04:14:05.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T04:14:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.139+0000 7f5be12a5640 1 -- 172.21.15.94:0/2685671524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc102dc0 msgr2=0x7f5bdc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.139+0000 7f5be12a5640 1 --2- 172.21.15.94:0/2685671524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc102dc0 0x7f5bdc1031a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009900 tx=0x7f5bcc02a480 comp rx=0 tx=0).stop 2024-09-19T04:14:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.139+0000 7f5be12a5640 1 -- 172.21.15.94:0/2685671524 shutdown_connections 2024-09-19T04:14:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.139+0000 7f5be12a5640 1 --2- 172.21.15.94:0/2685671524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc10c1a0 0x7f5bdc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.139+0000 7f5be12a5640 1 --2- 172.21.15.94:0/2685671524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bdc1036e0 0x7f5bdc10bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.139+0000 7f5be12a5640 1 --2- 172.21.15.94:0/2685671524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc102dc0 0x7f5bdc1031a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.139+0000 7f5be12a5640 1 -- 172.21.15.94:0/2685671524 >> 172.21.15.94:0/2685671524 conn(0x7f5bdc0fc7a0 msgr2=0x7f5bdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.139+0000 7f5be12a5640 1 -- 172.21.15.94:0/2685671524 shutdown_connections 2024-09-19T04:14:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.140+0000 7f5be12a5640 1 -- 172.21.15.94:0/2685671524 wait complete. 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.140+0000 7f5be12a5640 1 Processor -- start 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.140+0000 7f5be12a5640 1 -- start start 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.140+0000 7f5be12a5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc102dc0 0x7f5bdc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.140+0000 7f5be12a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc1036e0 0x7f5bdc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.140+0000 7f5be12a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bdc10c1a0 0x7f5bdc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.140+0000 7f5be12a5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc111e20 con 0x7f5bdc1036e0 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.140+0000 7f5be12a5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc111ca0 con 0x7f5bdc102dc0 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.140+0000 7f5be12a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc111fa0 con 0x7f5bdc10c1a0 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bda7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc1036e0 0x7f5bdc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bda7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc1036e0 0x7f5bdc1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50602/0 (socket says 172.21.15.94:50602) 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bda7fc640 1 -- 172.21.15.94:0/1960241670 learned_addr learned my addr 172.21.15.94:0/1960241670 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bdaffd640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc102dc0 0x7f5bdc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bdb7fe640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bdc10c1a0 0x7f5bdc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bda7fc640 1 -- 172.21.15.94:0/1960241670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bdc10c1a0 msgr2=0x7f5bdc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bda7fc640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bdc10c1a0 0x7f5bdc19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bda7fc640 1 -- 172.21.15.94:0/1960241670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc102dc0 msgr2=0x7f5bdc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bda7fc640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc102dc0 0x7f5bdc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bda7fc640 1 -- 172.21.15.94:0/1960241670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc19d350 con 0x7f5bdc1036e0 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bda7fc640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc1036e0 0x7f5bdc1988e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d950 tx=0x7f5bd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bc3fff640 1 -- 172.21.15.94:0/1960241670 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0017070 con 0x7f5bdc1036e0 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bc3fff640 1 -- 172.21.15.94:0/1960241670 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bd0009c30 con 0x7f5bdc1036e0 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5bc3fff640 1 -- 172.21.15.94:0/1960241670 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000f620 con 0x7f5bdc1036e0 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bdc19d640 con 0x7f5bdc1036e0 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.141+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bdc1a4f20 con 0x7f5bdc1036e0 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.142+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bdc1031a0 con 0x7f5bdc1036e0 2024-09-19T04:14:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.143+0000 7f5bc3fff640 1 -- 172.21.15.94:0/1960241670 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5bd00109a0 con 0x7f5bdc1036e0 2024-09-19T04:14:06.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.144+0000 7f5bc3fff640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5bc40a32d0 0x7f5bc40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.144+0000 7f5bdaffd640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5bc40a32d0 0x7f5bc40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.144+0000 7f5bc3fff640 1 -- 172.21.15.94:0/1960241670 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5bd00cf640 con 0x7f5bdc1036e0 2024-09-19T04:14:06.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.144+0000 7f5bdaffd640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5bc40a32d0 0x7f5bc40a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc031440 tx=0x7f5bcc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.148+0000 7f5bc3fff640 1 -- 172.21.15.94:0/1960241670 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5bd0009de0 con 0x7f5bdc1036e0 2024-09-19T04:14:06.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.323+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5bdc1072a0 con 0x7f5bdc1036e0 2024-09-19T04:14:06.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.324+0000 7f5bc3fff640 1 -- 172.21.15.94:0/1960241670 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5bd009bbb0 con 0x7f5bdc1036e0 2024-09-19T04:14:06.325 INFO:teuthology.orchestra.run.smithi094.stdout:184683593776 2024-09-19T04:14:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5bc40a32d0 msgr2=0x7f5bc40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5bc40a32d0 0x7f5bc40a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc031440 tx=0x7f5bcc005b50 comp rx=0 tx=0).stop 2024-09-19T04:14:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc1036e0 msgr2=0x7f5bdc1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc1036e0 0x7f5bdc1988e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d950 tx=0x7f5bd000de10 comp rx=0 tx=0).stop 2024-09-19T04:14:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 shutdown_connections 2024-09-19T04:14:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5bc40a32d0 0x7f5bc40a5790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bdc10c1a0 0x7f5bdc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bdc1036e0 0x7f5bdc1988e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 --2- 172.21.15.94:0/1960241670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc102dc0 0x7f5bdc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 >> 172.21.15.94:0/1960241670 conn(0x7f5bdc0fc7a0 msgr2=0x7f5bdc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 shutdown_connections 2024-09-19T04:14:06.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.326+0000 7f5be12a5640 1 -- 172.21.15.94:0/1960241670 wait complete. 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 -- 172.21.15.94:0/2980271552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a68102d30 msgr2=0x7f6a68103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/2980271552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a68102d30 0x7f6a68103110 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009930 tx=0x7f6a5802a440 comp rx=0 tx=0).stop 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 -- 172.21.15.94:0/2980271552 shutdown_connections 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/2980271552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a681082a0 0x7f6a6810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/2980271552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681036e0 0x7f6a68107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/2980271552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a68102d30 0x7f6a68103110 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 -- 172.21.15.94:0/2980271552 >> 172.21.15.94:0/2980271552 conn(0x7f6a680fc7d0 msgr2=0x7f6a680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 -- 172.21.15.94:0/2980271552 shutdown_connections 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 -- 172.21.15.94:0/2980271552 wait complete. 2024-09-19T04:14:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 Processor -- start 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.360+0000 7f6a6dc20640 1 -- start start 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a6dc20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68102d30 0x7f6a68198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a6dc20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a681036e0 0x7f6a68198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a6dc20640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a681082a0 0x7f6a6819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a6dc20640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a68111f90 con 0x7f6a681036e0 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a6dc20640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a68111e10 con 0x7f6a681082a0 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a6dc20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a68112110 con 0x7f6a68102d30 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a66ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a681036e0 0x7f6a68198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a66ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a681036e0 0x7f6a68198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50630/0 (socket says 172.21.15.94:50630) 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a66ffd640 1 -- 172.21.15.94:0/5226984 learned_addr learned my addr 172.21.15.94:0/5226984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a677fe640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68102d30 0x7f6a68198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a67fff640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a681082a0 0x7f6a6819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a66ffd640 1 -- 172.21.15.94:0/5226984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68102d30 msgr2=0x7f6a68198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a66ffd640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68102d30 0x7f6a68198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a66ffd640 1 -- 172.21.15.94:0/5226984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a681082a0 msgr2=0x7f6a6819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a66ffd640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a681082a0 0x7f6a6819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a66ffd640 1 -- 172.21.15.94:0/5226984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6819d540 con 0x7f6a681036e0 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a66ffd640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a681036e0 0x7f6a68198ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c006fd0 tx=0x7f6a5c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a64ff9640 1 -- 172.21.15.94:0/5226984 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c011830 con 0x7f6a681036e0 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a64ff9640 1 -- 172.21.15.94:0/5226984 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a5c016070 con 0x7f6a681036e0 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.361+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6819d830 con 0x7f6a681036e0 2024-09-19T04:14:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.362+0000 7f6a64ff9640 1 -- 172.21.15.94:0/5226984 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c010df0 con 0x7f6a681036e0 2024-09-19T04:14:06.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.362+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a681a5110 con 0x7f6a681036e0 2024-09-19T04:14:06.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.363+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a6806a850 con 0x7f6a681036e0 2024-09-19T04:14:06.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.364+0000 7f6a64ff9640 1 -- 172.21.15.94:0/5226984 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6a5c0119d0 con 0x7f6a681036e0 2024-09-19T04:14:06.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.364+0000 7f6a64ff9640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6a500a32d0 0x7f6a500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.365+0000 7f6a677fe640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6a500a32d0 0x7f6a500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.365+0000 7f6a64ff9640 1 -- 172.21.15.94:0/5226984 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6a5c0cef70 con 0x7f6a681036e0 2024-09-19T04:14:06.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.365+0000 7f6a677fe640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6a500a32d0 0x7f6a500a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009900 tx=0x7f6a58000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.369+0000 7f6a64ff9640 1 -- 172.21.15.94:0/5226984 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6a5c09bc40 con 0x7f6a681036e0 2024-09-19T04:14:06.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.542+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6a6810a2b0 con 0x7f6a681036e0 2024-09-19T04:14:06.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.543+0000 7f6a64ff9640 1 -- 172.21.15.94:0/5226984 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6a5c09b540 con 0x7f6a681036e0 2024-09-19T04:14:06.545 INFO:teuthology.orchestra.run.smithi094.stdout:283467841564 2024-09-19T04:14:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6a500a32d0 msgr2=0x7f6a500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6a500a32d0 0x7f6a500a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009900 tx=0x7f6a58000bc0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a681036e0 msgr2=0x7f6a68198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a681036e0 0x7f6a68198ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c006fd0 tx=0x7f6a5c00b8e0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 shutdown_connections 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f6a500a32d0 0x7f6a500a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a681082a0 0x7f6a6819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a681036e0 0x7f6a68198ad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 --2- 172.21.15.94:0/5226984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68102d30 0x7f6a68198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 >> 172.21.15.94:0/5226984 conn(0x7f6a680fc7d0 msgr2=0x7f6a68105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 shutdown_connections 2024-09-19T04:14:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:06.545+0000 7f6a6dc20640 1 -- 172.21.15.94:0/5226984 wait complete. 2024-09-19T04:14:07.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:07 smithi112 ceph-mon[35051]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T04:14:07.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:07 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1960241670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:14:07.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:07 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/5226984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:14:07.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:07 smithi094 ceph-mon[30972]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T04:14:07.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:07 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1960241670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:14:07.555 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:07 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/5226984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:14:07.581 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841562 got 283467841564 for osd.10 2024-09-19T04:14:07.581 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:07 smithi104 ceph-mon[34145]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T04:14:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:07 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1960241670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:14:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:07 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/5226984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:14:07.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.749+0000 7fe51ddeb640 1 -- 172.21.15.94:0/3532407622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518101de0 msgr2=0x7fe5181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.749+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/3532407622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518101de0 0x7fe5181021c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe508009900 tx=0x7fe50802a410 comp rx=0 tx=0).stop 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.749+0000 7fe51ddeb640 1 -- 172.21.15.94:0/3532407622 shutdown_connections 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.749+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/3532407622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe518106f10 0x7fe51810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.749+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/3532407622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe518102790 0x7fe5181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.749+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/3532407622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518101de0 0x7fe5181021c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.749+0000 7fe51ddeb640 1 -- 172.21.15.94:0/3532407622 >> 172.21.15.94:0/3532407622 conn(0x7fe5180fd8c0 msgr2=0x7fe5180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.749+0000 7fe51ddeb640 1 -- 172.21.15.94:0/3532407622 shutdown_connections 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.749+0000 7fe51ddeb640 1 -- 172.21.15.94:0/3532407622 wait complete. 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe51ddeb640 1 Processor -- start 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe51ddeb640 1 -- start start 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe51ddeb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe518101de0 0x7fe5180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe51ddeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe518102790 0x7fe518074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe51ddeb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518106f10 0x7fe518074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe51ddeb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe518111e40 con 0x7fe518106f10 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe51ddeb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe518111cc0 con 0x7fe518101de0 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe51ddeb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe518111fc0 con 0x7fe518102790 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe517fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518106f10 0x7fe518074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe517fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518106f10 0x7fe518074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50642/0 (socket says 172.21.15.94:50642) 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe517fff640 1 -- 172.21.15.94:0/2316458790 learned_addr learned my addr 172.21.15.94:0/2316458790 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe5177fe640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe518101de0 0x7fe5180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe50ffff640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe518102790 0x7fe518074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe517fff640 1 -- 172.21.15.94:0/2316458790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe518102790 msgr2=0x7fe518074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe517fff640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe518102790 0x7fe518074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe517fff640 1 -- 172.21.15.94:0/2316458790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe518101de0 msgr2=0x7fe5180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe517fff640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe518101de0 0x7fe5180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe517fff640 1 -- 172.21.15.94:0/2316458790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5181a6880 con 0x7fe518106f10 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe5177fe640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe518101de0 0x7fe5180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.750+0000 7fe517fff640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518106f10 0x7fe518074f70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cad0 tx=0x7fe51000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.751+0000 7fe5157fa640 1 -- 172.21.15.94:0/2316458790 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51000ae20 con 0x7fe518106f10 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.751+0000 7fe5157fa640 1 -- 172.21.15.94:0/2316458790 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe510005020 con 0x7fe518106f10 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.751+0000 7fe5157fa640 1 -- 172.21.15.94:0/2316458790 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51000f6b0 con 0x7fe518106f10 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.751+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5181a6b70 con 0x7fe518106f10 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.751+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5181a70b0 con 0x7fe518106f10 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.752+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5181021c0 con 0x7fe518106f10 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.753+0000 7fe5157fa640 1 -- 172.21.15.94:0/2316458790 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe5100051d0 con 0x7fe518106f10 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.753+0000 7fe5157fa640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe5040a3350 0x7fe5040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.754+0000 7fe5177fe640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe5040a3350 0x7fe5040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.754+0000 7fe5157fa640 1 -- 172.21.15.94:0/2316458790 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe5100cfed0 con 0x7fe518106f10 2024-09-19T04:14:07.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.754+0000 7fe5177fe640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe5040a3350 0x7fe5040a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe508000c00 tx=0x7fe508031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:07.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.757+0000 7fe5157fa640 1 -- 172.21.15.94:0/2316458790 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe510014030 con 0x7fe518106f10 2024-09-19T04:14:07.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.939+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe51802d7c0 con 0x7fe518106f10 2024-09-19T04:14:07.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.939+0000 7fe5157fa640 1 -- 172.21.15.94:0/2316458790 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe51009cb10 con 0x7fe518106f10 2024-09-19T04:14:07.940 INFO:teuthology.orchestra.run.smithi094.stdout:133143986234 2024-09-19T04:14:07.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe5040a3350 msgr2=0x7fe5040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe5040a3350 0x7fe5040a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe508000c00 tx=0x7fe508031400 comp rx=0 tx=0).stop 2024-09-19T04:14:07.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518106f10 msgr2=0x7fe518074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518106f10 0x7fe518074f70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cad0 tx=0x7fe51000cf90 comp rx=0 tx=0).stop 2024-09-19T04:14:07.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 shutdown_connections 2024-09-19T04:14:07.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fe5040a3350 0x7fe5040a5810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe518106f10 0x7fe518074f70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe518102790 0x7fe518074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 --2- 172.21.15.94:0/2316458790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe518101de0 0x7fe5180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 >> 172.21.15.94:0/2316458790 conn(0x7fe5180fd8c0 msgr2=0x7fe5180fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:07.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 shutdown_connections 2024-09-19T04:14:07.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:07.941+0000 7fe51ddeb640 1 -- 172.21.15.94:0/2316458790 wait complete. 2024-09-19T04:14:08.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:08 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2316458790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:14:08.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:08 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2316458790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:14:08.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:08 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2316458790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:14:08.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986232 got 133143986234 for osd.4 2024-09-19T04:14:08.667 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.045+0000 7fabed39e640 1 -- 172.21.15.94:0/3845580754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 msgr2=0x7fabe810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.045+0000 7fabed39e640 1 --2- 172.21.15.94:0/3845580754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 0x7fabe810b9a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009900 tx=0x7fabdc02a480 comp rx=0 tx=0).stop 2024-09-19T04:14:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.045+0000 7fabed39e640 1 -- 172.21.15.94:0/3845580754 shutdown_connections 2024-09-19T04:14:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.045+0000 7fabed39e640 1 --2- 172.21.15.94:0/3845580754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe810bfc0 0x7fabe810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.045+0000 7fabed39e640 1 --2- 172.21.15.94:0/3845580754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 0x7fabe810b9a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.045+0000 7fabed39e640 1 --2- 172.21.15.94:0/3845580754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe8102b90 0x7fabe8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.045+0000 7fabed39e640 1 -- 172.21.15.94:0/3845580754 >> 172.21.15.94:0/3845580754 conn(0x7fabe80fc610 msgr2=0x7fabe80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.045+0000 7fabed39e640 1 -- 172.21.15.94:0/3845580754 shutdown_connections 2024-09-19T04:14:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.045+0000 7fabed39e640 1 -- 172.21.15.94:0/3845580754 wait complete. 2024-09-19T04:14:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.046+0000 7fabed39e640 1 Processor -- start 2024-09-19T04:14:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.046+0000 7fabed39e640 1 -- start start 2024-09-19T04:14:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabed39e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe8102b90 0x7fabe8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabed39e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 0x7fabe8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabed39e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe810bfc0 0x7fabe819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabed39e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8111d30 con 0x7fabe81034b0 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabed39e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8111bb0 con 0x7fabe8102b90 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabed39e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8111eb0 con 0x7fabe810bfc0 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 0x7fabe8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 0x7fabe8198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50672/0 (socket says 172.21.15.94:50672) 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe77fe640 1 -- 172.21.15.94:0/3405747902 learned_addr learned my addr 172.21.15.94:0/3405747902 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabecb9d640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe810bfc0 0x7fabe819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe7fff640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe8102b90 0x7fabe8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe77fe640 1 -- 172.21.15.94:0/3405747902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe810bfc0 msgr2=0x7fabe819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe77fe640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe810bfc0 0x7fabe819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe77fe640 1 -- 172.21.15.94:0/3405747902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe8102b90 msgr2=0x7fabe8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe77fe640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe8102b90 0x7fabe8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe77fe640 1 -- 172.21.15.94:0/3405747902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe819d2e0 con 0x7fabe81034b0 2024-09-19T04:14:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe77fe640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 0x7fabe8198870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006010 tx=0x7fabdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe57fa640 1 -- 172.21.15.94:0/3405747902 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc03f030 con 0x7fabe81034b0 2024-09-19T04:14:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe819d570 con 0x7fabe81034b0 2024-09-19T04:14:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe57fa640 1 -- 172.21.15.94:0/3405747902 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabdc003800 con 0x7fabe81034b0 2024-09-19T04:14:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabe57fa640 1 -- 172.21.15.94:0/3405747902 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc035070 con 0x7fabe81034b0 2024-09-19T04:14:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.047+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe81a4eb0 con 0x7fabe81034b0 2024-09-19T04:14:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.048+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe806a850 con 0x7fabe81034b0 2024-09-19T04:14:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.050+0000 7fabe57fa640 1 -- 172.21.15.94:0/3405747902 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fabdc0039b0 con 0x7fabe81034b0 2024-09-19T04:14:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.050+0000 7fabe57fa640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fabd00a32d0 0x7fabd00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.050+0000 7fabe7fff640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fabd00a32d0 0x7fabd00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.050+0000 7fabe57fa640 1 -- 172.21.15.94:0/3405747902 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fabdc0f02c0 con 0x7fabe81034b0 2024-09-19T04:14:09.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.051+0000 7fabe7fff640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fabd00a32d0 0x7fabd00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009990 tx=0x7fabd8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:09.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.054+0000 7fabe57fa640 1 -- 172.21.15.94:0/3405747902 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fabdc0bce80 con 0x7fabe81034b0 2024-09-19T04:14:09.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.224+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fabe8107030 con 0x7fabe81034b0 2024-09-19T04:14:09.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.224+0000 7fabe57fa640 1 -- 172.21.15.94:0/3405747902 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fabdc0bc780 con 0x7fabe81034b0 2024-09-19T04:14:09.226 INFO:teuthology.orchestra.run.smithi094.stdout:154618822709 2024-09-19T04:14:09.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fabd00a32d0 msgr2=0x7fabd00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fabd00a32d0 0x7fabd00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009990 tx=0x7fabd8009110 comp rx=0 tx=0).stop 2024-09-19T04:14:09.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 msgr2=0x7fabe8198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 0x7fabe8198870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006010 tx=0x7fabdc0042f0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 shutdown_connections 2024-09-19T04:14:09.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fabd00a32d0 0x7fabd00a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe810bfc0 0x7fabe819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe81034b0 0x7fabe8198870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 --2- 172.21.15.94:0/3405747902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe8102b90 0x7fabe8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.226+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 >> 172.21.15.94:0/3405747902 conn(0x7fabe80fc610 msgr2=0x7fabe80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:09.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.227+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 shutdown_connections 2024-09-19T04:14:09.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.227+0000 7fabed39e640 1 -- 172.21.15.94:0/3405747902 wait complete. 2024-09-19T04:14:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.275+0000 7fd59b11f640 1 -- 172.21.15.94:0/2897773927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd594102da0 msgr2=0x7fd594103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.275+0000 7fd59b11f640 1 --2- 172.21.15.94:0/2897773927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd594102da0 0x7fd594103180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd588009900 tx=0x7fd58802a480 comp rx=0 tx=0).stop 2024-09-19T04:14:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 -- 172.21.15.94:0/2897773927 shutdown_connections 2024-09-19T04:14:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 --2- 172.21.15.94:0/2897773927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd59410c180 0x7fd59410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 --2- 172.21.15.94:0/2897773927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5941036c0 0x7fd59410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 --2- 172.21.15.94:0/2897773927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd594102da0 0x7fd594103180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 -- 172.21.15.94:0/2897773927 >> 172.21.15.94:0/2897773927 conn(0x7fd5940fc7a0 msgr2=0x7fd5940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 -- 172.21.15.94:0/2897773927 shutdown_connections 2024-09-19T04:14:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 -- 172.21.15.94:0/2897773927 wait complete. 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 Processor -- start 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 -- start start 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd594102da0 0x7fd594198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5941036c0 0x7fd594198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd59410c180 0x7fd59419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd594111e50 con 0x7fd5941036c0 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd594111cd0 con 0x7fd59410c180 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd59b11f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd594111fd0 con 0x7fd594102da0 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd593fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5941036c0 0x7fd594198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.276+0000 7fd593fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5941036c0 0x7fd594198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50688/0 (socket says 172.21.15.94:50688) 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd593fff640 1 -- 172.21.15.94:0/3108575247 learned_addr learned my addr 172.21.15.94:0/3108575247 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd598ebb640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd594102da0 0x7fd594198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd5996bc640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd59410c180 0x7fd59419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd593fff640 1 -- 172.21.15.94:0/3108575247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd594102da0 msgr2=0x7fd594198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd593fff640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd594102da0 0x7fd594198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd593fff640 1 -- 172.21.15.94:0/3108575247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd59410c180 msgr2=0x7fd59419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd593fff640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd59410c180 0x7fd59419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd593fff640 1 -- 172.21.15.94:0/3108575247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59419d2e0 con 0x7fd5941036c0 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd593fff640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5941036c0 0x7fd594198870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd58000c920 tx=0x7fd58000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd591ffb640 1 -- 172.21.15.94:0/3108575247 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580007c70 con 0x7fd5941036c0 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd591ffb640 1 -- 172.21.15.94:0/3108575247 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd580007e10 con 0x7fd5941036c0 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59419d5d0 con 0x7fd5941036c0 2024-09-19T04:14:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd591ffb640 1 -- 172.21.15.94:0/3108575247 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580018820 con 0x7fd5941036c0 2024-09-19T04:14:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.277+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5941a4eb0 con 0x7fd5941036c0 2024-09-19T04:14:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.278+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd594103180 con 0x7fd5941036c0 2024-09-19T04:14:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.279+0000 7fd591ffb640 1 -- 172.21.15.94:0/3108575247 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd5800189c0 con 0x7fd5941036c0 2024-09-19T04:14:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.280+0000 7fd591ffb640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd5840a32d0 0x7fd5840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.280+0000 7fd598ebb640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd5840a32d0 0x7fd5840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.280+0000 7fd591ffb640 1 -- 172.21.15.94:0/3108575247 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd5800cfd20 con 0x7fd5941036c0 2024-09-19T04:14:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.280+0000 7fd598ebb640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd5840a32d0 0x7fd5840a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd588031440 tx=0x7fd5880095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.285+0000 7fd591ffb640 1 -- 172.21.15.94:0/3108575247 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd580014030 con 0x7fd5941036c0 2024-09-19T04:14:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.454+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd594199690 con 0x7fd5941036c0 2024-09-19T04:14:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.454+0000 7fd591ffb640 1 -- 172.21.15.94:0/3108575247 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd594199690 con 0x7fd5941036c0 2024-09-19T04:14:09.456 INFO:teuthology.orchestra.run.smithi094.stdout:51539607625 2024-09-19T04:14:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd5840a32d0 msgr2=0x7fd5840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd5840a32d0 0x7fd5840a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd588031440 tx=0x7fd5880095f0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5941036c0 msgr2=0x7fd594198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5941036c0 0x7fd594198870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd58000c920 tx=0x7fd58000cde0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 shutdown_connections 2024-09-19T04:14:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd5840a32d0 0x7fd5840a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd59410c180 0x7fd59419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5941036c0 0x7fd594198870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 --2- 172.21.15.94:0/3108575247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd594102da0 0x7fd594198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 >> 172.21.15.94:0/3108575247 conn(0x7fd5940fc7a0 msgr2=0x7fd5940fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.456+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 shutdown_connections 2024-09-19T04:14:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:09.457+0000 7fd59b11f640 1 -- 172.21.15.94:0/3108575247 wait complete. 2024-09-19T04:14:09.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:09 smithi112 ceph-mon[35051]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-19T04:14:09.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:09 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3405747902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:14:09.546 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:09 smithi094 ceph-mon[30972]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-19T04:14:09.546 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:09 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3405747902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:14:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:09 smithi104 ceph-mon[34145]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-19T04:14:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:09 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3405747902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:14:10.152 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822707 got 154618822709 for osd.5 2024-09-19T04:14:10.152 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:10.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:10 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3108575247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:14:10.551 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:10 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3108575247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:14:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:10.568 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:10.568 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:10.569 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:10.602 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593773 got 184683593776 for osd.6 2024-09-19T04:14:10.602 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:10.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:10 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3108575247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:14:10.610 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:10.623 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:11.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.277+0000 7f8b401fb640 1 -- 172.21.15.94:0/3077605382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38100e20 msgr2=0x7f8b38101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.277+0000 7f8b401fb640 1 --2- 172.21.15.94:0/3077605382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38100e20 0x7f8b38101200 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009900 tx=0x7f8b3402a480 comp rx=0 tx=0).stop 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 -- 172.21.15.94:0/3077605382 shutdown_connections 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 --2- 172.21.15.94:0/3077605382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b38102ef0 0x7f8b3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 --2- 172.21.15.94:0/3077605382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b38102570 0x7f8b381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 --2- 172.21.15.94:0/3077605382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38100e20 0x7f8b38101200 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 -- 172.21.15.94:0/3077605382 >> 172.21.15.94:0/3077605382 conn(0x7f8b380fc7d0 msgr2=0x7f8b380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 -- 172.21.15.94:0/3077605382 shutdown_connections 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 -- 172.21.15.94:0/3077605382 wait complete. 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 Processor -- start 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 -- start start 2024-09-19T04:14:11.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b38100e20 0x7f8b381984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b38102570 0x7f8b381989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38102ef0 0x7f8b3819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38110e30 con 0x7f8b38102ef0 2024-09-19T04:14:11.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38110cb0 con 0x7f8b38100e20 2024-09-19T04:14:11.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.278+0000 7f8b401fb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38110fb0 con 0x7f8b38102570 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3e798640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38102ef0 0x7f8b3819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3e798640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38102ef0 0x7f8b3819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50700/0 (socket says 172.21.15.94:50700) 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3e798640 1 -- 172.21.15.94:0/1501574457 learned_addr learned my addr 172.21.15.94:0/1501574457 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3d796640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b38102570 0x7f8b381989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3df97640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b38100e20 0x7f8b381984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3e798640 1 -- 172.21.15.94:0/1501574457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b38102570 msgr2=0x7f8b381989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3e798640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b38102570 0x7f8b381989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3e798640 1 -- 172.21.15.94:0/1501574457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b38100e20 msgr2=0x7f8b381984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3e798640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b38100e20 0x7f8b381984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3e798640 1 -- 172.21.15.94:0/1501574457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3819d450 con 0x7f8b38102ef0 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3df97640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b38100e20 0x7f8b381984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b3e798640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38102ef0 0x7f8b3819cd70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000cad0 tx=0x7f8b3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b2effd640 1 -- 172.21.15.94:0/1501574457 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000ae20 con 0x7f8b38102ef0 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b2effd640 1 -- 172.21.15.94:0/1501574457 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b30005020 con 0x7f8b38102ef0 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3819d740 con 0x7f8b38102ef0 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b2effd640 1 -- 172.21.15.94:0/1501574457 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000f6b0 con 0x7f8b38102ef0 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.279+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b381a5020 con 0x7f8b38102ef0 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.280+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b38101200 con 0x7f8b38102ef0 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.281+0000 7f8b2effd640 1 -- 172.21.15.94:0/1501574457 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8b300051d0 con 0x7f8b38102ef0 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.282+0000 7f8b2effd640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8b240a32d0 0x7f8b240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.282+0000 7f8b3df97640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8b240a32d0 0x7f8b240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.282+0000 7f8b2effd640 1 -- 172.21.15.94:0/1501574457 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8b300cfa20 con 0x7f8b38102ef0 2024-09-19T04:14:11.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.283+0000 7f8b3df97640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8b240a32d0 0x7f8b240a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005ab0 tx=0x7f8b34005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:11.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.286+0000 7f8b2effd640 1 -- 172.21.15.94:0/1501574457 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8b30014030 con 0x7f8b38102ef0 2024-09-19T04:14:11.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:11 smithi094 ceph-mon[30972]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-19T04:14:11.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.455+0000 7fa8ace47640 1 -- 172.21.15.94:0/520917016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8102d30 msgr2=0x7fa8a8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.455+0000 7fa8ace47640 1 --2- 172.21.15.94:0/520917016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8102d30 0x7fa8a8103110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa898009900 tx=0x7fa89802a480 comp rx=0 tx=0).stop 2024-09-19T04:14:11.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.455+0000 7fa8ace47640 1 -- 172.21.15.94:0/520917016 shutdown_connections 2024-09-19T04:14:11.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.455+0000 7fa8ace47640 1 --2- 172.21.15.94:0/520917016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a810c160 0x7fa8a810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.455+0000 7fa8ace47640 1 --2- 172.21.15.94:0/520917016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8a8103650 0x7fa8a810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.455+0000 7fa8ace47640 1 --2- 172.21.15.94:0/520917016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8102d30 0x7fa8a8103110 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.455+0000 7fa8ace47640 1 -- 172.21.15.94:0/520917016 >> 172.21.15.94:0/520917016 conn(0x7fa8a80fc790 msgr2=0x7fa8a80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.455+0000 7fa8ace47640 1 -- 172.21.15.94:0/520917016 shutdown_connections 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.455+0000 7fa8ace47640 1 -- 172.21.15.94:0/520917016 wait complete. 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.456+0000 7fa8ace47640 1 Processor -- start 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.456+0000 7fa8ace47640 1 -- start start 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.456+0000 7fa8ace47640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a8102d30 0x7fa8a8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.456+0000 7fa8ace47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8103650 0x7fa8a81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.456+0000 7fa8ace47640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8a810c160 0x7fa8a819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.456+0000 7fa8ace47640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8111e10 con 0x7fa8a8103650 2024-09-19T04:14:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.456+0000 7fa8ace47640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8111c90 con 0x7fa8a810c160 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.456+0000 7fa8ace47640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8111f90 con 0x7fa8a8102d30 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8103650 0x7fa8a81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a8102d30 0x7fa8a8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8103650 0x7fa8a81988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50712/0 (socket says 172.21.15.94:50712) 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a5d9b640 1 -- 172.21.15.94:0/333938129 learned_addr learned my addr 172.21.15.94:0/333938129 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a8102d30 0x7fa8a8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:53592/0 (socket says 172.21.15.94:53592) 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a6d9d640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8a810c160 0x7fa8a819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a659c640 1 -- 172.21.15.94:0/333938129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8a810c160 msgr2=0x7fa8a819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a659c640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8a810c160 0x7fa8a819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a659c640 1 -- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8103650 msgr2=0x7fa8a81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a659c640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8103650 0x7fa8a81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a659c640 1 -- 172.21.15.94:0/333938129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a819d310 con 0x7fa8a8102d30 2024-09-19T04:14:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a5d9b640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8103650 0x7fa8a81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:14:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8a659c640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a8102d30 0x7fa8a8198360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa89802a9a0 tx=0x7fa898031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8977fe640 1 -- 172.21.15.94:0/333938129 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89803f030 con 0x7fa8a8102d30 2024-09-19T04:14:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a819d5a0 con 0x7fa8a8102d30 2024-09-19T04:14:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8977fe640 1 -- 172.21.15.94:0/333938129 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa898004490 con 0x7fa8a8102d30 2024-09-19T04:14:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8977fe640 1 -- 172.21.15.94:0/333938129 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898035070 con 0x7fa8a8102d30 2024-09-19T04:14:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.457+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a81a4ee0 con 0x7fa8a8102d30 2024-09-19T04:14:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.458+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a806b070 con 0x7fa8a8102d30 2024-09-19T04:14:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.460+0000 7fa8977fe640 1 -- 172.21.15.94:0/333938129 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa898042030 con 0x7fa8a8102d30 2024-09-19T04:14:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.461+0000 7fa8977fe640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8900a32d0 0x7fa8900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.461+0000 7fa8a5d9b640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8900a32d0 0x7fa8900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.461+0000 7fa8977fe640 1 -- 172.21.15.94:0/333938129 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa8980efcf0 con 0x7fa8a8102d30 2024-09-19T04:14:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.461+0000 7fa8a5d9b640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8900a32d0 0x7fa8900a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80fee00 tx=0x7fa89c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:11.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.462+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f8b381a5280 con 0x7f8b38102ef0 2024-09-19T04:14:11.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.463+0000 7f8b2effd640 1 -- 172.21.15.94:0/1501574457 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8b3009c5e0 con 0x7f8b38102ef0 2024-09-19T04:14:11.464 INFO:teuthology.orchestra.run.smithi094.stdout:313532612632 2024-09-19T04:14:11.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8b240a32d0 msgr2=0x7f8b240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8b240a32d0 0x7f8b240a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005ab0 tx=0x7f8b34005a20 comp rx=0 tx=0).stop 2024-09-19T04:14:11.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38102ef0 msgr2=0x7f8b3819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38102ef0 0x7f8b3819cd70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000cad0 tx=0x7f8b3000cf90 comp rx=0 tx=0).stop 2024-09-19T04:14:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 shutdown_connections 2024-09-19T04:14:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f8b240a32d0 0x7f8b240a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b38102ef0 0x7f8b3819cd70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b38102570 0x7f8b381989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 --2- 172.21.15.94:0/1501574457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b38100e20 0x7f8b381984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 >> 172.21.15.94:0/1501574457 conn(0x7f8b380fc7d0 msgr2=0x7f8b3806a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 shutdown_connections 2024-09-19T04:14:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7f8b401fb640 1 -- 172.21.15.94:0/1501574457 wait complete. 2024-09-19T04:14:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.465+0000 7fa8977fe640 1 -- 172.21.15.94:0/333938129 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8980bc260 con 0x7fa8a8102d30 2024-09-19T04:14:11.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:11 smithi112 ceph-mon[35051]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-19T04:14:11.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:11 smithi104 ceph-mon[34145]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-19T04:14:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.638+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa8a8107200 con 0x7fa8a8102d30 2024-09-19T04:14:11.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.638+0000 7fa8977fe640 1 -- 172.21.15.94:0/333938129 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa8980c0560 con 0x7fa8a8102d30 2024-09-19T04:14:11.640 INFO:teuthology.orchestra.run.smithi094.stdout:34359738446 2024-09-19T04:14:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8900a32d0 msgr2=0x7fa8900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8900a32d0 0x7fa8900a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80fee00 tx=0x7fa89c005f90 comp rx=0 tx=0).stop 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a8102d30 msgr2=0x7fa8a8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a8102d30 0x7fa8a8198360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa89802a9a0 tx=0x7fa898031a20 comp rx=0 tx=0).stop 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 shutdown_connections 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa8900a32d0 0x7fa8900a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8a810c160 0x7fa8a819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8103650 0x7fa8a81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 --2- 172.21.15.94:0/333938129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a8102d30 0x7fa8a8198360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 >> 172.21.15.94:0/333938129 conn(0x7fa8a80fc790 msgr2=0x7fa8a80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 shutdown_connections 2024-09-19T04:14:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:11.640+0000 7fa8ace47640 1 -- 172.21.15.94:0/333938129 wait complete. 2024-09-19T04:14:12.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.109+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/2067945023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 msgr2=0x7fc6bc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.109+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/2067945023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 0x7fc6bc101200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009900 tx=0x7fc6b802a480 comp rx=0 tx=0).stop 2024-09-19T04:14:12.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.109+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/2067945023 shutdown_connections 2024-09-19T04:14:12.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.109+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/2067945023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc102ef0 0x7fc6bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.109+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/2067945023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6bc102570 0x7fc6bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.109+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/2067945023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 0x7fc6bc101200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.109+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/2067945023 >> 172.21.15.94:0/2067945023 conn(0x7fc6bc0fc7d0 msgr2=0x7fc6bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:12.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.109+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/2067945023 shutdown_connections 2024-09-19T04:14:12.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.109+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/2067945023 wait complete. 2024-09-19T04:14:12.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c3a9a640 1 Processor -- start 2024-09-19T04:14:12.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c3a9a640 1 -- start start 2024-09-19T04:14:12.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c3a9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 0x7fc6bc197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c3a9a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6bc102570 0x7fc6bc1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c3a9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc102ef0 0x7fc6bc19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c3a9a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc110e50 con 0x7fc6bc100e20 2024-09-19T04:14:12.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c3a9a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc110cd0 con 0x7fc6bc102570 2024-09-19T04:14:12.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c3a9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc110fd0 con 0x7fc6bc102ef0 2024-09-19T04:14:12.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c1836640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 0x7fc6bc197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c1836640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 0x7fc6bc197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51958/0 (socket says 172.21.15.94:51958) 2024-09-19T04:14:12.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.110+0000 7fc6c1836640 1 -- 172.21.15.94:0/254937305 learned_addr learned my addr 172.21.15.94:0/254937305 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:12.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c2037640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc102ef0 0x7fc6bc19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c1035640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6bc102570 0x7fc6bc1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c1836640 1 -- 172.21.15.94:0/254937305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc102ef0 msgr2=0x7fc6bc19bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c1836640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc102ef0 0x7fc6bc19bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c1836640 1 -- 172.21.15.94:0/254937305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6bc102570 msgr2=0x7fc6bc1978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c1836640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6bc102570 0x7fc6bc1978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c1836640 1 -- 172.21.15.94:0/254937305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6bc19c310 con 0x7fc6bc100e20 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c1035640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6bc102570 0x7fc6bc1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c1836640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 0x7fc6bc197360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8006010 tx=0x7fc6b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6b2ffd640 1 -- 172.21.15.94:0/254937305 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b803f030 con 0x7fc6bc100e20 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6b2ffd640 1 -- 172.21.15.94:0/254937305 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b8003cb0 con 0x7fc6bc100e20 2024-09-19T04:14:12.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6bc19c5a0 con 0x7fc6bc100e20 2024-09-19T04:14:12.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6b2ffd640 1 -- 172.21.15.94:0/254937305 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8035070 con 0x7fc6bc100e20 2024-09-19T04:14:12.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.111+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6bc1a3ee0 con 0x7fc6bc100e20 2024-09-19T04:14:12.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.112+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6bc101200 con 0x7fc6bc100e20 2024-09-19T04:14:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.113+0000 7fc6b2ffd640 1 -- 172.21.15.94:0/254937305 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc6b8042030 con 0x7fc6bc100e20 2024-09-19T04:14:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.114+0000 7fc6b2ffd640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc6a80a32d0 0x7fc6a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.114+0000 7fc6c1035640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc6a80a32d0 0x7fc6a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.114+0000 7fc6b2ffd640 1 -- 172.21.15.94:0/254937305 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc6b80f00b0 con 0x7fc6bc100e20 2024-09-19T04:14:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.114+0000 7fc6c1035640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc6a80a32d0 0x7fc6a80a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0ffe00 tx=0x7fc6ac005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.118+0000 7fc6b2ffd640 1 -- 172.21.15.94:0/254937305 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc6b80bcc70 con 0x7fc6bc100e20 2024-09-19T04:14:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.288+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc6bc108b00 con 0x7fc6bc100e20 2024-09-19T04:14:12.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:12 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1501574457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:14:12.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:12 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/333938129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:14:12.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.288+0000 7fc6b2ffd640 1 -- 172.21.15.94:0/254937305 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc6b80dc020 con 0x7fc6bc100e20 2024-09-19T04:14:12.290 INFO:teuthology.orchestra.run.smithi094.stdout:103079215168 2024-09-19T04:14:12.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc6a80a32d0 msgr2=0x7fc6a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc6a80a32d0 0x7fc6a80a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0ffe00 tx=0x7fc6ac005e50 comp rx=0 tx=0).stop 2024-09-19T04:14:12.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 msgr2=0x7fc6bc197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 0x7fc6bc197360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8006010 tx=0x7fc6b80042f0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 shutdown_connections 2024-09-19T04:14:12.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fc6a80a32d0 0x7fc6a80a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc102ef0 0x7fc6bc19bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6bc102570 0x7fc6bc1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 --2- 172.21.15.94:0/254937305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6bc100e20 0x7fc6bc197360 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 >> 172.21.15.94:0/254937305 conn(0x7fc6bc0fc7d0 msgr2=0x7fc6bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:12.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 shutdown_connections 2024-09-19T04:14:12.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:12.290+0000 7fc6c3a9a640 1 -- 172.21.15.94:0/254937305 wait complete. 2024-09-19T04:14:12.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:12 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1501574457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:14:12.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:12 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/333938129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:14:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:12 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1501574457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:14:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:12 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/333938129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:14:12.962 INFO:tasks.cephadm.ceph_manager.ceph:need seq 51539607623 got 51539607625 for osd.1 2024-09-19T04:14:12.963 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:13.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:13 smithi112 ceph-mon[35051]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:13.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:13 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/254937305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:14:13.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:13 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:13.556 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:13 smithi094 ceph-mon[30972]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:13.556 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:13 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/254937305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:14:13.557 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:13 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:13.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.555+0000 7fec910e7640 1 -- 172.21.15.94:0/2570286498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 msgr2=0x7fec8c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.555+0000 7fec910e7640 1 --2- 172.21.15.94:0/2570286498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 0x7fec8c10bb50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009900 tx=0x7fec7c02a480 comp rx=0 tx=0).stop 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.555+0000 7fec910e7640 1 -- 172.21.15.94:0/2570286498 shutdown_connections 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.555+0000 7fec910e7640 1 --2- 172.21.15.94:0/2570286498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec8c10c170 0x7fec8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.555+0000 7fec910e7640 1 --2- 172.21.15.94:0/2570286498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 0x7fec8c10bb50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.555+0000 7fec910e7640 1 --2- 172.21.15.94:0/2570286498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec8c102d40 0x7fec8c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.555+0000 7fec910e7640 1 -- 172.21.15.94:0/2570286498 >> 172.21.15.94:0/2570286498 conn(0x7fec8c0fc7a0 msgr2=0x7fec8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.555+0000 7fec910e7640 1 -- 172.21.15.94:0/2570286498 shutdown_connections 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.555+0000 7fec910e7640 1 -- 172.21.15.94:0/2570286498 wait complete. 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec910e7640 1 Processor -- start 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec910e7640 1 -- start start 2024-09-19T04:14:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec910e7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec8c102d40 0x7fec8c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec910e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 0x7fec8c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec910e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec8c10c170 0x7fec8c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec910e7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c111e70 con 0x7fec8c103660 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec910e7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c111cf0 con 0x7fec8c102d40 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec910e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c111ff0 con 0x7fec8c10c170 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec8a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 0x7fec8c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec8a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 0x7fec8c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51988/0 (socket says 172.21.15.94:51988) 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec8a59c640 1 -- 172.21.15.94:0/2454949810 learned_addr learned my addr 172.21.15.94:0/2454949810 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec8b59e640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec8c10c170 0x7fec8c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.556+0000 7fec8ad9d640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec8c102d40 0x7fec8c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec8a59c640 1 -- 172.21.15.94:0/2454949810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec8c10c170 msgr2=0x7fec8c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec8a59c640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec8c10c170 0x7fec8c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec8a59c640 1 -- 172.21.15.94:0/2454949810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec8c102d40 msgr2=0x7fec8c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec8a59c640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec8c102d40 0x7fec8c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec8a59c640 1 -- 172.21.15.94:0/2454949810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c19d360 con 0x7fec8c103660 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec8a59c640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 0x7fec8c1988f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fec7c005ab0 tx=0x7fec7c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec83fff640 1 -- 172.21.15.94:0/2454949810 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c03f070 con 0x7fec8c103660 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8c19d5f0 con 0x7fec8c103660 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec83fff640 1 -- 172.21.15.94:0/2454949810 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec7c031d60 con 0x7fec8c103660 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec83fff640 1 -- 172.21.15.94:0/2454949810 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c035030 con 0x7fec8c103660 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.557+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8c1a4f30 con 0x7fec8c103660 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.558+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec8c06b070 con 0x7fec8c103660 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.559+0000 7fec83fff640 1 -- 172.21.15.94:0/2454949810 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fec7c042070 con 0x7fec8c103660 2024-09-19T04:14:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.560+0000 7fec83fff640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fec780a32d0 0x7fec780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.560+0000 7fec83fff640 1 -- 172.21.15.94:0/2454949810 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fec7c0f0870 con 0x7fec8c103660 2024-09-19T04:14:13.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.560+0000 7fec8ad9d640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fec780a32d0 0x7fec780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.560+0000 7fec8ad9d640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fec780a32d0 0x7fec780a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fec74005fd0 tx=0x7fec7400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:13.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.563+0000 7fec83fff640 1 -- 172.21.15.94:0/2454949810 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec7c0bcde0 con 0x7fec8c103660 2024-09-19T04:14:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:13 smithi104 ceph-mon[34145]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:13 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/254937305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:14:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:13 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:13.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.734+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fec8c103120 con 0x7fec8c103660 2024-09-19T04:14:13.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.734+0000 7fec83fff640 1 -- 172.21.15.94:0/2454949810 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fec8c103120 con 0x7fec8c103660 2024-09-19T04:14:13.736 INFO:teuthology.orchestra.run.smithi094.stdout:231928234023 2024-09-19T04:14:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fec780a32d0 msgr2=0x7fec780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fec780a32d0 0x7fec780a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fec74005fd0 tx=0x7fec7400b390 comp rx=0 tx=0).stop 2024-09-19T04:14:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 msgr2=0x7fec8c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 0x7fec8c1988f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fec7c005ab0 tx=0x7fec7c003c80 comp rx=0 tx=0).stop 2024-09-19T04:14:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 shutdown_connections 2024-09-19T04:14:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fec780a32d0 0x7fec780a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec8c10c170 0x7fec8c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec8c103660 0x7fec8c1988f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 --2- 172.21.15.94:0/2454949810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec8c102d40 0x7fec8c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.736+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 >> 172.21.15.94:0/2454949810 conn(0x7fec8c0fc7a0 msgr2=0x7fec8c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:13.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.737+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 shutdown_connections 2024-09-19T04:14:13.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:13.737+0000 7fec910e7640 1 -- 172.21.15.94:0/2454949810 wait complete. 2024-09-19T04:14:14.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215165 got 103079215168 for osd.3 2024-09-19T04:14:14.006 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:14.446 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612630 got 313532612632 for osd.11 2024-09-19T04:14:14.446 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:14.527 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:14 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2454949810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:14:14.527 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:14 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.527 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:14 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.527 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:14 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.527 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:14 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:14 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2454949810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:14:14.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:14 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:14 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:14 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:14 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:14 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2454949810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:14:14.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:14 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:14 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:14 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:14 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:14.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.628+0000 7f045880e640 1 -- 172.21.15.94:0/1200589606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04541082a0 msgr2=0x7f045410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.628+0000 7f045880e640 1 --2- 172.21.15.94:0/1200589606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04541082a0 0x7f045410ec40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f044c009cd0 tx=0x7f044c02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.628+0000 7f045880e640 1 -- 172.21.15.94:0/1200589606 shutdown_connections 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.628+0000 7f045880e640 1 --2- 172.21.15.94:0/1200589606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04541082a0 0x7f045410ec40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.628+0000 7f045880e640 1 --2- 172.21.15.94:0/1200589606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04541036e0 0x7f0454107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.628+0000 7f045880e640 1 --2- 172.21.15.94:0/1200589606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0454102d30 0x7f0454103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.628+0000 7f045880e640 1 -- 172.21.15.94:0/1200589606 >> 172.21.15.94:0/1200589606 conn(0x7f04540fc7d0 msgr2=0x7f04540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.628+0000 7f045880e640 1 -- 172.21.15.94:0/1200589606 shutdown_connections 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.628+0000 7f045880e640 1 -- 172.21.15.94:0/1200589606 wait complete. 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045880e640 1 Processor -- start 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045880e640 1 -- start start 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045880e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0454102d30 0x7f04541984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045880e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04541036e0 0x7f04541989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045880e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04541082a0 0x7f045419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045880e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0454111f50 con 0x7f0454102d30 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045880e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0454111dd0 con 0x7f04541082a0 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045880e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04541120d0 con 0x7f04541036e0 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0454102d30 0x7f04541984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0454102d30 0x7f04541984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52006/0 (socket says 172.21.15.94:52006) 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.629+0000 7f045259c640 1 -- 172.21.15.94:0/1094324974 learned_addr learned my addr 172.21.15.94:0/1094324974 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f0451d9b640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04541036e0 0x7f04541989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f0452d9d640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04541082a0 0x7f045419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f045259c640 1 -- 172.21.15.94:0/1094324974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04541036e0 msgr2=0x7f04541989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f045259c640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04541036e0 0x7f04541989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f045259c640 1 -- 172.21.15.94:0/1094324974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04541082a0 msgr2=0x7f045419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f045259c640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04541082a0 0x7f045419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f045259c640 1 -- 172.21.15.94:0/1094324974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045419d460 con 0x7f0454102d30 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f0452d9d640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04541082a0 0x7f045419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f0451d9b640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04541036e0 0x7f04541989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f045259c640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0454102d30 0x7f04541984b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f044400d770 tx=0x7f044400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f04437fe640 1 -- 172.21.15.94:0/1094324974 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444017070 con 0x7f0454102d30 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045419d750 con 0x7f0454102d30 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f04437fe640 1 -- 172.21.15.94:0/1094324974 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04440054d0 con 0x7f0454102d30 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f04437fe640 1 -- 172.21.15.94:0/1094324974 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444004cf0 con 0x7f0454102d30 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.630+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04541a5030 con 0x7f0454102d30 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.631+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0454103110 con 0x7f0454102d30 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.632+0000 7f04437fe640 1 -- 172.21.15.94:0/1094324974 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0444004e90 con 0x7f0454102d30 2024-09-19T04:14:14.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.633+0000 7f04437fe640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f043c0a32d0 0x7f043c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.633+0000 7f0451d9b640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f043c0a32d0 0x7f043c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.633+0000 7f04437fe640 1 -- 172.21.15.94:0/1094324974 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f04440cf5c0 con 0x7f0454102d30 2024-09-19T04:14:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.633+0000 7f0451d9b640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f043c0a32d0 0x7f043c0a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f04540fee40 tx=0x7f0448005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.637+0000 7f04437fe640 1 -- 172.21.15.94:0/1094324974 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f044409c180 con 0x7f0454102d30 2024-09-19T04:14:14.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.810+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f045410a2b0 con 0x7f0454102d30 2024-09-19T04:14:14.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.810+0000 7f04437fe640 1 -- 172.21.15.94:0/1094324974 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f044409ba80 con 0x7f0454102d30 2024-09-19T04:14:14.813 INFO:teuthology.orchestra.run.smithi094.stdout:201863462955 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f043c0a32d0 msgr2=0x7f043c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f043c0a32d0 0x7f043c0a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f04540fee40 tx=0x7f0448005f90 comp rx=0 tx=0).stop 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0454102d30 msgr2=0x7f04541984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0454102d30 0x7f04541984b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f044400d770 tx=0x7f044400dc30 comp rx=0 tx=0).stop 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 shutdown_connections 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f043c0a32d0 0x7f043c0a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04541082a0 0x7f045419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04541036e0 0x7f04541989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 --2- 172.21.15.94:0/1094324974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0454102d30 0x7f04541984b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 >> 172.21.15.94:0/1094324974 conn(0x7f04540fc7d0 msgr2=0x7f0454105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 shutdown_connections 2024-09-19T04:14:14.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:14.812+0000 7f045880e640 1 -- 172.21.15.94:0/1094324974 wait complete. 2024-09-19T04:14:15.404 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462953 got 201863462955 for osd.7 2024-09-19T04:14:15.404 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:15.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:15 smithi112 ceph-mon[35051]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:15.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:15 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1094324974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:14:15.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:15 smithi094 ceph-mon[30972]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:15.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:15 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1094324974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:14:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:15 smithi104 ceph-mon[34145]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:15 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1094324974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:14:15.972 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234020 got 231928234023 for osd.8 2024-09-19T04:14:15.973 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:16.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.406+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2144818481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4102d90 msgr2=0x7fa1a4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.406+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2144818481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4102d90 0x7fa1a4103170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009930 tx=0x7fa19c02a420 comp rx=0 tx=0).stop 2024-09-19T04:14:16.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.406+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2144818481 shutdown_connections 2024-09-19T04:14:16.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.406+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2144818481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a4108290 0x7fa1a410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.406+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2144818481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4103740 0x7fa1a4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.406+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2144818481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4102d90 0x7fa1a4103170 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.406+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2144818481 >> 172.21.15.94:0/2144818481 conn(0x7fa1a40fc7d0 msgr2=0x7fa1a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.406+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2144818481 shutdown_connections 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.406+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2144818481 wait complete. 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1ab4fb640 1 Processor -- start 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1ab4fb640 1 -- start start 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1ab4fb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4102d90 0x7fa1a41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1ab4fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a4103740 0x7fa1a4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1ab4fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4108290 0x7fa1a419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1ab4fb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4111f40 con 0x7fa1a4108290 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1ab4fb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4111dc0 con 0x7fa1a4102d90 2024-09-19T04:14:16.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1ab4fb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41120c0 con 0x7fa1a4103740 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1a9a98640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4108290 0x7fa1a419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1a9a98640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4108290 0x7fa1a419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52022/0 (socket says 172.21.15.94:52022) 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1a9a98640 1 -- 172.21.15.94:0/2354993926 learned_addr learned my addr 172.21.15.94:0/2354993926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.407+0000 7fa1a9297640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4102d90 0x7fa1a41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a8a96640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a4103740 0x7fa1a4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a9a98640 1 -- 172.21.15.94:0/2354993926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a4103740 msgr2=0x7fa1a4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a9a98640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a4103740 0x7fa1a4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a9a98640 1 -- 172.21.15.94:0/2354993926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4102d90 msgr2=0x7fa1a41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a9a98640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4102d90 0x7fa1a41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a9a98640 1 -- 172.21.15.94:0/2354993926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a419d490 con 0x7fa1a4108290 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a9a98640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4108290 0x7fa1a419cdb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa19400eaa0 tx=0x7fa19400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:16.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a27fc640 1 -- 172.21.15.94:0/2354993926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19400cdb0 con 0x7fa1a4108290 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a419d780 con 0x7fa1a4108290 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a27fc640 1 -- 172.21.15.94:0/2354993926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa194005020 con 0x7fa1a4108290 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1a27fc640 1 -- 172.21.15.94:0/2354993926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940116d0 con 0x7fa1a4108290 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.408+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41a5060 con 0x7fa1a4108290 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.409+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a406a7f0 con 0x7fa1a4108290 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.410+0000 7fa1a27fc640 1 -- 172.21.15.94:0/2354993926 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa1940051d0 con 0x7fa1a4108290 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.411+0000 7fa1a27fc640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa18c0a32d0 0x7fa18c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.411+0000 7fa1a9297640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa18c0a32d0 0x7fa18c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.411+0000 7fa1a27fc640 1 -- 172.21.15.94:0/2354993926 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa1940cfef0 con 0x7fa1a4108290 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.412+0000 7fa1a9297640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa18c0a32d0 0x7fa18c0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009900 tx=0x7fa19c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:16.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.415+0000 7fa1a27fc640 1 -- 172.21.15.94:0/2354993926 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa194016030 con 0x7fa1a4108290 2024-09-19T04:14:16.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.587+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa1a4103170 con 0x7fa1a4108290 2024-09-19T04:14:16.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.587+0000 7fa1a27fc640 1 -- 172.21.15.94:0/2354993926 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1a4103170 con 0x7fa1a4108290 2024-09-19T04:14:16.590 INFO:teuthology.orchestra.run.smithi094.stdout:68719476806 2024-09-19T04:14:16.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa18c0a32d0 msgr2=0x7fa18c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa18c0a32d0 0x7fa18c0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009900 tx=0x7fa19c000bc0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4108290 msgr2=0x7fa1a419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4108290 0x7fa1a419cdb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa19400eaa0 tx=0x7fa19400ef60 comp rx=0 tx=0).stop 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 shutdown_connections 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa18c0a32d0 0x7fa18c0a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4108290 0x7fa1a419cdb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a4103740 0x7fa1a4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 --2- 172.21.15.94:0/2354993926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4102d90 0x7fa1a41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 >> 172.21.15.94:0/2354993926 conn(0x7fa1a40fc7d0 msgr2=0x7fa1a4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 shutdown_connections 2024-09-19T04:14:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.590+0000 7fa1ab4fb640 1 -- 172.21.15.94:0/2354993926 wait complete. 2024-09-19T04:14:16.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.638+0000 7fb16f9f6640 1 -- 172.21.15.94:0/611721741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 msgr2=0x7fb16810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.638+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/611721741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 0x7fb16810da40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002a480 comp rx=0 tx=0).stop 2024-09-19T04:14:16.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.638+0000 7fb16f9f6640 1 -- 172.21.15.94:0/611721741 shutdown_connections 2024-09-19T04:14:16.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.638+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/611721741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 0x7fb16810da40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.638+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/611721741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb168102570 0x7fb1681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.638+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/611721741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb168100e20 0x7fb168101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.638+0000 7fb16f9f6640 1 -- 172.21.15.94:0/611721741 >> 172.21.15.94:0/611721741 conn(0x7fb1680fc7d0 msgr2=0x7fb1680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.638+0000 7fb16f9f6640 1 -- 172.21.15.94:0/611721741 shutdown_connections 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.638+0000 7fb16f9f6640 1 -- 172.21.15.94:0/611721741 wait complete. 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16f9f6640 1 Processor -- start 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16f9f6640 1 -- start start 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16f9f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb168100e20 0x7fb16819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16f9f6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb168102570 0x7fb16819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16f9f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 0x7fb16819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16f9f6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb168110e50 con 0x7fb168102ef0 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16f9f6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb168110cd0 con 0x7fb168102570 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16f9f6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb168110fd0 con 0x7fb168100e20 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16df93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 0x7fb16819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16df93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 0x7fb16819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52044/0 (socket says 172.21.15.94:52044) 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16df93640 1 -- 172.21.15.94:0/1435334563 learned_addr learned my addr 172.21.15.94:0/1435334563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16d792640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb168100e20 0x7fb16819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16cf91640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb168102570 0x7fb16819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16df93640 1 -- 172.21.15.94:0/1435334563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb168100e20 msgr2=0x7fb16819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16df93640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb168100e20 0x7fb16819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16df93640 1 -- 172.21.15.94:0/1435334563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb168102570 msgr2=0x7fb16819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16df93640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb168102570 0x7fb16819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16df93640 1 -- 172.21.15.94:0/1435334563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16819f650 con 0x7fb168102ef0 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16d792640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb168100e20 0x7fb16819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16cf91640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb168102570 0x7fb16819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.639+0000 7fb16df93640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 0x7fb16819ef70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb160005ab0 tx=0x7fb1600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.640+0000 7fb1667fc640 1 -- 172.21.15.94:0/1435334563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16003f030 con 0x7fb168102ef0 2024-09-19T04:14:16.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.640+0000 7fb1667fc640 1 -- 172.21.15.94:0/1435334563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb160003cb0 con 0x7fb168102ef0 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.640+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16819f8e0 con 0x7fb168102ef0 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.640+0000 7fb1667fc640 1 -- 172.21.15.94:0/1435334563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160035070 con 0x7fb168102ef0 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.640+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1681a7220 con 0x7fb168102ef0 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.641+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb168101200 con 0x7fb168102ef0 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.642+0000 7fb1667fc640 1 -- 172.21.15.94:0/1435334563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb160042030 con 0x7fb168102ef0 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.642+0000 7fb1667fc640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb1500a3350 0x7fb1500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.642+0000 7fb16d792640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb1500a3350 0x7fb1500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.642+0000 7fb1667fc640 1 -- 172.21.15.94:0/1435334563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb1600f01f0 con 0x7fb168102ef0 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.643+0000 7fb16d792640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb1500a3350 0x7fb1500a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb158007eb0 tx=0x7fb1580073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:16.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.646+0000 7fb1667fc640 1 -- 172.21.15.94:0/1435334563 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb1600bcdb0 con 0x7fb168102ef0 2024-09-19T04:14:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.817+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb1681a73f0 con 0x7fb168102ef0 2024-09-19T04:14:16.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.817+0000 7fb1667fc640 1 -- 172.21.15.94:0/1435334563 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb1600bc6b0 con 0x7fb168102ef0 2024-09-19T04:14:16.820 INFO:teuthology.orchestra.run.smithi094.stdout:261993005091 2024-09-19T04:14:16.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb1500a3350 msgr2=0x7fb1500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb1500a3350 0x7fb1500a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb158007eb0 tx=0x7fb1580073c0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 msgr2=0x7fb16819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 0x7fb16819ef70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb160005ab0 tx=0x7fb1600042f0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 shutdown_connections 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb1500a3350 0x7fb1500a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb168102ef0 0x7fb16819ef70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb168102570 0x7fb16819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 --2- 172.21.15.94:0/1435334563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb168100e20 0x7fb16819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 >> 172.21.15.94:0/1435334563 conn(0x7fb1680fc7d0 msgr2=0x7fb1680ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 shutdown_connections 2024-09-19T04:14:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:16.820+0000 7fb16f9f6640 1 -- 172.21.15.94:0/1435334563 wait complete. 2024-09-19T04:14:17.480 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:17 smithi094 ceph-mon[30972]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:17.480 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:17 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2354993926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:14:17.480 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:17 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1435334563' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:14:17.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476803 got 68719476806 for osd.2 2024-09-19T04:14:17.508 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:17.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:17 smithi112 ceph-mon[35051]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:17.537 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:17 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2354993926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:14:17.537 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:17 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1435334563' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:14:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:17 smithi104 ceph-mon[34145]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:17 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2354993926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:14:17.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:17 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1435334563' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:14:17.983 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005088 got 261993005091 for osd.9 2024-09-19T04:14:17.983 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:18.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738446 for osd.0 2024-09-19T04:14:18.502 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:18.502 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T04:14:18.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:14:19.298 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:19.421 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:19 smithi094 ceph-mon[30972]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:19.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:19 smithi112 ceph-mon[35051]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:19.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:19 smithi104 ceph-mon[34145]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:19.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.790+0000 7f9959dbf640 1 -- 172.21.15.94:0/354776768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9954102db0 msgr2=0x7f9954103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:19.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.790+0000 7f9959dbf640 1 --2- 172.21.15.94:0/354776768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9954102db0 0x7f9954103190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9944009930 tx=0x7f994402a420 comp rx=0 tx=0).stop 2024-09-19T04:14:19.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.790+0000 7f9959dbf640 1 -- 172.21.15.94:0/354776768 shutdown_connections 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.790+0000 7f9959dbf640 1 --2- 172.21.15.94:0/354776768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99541082e0 0x7f995410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.790+0000 7f9959dbf640 1 --2- 172.21.15.94:0/354776768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9954103760 0x7f9954107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.790+0000 7f9959dbf640 1 --2- 172.21.15.94:0/354776768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9954102db0 0x7f9954103190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.790+0000 7f9959dbf640 1 -- 172.21.15.94:0/354776768 >> 172.21.15.94:0/354776768 conn(0x7f99540fc7d0 msgr2=0x7f99540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.790+0000 7f9959dbf640 1 -- 172.21.15.94:0/354776768 shutdown_connections 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.790+0000 7f9959dbf640 1 -- 172.21.15.94:0/354776768 wait complete. 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.791+0000 7f9959dbf640 1 Processor -- start 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.791+0000 7f9959dbf640 1 -- start start 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.791+0000 7f9959dbf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9954102db0 0x7f99541984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.791+0000 7f9959dbf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9954103760 0x7f9954198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.791+0000 7f9959dbf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99541082e0 0x7f995419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.791+0000 7f9959dbf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9954111fb0 con 0x7f99541082e0 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.791+0000 7f9959dbf640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9954111e30 con 0x7f9954102db0 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.791+0000 7f9959dbf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9954112130 con 0x7f9954103760 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9953fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99541082e0 0x7f995419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9953fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99541082e0 0x7f995419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52058/0 (socket says 172.21.15.94:52058) 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9953fff640 1 -- 172.21.15.94:0/577852142 learned_addr learned my addr 172.21.15.94:0/577852142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f99537fe640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9954102db0 0x7f99541984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9952ffd640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9954103760 0x7f9954198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9953fff640 1 -- 172.21.15.94:0/577852142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9954103760 msgr2=0x7f9954198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9953fff640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9954103760 0x7f9954198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9953fff640 1 -- 172.21.15.94:0/577852142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9954102db0 msgr2=0x7f99541984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9953fff640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9954102db0 0x7f99541984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9953fff640 1 -- 172.21.15.94:0/577852142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995419d4a0 con 0x7f99541082e0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9953fff640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99541082e0 0x7f995419cdc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f994c00eb10 tx=0x7f994c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9950ff9640 1 -- 172.21.15.94:0/577852142 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c004b20 con 0x7f99541082e0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995419d790 con 0x7f99541082e0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9950ff9640 1 -- 172.21.15.94:0/577852142 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f994c004cc0 con 0x7f99541082e0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9950ff9640 1 -- 172.21.15.94:0/577852142 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c010690 con 0x7f99541082e0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.792+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99541a5070 con 0x7f99541082e0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.793+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995406a7d0 con 0x7f99541082e0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.795+0000 7f9950ff9640 1 -- 172.21.15.94:0/577852142 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f994c005000 con 0x7f99541082e0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.796+0000 7f9950ff9640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f993c0a32d0 0x7f993c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.796+0000 7f9950ff9640 1 -- 172.21.15.94:0/577852142 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f994c0d0450 con 0x7f99541082e0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.796+0000 7f99537fe640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f993c0a32d0 0x7f993c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.796+0000 7f99537fe640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f993c0a32d0 0x7f993c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9944006010 tx=0x7f9944005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.798+0000 7f9950ff9640 1 -- 172.21.15.94:0/577852142 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f994c09d010 con 0x7f99541082e0 2024-09-19T04:14:19.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.981+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f995410a330 con 0x7f993c0a32d0 2024-09-19T04:14:19.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.991+0000 7f9950ff9640 1 -- 172.21.15.94:0/577852142 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258717 (secure 0 0 0) 0x7f9944004410 con 0x7f993c0a32d0 2024-09-19T04:14:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:19.997 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.994+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f993c0a32d0 msgr2=0x7f993c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.994+0000 7f9959dbf640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f993c0a32d0 0x7f993c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9944006010 tx=0x7f9944005e80 comp rx=0 tx=0).stop 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.994+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99541082e0 msgr2=0x7f995419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.994+0000 7f9959dbf640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99541082e0 0x7f995419cdc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f994c00eb10 tx=0x7f994c00c2d0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.995+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 shutdown_connections 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.995+0000 7f9959dbf640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f993c0a32d0 0x7f993c0a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.995+0000 7f9959dbf640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99541082e0 0x7f995419cdc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.995+0000 7f9959dbf640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9954103760 0x7f9954198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.995+0000 7f9959dbf640 1 --2- 172.21.15.94:0/577852142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9954102db0 0x7f99541984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.995+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 >> 172.21.15.94:0/577852142 conn(0x7f99540fc7d0 msgr2=0x7f9954105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.995+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 shutdown_connections 2024-09-19T04:14:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:19.995+0000 7f9959dbf640 1 -- 172.21.15.94:0/577852142 wait complete. 2024-09-19T04:14:20.769 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-19T04:14:18.145544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654762,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":449,"num_read_kb":342,"num_write":345,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399252,"kb_used_data":12540,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542572,"statfs":{"total":1151940427776,"available":1151531593728,"internally_reserved":0,"allocated":12840960,"data_stored":7133538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002802"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501952+0000","last_change":"2024-09-19T04:13:16.451031+0000","last_active":"2024-09-19T04:13:31.501952+0000","last_peered":"2024-09-19T04:13:31.501952+0000","last_clean":"2024-09-19T04:13:31.501952+0000","last_became_active":"2024-09-19T04:13:16.450713+0000","last_became_peered":"2024-09-19T04:13:16.450713+0000","last_unstale":"2024-09-19T04:13:31.501952+0000","last_undegraded":"2024-09-19T04:13:31.501952+0000","last_fullsized":"2024-09-19T04:13:31.501952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:59.485415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735600+0000","last_change":"2024-09-19T04:13:16.455689+0000","last_active":"2024-09-19T04:13:31.735600+0000","last_peered":"2024-09-19T04:13:31.735600+0000","last_clean":"2024-09-19T04:13:31.735600+0000","last_became_active":"2024-09-19T04:13:16.455154+0000","last_became_peered":"2024-09-19T04:13:16.455154+0000","last_unstale":"2024-09-19T04:13:31.735600+0000","last_undegraded":"2024-09-19T04:13:31.735600+0000","last_fullsized":"2024-09-19T04:13:31.735600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:57.610619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735526+0000","last_change":"2024-09-19T04:13:16.447392+0000","last_active":"2024-09-19T04:13:31.735526+0000","last_peered":"2024-09-19T04:13:31.735526+0000","last_clean":"2024-09-19T04:13:31.735526+0000","last_became_active":"2024-09-19T04:13:16.447088+0000","last_became_peered":"2024-09-19T04:13:16.447088+0000","last_unstale":"2024-09-19T04:13:31.735526+0000","last_undegraded":"2024-09-19T04:13:31.735526+0000","last_fullsized":"2024-09-19T04:13:31.735526+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:16.591420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504467+0000","last_change":"2024-09-19T04:13:16.452642+0000","last_active":"2024-09-19T04:13:31.504467+0000","last_peered":"2024-09-19T04:13:31.504467+0000","last_clean":"2024-09-19T04:13:31.504467+0000","last_became_active":"2024-09-19T04:13:16.452329+0000","last_became_peered":"2024-09-19T04:13:16.452329+0000","last_unstale":"2024-09-19T04:13:31.504467+0000","last_undegraded":"2024-09-19T04:13:31.504467+0000","last_fullsized":"2024-09-19T04:13:31.504467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:54.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505930+0000","last_change":"2024-09-19T04:13:16.447171+0000","last_active":"2024-09-19T04:13:31.505930+0000","last_peered":"2024-09-19T04:13:31.505930+0000","last_clean":"2024-09-19T04:13:31.505930+0000","last_became_active":"2024-09-19T04:13:16.446835+0000","last_became_peered":"2024-09-19T04:13:16.446835+0000","last_unstale":"2024-09-19T04:13:31.505930+0000","last_undegraded":"2024-09-19T04:13:31.505930+0000","last_fullsized":"2024-09-19T04:13:31.505930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:41.892118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.507665+0000","last_change":"2024-09-19T04:13:31.507665+0000","last_active":"2024-09-19T04:13:31.507665+0000","last_peered":"2024-09-19T04:13:31.507665+0000","last_clean":"2024-09-19T04:13:31.507665+0000","last_became_active":"2024-09-19T04:13:31.507329+0000","last_became_peered":"2024-09-19T04:13:31.507329+0000","last_unstale":"2024-09-19T04:13:31.507665+0000","last_undegraded":"2024-09-19T04:13:31.507665+0000","last_fullsized":"2024-09-19T04:13:31.507665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:35.832776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500742+0000","last_change":"2024-09-19T04:13:16.454568+0000","last_active":"2024-09-19T04:13:31.500742+0000","last_peered":"2024-09-19T04:13:31.500742+0000","last_clean":"2024-09-19T04:13:31.500742+0000","last_became_active":"2024-09-19T04:13:16.454230+0000","last_became_peered":"2024-09-19T04:13:16.454230+0000","last_unstale":"2024-09-19T04:13:31.500742+0000","last_undegraded":"2024-09-19T04:13:31.500742+0000","last_fullsized":"2024-09-19T04:13:31.500742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:23.401316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501235+0000","last_change":"2024-09-19T04:13:16.449223+0000","last_active":"2024-09-19T04:13:31.501235+0000","last_peered":"2024-09-19T04:13:31.501235+0000","last_clean":"2024-09-19T04:13:31.501235+0000","last_became_active":"2024-09-19T04:13:16.448665+0000","last_became_peered":"2024-09-19T04:13:16.448665+0000","last_unstale":"2024-09-19T04:13:31.501235+0000","last_undegraded":"2024-09-19T04:13:31.501235+0000","last_fullsized":"2024-09-19T04:13:31.501235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:29.619847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.701069+0000","last_change":"2024-09-19T04:13:16.458203+0000","last_active":"2024-09-19T04:13:31.701069+0000","last_peered":"2024-09-19T04:13:31.701069+0000","last_clean":"2024-09-19T04:13:31.701069+0000","last_became_active":"2024-09-19T04:13:16.457544+0000","last_became_peered":"2024-09-19T04:13:16.457544+0000","last_unstale":"2024-09-19T04:13:31.701069+0000","last_undegraded":"2024-09-19T04:13:31.701069+0000","last_fullsized":"2024-09-19T04:13:31.701069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:47.611695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500145+0000","last_change":"2024-09-19T04:13:16.454213+0000","last_active":"2024-09-19T04:13:31.500145+0000","last_peered":"2024-09-19T04:13:31.500145+0000","last_clean":"2024-09-19T04:13:31.500145+0000","last_became_active":"2024-09-19T04:13:16.453047+0000","last_became_peered":"2024-09-19T04:13:16.453047+0000","last_unstale":"2024-09-19T04:13:31.500145+0000","last_undegraded":"2024-09-19T04:13:31.500145+0000","last_fullsized":"2024-09-19T04:13:31.500145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:27.425259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501213+0000","last_change":"2024-09-19T04:13:16.458195+0000","last_active":"2024-09-19T04:13:31.501213+0000","last_peered":"2024-09-19T04:13:31.501213+0000","last_clean":"2024-09-19T04:13:31.501213+0000","last_became_active":"2024-09-19T04:13:16.457891+0000","last_became_peered":"2024-09-19T04:13:16.457891+0000","last_unstale":"2024-09-19T04:13:31.501213+0000","last_undegraded":"2024-09-19T04:13:31.501213+0000","last_fullsized":"2024-09-19T04:13:31.501213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:53.833249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.508474+0000","last_change":"2024-09-19T04:13:31.508117+0000","last_active":"2024-09-19T04:13:31.508474+0000","last_peered":"2024-09-19T04:13:31.508474+0000","last_clean":"2024-09-19T04:13:31.508474+0000","last_became_active":"2024-09-19T04:13:31.507693+0000","last_became_peered":"2024-09-19T04:13:31.507693+0000","last_unstale":"2024-09-19T04:13:31.508474+0000","last_undegraded":"2024-09-19T04:13:31.508474+0000","last_fullsized":"2024-09-19T04:13:31.508474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:58.447965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499011+0000","last_change":"2024-09-19T04:13:16.440079+0000","last_active":"2024-09-19T04:13:31.499011+0000","last_peered":"2024-09-19T04:13:31.499011+0000","last_clean":"2024-09-19T04:13:31.499011+0000","last_became_active":"2024-09-19T04:13:16.439474+0000","last_became_peered":"2024-09-19T04:13:16.439474+0000","last_unstale":"2024-09-19T04:13:31.499011+0000","last_undegraded":"2024-09-19T04:13:31.499011+0000","last_fullsized":"2024-09-19T04:13:31.499011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:11.151412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735816+0000","last_change":"2024-09-19T04:13:16.449958+0000","last_active":"2024-09-19T04:13:31.735816+0000","last_peered":"2024-09-19T04:13:31.735816+0000","last_clean":"2024-09-19T04:13:31.735816+0000","last_became_active":"2024-09-19T04:13:16.449665+0000","last_became_peered":"2024-09-19T04:13:16.449665+0000","last_unstale":"2024-09-19T04:13:31.735816+0000","last_undegraded":"2024-09-19T04:13:31.735816+0000","last_fullsized":"2024-09-19T04:13:31.735816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:24.023779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735453+0000","last_change":"2024-09-19T04:13:16.454333+0000","last_active":"2024-09-19T04:13:31.735453+0000","last_peered":"2024-09-19T04:13:31.735453+0000","last_clean":"2024-09-19T04:13:31.735453+0000","last_became_active":"2024-09-19T04:13:16.454066+0000","last_became_peered":"2024-09-19T04:13:16.454066+0000","last_unstale":"2024-09-19T04:13:31.735453+0000","last_undegraded":"2024-09-19T04:13:31.735453+0000","last_fullsized":"2024-09-19T04:13:31.735453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:16.815173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700446+0000","last_change":"2024-09-19T04:13:16.451864+0000","last_active":"2024-09-19T04:13:31.700446+0000","last_peered":"2024-09-19T04:13:31.700446+0000","last_clean":"2024-09-19T04:13:31.700446+0000","last_became_active":"2024-09-19T04:13:16.450995+0000","last_became_peered":"2024-09-19T04:13:16.450995+0000","last_unstale":"2024-09-19T04:13:31.700446+0000","last_undegraded":"2024-09-19T04:13:31.700446+0000","last_fullsized":"2024-09-19T04:13:31.700446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:40.750221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502172+0000","last_change":"2024-09-19T04:13:16.453176+0000","last_active":"2024-09-19T04:13:31.502172+0000","last_peered":"2024-09-19T04:13:31.502172+0000","last_clean":"2024-09-19T04:13:31.502172+0000","last_became_active":"2024-09-19T04:13:16.452788+0000","last_became_peered":"2024-09-19T04:13:16.452788+0000","last_unstale":"2024-09-19T04:13:31.502172+0000","last_undegraded":"2024-09-19T04:13:31.502172+0000","last_fullsized":"2024-09-19T04:13:31.502172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:47.099180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501446+0000","last_change":"2024-09-19T04:13:16.449167+0000","last_active":"2024-09-19T04:13:31.501446+0000","last_peered":"2024-09-19T04:13:31.501446+0000","last_clean":"2024-09-19T04:13:31.501446+0000","last_became_active":"2024-09-19T04:13:16.448579+0000","last_became_peered":"2024-09-19T04:13:16.448579+0000","last_unstale":"2024-09-19T04:13:31.501446+0000","last_undegraded":"2024-09-19T04:13:31.501446+0000","last_fullsized":"2024-09-19T04:13:31.501446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:22:40.023000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505138+0000","last_change":"2024-09-19T04:13:16.445449+0000","last_active":"2024-09-19T04:13:31.505138+0000","last_peered":"2024-09-19T04:13:31.505138+0000","last_clean":"2024-09-19T04:13:31.505138+0000","last_became_active":"2024-09-19T04:13:16.445028+0000","last_became_peered":"2024-09-19T04:13:16.445028+0000","last_unstale":"2024-09-19T04:13:31.505138+0000","last_undegraded":"2024-09-19T04:13:31.505138+0000","last_fullsized":"2024-09-19T04:13:31.505138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:13.095348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501714+0000","last_change":"2024-09-19T04:13:16.454033+0000","last_active":"2024-09-19T04:13:31.501714+0000","last_peered":"2024-09-19T04:13:31.501714+0000","last_clean":"2024-09-19T04:13:31.501714+0000","last_became_active":"2024-09-19T04:13:16.453592+0000","last_became_peered":"2024-09-19T04:13:16.453592+0000","last_unstale":"2024-09-19T04:13:31.501714+0000","last_undegraded":"2024-09-19T04:13:31.501714+0000","last_fullsized":"2024-09-19T04:13:31.501714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.304478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499027+0000","last_change":"2024-09-19T04:13:16.448763+0000","last_active":"2024-09-19T04:13:31.499027+0000","last_peered":"2024-09-19T04:13:31.499027+0000","last_clean":"2024-09-19T04:13:31.499027+0000","last_became_active":"2024-09-19T04:13:16.448459+0000","last_became_peered":"2024-09-19T04:13:16.448459+0000","last_unstale":"2024-09-19T04:13:31.499027+0000","last_undegraded":"2024-09-19T04:13:31.499027+0000","last_fullsized":"2024-09-19T04:13:31.499027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700507+0000","last_change":"2024-09-19T04:13:16.451489+0000","last_active":"2024-09-19T04:13:31.700507+0000","last_peered":"2024-09-19T04:13:31.700507+0000","last_clean":"2024-09-19T04:13:31.700507+0000","last_became_active":"2024-09-19T04:13:16.451176+0000","last_became_peered":"2024-09-19T04:13:16.451176+0000","last_unstale":"2024-09-19T04:13:31.700507+0000","last_undegraded":"2024-09-19T04:13:31.700507+0000","last_fullsized":"2024-09-19T04:13:31.700507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.921102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501314+0000","last_change":"2024-09-19T04:13:16.454551+0000","last_active":"2024-09-19T04:13:31.501314+0000","last_peered":"2024-09-19T04:13:31.501314+0000","last_clean":"2024-09-19T04:13:31.501314+0000","last_became_active":"2024-09-19T04:13:16.453976+0000","last_became_peered":"2024-09-19T04:13:16.453976+0000","last_unstale":"2024-09-19T04:13:31.501314+0000","last_undegraded":"2024-09-19T04:13:31.501314+0000","last_fullsized":"2024-09-19T04:13:31.501314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:06.504615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504331+0000","last_change":"2024-09-19T04:13:31.503222+0000","last_active":"2024-09-19T04:13:31.504331+0000","last_peered":"2024-09-19T04:13:31.504331+0000","last_clean":"2024-09-19T04:13:31.504331+0000","last_became_active":"2024-09-19T04:13:31.502873+0000","last_became_peered":"2024-09-19T04:13:31.502873+0000","last_unstale":"2024-09-19T04:13:31.504331+0000","last_undegraded":"2024-09-19T04:13:31.504331+0000","last_fullsized":"2024-09-19T04:13:31.504331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:54.559592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505834+0000","last_change":"2024-09-19T04:13:31.505834+0000","last_active":"2024-09-19T04:13:31.505834+0000","last_peered":"2024-09-19T04:13:31.505834+0000","last_clean":"2024-09-19T04:13:31.505834+0000","last_became_active":"2024-09-19T04:13:31.505501+0000","last_became_peered":"2024-09-19T04:13:31.505501+0000","last_unstale":"2024-09-19T04:13:31.505834+0000","last_undegraded":"2024-09-19T04:13:31.505834+0000","last_fullsized":"2024-09-19T04:13:31.505834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:45.136128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504841+0000","last_change":"2024-09-19T04:13:16.449237+0000","last_active":"2024-09-19T04:13:31.504841+0000","last_peered":"2024-09-19T04:13:31.504841+0000","last_clean":"2024-09-19T04:13:31.504841+0000","last_became_active":"2024-09-19T04:13:16.448896+0000","last_became_peered":"2024-09-19T04:13:16.448896+0000","last_unstale":"2024-09-19T04:13:31.504841+0000","last_undegraded":"2024-09-19T04:13:31.504841+0000","last_fullsized":"2024-09-19T04:13:31.504841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:38.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501835+0000","last_change":"2024-09-19T04:13:16.456053+0000","last_active":"2024-09-19T04:13:31.501835+0000","last_peered":"2024-09-19T04:13:31.501835+0000","last_clean":"2024-09-19T04:13:31.501835+0000","last_became_active":"2024-09-19T04:13:16.454775+0000","last_became_peered":"2024-09-19T04:13:16.454775+0000","last_unstale":"2024-09-19T04:13:31.501835+0000","last_undegraded":"2024-09-19T04:13:31.501835+0000","last_fullsized":"2024-09-19T04:13:31.501835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.902036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499121+0000","last_change":"2024-09-19T04:13:16.451473+0000","last_active":"2024-09-19T04:13:31.499121+0000","last_peered":"2024-09-19T04:13:31.499121+0000","last_clean":"2024-09-19T04:13:31.499121+0000","last_became_active":"2024-09-19T04:13:16.451150+0000","last_became_peered":"2024-09-19T04:13:16.451150+0000","last_unstale":"2024-09-19T04:13:31.499121+0000","last_undegraded":"2024-09-19T04:13:31.499121+0000","last_fullsized":"2024-09-19T04:13:31.499121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:59.332745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735656+0000","last_change":"2024-09-19T04:13:16.452017+0000","last_active":"2024-09-19T04:13:31.735656+0000","last_peered":"2024-09-19T04:13:31.735656+0000","last_clean":"2024-09-19T04:13:31.735656+0000","last_became_active":"2024-09-19T04:13:16.450715+0000","last_became_peered":"2024-09-19T04:13:16.450715+0000","last_unstale":"2024-09-19T04:13:31.735656+0000","last_undegraded":"2024-09-19T04:13:31.735656+0000","last_fullsized":"2024-09-19T04:13:31.735656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:43.548770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.736597+0000","last_change":"2024-09-19T04:13:31.736597+0000","last_active":"2024-09-19T04:13:31.736597+0000","last_peered":"2024-09-19T04:13:31.736597+0000","last_clean":"2024-09-19T04:13:31.736597+0000","last_became_active":"2024-09-19T04:13:31.736301+0000","last_became_peered":"2024-09-19T04:13:31.736301+0000","last_unstale":"2024-09-19T04:13:31.736597+0000","last_undegraded":"2024-09-19T04:13:31.736597+0000","last_fullsized":"2024-09-19T04:13:31.736597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:06.614117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.509742+0000","last_change":"2024-09-19T04:13:31.509742+0000","last_active":"2024-09-19T04:13:31.509742+0000","last_peered":"2024-09-19T04:13:31.509742+0000","last_clean":"2024-09-19T04:13:31.509742+0000","last_became_active":"2024-09-19T04:13:31.509363+0000","last_became_peered":"2024-09-19T04:13:31.509363+0000","last_unstale":"2024-09-19T04:13:31.509742+0000","last_undegraded":"2024-09-19T04:13:31.509742+0000","last_fullsized":"2024-09-19T04:13:31.509742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.094930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501525+0000","last_change":"2024-09-19T04:13:16.449459+0000","last_active":"2024-09-19T04:13:31.501525+0000","last_peered":"2024-09-19T04:13:31.501525+0000","last_clean":"2024-09-19T04:13:31.501525+0000","last_became_active":"2024-09-19T04:13:16.449119+0000","last_became_peered":"2024-09-19T04:13:16.449119+0000","last_unstale":"2024-09-19T04:13:31.501525+0000","last_undegraded":"2024-09-19T04:13:31.501525+0000","last_fullsized":"2024-09-19T04:13:31.501525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:26:26.941609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504930+0000","last_change":"2024-09-19T04:13:16.448939+0000","last_active":"2024-09-19T04:13:31.504930+0000","last_peered":"2024-09-19T04:13:31.504930+0000","last_clean":"2024-09-19T04:13:31.504930+0000","last_became_active":"2024-09-19T04:13:16.448619+0000","last_became_peered":"2024-09-19T04:13:16.448619+0000","last_unstale":"2024-09-19T04:13:31.504930+0000","last_undegraded":"2024-09-19T04:13:31.504930+0000","last_fullsized":"2024-09-19T04:13:31.504930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:41.017404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:12.563797+0000","last_change":"2024-09-19T04:13:14.430467+0000","last_active":"2024-09-19T04:14:12.563797+0000","last_peered":"2024-09-19T04:14:12.563797+0000","last_clean":"2024-09-19T04:14:12.563797+0000","last_became_active":"2024-09-19T04:13:14.430095+0000","last_became_peered":"2024-09-19T04:13:14.430095+0000","last_unstale":"2024-09-19T04:14:12.563797+0000","last_undegraded":"2024-09-19T04:14:12.563797+0000","last_fullsized":"2024-09-19T04:14:12.563797+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:50.471745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506178+0000","last_change":"2024-09-19T04:13:31.505798+0000","last_active":"2024-09-19T04:13:31.506178+0000","last_peered":"2024-09-19T04:13:31.506178+0000","last_clean":"2024-09-19T04:13:31.506178+0000","last_became_active":"2024-09-19T04:13:31.505469+0000","last_became_peered":"2024-09-19T04:13:31.505469+0000","last_unstale":"2024-09-19T04:13:31.506178+0000","last_undegraded":"2024-09-19T04:13:31.506178+0000","last_fullsized":"2024-09-19T04:13:31.506178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:13:42.405075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499636+0000","last_change":"2024-09-19T04:13:14.426443+0000","last_active":"2024-09-19T04:13:31.499636+0000","last_peered":"2024-09-19T04:13:31.499636+0000","last_clean":"2024-09-19T04:13:31.499636+0000","last_became_active":"2024-09-19T04:13:14.426103+0000","last_became_peered":"2024-09-19T04:13:14.426103+0000","last_unstale":"2024-09-19T04:13:31.499636+0000","last_undegraded":"2024-09-19T04:13:31.499636+0000","last_fullsized":"2024-09-19T04:13:31.499636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:28.461962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498768+0000","last_change":"2024-09-19T04:13:14.424439+0000","last_active":"2024-09-19T04:13:31.498768+0000","last_peered":"2024-09-19T04:13:31.498768+0000","last_clean":"2024-09-19T04:13:31.498768+0000","last_became_active":"2024-09-19T04:13:14.423975+0000","last_became_peered":"2024-09-19T04:13:14.423975+0000","last_unstale":"2024-09-19T04:13:31.498768+0000","last_undegraded":"2024-09-19T04:13:31.498768+0000","last_fullsized":"2024-09-19T04:13:31.498768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:12.032125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501447+0000","last_change":"2024-09-19T04:13:16.444865+0000","last_active":"2024-09-19T04:13:31.501447+0000","last_peered":"2024-09-19T04:13:31.501447+0000","last_clean":"2024-09-19T04:13:31.501447+0000","last_became_active":"2024-09-19T04:13:16.444556+0000","last_became_peered":"2024-09-19T04:13:16.444556+0000","last_unstale":"2024-09-19T04:13:31.501447+0000","last_undegraded":"2024-09-19T04:13:31.501447+0000","last_fullsized":"2024-09-19T04:13:31.501447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:38.025604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505570+0000","last_change":"2024-09-19T04:13:14.429018+0000","last_active":"2024-09-19T04:13:31.505570+0000","last_peered":"2024-09-19T04:13:31.505570+0000","last_clean":"2024-09-19T04:13:31.505570+0000","last_became_active":"2024-09-19T04:13:14.428383+0000","last_became_peered":"2024-09-19T04:13:14.428383+0000","last_unstale":"2024-09-19T04:13:31.505570+0000","last_undegraded":"2024-09-19T04:13:31.505570+0000","last_fullsized":"2024-09-19T04:13:31.505570+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:46.190421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501335+0000","last_change":"2024-09-19T04:13:16.449013+0000","last_active":"2024-09-19T04:13:31.501335+0000","last_peered":"2024-09-19T04:13:31.501335+0000","last_clean":"2024-09-19T04:13:31.501335+0000","last_became_active":"2024-09-19T04:13:16.448703+0000","last_became_peered":"2024-09-19T04:13:16.448703+0000","last_unstale":"2024-09-19T04:13:31.501335+0000","last_undegraded":"2024-09-19T04:13:31.501335+0000","last_fullsized":"2024-09-19T04:13:31.501335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:41.231759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":72,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:11.562015+0000","last_change":"2024-09-19T04:13:14.430050+0000","last_active":"2024-09-19T04:14:11.562015+0000","last_peered":"2024-09-19T04:14:11.562015+0000","last_clean":"2024-09-19T04:14:11.562015+0000","last_became_active":"2024-09-19T04:13:14.429710+0000","last_became_peered":"2024-09-19T04:13:14.429710+0000","last_unstale":"2024-09-19T04:14:11.562015+0000","last_undegraded":"2024-09-19T04:14:11.562015+0000","last_fullsized":"2024-09-19T04:14:11.562015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:55.796080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700932+0000","last_change":"2024-09-19T04:13:16.454431+0000","last_active":"2024-09-19T04:13:31.700932+0000","last_peered":"2024-09-19T04:13:31.700932+0000","last_clean":"2024-09-19T04:13:31.700932+0000","last_became_active":"2024-09-19T04:13:16.453033+0000","last_became_peered":"2024-09-19T04:13:16.453033+0000","last_unstale":"2024-09-19T04:13:31.700932+0000","last_undegraded":"2024-09-19T04:13:31.700932+0000","last_fullsized":"2024-09-19T04:13:31.700932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:52.817740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501175+0000","last_change":"2024-09-19T04:13:14.428949+0000","last_active":"2024-09-19T04:13:31.501175+0000","last_peered":"2024-09-19T04:13:31.501175+0000","last_clean":"2024-09-19T04:13:31.501175+0000","last_became_active":"2024-09-19T04:13:14.428525+0000","last_became_peered":"2024-09-19T04:13:14.428525+0000","last_unstale":"2024-09-19T04:13:31.501175+0000","last_undegraded":"2024-09-19T04:13:31.501175+0000","last_fullsized":"2024-09-19T04:13:31.501175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:30.296730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735848+0000","last_change":"2024-09-19T04:13:16.454077+0000","last_active":"2024-09-19T04:13:31.735848+0000","last_peered":"2024-09-19T04:13:31.735848+0000","last_clean":"2024-09-19T04:13:31.735848+0000","last_became_active":"2024-09-19T04:13:16.453791+0000","last_became_peered":"2024-09-19T04:13:16.453791+0000","last_unstale":"2024-09-19T04:13:31.735848+0000","last_undegraded":"2024-09-19T04:13:31.735848+0000","last_fullsized":"2024-09-19T04:13:31.735848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:29.426475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505069+0000","last_change":"2024-09-19T04:13:14.620179+0000","last_active":"2024-09-19T04:13:31.505069+0000","last_peered":"2024-09-19T04:13:31.505069+0000","last_clean":"2024-09-19T04:13:31.505069+0000","last_became_active":"2024-09-19T04:13:14.619758+0000","last_became_peered":"2024-09-19T04:13:14.619758+0000","last_unstale":"2024-09-19T04:13:31.505069+0000","last_undegraded":"2024-09-19T04:13:31.505069+0000","last_fullsized":"2024-09-19T04:13:31.505069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:11.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700712+0000","last_change":"2024-09-19T04:13:16.453036+0000","last_active":"2024-09-19T04:13:31.700712+0000","last_peered":"2024-09-19T04:13:31.700712+0000","last_clean":"2024-09-19T04:13:31.700712+0000","last_became_active":"2024-09-19T04:13:16.452736+0000","last_became_peered":"2024-09-19T04:13:16.452736+0000","last_unstale":"2024-09-19T04:13:31.700712+0000","last_undegraded":"2024-09-19T04:13:31.700712+0000","last_fullsized":"2024-09-19T04:13:31.700712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:30.003652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'28","reported_seq":152,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:12.570080+0000","last_change":"2024-09-19T04:13:14.428715+0000","last_active":"2024-09-19T04:14:12.570080+0000","last_peered":"2024-09-19T04:14:12.570080+0000","last_clean":"2024-09-19T04:14:12.570080+0000","last_became_active":"2024-09-19T04:13:14.428409+0000","last_became_peered":"2024-09-19T04:13:14.428409+0000","last_unstale":"2024-09-19T04:14:12.570080+0000","last_undegraded":"2024-09-19T04:14:12.570080+0000","last_fullsized":"2024-09-19T04:14:12.570080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:23.130857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":102,"num_read_kb":109,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504291+0000","last_change":"2024-09-19T04:13:16.454512+0000","last_active":"2024-09-19T04:13:31.504291+0000","last_peered":"2024-09-19T04:13:31.504291+0000","last_clean":"2024-09-19T04:13:31.504291+0000","last_became_active":"2024-09-19T04:13:16.454193+0000","last_became_peered":"2024-09-19T04:13:16.454193+0000","last_unstale":"2024-09-19T04:13:31.504291+0000","last_undegraded":"2024-09-19T04:13:31.504291+0000","last_fullsized":"2024-09-19T04:13:31.504291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:11.825938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.515863+0000","last_change":"2024-09-19T04:13:31.515448+0000","last_active":"2024-09-19T04:13:31.515863+0000","last_peered":"2024-09-19T04:13:31.515863+0000","last_clean":"2024-09-19T04:13:31.515863+0000","last_became_active":"2024-09-19T04:13:31.506099+0000","last_became_peered":"2024-09-19T04:13:31.506099+0000","last_unstale":"2024-09-19T04:13:31.515863+0000","last_undegraded":"2024-09-19T04:13:31.515863+0000","last_fullsized":"2024-09-19T04:13:31.515863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:21.462488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498861+0000","last_change":"2024-09-19T04:13:16.447607+0000","last_active":"2024-09-19T04:13:31.498861+0000","last_peered":"2024-09-19T04:13:31.498861+0000","last_clean":"2024-09-19T04:13:31.498861+0000","last_became_active":"2024-09-19T04:13:16.447029+0000","last_became_peered":"2024-09-19T04:13:16.447029+0000","last_unstale":"2024-09-19T04:13:31.498861+0000","last_undegraded":"2024-09-19T04:13:31.498861+0000","last_fullsized":"2024-09-19T04:13:31.498861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:52.418104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505867+0000","last_change":"2024-09-19T04:13:14.428856+0000","last_active":"2024-09-19T04:13:31.505867+0000","last_peered":"2024-09-19T04:13:31.505867+0000","last_clean":"2024-09-19T04:13:31.505867+0000","last_became_active":"2024-09-19T04:13:14.428510+0000","last_became_peered":"2024-09-19T04:13:14.428510+0000","last_unstale":"2024-09-19T04:13:31.505867+0000","last_undegraded":"2024-09-19T04:13:31.505867+0000","last_fullsized":"2024-09-19T04:13:31.505867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:49.633325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501063+0000","last_change":"2024-09-19T04:13:16.455628+0000","last_active":"2024-09-19T04:13:31.501063+0000","last_peered":"2024-09-19T04:13:31.501063+0000","last_clean":"2024-09-19T04:13:31.501063+0000","last_became_active":"2024-09-19T04:13:16.453817+0000","last_became_peered":"2024-09-19T04:13:16.453817+0000","last_unstale":"2024-09-19T04:13:31.501063+0000","last_undegraded":"2024-09-19T04:13:31.501063+0000","last_fullsized":"2024-09-19T04:13:31.501063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.832806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'4","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:13.564396+0000","last_change":"2024-09-19T04:13:14.424924+0000","last_active":"2024-09-19T04:14:13.564396+0000","last_peered":"2024-09-19T04:14:13.564396+0000","last_clean":"2024-09-19T04:14:13.564396+0000","last_became_active":"2024-09-19T04:13:14.424627+0000","last_became_peered":"2024-09-19T04:13:14.424627+0000","last_unstale":"2024-09-19T04:14:13.564396+0000","last_undegraded":"2024-09-19T04:14:13.564396+0000","last_fullsized":"2024-09-19T04:14:13.564396+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:49.830205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500550+0000","last_change":"2024-09-19T04:13:16.455365+0000","last_active":"2024-09-19T04:13:31.500550+0000","last_peered":"2024-09-19T04:13:31.500550+0000","last_clean":"2024-09-19T04:13:31.500550+0000","last_became_active":"2024-09-19T04:13:16.455029+0000","last_became_peered":"2024-09-19T04:13:16.455029+0000","last_unstale":"2024-09-19T04:13:31.500550+0000","last_undegraded":"2024-09-19T04:13:31.500550+0000","last_fullsized":"2024-09-19T04:13:31.500550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:20.947391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504041+0000","last_change":"2024-09-19T04:13:14.620462+0000","last_active":"2024-09-19T04:13:31.504041+0000","last_peered":"2024-09-19T04:13:31.504041+0000","last_clean":"2024-09-19T04:13:31.504041+0000","last_became_active":"2024-09-19T04:13:14.620103+0000","last_became_peered":"2024-09-19T04:13:14.620103+0000","last_unstale":"2024-09-19T04:13:31.504041+0000","last_undegraded":"2024-09-19T04:13:31.504041+0000","last_fullsized":"2024-09-19T04:13:31.504041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:14.496075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506534+0000","last_change":"2024-09-19T04:13:16.450222+0000","last_active":"2024-09-19T04:13:31.506534+0000","last_peered":"2024-09-19T04:13:31.506534+0000","last_clean":"2024-09-19T04:13:31.506534+0000","last_became_active":"2024-09-19T04:13:16.449916+0000","last_became_peered":"2024-09-19T04:13:16.449916+0000","last_unstale":"2024-09-19T04:13:31.506534+0000","last_undegraded":"2024-09-19T04:13:31.506534+0000","last_fullsized":"2024-09-19T04:13:31.506534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:31.886342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.519612+0000","last_change":"2024-09-19T04:13:31.519190+0000","last_active":"2024-09-19T04:13:31.519612+0000","last_peered":"2024-09-19T04:13:31.519612+0000","last_clean":"2024-09-19T04:13:31.519612+0000","last_became_active":"2024-09-19T04:13:31.508605+0000","last_became_peered":"2024-09-19T04:13:31.508605+0000","last_unstale":"2024-09-19T04:13:31.519612+0000","last_undegraded":"2024-09-19T04:13:31.519612+0000","last_fullsized":"2024-09-19T04:13:31.519612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:51.437135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502356+0000","last_change":"2024-09-19T04:13:16.450848+0000","last_active":"2024-09-19T04:13:31.502356+0000","last_peered":"2024-09-19T04:13:31.502356+0000","last_clean":"2024-09-19T04:13:31.502356+0000","last_became_active":"2024-09-19T04:13:16.450496+0000","last_became_peered":"2024-09-19T04:13:16.450496+0000","last_unstale":"2024-09-19T04:13:31.502356+0000","last_undegraded":"2024-09-19T04:13:31.502356+0000","last_fullsized":"2024-09-19T04:13:31.502356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:49:00.346998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499054+0000","last_change":"2024-09-19T04:13:14.428654+0000","last_active":"2024-09-19T04:13:31.499054+0000","last_peered":"2024-09-19T04:13:31.499054+0000","last_clean":"2024-09-19T04:13:31.499054+0000","last_became_active":"2024-09-19T04:13:14.428297+0000","last_became_peered":"2024-09-19T04:13:14.428297+0000","last_unstale":"2024-09-19T04:13:31.499054+0000","last_undegraded":"2024-09-19T04:13:31.499054+0000","last_fullsized":"2024-09-19T04:13:31.499054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:30.834262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501214+0000","last_change":"2024-09-19T04:13:16.456002+0000","last_active":"2024-09-19T04:13:31.501214+0000","last_peered":"2024-09-19T04:13:31.501214+0000","last_clean":"2024-09-19T04:13:31.501214+0000","last_became_active":"2024-09-19T04:13:16.455463+0000","last_became_peered":"2024-09-19T04:13:16.455463+0000","last_unstale":"2024-09-19T04:13:31.501214+0000","last_undegraded":"2024-09-19T04:13:31.501214+0000","last_fullsized":"2024-09-19T04:13:31.501214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:47.266364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505707+0000","last_change":"2024-09-19T04:13:14.429555+0000","last_active":"2024-09-19T04:13:31.505707+0000","last_peered":"2024-09-19T04:13:31.505707+0000","last_clean":"2024-09-19T04:13:31.505707+0000","last_became_active":"2024-09-19T04:13:14.429224+0000","last_became_peered":"2024-09-19T04:13:14.429224+0000","last_unstale":"2024-09-19T04:13:31.505707+0000","last_undegraded":"2024-09-19T04:13:31.505707+0000","last_fullsized":"2024-09-19T04:13:31.505707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:35.304750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501182+0000","last_change":"2024-09-19T04:13:16.454337+0000","last_active":"2024-09-19T04:13:31.501182+0000","last_peered":"2024-09-19T04:13:31.501182+0000","last_clean":"2024-09-19T04:13:31.501182+0000","last_became_active":"2024-09-19T04:13:16.453831+0000","last_became_peered":"2024-09-19T04:13:16.453831+0000","last_unstale":"2024-09-19T04:13:31.501182+0000","last_undegraded":"2024-09-19T04:13:31.501182+0000","last_fullsized":"2024-09-19T04:13:31.501182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:41.751933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506399+0000","last_change":"2024-09-19T04:13:14.620406+0000","last_active":"2024-09-19T04:13:31.506399+0000","last_peered":"2024-09-19T04:13:31.506399+0000","last_clean":"2024-09-19T04:13:31.506399+0000","last_became_active":"2024-09-19T04:13:14.620050+0000","last_became_peered":"2024-09-19T04:13:14.620050+0000","last_unstale":"2024-09-19T04:13:31.506399+0000","last_undegraded":"2024-09-19T04:13:31.506399+0000","last_fullsized":"2024-09-19T04:13:31.506399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:35.828760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501221+0000","last_change":"2024-09-19T04:13:16.459016+0000","last_active":"2024-09-19T04:13:31.501221+0000","last_peered":"2024-09-19T04:13:31.501221+0000","last_clean":"2024-09-19T04:13:31.501221+0000","last_became_active":"2024-09-19T04:13:16.458674+0000","last_became_peered":"2024-09-19T04:13:16.458674+0000","last_unstale":"2024-09-19T04:13:31.501221+0000","last_undegraded":"2024-09-19T04:13:31.501221+0000","last_fullsized":"2024-09-19T04:13:31.501221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:55.894456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":346,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500892+0000","last_change":"2024-09-19T04:12:05.913704+0000","last_active":"2024-09-19T04:13:31.500892+0000","last_peered":"2024-09-19T04:13:31.500892+0000","last_clean":"2024-09-19T04:13:31.500892+0000","last_became_active":"2024-09-19T04:12:05.913299+0000","last_became_peered":"2024-09-19T04:12:05.913299+0000","last_unstale":"2024-09-19T04:13:31.500892+0000","last_undegraded":"2024-09-19T04:13:31.500892+0000","last_fullsized":"2024-09-19T04:13:31.500892+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_clean_scrub_stamp":"2024-09-19T04:08:46.646000+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.638160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504518+0000","last_change":"2024-09-19T04:13:14.423962+0000","last_active":"2024-09-19T04:13:31.504518+0000","last_peered":"2024-09-19T04:13:31.504518+0000","last_clean":"2024-09-19T04:13:31.504518+0000","last_became_active":"2024-09-19T04:13:14.423591+0000","last_became_peered":"2024-09-19T04:13:14.423591+0000","last_unstale":"2024-09-19T04:13:31.504518+0000","last_undegraded":"2024-09-19T04:13:31.504518+0000","last_fullsized":"2024-09-19T04:13:31.504518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:02.239975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501743+0000","last_change":"2024-09-19T04:13:16.447113+0000","last_active":"2024-09-19T04:13:31.501743+0000","last_peered":"2024-09-19T04:13:31.501743+0000","last_clean":"2024-09-19T04:13:31.501743+0000","last_became_active":"2024-09-19T04:13:16.446792+0000","last_became_peered":"2024-09-19T04:13:16.446792+0000","last_unstale":"2024-09-19T04:13:31.501743+0000","last_undegraded":"2024-09-19T04:13:31.501743+0000","last_fullsized":"2024-09-19T04:13:31.501743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:28:21.527010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500749+0000","last_change":"2024-09-19T04:13:14.432663+0000","last_active":"2024-09-19T04:13:31.500749+0000","last_peered":"2024-09-19T04:13:31.500749+0000","last_clean":"2024-09-19T04:13:31.500749+0000","last_became_active":"2024-09-19T04:13:14.432318+0000","last_became_peered":"2024-09-19T04:13:14.432318+0000","last_unstale":"2024-09-19T04:13:31.500749+0000","last_undegraded":"2024-09-19T04:13:31.500749+0000","last_fullsized":"2024-09-19T04:13:31.500749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:35.787596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498761+0000","last_change":"2024-09-19T04:13:16.451010+0000","last_active":"2024-09-19T04:13:31.498761+0000","last_peered":"2024-09-19T04:13:31.498761+0000","last_clean":"2024-09-19T04:13:31.498761+0000","last_became_active":"2024-09-19T04:13:16.450708+0000","last_became_peered":"2024-09-19T04:13:16.450708+0000","last_unstale":"2024-09-19T04:13:31.498761+0000","last_undegraded":"2024-09-19T04:13:31.498761+0000","last_fullsized":"2024-09-19T04:13:31.498761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:30.108541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:12.562817+0000","last_change":"2024-09-19T04:13:14.620037+0000","last_active":"2024-09-19T04:14:12.562817+0000","last_peered":"2024-09-19T04:14:12.562817+0000","last_clean":"2024-09-19T04:14:12.562817+0000","last_became_active":"2024-09-19T04:13:14.619686+0000","last_became_peered":"2024-09-19T04:13:14.619686+0000","last_unstale":"2024-09-19T04:14:12.562817+0000","last_undegraded":"2024-09-19T04:14:12.562817+0000","last_fullsized":"2024-09-19T04:14:12.562817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:32.490708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504838+0000","last_change":"2024-09-19T04:13:16.446429+0000","last_active":"2024-09-19T04:13:31.504838+0000","last_peered":"2024-09-19T04:13:31.504838+0000","last_clean":"2024-09-19T04:13:31.504838+0000","last_became_active":"2024-09-19T04:13:16.446075+0000","last_became_peered":"2024-09-19T04:13:16.446075+0000","last_unstale":"2024-09-19T04:13:31.504838+0000","last_undegraded":"2024-09-19T04:13:31.504838+0000","last_fullsized":"2024-09-19T04:13:31.504838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:03.613926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501031+0000","last_change":"2024-09-19T04:13:14.429899+0000","last_active":"2024-09-19T04:13:31.501031+0000","last_peered":"2024-09-19T04:13:31.501031+0000","last_clean":"2024-09-19T04:13:31.501031+0000","last_became_active":"2024-09-19T04:13:14.429571+0000","last_became_peered":"2024-09-19T04:13:14.429571+0000","last_unstale":"2024-09-19T04:13:31.501031+0000","last_undegraded":"2024-09-19T04:13:31.501031+0000","last_fullsized":"2024-09-19T04:13:31.501031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:37.017431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499733+0000","last_change":"2024-09-19T04:13:16.458267+0000","last_active":"2024-09-19T04:13:31.499733+0000","last_peered":"2024-09-19T04:13:31.499733+0000","last_clean":"2024-09-19T04:13:31.499733+0000","last_became_active":"2024-09-19T04:13:16.457911+0000","last_became_peered":"2024-09-19T04:13:16.457911+0000","last_unstale":"2024-09-19T04:13:31.499733+0000","last_undegraded":"2024-09-19T04:13:31.499733+0000","last_fullsized":"2024-09-19T04:13:31.499733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:32.465923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501677+0000","last_change":"2024-09-19T04:13:14.621552+0000","last_active":"2024-09-19T04:13:31.501677+0000","last_peered":"2024-09-19T04:13:31.501677+0000","last_clean":"2024-09-19T04:13:31.501677+0000","last_became_active":"2024-09-19T04:13:14.620592+0000","last_became_peered":"2024-09-19T04:13:14.620592+0000","last_unstale":"2024-09-19T04:13:31.501677+0000","last_undegraded":"2024-09-19T04:13:31.501677+0000","last_fullsized":"2024-09-19T04:13:31.501677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:10.055202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500150+0000","last_change":"2024-09-19T04:13:16.443117+0000","last_active":"2024-09-19T04:13:31.500150+0000","last_peered":"2024-09-19T04:13:31.500150+0000","last_clean":"2024-09-19T04:13:31.500150+0000","last_became_active":"2024-09-19T04:13:16.442808+0000","last_became_peered":"2024-09-19T04:13:16.442808+0000","last_unstale":"2024-09-19T04:13:31.500150+0000","last_undegraded":"2024-09-19T04:13:31.500150+0000","last_fullsized":"2024-09-19T04:13:31.500150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:40.274320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504531+0000","last_change":"2024-09-19T04:13:14.620618+0000","last_active":"2024-09-19T04:13:31.504531+0000","last_peered":"2024-09-19T04:13:31.504531+0000","last_clean":"2024-09-19T04:13:31.504531+0000","last_became_active":"2024-09-19T04:13:14.620290+0000","last_became_peered":"2024-09-19T04:13:14.620290+0000","last_unstale":"2024-09-19T04:13:31.504531+0000","last_undegraded":"2024-09-19T04:13:31.504531+0000","last_fullsized":"2024-09-19T04:13:31.504531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:51.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499218+0000","last_change":"2024-09-19T04:13:16.443498+0000","last_active":"2024-09-19T04:13:31.499218+0000","last_peered":"2024-09-19T04:13:31.499218+0000","last_clean":"2024-09-19T04:13:31.499218+0000","last_became_active":"2024-09-19T04:13:16.443104+0000","last_became_peered":"2024-09-19T04:13:16.443104+0000","last_unstale":"2024-09-19T04:13:31.499218+0000","last_undegraded":"2024-09-19T04:13:31.499218+0000","last_fullsized":"2024-09-19T04:13:31.499218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:10.733570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:40.455994+0000","last_change":"2024-09-19T04:13:14.425881+0000","last_active":"2024-09-19T04:13:40.455994+0000","last_peered":"2024-09-19T04:13:40.455994+0000","last_clean":"2024-09-19T04:13:40.455994+0000","last_became_active":"2024-09-19T04:13:14.425542+0000","last_became_peered":"2024-09-19T04:13:14.425542+0000","last_unstale":"2024-09-19T04:13:40.455994+0000","last_undegraded":"2024-09-19T04:13:40.455994+0000","last_fullsized":"2024-09-19T04:13:40.455994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:44.250890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502301+0000","last_change":"2024-09-19T04:13:16.454672+0000","last_active":"2024-09-19T04:13:31.502301+0000","last_peered":"2024-09-19T04:13:31.502301+0000","last_clean":"2024-09-19T04:13:31.502301+0000","last_became_active":"2024-09-19T04:13:16.454003+0000","last_became_peered":"2024-09-19T04:13:16.454003+0000","last_unstale":"2024-09-19T04:13:31.502301+0000","last_undegraded":"2024-09-19T04:13:31.502301+0000","last_fullsized":"2024-09-19T04:13:31.502301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:26.849617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504465+0000","last_change":"2024-09-19T04:13:14.432312+0000","last_active":"2024-09-19T04:13:31.504465+0000","last_peered":"2024-09-19T04:13:31.504465+0000","last_clean":"2024-09-19T04:13:31.504465+0000","last_became_active":"2024-09-19T04:13:14.431994+0000","last_became_peered":"2024-09-19T04:13:14.431994+0000","last_unstale":"2024-09-19T04:13:31.504465+0000","last_undegraded":"2024-09-19T04:13:31.504465+0000","last_fullsized":"2024-09-19T04:13:31.504465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:00.410400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735768+0000","last_change":"2024-09-19T04:13:16.455312+0000","last_active":"2024-09-19T04:13:31.735768+0000","last_peered":"2024-09-19T04:13:31.735768+0000","last_clean":"2024-09-19T04:13:31.735768+0000","last_became_active":"2024-09-19T04:13:16.454990+0000","last_became_peered":"2024-09-19T04:13:16.454990+0000","last_unstale":"2024-09-19T04:13:31.735768+0000","last_undegraded":"2024-09-19T04:13:31.735768+0000","last_fullsized":"2024-09-19T04:13:31.735768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:09.207395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700505+0000","last_change":"2024-09-19T04:13:14.620316+0000","last_active":"2024-09-19T04:13:31.700505+0000","last_peered":"2024-09-19T04:13:31.700505+0000","last_clean":"2024-09-19T04:13:31.700505+0000","last_became_active":"2024-09-19T04:13:14.619979+0000","last_became_peered":"2024-09-19T04:13:14.619979+0000","last_unstale":"2024-09-19T04:13:31.700505+0000","last_undegraded":"2024-09-19T04:13:31.700505+0000","last_fullsized":"2024-09-19T04:13:31.700505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:24.382144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505808+0000","last_change":"2024-09-19T04:13:16.455684+0000","last_active":"2024-09-19T04:13:31.505808+0000","last_peered":"2024-09-19T04:13:31.505808+0000","last_clean":"2024-09-19T04:13:31.505808+0000","last_became_active":"2024-09-19T04:13:16.455355+0000","last_became_peered":"2024-09-19T04:13:16.455355+0000","last_unstale":"2024-09-19T04:13:31.505808+0000","last_undegraded":"2024-09-19T04:13:31.505808+0000","last_fullsized":"2024-09-19T04:13:31.505808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:09.657447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501550+0000","last_change":"2024-09-19T04:13:14.423599+0000","last_active":"2024-09-19T04:13:31.501550+0000","last_peered":"2024-09-19T04:13:31.501550+0000","last_clean":"2024-09-19T04:13:31.501550+0000","last_became_active":"2024-09-19T04:13:14.423282+0000","last_became_peered":"2024-09-19T04:13:14.423282+0000","last_unstale":"2024-09-19T04:13:31.501550+0000","last_undegraded":"2024-09-19T04:13:31.501550+0000","last_fullsized":"2024-09-19T04:13:31.501550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.975368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501564+0000","last_change":"2024-09-19T04:13:16.450435+0000","last_active":"2024-09-19T04:13:31.501564+0000","last_peered":"2024-09-19T04:13:31.501564+0000","last_clean":"2024-09-19T04:13:31.501564+0000","last_became_active":"2024-09-19T04:13:16.450087+0000","last_became_peered":"2024-09-19T04:13:16.450087+0000","last_unstale":"2024-09-19T04:13:31.501564+0000","last_undegraded":"2024-09-19T04:13:31.501564+0000","last_fullsized":"2024-09-19T04:13:31.501564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:34:51.764776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504466+0000","last_change":"2024-09-19T04:13:14.432868+0000","last_active":"2024-09-19T04:13:31.504466+0000","last_peered":"2024-09-19T04:13:31.504466+0000","last_clean":"2024-09-19T04:13:31.504466+0000","last_became_active":"2024-09-19T04:13:14.432507+0000","last_became_peered":"2024-09-19T04:13:14.432507+0000","last_unstale":"2024-09-19T04:13:31.504466+0000","last_undegraded":"2024-09-19T04:13:31.504466+0000","last_fullsized":"2024-09-19T04:13:31.504466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:14.773711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501610+0000","last_change":"2024-09-19T04:13:16.450061+0000","last_active":"2024-09-19T04:13:31.501610+0000","last_peered":"2024-09-19T04:13:31.501610+0000","last_clean":"2024-09-19T04:13:31.501610+0000","last_became_active":"2024-09-19T04:13:16.448829+0000","last_became_peered":"2024-09-19T04:13:16.448829+0000","last_unstale":"2024-09-19T04:13:31.501610+0000","last_undegraded":"2024-09-19T04:13:31.501610+0000","last_fullsized":"2024-09-19T04:13:31.501610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:31.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506029+0000","last_change":"2024-09-19T04:13:14.431574+0000","last_active":"2024-09-19T04:13:31.506029+0000","last_peered":"2024-09-19T04:13:31.506029+0000","last_clean":"2024-09-19T04:13:31.506029+0000","last_became_active":"2024-09-19T04:13:14.431202+0000","last_became_peered":"2024-09-19T04:13:14.431202+0000","last_unstale":"2024-09-19T04:13:31.506029+0000","last_undegraded":"2024-09-19T04:13:31.506029+0000","last_fullsized":"2024-09-19T04:13:31.506029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:58.167201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500887+0000","last_change":"2024-09-19T04:13:16.452688+0000","last_active":"2024-09-19T04:13:31.500887+0000","last_peered":"2024-09-19T04:13:31.500887+0000","last_clean":"2024-09-19T04:13:31.500887+0000","last_became_active":"2024-09-19T04:13:16.452342+0000","last_became_peered":"2024-09-19T04:13:16.452342+0000","last_unstale":"2024-09-19T04:13:31.500887+0000","last_undegraded":"2024-09-19T04:13:31.500887+0000","last_fullsized":"2024-09-19T04:13:31.500887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:03.162688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504711+0000","last_change":"2024-09-19T04:13:16.449402+0000","last_active":"2024-09-19T04:13:31.504711+0000","last_peered":"2024-09-19T04:13:31.504711+0000","last_clean":"2024-09-19T04:13:31.504711+0000","last_became_active":"2024-09-19T04:13:16.449108+0000","last_became_peered":"2024-09-19T04:13:16.449108+0000","last_unstale":"2024-09-19T04:13:31.504711+0000","last_undegraded":"2024-09-19T04:13:31.504711+0000","last_fullsized":"2024-09-19T04:13:31.504711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:58.874638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505684+0000","last_change":"2024-09-19T04:13:14.430684+0000","last_active":"2024-09-19T04:13:31.505684+0000","last_peered":"2024-09-19T04:13:31.505684+0000","last_clean":"2024-09-19T04:13:31.505684+0000","last_became_active":"2024-09-19T04:13:14.430137+0000","last_became_peered":"2024-09-19T04:13:14.430137+0000","last_unstale":"2024-09-19T04:13:31.505684+0000","last_undegraded":"2024-09-19T04:13:31.505684+0000","last_fullsized":"2024-09-19T04:13:31.505684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:39.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700648+0000","last_change":"2024-09-19T04:13:16.452491+0000","last_active":"2024-09-19T04:13:31.700648+0000","last_peered":"2024-09-19T04:13:31.700648+0000","last_clean":"2024-09-19T04:13:31.700648+0000","last_became_active":"2024-09-19T04:13:16.452204+0000","last_became_peered":"2024-09-19T04:13:16.452204+0000","last_unstale":"2024-09-19T04:13:31.700648+0000","last_undegraded":"2024-09-19T04:13:31.700648+0000","last_fullsized":"2024-09-19T04:13:31.700648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:10.113510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504401+0000","last_change":"2024-09-19T04:13:14.431608+0000","last_active":"2024-09-19T04:13:31.504401+0000","last_peered":"2024-09-19T04:13:31.504401+0000","last_clean":"2024-09-19T04:13:31.504401+0000","last_became_active":"2024-09-19T04:13:14.431111+0000","last_became_peered":"2024-09-19T04:13:14.431111+0000","last_unstale":"2024-09-19T04:13:31.504401+0000","last_undegraded":"2024-09-19T04:13:31.504401+0000","last_fullsized":"2024-09-19T04:13:31.504401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:03:41.125315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735802+0000","last_change":"2024-09-19T04:13:16.457867+0000","last_active":"2024-09-19T04:13:31.735802+0000","last_peered":"2024-09-19T04:13:31.735802+0000","last_clean":"2024-09-19T04:13:31.735802+0000","last_became_active":"2024-09-19T04:13:16.457582+0000","last_became_peered":"2024-09-19T04:13:16.457582+0000","last_unstale":"2024-09-19T04:13:31.735802+0000","last_undegraded":"2024-09-19T04:13:31.735802+0000","last_fullsized":"2024-09-19T04:13:31.735802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:58.076250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501886+0000","last_change":"2024-09-19T04:13:14.620070+0000","last_active":"2024-09-19T04:13:31.501886+0000","last_peered":"2024-09-19T04:13:31.501886+0000","last_clean":"2024-09-19T04:13:31.501886+0000","last_became_active":"2024-09-19T04:13:14.619734+0000","last_became_peered":"2024-09-19T04:13:14.619734+0000","last_unstale":"2024-09-19T04:13:31.501886+0000","last_undegraded":"2024-09-19T04:13:31.501886+0000","last_fullsized":"2024-09-19T04:13:31.501886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:44:02.818859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501786+0000","last_change":"2024-09-19T04:13:16.443832+0000","last_active":"2024-09-19T04:13:31.501786+0000","last_peered":"2024-09-19T04:13:31.501786+0000","last_clean":"2024-09-19T04:13:31.501786+0000","last_became_active":"2024-09-19T04:13:16.442690+0000","last_became_peered":"2024-09-19T04:13:16.442690+0000","last_unstale":"2024-09-19T04:13:31.501786+0000","last_undegraded":"2024-09-19T04:13:31.501786+0000","last_fullsized":"2024-09-19T04:13:31.501786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:39.535628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501657+0000","last_change":"2024-09-19T04:13:14.427186+0000","last_active":"2024-09-19T04:13:31.501657+0000","last_peered":"2024-09-19T04:13:31.501657+0000","last_clean":"2024-09-19T04:13:31.501657+0000","last_became_active":"2024-09-19T04:13:14.426949+0000","last_became_peered":"2024-09-19T04:13:14.426949+0000","last_unstale":"2024-09-19T04:13:31.501657+0000","last_undegraded":"2024-09-19T04:13:31.501657+0000","last_fullsized":"2024-09-19T04:13:31.501657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:26.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":313,"num_read_kb":223,"num_write":162,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":431409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841566,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1082991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005091,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40388,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704764,"statfs":{"total":95995035648,"available":95953678336,"internally_reserved":0,"allocated":905216,"data_stored":431589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234025,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":431487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1023834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":892928,"data_stored":430978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1511424,"data_stored":1023125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":431808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":431289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476806,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":492244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":490535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":913408,"data_stored":432249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:14:20.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:14:21.038 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:21.519 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:21 smithi094 ceph-mon[30972]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:14:21.520 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:21 smithi094 ceph-mon[30972]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:21.520 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:21.520 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:21 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:21.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:21 smithi112 ceph-mon[35051]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:14:21.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:21 smithi112 ceph-mon[35051]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:21.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:21.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:21 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:21.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.597+0000 7f2923c67640 1 -- 172.21.15.94:0/909945991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c1027a0 msgr2=0x7f291c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.597+0000 7f2923c67640 1 --2- 172.21.15.94:0/909945991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c1027a0 0x7f291c1067f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c02a410 comp rx=0 tx=0).stop 2024-09-19T04:14:21.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.597+0000 7f2923c67640 1 -- 172.21.15.94:0/909945991 shutdown_connections 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.597+0000 7f2923c67640 1 --2- 172.21.15.94:0/909945991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f291c106f20 0x7f291c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.597+0000 7f2923c67640 1 --2- 172.21.15.94:0/909945991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c1027a0 0x7f291c1067f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.597+0000 7f2923c67640 1 --2- 172.21.15.94:0/909945991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c101df0 0x7f291c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.597+0000 7f2923c67640 1 -- 172.21.15.94:0/909945991 >> 172.21.15.94:0/909945991 conn(0x7f291c0fd8f0 msgr2=0x7f291c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 -- 172.21.15.94:0/909945991 shutdown_connections 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 -- 172.21.15.94:0/909945991 wait complete. 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 Processor -- start 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 -- start start 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c101df0 0x7f291c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f291c1027a0 0x7f291c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c106f20 0x7f291c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f291c111e80 con 0x7f291c101df0 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f291c111d00 con 0x7f291c1027a0 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2923c67640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f291c112000 con 0x7f291c106f20 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2921a03640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c101df0 0x7f291c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.598+0000 7f2921a03640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c101df0 0x7f291c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52064/0 (socket says 172.21.15.94:52064) 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2921a03640 1 -- 172.21.15.94:0/2340709801 learned_addr learned my addr 172.21.15.94:0/2340709801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2921202640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f291c1027a0 0x7f291c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2922204640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c106f20 0x7f291c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2921a03640 1 -- 172.21.15.94:0/2340709801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c106f20 msgr2=0x7f291c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2921a03640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c106f20 0x7f291c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2921a03640 1 -- 172.21.15.94:0/2340709801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f291c1027a0 msgr2=0x7f291c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2921a03640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f291c1027a0 0x7f291c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2921a03640 1 -- 172.21.15.94:0/2340709801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c19d390 con 0x7f291c101df0 2024-09-19T04:14:21.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2922204640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c106f20 0x7f291c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:21.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2921a03640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c101df0 0x7f291c1983e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f291400d770 tx=0x7f291400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2912ffd640 1 -- 172.21.15.94:0/2340709801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914017070 con 0x7f291c101df0 2024-09-19T04:14:21.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2912ffd640 1 -- 172.21.15.94:0/2340709801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29140054d0 con 0x7f291c101df0 2024-09-19T04:14:21.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c19d680 con 0x7f291c101df0 2024-09-19T04:14:21.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2912ffd640 1 -- 172.21.15.94:0/2340709801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914004cf0 con 0x7f291c101df0 2024-09-19T04:14:21.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.599+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1a4f60 con 0x7f291c101df0 2024-09-19T04:14:21.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:21 smithi104 ceph-mon[34145]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:14:21.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:21 smithi104 ceph-mon[34145]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:21.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:21.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:21 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:21.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.600+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c1032d0 con 0x7f291c101df0 2024-09-19T04:14:21.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.601+0000 7f2912ffd640 1 -- 172.21.15.94:0/2340709801 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2914004e90 con 0x7f291c101df0 2024-09-19T04:14:21.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.602+0000 7f2912ffd640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29080a32d0 0x7f29080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.602+0000 7f2921202640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29080a32d0 0x7f29080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.602+0000 7f2912ffd640 1 -- 172.21.15.94:0/2340709801 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f29140cf5c0 con 0x7f291c101df0 2024-09-19T04:14:21.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.602+0000 7f2921202640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29080a32d0 0x7f29080a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f290c000c00 tx=0x7f290c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.606+0000 7f2912ffd640 1 -- 172.21.15.94:0/2340709801 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f291409c180 con 0x7f291c101df0 2024-09-19T04:14:21.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.780+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f291c062630 con 0x7f29080a32d0 2024-09-19T04:14:21.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.787+0000 7f2912ffd640 1 -- 172.21.15.94:0/2340709801 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258717 (secure 0 0 0) 0x7f291c062630 con 0x7f29080a32d0 2024-09-19T04:14:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:21.791 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29080a32d0 msgr2=0x7f29080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29080a32d0 0x7f29080a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f290c000c00 tx=0x7f290c031400 comp rx=0 tx=0).stop 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c101df0 msgr2=0x7f291c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c101df0 0x7f291c1983e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f291400d770 tx=0x7f291400dc30 comp rx=0 tx=0).stop 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 shutdown_connections 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f29080a32d0 0x7f29080a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c106f20 0x7f291c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f291c1027a0 0x7f291c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 --2- 172.21.15.94:0/2340709801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c101df0 0x7f291c1983e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 >> 172.21.15.94:0/2340709801 conn(0x7f291c0fd8f0 msgr2=0x7f291c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:21.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 shutdown_connections 2024-09-19T04:14:21.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:21.790+0000 7f2923c67640 1 -- 172.21.15.94:0/2340709801 wait complete. 2024-09-19T04:14:22.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:22.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:22.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:22 smithi094 ceph-mon[30972]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:22.459 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-19T04:14:20.145954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654762,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":474,"num_read_kb":357,"num_write":345,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399252,"kb_used_data":12540,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542572,"statfs":{"total":1151940427776,"available":1151531593728,"internally_reserved":0,"allocated":12840960,"data_stored":7133538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002731"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501952+0000","last_change":"2024-09-19T04:13:16.451031+0000","last_active":"2024-09-19T04:13:31.501952+0000","last_peered":"2024-09-19T04:13:31.501952+0000","last_clean":"2024-09-19T04:13:31.501952+0000","last_became_active":"2024-09-19T04:13:16.450713+0000","last_became_peered":"2024-09-19T04:13:16.450713+0000","last_unstale":"2024-09-19T04:13:31.501952+0000","last_undegraded":"2024-09-19T04:13:31.501952+0000","last_fullsized":"2024-09-19T04:13:31.501952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:59.485415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735600+0000","last_change":"2024-09-19T04:13:16.455689+0000","last_active":"2024-09-19T04:13:31.735600+0000","last_peered":"2024-09-19T04:13:31.735600+0000","last_clean":"2024-09-19T04:13:31.735600+0000","last_became_active":"2024-09-19T04:13:16.455154+0000","last_became_peered":"2024-09-19T04:13:16.455154+0000","last_unstale":"2024-09-19T04:13:31.735600+0000","last_undegraded":"2024-09-19T04:13:31.735600+0000","last_fullsized":"2024-09-19T04:13:31.735600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:57.610619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735526+0000","last_change":"2024-09-19T04:13:16.447392+0000","last_active":"2024-09-19T04:13:31.735526+0000","last_peered":"2024-09-19T04:13:31.735526+0000","last_clean":"2024-09-19T04:13:31.735526+0000","last_became_active":"2024-09-19T04:13:16.447088+0000","last_became_peered":"2024-09-19T04:13:16.447088+0000","last_unstale":"2024-09-19T04:13:31.735526+0000","last_undegraded":"2024-09-19T04:13:31.735526+0000","last_fullsized":"2024-09-19T04:13:31.735526+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:16.591420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504467+0000","last_change":"2024-09-19T04:13:16.452642+0000","last_active":"2024-09-19T04:13:31.504467+0000","last_peered":"2024-09-19T04:13:31.504467+0000","last_clean":"2024-09-19T04:13:31.504467+0000","last_became_active":"2024-09-19T04:13:16.452329+0000","last_became_peered":"2024-09-19T04:13:16.452329+0000","last_unstale":"2024-09-19T04:13:31.504467+0000","last_undegraded":"2024-09-19T04:13:31.504467+0000","last_fullsized":"2024-09-19T04:13:31.504467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:54.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505930+0000","last_change":"2024-09-19T04:13:16.447171+0000","last_active":"2024-09-19T04:13:31.505930+0000","last_peered":"2024-09-19T04:13:31.505930+0000","last_clean":"2024-09-19T04:13:31.505930+0000","last_became_active":"2024-09-19T04:13:16.446835+0000","last_became_peered":"2024-09-19T04:13:16.446835+0000","last_unstale":"2024-09-19T04:13:31.505930+0000","last_undegraded":"2024-09-19T04:13:31.505930+0000","last_fullsized":"2024-09-19T04:13:31.505930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:41.892118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.507665+0000","last_change":"2024-09-19T04:13:31.507665+0000","last_active":"2024-09-19T04:13:31.507665+0000","last_peered":"2024-09-19T04:13:31.507665+0000","last_clean":"2024-09-19T04:13:31.507665+0000","last_became_active":"2024-09-19T04:13:31.507329+0000","last_became_peered":"2024-09-19T04:13:31.507329+0000","last_unstale":"2024-09-19T04:13:31.507665+0000","last_undegraded":"2024-09-19T04:13:31.507665+0000","last_fullsized":"2024-09-19T04:13:31.507665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:35.832776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500742+0000","last_change":"2024-09-19T04:13:16.454568+0000","last_active":"2024-09-19T04:13:31.500742+0000","last_peered":"2024-09-19T04:13:31.500742+0000","last_clean":"2024-09-19T04:13:31.500742+0000","last_became_active":"2024-09-19T04:13:16.454230+0000","last_became_peered":"2024-09-19T04:13:16.454230+0000","last_unstale":"2024-09-19T04:13:31.500742+0000","last_undegraded":"2024-09-19T04:13:31.500742+0000","last_fullsized":"2024-09-19T04:13:31.500742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:23.401316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501235+0000","last_change":"2024-09-19T04:13:16.449223+0000","last_active":"2024-09-19T04:13:31.501235+0000","last_peered":"2024-09-19T04:13:31.501235+0000","last_clean":"2024-09-19T04:13:31.501235+0000","last_became_active":"2024-09-19T04:13:16.448665+0000","last_became_peered":"2024-09-19T04:13:16.448665+0000","last_unstale":"2024-09-19T04:13:31.501235+0000","last_undegraded":"2024-09-19T04:13:31.501235+0000","last_fullsized":"2024-09-19T04:13:31.501235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:29.619847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.701069+0000","last_change":"2024-09-19T04:13:16.458203+0000","last_active":"2024-09-19T04:13:31.701069+0000","last_peered":"2024-09-19T04:13:31.701069+0000","last_clean":"2024-09-19T04:13:31.701069+0000","last_became_active":"2024-09-19T04:13:16.457544+0000","last_became_peered":"2024-09-19T04:13:16.457544+0000","last_unstale":"2024-09-19T04:13:31.701069+0000","last_undegraded":"2024-09-19T04:13:31.701069+0000","last_fullsized":"2024-09-19T04:13:31.701069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:47.611695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500145+0000","last_change":"2024-09-19T04:13:16.454213+0000","last_active":"2024-09-19T04:13:31.500145+0000","last_peered":"2024-09-19T04:13:31.500145+0000","last_clean":"2024-09-19T04:13:31.500145+0000","last_became_active":"2024-09-19T04:13:16.453047+0000","last_became_peered":"2024-09-19T04:13:16.453047+0000","last_unstale":"2024-09-19T04:13:31.500145+0000","last_undegraded":"2024-09-19T04:13:31.500145+0000","last_fullsized":"2024-09-19T04:13:31.500145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:27.425259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501213+0000","last_change":"2024-09-19T04:13:16.458195+0000","last_active":"2024-09-19T04:13:31.501213+0000","last_peered":"2024-09-19T04:13:31.501213+0000","last_clean":"2024-09-19T04:13:31.501213+0000","last_became_active":"2024-09-19T04:13:16.457891+0000","last_became_peered":"2024-09-19T04:13:16.457891+0000","last_unstale":"2024-09-19T04:13:31.501213+0000","last_undegraded":"2024-09-19T04:13:31.501213+0000","last_fullsized":"2024-09-19T04:13:31.501213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:53.833249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.508474+0000","last_change":"2024-09-19T04:13:31.508117+0000","last_active":"2024-09-19T04:13:31.508474+0000","last_peered":"2024-09-19T04:13:31.508474+0000","last_clean":"2024-09-19T04:13:31.508474+0000","last_became_active":"2024-09-19T04:13:31.507693+0000","last_became_peered":"2024-09-19T04:13:31.507693+0000","last_unstale":"2024-09-19T04:13:31.508474+0000","last_undegraded":"2024-09-19T04:13:31.508474+0000","last_fullsized":"2024-09-19T04:13:31.508474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:58.447965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499011+0000","last_change":"2024-09-19T04:13:16.440079+0000","last_active":"2024-09-19T04:13:31.499011+0000","last_peered":"2024-09-19T04:13:31.499011+0000","last_clean":"2024-09-19T04:13:31.499011+0000","last_became_active":"2024-09-19T04:13:16.439474+0000","last_became_peered":"2024-09-19T04:13:16.439474+0000","last_unstale":"2024-09-19T04:13:31.499011+0000","last_undegraded":"2024-09-19T04:13:31.499011+0000","last_fullsized":"2024-09-19T04:13:31.499011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:11.151412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735816+0000","last_change":"2024-09-19T04:13:16.449958+0000","last_active":"2024-09-19T04:13:31.735816+0000","last_peered":"2024-09-19T04:13:31.735816+0000","last_clean":"2024-09-19T04:13:31.735816+0000","last_became_active":"2024-09-19T04:13:16.449665+0000","last_became_peered":"2024-09-19T04:13:16.449665+0000","last_unstale":"2024-09-19T04:13:31.735816+0000","last_undegraded":"2024-09-19T04:13:31.735816+0000","last_fullsized":"2024-09-19T04:13:31.735816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:24.023779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735453+0000","last_change":"2024-09-19T04:13:16.454333+0000","last_active":"2024-09-19T04:13:31.735453+0000","last_peered":"2024-09-19T04:13:31.735453+0000","last_clean":"2024-09-19T04:13:31.735453+0000","last_became_active":"2024-09-19T04:13:16.454066+0000","last_became_peered":"2024-09-19T04:13:16.454066+0000","last_unstale":"2024-09-19T04:13:31.735453+0000","last_undegraded":"2024-09-19T04:13:31.735453+0000","last_fullsized":"2024-09-19T04:13:31.735453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:16.815173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700446+0000","last_change":"2024-09-19T04:13:16.451864+0000","last_active":"2024-09-19T04:13:31.700446+0000","last_peered":"2024-09-19T04:13:31.700446+0000","last_clean":"2024-09-19T04:13:31.700446+0000","last_became_active":"2024-09-19T04:13:16.450995+0000","last_became_peered":"2024-09-19T04:13:16.450995+0000","last_unstale":"2024-09-19T04:13:31.700446+0000","last_undegraded":"2024-09-19T04:13:31.700446+0000","last_fullsized":"2024-09-19T04:13:31.700446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:40.750221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502172+0000","last_change":"2024-09-19T04:13:16.453176+0000","last_active":"2024-09-19T04:13:31.502172+0000","last_peered":"2024-09-19T04:13:31.502172+0000","last_clean":"2024-09-19T04:13:31.502172+0000","last_became_active":"2024-09-19T04:13:16.452788+0000","last_became_peered":"2024-09-19T04:13:16.452788+0000","last_unstale":"2024-09-19T04:13:31.502172+0000","last_undegraded":"2024-09-19T04:13:31.502172+0000","last_fullsized":"2024-09-19T04:13:31.502172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:47.099180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501446+0000","last_change":"2024-09-19T04:13:16.449167+0000","last_active":"2024-09-19T04:13:31.501446+0000","last_peered":"2024-09-19T04:13:31.501446+0000","last_clean":"2024-09-19T04:13:31.501446+0000","last_became_active":"2024-09-19T04:13:16.448579+0000","last_became_peered":"2024-09-19T04:13:16.448579+0000","last_unstale":"2024-09-19T04:13:31.501446+0000","last_undegraded":"2024-09-19T04:13:31.501446+0000","last_fullsized":"2024-09-19T04:13:31.501446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:22:40.023000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505138+0000","last_change":"2024-09-19T04:13:16.445449+0000","last_active":"2024-09-19T04:13:31.505138+0000","last_peered":"2024-09-19T04:13:31.505138+0000","last_clean":"2024-09-19T04:13:31.505138+0000","last_became_active":"2024-09-19T04:13:16.445028+0000","last_became_peered":"2024-09-19T04:13:16.445028+0000","last_unstale":"2024-09-19T04:13:31.505138+0000","last_undegraded":"2024-09-19T04:13:31.505138+0000","last_fullsized":"2024-09-19T04:13:31.505138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:13.095348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501714+0000","last_change":"2024-09-19T04:13:16.454033+0000","last_active":"2024-09-19T04:13:31.501714+0000","last_peered":"2024-09-19T04:13:31.501714+0000","last_clean":"2024-09-19T04:13:31.501714+0000","last_became_active":"2024-09-19T04:13:16.453592+0000","last_became_peered":"2024-09-19T04:13:16.453592+0000","last_unstale":"2024-09-19T04:13:31.501714+0000","last_undegraded":"2024-09-19T04:13:31.501714+0000","last_fullsized":"2024-09-19T04:13:31.501714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.304478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499027+0000","last_change":"2024-09-19T04:13:16.448763+0000","last_active":"2024-09-19T04:13:31.499027+0000","last_peered":"2024-09-19T04:13:31.499027+0000","last_clean":"2024-09-19T04:13:31.499027+0000","last_became_active":"2024-09-19T04:13:16.448459+0000","last_became_peered":"2024-09-19T04:13:16.448459+0000","last_unstale":"2024-09-19T04:13:31.499027+0000","last_undegraded":"2024-09-19T04:13:31.499027+0000","last_fullsized":"2024-09-19T04:13:31.499027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700507+0000","last_change":"2024-09-19T04:13:16.451489+0000","last_active":"2024-09-19T04:13:31.700507+0000","last_peered":"2024-09-19T04:13:31.700507+0000","last_clean":"2024-09-19T04:13:31.700507+0000","last_became_active":"2024-09-19T04:13:16.451176+0000","last_became_peered":"2024-09-19T04:13:16.451176+0000","last_unstale":"2024-09-19T04:13:31.700507+0000","last_undegraded":"2024-09-19T04:13:31.700507+0000","last_fullsized":"2024-09-19T04:13:31.700507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.921102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501314+0000","last_change":"2024-09-19T04:13:16.454551+0000","last_active":"2024-09-19T04:13:31.501314+0000","last_peered":"2024-09-19T04:13:31.501314+0000","last_clean":"2024-09-19T04:13:31.501314+0000","last_became_active":"2024-09-19T04:13:16.453976+0000","last_became_peered":"2024-09-19T04:13:16.453976+0000","last_unstale":"2024-09-19T04:13:31.501314+0000","last_undegraded":"2024-09-19T04:13:31.501314+0000","last_fullsized":"2024-09-19T04:13:31.501314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:06.504615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504331+0000","last_change":"2024-09-19T04:13:31.503222+0000","last_active":"2024-09-19T04:13:31.504331+0000","last_peered":"2024-09-19T04:13:31.504331+0000","last_clean":"2024-09-19T04:13:31.504331+0000","last_became_active":"2024-09-19T04:13:31.502873+0000","last_became_peered":"2024-09-19T04:13:31.502873+0000","last_unstale":"2024-09-19T04:13:31.504331+0000","last_undegraded":"2024-09-19T04:13:31.504331+0000","last_fullsized":"2024-09-19T04:13:31.504331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:54.559592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505834+0000","last_change":"2024-09-19T04:13:31.505834+0000","last_active":"2024-09-19T04:13:31.505834+0000","last_peered":"2024-09-19T04:13:31.505834+0000","last_clean":"2024-09-19T04:13:31.505834+0000","last_became_active":"2024-09-19T04:13:31.505501+0000","last_became_peered":"2024-09-19T04:13:31.505501+0000","last_unstale":"2024-09-19T04:13:31.505834+0000","last_undegraded":"2024-09-19T04:13:31.505834+0000","last_fullsized":"2024-09-19T04:13:31.505834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:45.136128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504841+0000","last_change":"2024-09-19T04:13:16.449237+0000","last_active":"2024-09-19T04:13:31.504841+0000","last_peered":"2024-09-19T04:13:31.504841+0000","last_clean":"2024-09-19T04:13:31.504841+0000","last_became_active":"2024-09-19T04:13:16.448896+0000","last_became_peered":"2024-09-19T04:13:16.448896+0000","last_unstale":"2024-09-19T04:13:31.504841+0000","last_undegraded":"2024-09-19T04:13:31.504841+0000","last_fullsized":"2024-09-19T04:13:31.504841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:38.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501835+0000","last_change":"2024-09-19T04:13:16.456053+0000","last_active":"2024-09-19T04:13:31.501835+0000","last_peered":"2024-09-19T04:13:31.501835+0000","last_clean":"2024-09-19T04:13:31.501835+0000","last_became_active":"2024-09-19T04:13:16.454775+0000","last_became_peered":"2024-09-19T04:13:16.454775+0000","last_unstale":"2024-09-19T04:13:31.501835+0000","last_undegraded":"2024-09-19T04:13:31.501835+0000","last_fullsized":"2024-09-19T04:13:31.501835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.902036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499121+0000","last_change":"2024-09-19T04:13:16.451473+0000","last_active":"2024-09-19T04:13:31.499121+0000","last_peered":"2024-09-19T04:13:31.499121+0000","last_clean":"2024-09-19T04:13:31.499121+0000","last_became_active":"2024-09-19T04:13:16.451150+0000","last_became_peered":"2024-09-19T04:13:16.451150+0000","last_unstale":"2024-09-19T04:13:31.499121+0000","last_undegraded":"2024-09-19T04:13:31.499121+0000","last_fullsized":"2024-09-19T04:13:31.499121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:59.332745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735656+0000","last_change":"2024-09-19T04:13:16.452017+0000","last_active":"2024-09-19T04:13:31.735656+0000","last_peered":"2024-09-19T04:13:31.735656+0000","last_clean":"2024-09-19T04:13:31.735656+0000","last_became_active":"2024-09-19T04:13:16.450715+0000","last_became_peered":"2024-09-19T04:13:16.450715+0000","last_unstale":"2024-09-19T04:13:31.735656+0000","last_undegraded":"2024-09-19T04:13:31.735656+0000","last_fullsized":"2024-09-19T04:13:31.735656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:43.548770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.736597+0000","last_change":"2024-09-19T04:13:31.736597+0000","last_active":"2024-09-19T04:13:31.736597+0000","last_peered":"2024-09-19T04:13:31.736597+0000","last_clean":"2024-09-19T04:13:31.736597+0000","last_became_active":"2024-09-19T04:13:31.736301+0000","last_became_peered":"2024-09-19T04:13:31.736301+0000","last_unstale":"2024-09-19T04:13:31.736597+0000","last_undegraded":"2024-09-19T04:13:31.736597+0000","last_fullsized":"2024-09-19T04:13:31.736597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:06.614117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.509742+0000","last_change":"2024-09-19T04:13:31.509742+0000","last_active":"2024-09-19T04:13:31.509742+0000","last_peered":"2024-09-19T04:13:31.509742+0000","last_clean":"2024-09-19T04:13:31.509742+0000","last_became_active":"2024-09-19T04:13:31.509363+0000","last_became_peered":"2024-09-19T04:13:31.509363+0000","last_unstale":"2024-09-19T04:13:31.509742+0000","last_undegraded":"2024-09-19T04:13:31.509742+0000","last_fullsized":"2024-09-19T04:13:31.509742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.094930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501525+0000","last_change":"2024-09-19T04:13:16.449459+0000","last_active":"2024-09-19T04:13:31.501525+0000","last_peered":"2024-09-19T04:13:31.501525+0000","last_clean":"2024-09-19T04:13:31.501525+0000","last_became_active":"2024-09-19T04:13:16.449119+0000","last_became_peered":"2024-09-19T04:13:16.449119+0000","last_unstale":"2024-09-19T04:13:31.501525+0000","last_undegraded":"2024-09-19T04:13:31.501525+0000","last_fullsized":"2024-09-19T04:13:31.501525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:26:26.941609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504930+0000","last_change":"2024-09-19T04:13:16.448939+0000","last_active":"2024-09-19T04:13:31.504930+0000","last_peered":"2024-09-19T04:13:31.504930+0000","last_clean":"2024-09-19T04:13:31.504930+0000","last_became_active":"2024-09-19T04:13:16.448619+0000","last_became_peered":"2024-09-19T04:13:16.448619+0000","last_unstale":"2024-09-19T04:13:31.504930+0000","last_undegraded":"2024-09-19T04:13:31.504930+0000","last_fullsized":"2024-09-19T04:13:31.504930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:41.017404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":84,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.570074+0000","last_change":"2024-09-19T04:13:14.430467+0000","last_active":"2024-09-19T04:14:17.570074+0000","last_peered":"2024-09-19T04:14:17.570074+0000","last_clean":"2024-09-19T04:14:17.570074+0000","last_became_active":"2024-09-19T04:13:14.430095+0000","last_became_peered":"2024-09-19T04:13:14.430095+0000","last_unstale":"2024-09-19T04:14:17.570074+0000","last_undegraded":"2024-09-19T04:14:17.570074+0000","last_fullsized":"2024-09-19T04:14:17.570074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:50.471745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506178+0000","last_change":"2024-09-19T04:13:31.505798+0000","last_active":"2024-09-19T04:13:31.506178+0000","last_peered":"2024-09-19T04:13:31.506178+0000","last_clean":"2024-09-19T04:13:31.506178+0000","last_became_active":"2024-09-19T04:13:31.505469+0000","last_became_peered":"2024-09-19T04:13:31.505469+0000","last_unstale":"2024-09-19T04:13:31.506178+0000","last_undegraded":"2024-09-19T04:13:31.506178+0000","last_fullsized":"2024-09-19T04:13:31.506178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:13:42.405075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499636+0000","last_change":"2024-09-19T04:13:14.426443+0000","last_active":"2024-09-19T04:13:31.499636+0000","last_peered":"2024-09-19T04:13:31.499636+0000","last_clean":"2024-09-19T04:13:31.499636+0000","last_became_active":"2024-09-19T04:13:14.426103+0000","last_became_peered":"2024-09-19T04:13:14.426103+0000","last_unstale":"2024-09-19T04:13:31.499636+0000","last_undegraded":"2024-09-19T04:13:31.499636+0000","last_fullsized":"2024-09-19T04:13:31.499636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:28.461962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498768+0000","last_change":"2024-09-19T04:13:14.424439+0000","last_active":"2024-09-19T04:13:31.498768+0000","last_peered":"2024-09-19T04:13:31.498768+0000","last_clean":"2024-09-19T04:13:31.498768+0000","last_became_active":"2024-09-19T04:13:14.423975+0000","last_became_peered":"2024-09-19T04:13:14.423975+0000","last_unstale":"2024-09-19T04:13:31.498768+0000","last_undegraded":"2024-09-19T04:13:31.498768+0000","last_fullsized":"2024-09-19T04:13:31.498768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:12.032125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501447+0000","last_change":"2024-09-19T04:13:16.444865+0000","last_active":"2024-09-19T04:13:31.501447+0000","last_peered":"2024-09-19T04:13:31.501447+0000","last_clean":"2024-09-19T04:13:31.501447+0000","last_became_active":"2024-09-19T04:13:16.444556+0000","last_became_peered":"2024-09-19T04:13:16.444556+0000","last_unstale":"2024-09-19T04:13:31.501447+0000","last_undegraded":"2024-09-19T04:13:31.501447+0000","last_fullsized":"2024-09-19T04:13:31.501447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:38.025604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505570+0000","last_change":"2024-09-19T04:13:14.429018+0000","last_active":"2024-09-19T04:13:31.505570+0000","last_peered":"2024-09-19T04:13:31.505570+0000","last_clean":"2024-09-19T04:13:31.505570+0000","last_became_active":"2024-09-19T04:13:14.428383+0000","last_became_peered":"2024-09-19T04:13:14.428383+0000","last_unstale":"2024-09-19T04:13:31.505570+0000","last_undegraded":"2024-09-19T04:13:31.505570+0000","last_fullsized":"2024-09-19T04:13:31.505570+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:46.190421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501335+0000","last_change":"2024-09-19T04:13:16.449013+0000","last_active":"2024-09-19T04:13:31.501335+0000","last_peered":"2024-09-19T04:13:31.501335+0000","last_clean":"2024-09-19T04:13:31.501335+0000","last_became_active":"2024-09-19T04:13:16.448703+0000","last_became_peered":"2024-09-19T04:13:16.448703+0000","last_unstale":"2024-09-19T04:13:31.501335+0000","last_undegraded":"2024-09-19T04:13:31.501335+0000","last_fullsized":"2024-09-19T04:13:31.501335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:41.231759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":77,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:16.568087+0000","last_change":"2024-09-19T04:13:14.430050+0000","last_active":"2024-09-19T04:14:16.568087+0000","last_peered":"2024-09-19T04:14:16.568087+0000","last_clean":"2024-09-19T04:14:16.568087+0000","last_became_active":"2024-09-19T04:13:14.429710+0000","last_became_peered":"2024-09-19T04:13:14.429710+0000","last_unstale":"2024-09-19T04:14:16.568087+0000","last_undegraded":"2024-09-19T04:14:16.568087+0000","last_fullsized":"2024-09-19T04:14:16.568087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:55.796080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700932+0000","last_change":"2024-09-19T04:13:16.454431+0000","last_active":"2024-09-19T04:13:31.700932+0000","last_peered":"2024-09-19T04:13:31.700932+0000","last_clean":"2024-09-19T04:13:31.700932+0000","last_became_active":"2024-09-19T04:13:16.453033+0000","last_became_peered":"2024-09-19T04:13:16.453033+0000","last_unstale":"2024-09-19T04:13:31.700932+0000","last_undegraded":"2024-09-19T04:13:31.700932+0000","last_fullsized":"2024-09-19T04:13:31.700932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:52.817740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501175+0000","last_change":"2024-09-19T04:13:14.428949+0000","last_active":"2024-09-19T04:13:31.501175+0000","last_peered":"2024-09-19T04:13:31.501175+0000","last_clean":"2024-09-19T04:13:31.501175+0000","last_became_active":"2024-09-19T04:13:14.428525+0000","last_became_peered":"2024-09-19T04:13:14.428525+0000","last_unstale":"2024-09-19T04:13:31.501175+0000","last_undegraded":"2024-09-19T04:13:31.501175+0000","last_fullsized":"2024-09-19T04:13:31.501175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:30.296730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735848+0000","last_change":"2024-09-19T04:13:16.454077+0000","last_active":"2024-09-19T04:13:31.735848+0000","last_peered":"2024-09-19T04:13:31.735848+0000","last_clean":"2024-09-19T04:13:31.735848+0000","last_became_active":"2024-09-19T04:13:16.453791+0000","last_became_peered":"2024-09-19T04:13:16.453791+0000","last_unstale":"2024-09-19T04:13:31.735848+0000","last_undegraded":"2024-09-19T04:13:31.735848+0000","last_fullsized":"2024-09-19T04:13:31.735848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:29.426475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505069+0000","last_change":"2024-09-19T04:13:14.620179+0000","last_active":"2024-09-19T04:13:31.505069+0000","last_peered":"2024-09-19T04:13:31.505069+0000","last_clean":"2024-09-19T04:13:31.505069+0000","last_became_active":"2024-09-19T04:13:14.619758+0000","last_became_peered":"2024-09-19T04:13:14.619758+0000","last_unstale":"2024-09-19T04:13:31.505069+0000","last_undegraded":"2024-09-19T04:13:31.505069+0000","last_fullsized":"2024-09-19T04:13:31.505069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:11.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700712+0000","last_change":"2024-09-19T04:13:16.453036+0000","last_active":"2024-09-19T04:13:31.700712+0000","last_peered":"2024-09-19T04:13:31.700712+0000","last_clean":"2024-09-19T04:13:31.700712+0000","last_became_active":"2024-09-19T04:13:16.452736+0000","last_became_peered":"2024-09-19T04:13:16.452736+0000","last_unstale":"2024-09-19T04:13:31.700712+0000","last_undegraded":"2024-09-19T04:13:31.700712+0000","last_fullsized":"2024-09-19T04:13:31.700712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:30.003652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'28","reported_seq":162,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.575822+0000","last_change":"2024-09-19T04:13:14.428715+0000","last_active":"2024-09-19T04:14:17.575822+0000","last_peered":"2024-09-19T04:14:17.575822+0000","last_clean":"2024-09-19T04:14:17.575822+0000","last_became_active":"2024-09-19T04:13:14.428409+0000","last_became_peered":"2024-09-19T04:13:14.428409+0000","last_unstale":"2024-09-19T04:14:17.575822+0000","last_undegraded":"2024-09-19T04:14:17.575822+0000","last_fullsized":"2024-09-19T04:14:17.575822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:23.130857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":112,"num_read_kb":114,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504291+0000","last_change":"2024-09-19T04:13:16.454512+0000","last_active":"2024-09-19T04:13:31.504291+0000","last_peered":"2024-09-19T04:13:31.504291+0000","last_clean":"2024-09-19T04:13:31.504291+0000","last_became_active":"2024-09-19T04:13:16.454193+0000","last_became_peered":"2024-09-19T04:13:16.454193+0000","last_unstale":"2024-09-19T04:13:31.504291+0000","last_undegraded":"2024-09-19T04:13:31.504291+0000","last_fullsized":"2024-09-19T04:13:31.504291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:11.825938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.515863+0000","last_change":"2024-09-19T04:13:31.515448+0000","last_active":"2024-09-19T04:13:31.515863+0000","last_peered":"2024-09-19T04:13:31.515863+0000","last_clean":"2024-09-19T04:13:31.515863+0000","last_became_active":"2024-09-19T04:13:31.506099+0000","last_became_peered":"2024-09-19T04:13:31.506099+0000","last_unstale":"2024-09-19T04:13:31.515863+0000","last_undegraded":"2024-09-19T04:13:31.515863+0000","last_fullsized":"2024-09-19T04:13:31.515863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:21.462488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498861+0000","last_change":"2024-09-19T04:13:16.447607+0000","last_active":"2024-09-19T04:13:31.498861+0000","last_peered":"2024-09-19T04:13:31.498861+0000","last_clean":"2024-09-19T04:13:31.498861+0000","last_became_active":"2024-09-19T04:13:16.447029+0000","last_became_peered":"2024-09-19T04:13:16.447029+0000","last_unstale":"2024-09-19T04:13:31.498861+0000","last_undegraded":"2024-09-19T04:13:31.498861+0000","last_fullsized":"2024-09-19T04:13:31.498861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:52.418104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505867+0000","last_change":"2024-09-19T04:13:14.428856+0000","last_active":"2024-09-19T04:13:31.505867+0000","last_peered":"2024-09-19T04:13:31.505867+0000","last_clean":"2024-09-19T04:13:31.505867+0000","last_became_active":"2024-09-19T04:13:14.428510+0000","last_became_peered":"2024-09-19T04:13:14.428510+0000","last_unstale":"2024-09-19T04:13:31.505867+0000","last_undegraded":"2024-09-19T04:13:31.505867+0000","last_fullsized":"2024-09-19T04:13:31.505867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:49.633325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501063+0000","last_change":"2024-09-19T04:13:16.455628+0000","last_active":"2024-09-19T04:13:31.501063+0000","last_peered":"2024-09-19T04:13:31.501063+0000","last_clean":"2024-09-19T04:13:31.501063+0000","last_became_active":"2024-09-19T04:13:16.453817+0000","last_became_peered":"2024-09-19T04:13:16.453817+0000","last_unstale":"2024-09-19T04:13:31.501063+0000","last_undegraded":"2024-09-19T04:13:31.501063+0000","last_fullsized":"2024-09-19T04:13:31.501063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.832806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'4","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:13.564396+0000","last_change":"2024-09-19T04:13:14.424924+0000","last_active":"2024-09-19T04:14:13.564396+0000","last_peered":"2024-09-19T04:14:13.564396+0000","last_clean":"2024-09-19T04:14:13.564396+0000","last_became_active":"2024-09-19T04:13:14.424627+0000","last_became_peered":"2024-09-19T04:13:14.424627+0000","last_unstale":"2024-09-19T04:14:13.564396+0000","last_undegraded":"2024-09-19T04:14:13.564396+0000","last_fullsized":"2024-09-19T04:14:13.564396+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:49.830205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500550+0000","last_change":"2024-09-19T04:13:16.455365+0000","last_active":"2024-09-19T04:13:31.500550+0000","last_peered":"2024-09-19T04:13:31.500550+0000","last_clean":"2024-09-19T04:13:31.500550+0000","last_became_active":"2024-09-19T04:13:16.455029+0000","last_became_peered":"2024-09-19T04:13:16.455029+0000","last_unstale":"2024-09-19T04:13:31.500550+0000","last_undegraded":"2024-09-19T04:13:31.500550+0000","last_fullsized":"2024-09-19T04:13:31.500550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:20.947391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504041+0000","last_change":"2024-09-19T04:13:14.620462+0000","last_active":"2024-09-19T04:13:31.504041+0000","last_peered":"2024-09-19T04:13:31.504041+0000","last_clean":"2024-09-19T04:13:31.504041+0000","last_became_active":"2024-09-19T04:13:14.620103+0000","last_became_peered":"2024-09-19T04:13:14.620103+0000","last_unstale":"2024-09-19T04:13:31.504041+0000","last_undegraded":"2024-09-19T04:13:31.504041+0000","last_fullsized":"2024-09-19T04:13:31.504041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:14.496075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506534+0000","last_change":"2024-09-19T04:13:16.450222+0000","last_active":"2024-09-19T04:13:31.506534+0000","last_peered":"2024-09-19T04:13:31.506534+0000","last_clean":"2024-09-19T04:13:31.506534+0000","last_became_active":"2024-09-19T04:13:16.449916+0000","last_became_peered":"2024-09-19T04:13:16.449916+0000","last_unstale":"2024-09-19T04:13:31.506534+0000","last_undegraded":"2024-09-19T04:13:31.506534+0000","last_fullsized":"2024-09-19T04:13:31.506534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:31.886342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.519612+0000","last_change":"2024-09-19T04:13:31.519190+0000","last_active":"2024-09-19T04:13:31.519612+0000","last_peered":"2024-09-19T04:13:31.519612+0000","last_clean":"2024-09-19T04:13:31.519612+0000","last_became_active":"2024-09-19T04:13:31.508605+0000","last_became_peered":"2024-09-19T04:13:31.508605+0000","last_unstale":"2024-09-19T04:13:31.519612+0000","last_undegraded":"2024-09-19T04:13:31.519612+0000","last_fullsized":"2024-09-19T04:13:31.519612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:51.437135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502356+0000","last_change":"2024-09-19T04:13:16.450848+0000","last_active":"2024-09-19T04:13:31.502356+0000","last_peered":"2024-09-19T04:13:31.502356+0000","last_clean":"2024-09-19T04:13:31.502356+0000","last_became_active":"2024-09-19T04:13:16.450496+0000","last_became_peered":"2024-09-19T04:13:16.450496+0000","last_unstale":"2024-09-19T04:13:31.502356+0000","last_undegraded":"2024-09-19T04:13:31.502356+0000","last_fullsized":"2024-09-19T04:13:31.502356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:49:00.346998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499054+0000","last_change":"2024-09-19T04:13:14.428654+0000","last_active":"2024-09-19T04:13:31.499054+0000","last_peered":"2024-09-19T04:13:31.499054+0000","last_clean":"2024-09-19T04:13:31.499054+0000","last_became_active":"2024-09-19T04:13:14.428297+0000","last_became_peered":"2024-09-19T04:13:14.428297+0000","last_unstale":"2024-09-19T04:13:31.499054+0000","last_undegraded":"2024-09-19T04:13:31.499054+0000","last_fullsized":"2024-09-19T04:13:31.499054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:30.834262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501214+0000","last_change":"2024-09-19T04:13:16.456002+0000","last_active":"2024-09-19T04:13:31.501214+0000","last_peered":"2024-09-19T04:13:31.501214+0000","last_clean":"2024-09-19T04:13:31.501214+0000","last_became_active":"2024-09-19T04:13:16.455463+0000","last_became_peered":"2024-09-19T04:13:16.455463+0000","last_unstale":"2024-09-19T04:13:31.501214+0000","last_undegraded":"2024-09-19T04:13:31.501214+0000","last_fullsized":"2024-09-19T04:13:31.501214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:47.266364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505707+0000","last_change":"2024-09-19T04:13:14.429555+0000","last_active":"2024-09-19T04:13:31.505707+0000","last_peered":"2024-09-19T04:13:31.505707+0000","last_clean":"2024-09-19T04:13:31.505707+0000","last_became_active":"2024-09-19T04:13:14.429224+0000","last_became_peered":"2024-09-19T04:13:14.429224+0000","last_unstale":"2024-09-19T04:13:31.505707+0000","last_undegraded":"2024-09-19T04:13:31.505707+0000","last_fullsized":"2024-09-19T04:13:31.505707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:35.304750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501182+0000","last_change":"2024-09-19T04:13:16.454337+0000","last_active":"2024-09-19T04:13:31.501182+0000","last_peered":"2024-09-19T04:13:31.501182+0000","last_clean":"2024-09-19T04:13:31.501182+0000","last_became_active":"2024-09-19T04:13:16.453831+0000","last_became_peered":"2024-09-19T04:13:16.453831+0000","last_unstale":"2024-09-19T04:13:31.501182+0000","last_undegraded":"2024-09-19T04:13:31.501182+0000","last_fullsized":"2024-09-19T04:13:31.501182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:41.751933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506399+0000","last_change":"2024-09-19T04:13:14.620406+0000","last_active":"2024-09-19T04:13:31.506399+0000","last_peered":"2024-09-19T04:13:31.506399+0000","last_clean":"2024-09-19T04:13:31.506399+0000","last_became_active":"2024-09-19T04:13:14.620050+0000","last_became_peered":"2024-09-19T04:13:14.620050+0000","last_unstale":"2024-09-19T04:13:31.506399+0000","last_undegraded":"2024-09-19T04:13:31.506399+0000","last_fullsized":"2024-09-19T04:13:31.506399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:35.828760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501221+0000","last_change":"2024-09-19T04:13:16.459016+0000","last_active":"2024-09-19T04:13:31.501221+0000","last_peered":"2024-09-19T04:13:31.501221+0000","last_clean":"2024-09-19T04:13:31.501221+0000","last_became_active":"2024-09-19T04:13:16.458674+0000","last_became_peered":"2024-09-19T04:13:16.458674+0000","last_unstale":"2024-09-19T04:13:31.501221+0000","last_undegraded":"2024-09-19T04:13:31.501221+0000","last_fullsized":"2024-09-19T04:13:31.501221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:55.894456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":346,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500892+0000","last_change":"2024-09-19T04:12:05.913704+0000","last_active":"2024-09-19T04:13:31.500892+0000","last_peered":"2024-09-19T04:13:31.500892+0000","last_clean":"2024-09-19T04:13:31.500892+0000","last_became_active":"2024-09-19T04:12:05.913299+0000","last_became_peered":"2024-09-19T04:12:05.913299+0000","last_unstale":"2024-09-19T04:13:31.500892+0000","last_undegraded":"2024-09-19T04:13:31.500892+0000","last_fullsized":"2024-09-19T04:13:31.500892+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_clean_scrub_stamp":"2024-09-19T04:08:46.646000+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.638160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504518+0000","last_change":"2024-09-19T04:13:14.423962+0000","last_active":"2024-09-19T04:13:31.504518+0000","last_peered":"2024-09-19T04:13:31.504518+0000","last_clean":"2024-09-19T04:13:31.504518+0000","last_became_active":"2024-09-19T04:13:14.423591+0000","last_became_peered":"2024-09-19T04:13:14.423591+0000","last_unstale":"2024-09-19T04:13:31.504518+0000","last_undegraded":"2024-09-19T04:13:31.504518+0000","last_fullsized":"2024-09-19T04:13:31.504518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:02.239975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501743+0000","last_change":"2024-09-19T04:13:16.447113+0000","last_active":"2024-09-19T04:13:31.501743+0000","last_peered":"2024-09-19T04:13:31.501743+0000","last_clean":"2024-09-19T04:13:31.501743+0000","last_became_active":"2024-09-19T04:13:16.446792+0000","last_became_peered":"2024-09-19T04:13:16.446792+0000","last_unstale":"2024-09-19T04:13:31.501743+0000","last_undegraded":"2024-09-19T04:13:31.501743+0000","last_fullsized":"2024-09-19T04:13:31.501743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:28:21.527010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500749+0000","last_change":"2024-09-19T04:13:14.432663+0000","last_active":"2024-09-19T04:13:31.500749+0000","last_peered":"2024-09-19T04:13:31.500749+0000","last_clean":"2024-09-19T04:13:31.500749+0000","last_became_active":"2024-09-19T04:13:14.432318+0000","last_became_peered":"2024-09-19T04:13:14.432318+0000","last_unstale":"2024-09-19T04:13:31.500749+0000","last_undegraded":"2024-09-19T04:13:31.500749+0000","last_fullsized":"2024-09-19T04:13:31.500749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:35.787596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.498761+0000","last_change":"2024-09-19T04:13:16.451010+0000","last_active":"2024-09-19T04:13:31.498761+0000","last_peered":"2024-09-19T04:13:31.498761+0000","last_clean":"2024-09-19T04:13:31.498761+0000","last_became_active":"2024-09-19T04:13:16.450708+0000","last_became_peered":"2024-09-19T04:13:16.450708+0000","last_unstale":"2024-09-19T04:13:31.498761+0000","last_undegraded":"2024-09-19T04:13:31.498761+0000","last_fullsized":"2024-09-19T04:13:31.498761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:30.108541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.569345+0000","last_change":"2024-09-19T04:13:14.620037+0000","last_active":"2024-09-19T04:14:17.569345+0000","last_peered":"2024-09-19T04:14:17.569345+0000","last_clean":"2024-09-19T04:14:17.569345+0000","last_became_active":"2024-09-19T04:13:14.619686+0000","last_became_peered":"2024-09-19T04:13:14.619686+0000","last_unstale":"2024-09-19T04:14:17.569345+0000","last_undegraded":"2024-09-19T04:14:17.569345+0000","last_fullsized":"2024-09-19T04:14:17.569345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:32.490708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504838+0000","last_change":"2024-09-19T04:13:16.446429+0000","last_active":"2024-09-19T04:13:31.504838+0000","last_peered":"2024-09-19T04:13:31.504838+0000","last_clean":"2024-09-19T04:13:31.504838+0000","last_became_active":"2024-09-19T04:13:16.446075+0000","last_became_peered":"2024-09-19T04:13:16.446075+0000","last_unstale":"2024-09-19T04:13:31.504838+0000","last_undegraded":"2024-09-19T04:13:31.504838+0000","last_fullsized":"2024-09-19T04:13:31.504838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:03.613926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501031+0000","last_change":"2024-09-19T04:13:14.429899+0000","last_active":"2024-09-19T04:13:31.501031+0000","last_peered":"2024-09-19T04:13:31.501031+0000","last_clean":"2024-09-19T04:13:31.501031+0000","last_became_active":"2024-09-19T04:13:14.429571+0000","last_became_peered":"2024-09-19T04:13:14.429571+0000","last_unstale":"2024-09-19T04:13:31.501031+0000","last_undegraded":"2024-09-19T04:13:31.501031+0000","last_fullsized":"2024-09-19T04:13:31.501031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:37.017431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499733+0000","last_change":"2024-09-19T04:13:16.458267+0000","last_active":"2024-09-19T04:13:31.499733+0000","last_peered":"2024-09-19T04:13:31.499733+0000","last_clean":"2024-09-19T04:13:31.499733+0000","last_became_active":"2024-09-19T04:13:16.457911+0000","last_became_peered":"2024-09-19T04:13:16.457911+0000","last_unstale":"2024-09-19T04:13:31.499733+0000","last_undegraded":"2024-09-19T04:13:31.499733+0000","last_fullsized":"2024-09-19T04:13:31.499733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:32.465923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501677+0000","last_change":"2024-09-19T04:13:14.621552+0000","last_active":"2024-09-19T04:13:31.501677+0000","last_peered":"2024-09-19T04:13:31.501677+0000","last_clean":"2024-09-19T04:13:31.501677+0000","last_became_active":"2024-09-19T04:13:14.620592+0000","last_became_peered":"2024-09-19T04:13:14.620592+0000","last_unstale":"2024-09-19T04:13:31.501677+0000","last_undegraded":"2024-09-19T04:13:31.501677+0000","last_fullsized":"2024-09-19T04:13:31.501677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:10.055202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500150+0000","last_change":"2024-09-19T04:13:16.443117+0000","last_active":"2024-09-19T04:13:31.500150+0000","last_peered":"2024-09-19T04:13:31.500150+0000","last_clean":"2024-09-19T04:13:31.500150+0000","last_became_active":"2024-09-19T04:13:16.442808+0000","last_became_peered":"2024-09-19T04:13:16.442808+0000","last_unstale":"2024-09-19T04:13:31.500150+0000","last_undegraded":"2024-09-19T04:13:31.500150+0000","last_fullsized":"2024-09-19T04:13:31.500150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:40.274320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504531+0000","last_change":"2024-09-19T04:13:14.620618+0000","last_active":"2024-09-19T04:13:31.504531+0000","last_peered":"2024-09-19T04:13:31.504531+0000","last_clean":"2024-09-19T04:13:31.504531+0000","last_became_active":"2024-09-19T04:13:14.620290+0000","last_became_peered":"2024-09-19T04:13:14.620290+0000","last_unstale":"2024-09-19T04:13:31.504531+0000","last_undegraded":"2024-09-19T04:13:31.504531+0000","last_fullsized":"2024-09-19T04:13:31.504531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:51.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.499218+0000","last_change":"2024-09-19T04:13:16.443498+0000","last_active":"2024-09-19T04:13:31.499218+0000","last_peered":"2024-09-19T04:13:31.499218+0000","last_clean":"2024-09-19T04:13:31.499218+0000","last_became_active":"2024-09-19T04:13:16.443104+0000","last_became_peered":"2024-09-19T04:13:16.443104+0000","last_unstale":"2024-09-19T04:13:31.499218+0000","last_undegraded":"2024-09-19T04:13:31.499218+0000","last_fullsized":"2024-09-19T04:13:31.499218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:10.733570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:40.455994+0000","last_change":"2024-09-19T04:13:14.425881+0000","last_active":"2024-09-19T04:13:40.455994+0000","last_peered":"2024-09-19T04:13:40.455994+0000","last_clean":"2024-09-19T04:13:40.455994+0000","last_became_active":"2024-09-19T04:13:14.425542+0000","last_became_peered":"2024-09-19T04:13:14.425542+0000","last_unstale":"2024-09-19T04:13:40.455994+0000","last_undegraded":"2024-09-19T04:13:40.455994+0000","last_fullsized":"2024-09-19T04:13:40.455994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:44.250890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.502301+0000","last_change":"2024-09-19T04:13:16.454672+0000","last_active":"2024-09-19T04:13:31.502301+0000","last_peered":"2024-09-19T04:13:31.502301+0000","last_clean":"2024-09-19T04:13:31.502301+0000","last_became_active":"2024-09-19T04:13:16.454003+0000","last_became_peered":"2024-09-19T04:13:16.454003+0000","last_unstale":"2024-09-19T04:13:31.502301+0000","last_undegraded":"2024-09-19T04:13:31.502301+0000","last_fullsized":"2024-09-19T04:13:31.502301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:26.849617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504465+0000","last_change":"2024-09-19T04:13:14.432312+0000","last_active":"2024-09-19T04:13:31.504465+0000","last_peered":"2024-09-19T04:13:31.504465+0000","last_clean":"2024-09-19T04:13:31.504465+0000","last_became_active":"2024-09-19T04:13:14.431994+0000","last_became_peered":"2024-09-19T04:13:14.431994+0000","last_unstale":"2024-09-19T04:13:31.504465+0000","last_undegraded":"2024-09-19T04:13:31.504465+0000","last_fullsized":"2024-09-19T04:13:31.504465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:00.410400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735768+0000","last_change":"2024-09-19T04:13:16.455312+0000","last_active":"2024-09-19T04:13:31.735768+0000","last_peered":"2024-09-19T04:13:31.735768+0000","last_clean":"2024-09-19T04:13:31.735768+0000","last_became_active":"2024-09-19T04:13:16.454990+0000","last_became_peered":"2024-09-19T04:13:16.454990+0000","last_unstale":"2024-09-19T04:13:31.735768+0000","last_undegraded":"2024-09-19T04:13:31.735768+0000","last_fullsized":"2024-09-19T04:13:31.735768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:09.207395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700505+0000","last_change":"2024-09-19T04:13:14.620316+0000","last_active":"2024-09-19T04:13:31.700505+0000","last_peered":"2024-09-19T04:13:31.700505+0000","last_clean":"2024-09-19T04:13:31.700505+0000","last_became_active":"2024-09-19T04:13:14.619979+0000","last_became_peered":"2024-09-19T04:13:14.619979+0000","last_unstale":"2024-09-19T04:13:31.700505+0000","last_undegraded":"2024-09-19T04:13:31.700505+0000","last_fullsized":"2024-09-19T04:13:31.700505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:24.382144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505808+0000","last_change":"2024-09-19T04:13:16.455684+0000","last_active":"2024-09-19T04:13:31.505808+0000","last_peered":"2024-09-19T04:13:31.505808+0000","last_clean":"2024-09-19T04:13:31.505808+0000","last_became_active":"2024-09-19T04:13:16.455355+0000","last_became_peered":"2024-09-19T04:13:16.455355+0000","last_unstale":"2024-09-19T04:13:31.505808+0000","last_undegraded":"2024-09-19T04:13:31.505808+0000","last_fullsized":"2024-09-19T04:13:31.505808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:09.657447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501550+0000","last_change":"2024-09-19T04:13:14.423599+0000","last_active":"2024-09-19T04:13:31.501550+0000","last_peered":"2024-09-19T04:13:31.501550+0000","last_clean":"2024-09-19T04:13:31.501550+0000","last_became_active":"2024-09-19T04:13:14.423282+0000","last_became_peered":"2024-09-19T04:13:14.423282+0000","last_unstale":"2024-09-19T04:13:31.501550+0000","last_undegraded":"2024-09-19T04:13:31.501550+0000","last_fullsized":"2024-09-19T04:13:31.501550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.975368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501564+0000","last_change":"2024-09-19T04:13:16.450435+0000","last_active":"2024-09-19T04:13:31.501564+0000","last_peered":"2024-09-19T04:13:31.501564+0000","last_clean":"2024-09-19T04:13:31.501564+0000","last_became_active":"2024-09-19T04:13:16.450087+0000","last_became_peered":"2024-09-19T04:13:16.450087+0000","last_unstale":"2024-09-19T04:13:31.501564+0000","last_undegraded":"2024-09-19T04:13:31.501564+0000","last_fullsized":"2024-09-19T04:13:31.501564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:34:51.764776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504466+0000","last_change":"2024-09-19T04:13:14.432868+0000","last_active":"2024-09-19T04:13:31.504466+0000","last_peered":"2024-09-19T04:13:31.504466+0000","last_clean":"2024-09-19T04:13:31.504466+0000","last_became_active":"2024-09-19T04:13:14.432507+0000","last_became_peered":"2024-09-19T04:13:14.432507+0000","last_unstale":"2024-09-19T04:13:31.504466+0000","last_undegraded":"2024-09-19T04:13:31.504466+0000","last_fullsized":"2024-09-19T04:13:31.504466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:14.773711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501610+0000","last_change":"2024-09-19T04:13:16.450061+0000","last_active":"2024-09-19T04:13:31.501610+0000","last_peered":"2024-09-19T04:13:31.501610+0000","last_clean":"2024-09-19T04:13:31.501610+0000","last_became_active":"2024-09-19T04:13:16.448829+0000","last_became_peered":"2024-09-19T04:13:16.448829+0000","last_unstale":"2024-09-19T04:13:31.501610+0000","last_undegraded":"2024-09-19T04:13:31.501610+0000","last_fullsized":"2024-09-19T04:13:31.501610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:31.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.506029+0000","last_change":"2024-09-19T04:13:14.431574+0000","last_active":"2024-09-19T04:13:31.506029+0000","last_peered":"2024-09-19T04:13:31.506029+0000","last_clean":"2024-09-19T04:13:31.506029+0000","last_became_active":"2024-09-19T04:13:14.431202+0000","last_became_peered":"2024-09-19T04:13:14.431202+0000","last_unstale":"2024-09-19T04:13:31.506029+0000","last_undegraded":"2024-09-19T04:13:31.506029+0000","last_fullsized":"2024-09-19T04:13:31.506029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:58.167201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.500887+0000","last_change":"2024-09-19T04:13:16.452688+0000","last_active":"2024-09-19T04:13:31.500887+0000","last_peered":"2024-09-19T04:13:31.500887+0000","last_clean":"2024-09-19T04:13:31.500887+0000","last_became_active":"2024-09-19T04:13:16.452342+0000","last_became_peered":"2024-09-19T04:13:16.452342+0000","last_unstale":"2024-09-19T04:13:31.500887+0000","last_undegraded":"2024-09-19T04:13:31.500887+0000","last_fullsized":"2024-09-19T04:13:31.500887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:03.162688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504711+0000","last_change":"2024-09-19T04:13:16.449402+0000","last_active":"2024-09-19T04:13:31.504711+0000","last_peered":"2024-09-19T04:13:31.504711+0000","last_clean":"2024-09-19T04:13:31.504711+0000","last_became_active":"2024-09-19T04:13:16.449108+0000","last_became_peered":"2024-09-19T04:13:16.449108+0000","last_unstale":"2024-09-19T04:13:31.504711+0000","last_undegraded":"2024-09-19T04:13:31.504711+0000","last_fullsized":"2024-09-19T04:13:31.504711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:58.874638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.505684+0000","last_change":"2024-09-19T04:13:14.430684+0000","last_active":"2024-09-19T04:13:31.505684+0000","last_peered":"2024-09-19T04:13:31.505684+0000","last_clean":"2024-09-19T04:13:31.505684+0000","last_became_active":"2024-09-19T04:13:14.430137+0000","last_became_peered":"2024-09-19T04:13:14.430137+0000","last_unstale":"2024-09-19T04:13:31.505684+0000","last_undegraded":"2024-09-19T04:13:31.505684+0000","last_fullsized":"2024-09-19T04:13:31.505684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:39.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.700648+0000","last_change":"2024-09-19T04:13:16.452491+0000","last_active":"2024-09-19T04:13:31.700648+0000","last_peered":"2024-09-19T04:13:31.700648+0000","last_clean":"2024-09-19T04:13:31.700648+0000","last_became_active":"2024-09-19T04:13:16.452204+0000","last_became_peered":"2024-09-19T04:13:16.452204+0000","last_unstale":"2024-09-19T04:13:31.700648+0000","last_undegraded":"2024-09-19T04:13:31.700648+0000","last_fullsized":"2024-09-19T04:13:31.700648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:10.113510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.504401+0000","last_change":"2024-09-19T04:13:14.431608+0000","last_active":"2024-09-19T04:13:31.504401+0000","last_peered":"2024-09-19T04:13:31.504401+0000","last_clean":"2024-09-19T04:13:31.504401+0000","last_became_active":"2024-09-19T04:13:14.431111+0000","last_became_peered":"2024-09-19T04:13:14.431111+0000","last_unstale":"2024-09-19T04:13:31.504401+0000","last_undegraded":"2024-09-19T04:13:31.504401+0000","last_fullsized":"2024-09-19T04:13:31.504401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:03:41.125315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.735802+0000","last_change":"2024-09-19T04:13:16.457867+0000","last_active":"2024-09-19T04:13:31.735802+0000","last_peered":"2024-09-19T04:13:31.735802+0000","last_clean":"2024-09-19T04:13:31.735802+0000","last_became_active":"2024-09-19T04:13:16.457582+0000","last_became_peered":"2024-09-19T04:13:16.457582+0000","last_unstale":"2024-09-19T04:13:31.735802+0000","last_undegraded":"2024-09-19T04:13:31.735802+0000","last_fullsized":"2024-09-19T04:13:31.735802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:58.076250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501886+0000","last_change":"2024-09-19T04:13:14.620070+0000","last_active":"2024-09-19T04:13:31.501886+0000","last_peered":"2024-09-19T04:13:31.501886+0000","last_clean":"2024-09-19T04:13:31.501886+0000","last_became_active":"2024-09-19T04:13:14.619734+0000","last_became_peered":"2024-09-19T04:13:14.619734+0000","last_unstale":"2024-09-19T04:13:31.501886+0000","last_undegraded":"2024-09-19T04:13:31.501886+0000","last_fullsized":"2024-09-19T04:13:31.501886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:44:02.818859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501786+0000","last_change":"2024-09-19T04:13:16.443832+0000","last_active":"2024-09-19T04:13:31.501786+0000","last_peered":"2024-09-19T04:13:31.501786+0000","last_clean":"2024-09-19T04:13:31.501786+0000","last_became_active":"2024-09-19T04:13:16.442690+0000","last_became_peered":"2024-09-19T04:13:16.442690+0000","last_unstale":"2024-09-19T04:13:31.501786+0000","last_undegraded":"2024-09-19T04:13:31.501786+0000","last_fullsized":"2024-09-19T04:13:31.501786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:39.535628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.501657+0000","last_change":"2024-09-19T04:13:14.427186+0000","last_active":"2024-09-19T04:13:31.501657+0000","last_peered":"2024-09-19T04:13:31.501657+0000","last_clean":"2024-09-19T04:13:31.501657+0000","last_became_active":"2024-09-19T04:13:14.426949+0000","last_became_peered":"2024-09-19T04:13:14.426949+0000","last_unstale":"2024-09-19T04:13:31.501657+0000","last_undegraded":"2024-09-19T04:13:31.501657+0000","last_fullsized":"2024-09-19T04:13:31.501657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:26.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":338,"num_read_kb":238,"num_write":162,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":431409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841567,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1082991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005092,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40388,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704764,"statfs":{"total":95995035648,"available":95953678336,"internally_reserved":0,"allocated":905216,"data_stored":431589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234025,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":431487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1023834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593779,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":892928,"data_stored":430978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1511424,"data_stored":1023125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":431808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":431289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476806,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":492244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":490535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":913408,"data_stored":432249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:14:22.463 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T04:14:22.463 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T04:14:22.463 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T04:14:22.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T04:14:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:22.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:22 smithi112 ceph-mon[35051]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:22.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:22.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:22.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:22 smithi104 ceph-mon[34145]: from='mgr.14150 172.21.15.94:0/352689071' entity='mgr.x' 2024-09-19T04:14:22.708 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.161+0000 7eff5e132640 1 -- 172.21.15.94:0/927421915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102ef0 msgr2=0x7eff5810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.161+0000 7eff5e132640 1 --2- 172.21.15.94:0/927421915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102ef0 0x7eff5810da40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7eff50009ca0 tx=0x7eff5002a950 comp rx=0 tx=0).stop 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.161+0000 7eff5e132640 1 -- 172.21.15.94:0/927421915 shutdown_connections 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.161+0000 7eff5e132640 1 --2- 172.21.15.94:0/927421915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102ef0 0x7eff5810da40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.161+0000 7eff5e132640 1 --2- 172.21.15.94:0/927421915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff58102570 0x7eff581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.161+0000 7eff5e132640 1 --2- 172.21.15.94:0/927421915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff58100e20 0x7eff58101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.161+0000 7eff5e132640 1 -- 172.21.15.94:0/927421915 >> 172.21.15.94:0/927421915 conn(0x7eff580fc7d0 msgr2=0x7eff580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.161+0000 7eff5e132640 1 -- 172.21.15.94:0/927421915 shutdown_connections 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.161+0000 7eff5e132640 1 -- 172.21.15.94:0/927421915 wait complete. 2024-09-19T04:14:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff5e132640 1 Processor -- start 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff5e132640 1 -- start start 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff5e132640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff58100e20 0x7eff581983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff5e132640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102570 0x7eff58198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff5e132640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff58102ef0 0x7eff5819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff5e132640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff58110e50 con 0x7eff58102570 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff5e132640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff58110cd0 con 0x7eff58100e20 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff5e132640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff58110fd0 con 0x7eff58102ef0 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102570 0x7eff58198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102570 0x7eff58198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37784/0 (socket says 172.21.15.94:37784) 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.162+0000 7eff56ffd640 1 -- 172.21.15.94:0/249716075 learned_addr learned my addr 172.21.15.94:0/249716075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff577fe640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff58100e20 0x7eff581983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff57fff640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff58102ef0 0x7eff5819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff56ffd640 1 -- 172.21.15.94:0/249716075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff58102ef0 msgr2=0x7eff5819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff56ffd640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff58102ef0 0x7eff5819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff56ffd640 1 -- 172.21.15.94:0/249716075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff58100e20 msgr2=0x7eff581983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff56ffd640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff58100e20 0x7eff581983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff56ffd640 1 -- 172.21.15.94:0/249716075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5819d3a0 con 0x7eff58102570 2024-09-19T04:14:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff56ffd640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102570 0x7eff58198930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b4a0 tx=0x7eff4c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff54ff9640 1 -- 172.21.15.94:0/249716075 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c014070 con 0x7eff58102570 2024-09-19T04:14:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5819d690 con 0x7eff58102570 2024-09-19T04:14:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff54ff9640 1 -- 172.21.15.94:0/249716075 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff4c010ac0 con 0x7eff58102570 2024-09-19T04:14:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff54ff9640 1 -- 172.21.15.94:0/249716075 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c017c30 con 0x7eff58102570 2024-09-19T04:14:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.163+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff581a4f70 con 0x7eff58102570 2024-09-19T04:14:23.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.164+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff58101200 con 0x7eff58102570 2024-09-19T04:14:23.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.165+0000 7eff54ff9640 1 -- 172.21.15.94:0/249716075 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7eff4c017070 con 0x7eff58102570 2024-09-19T04:14:23.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.166+0000 7eff54ff9640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff400a32d0 0x7eff400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:23.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.166+0000 7eff577fe640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff400a32d0 0x7eff400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:23.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.166+0000 7eff54ff9640 1 -- 172.21.15.94:0/249716075 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7eff4c0cf140 con 0x7eff58102570 2024-09-19T04:14:23.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.166+0000 7eff577fe640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff400a32d0 0x7eff400a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff480058c0 tx=0x7eff4800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:23.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.169+0000 7eff54ff9640 1 -- 172.21.15.94:0/249716075 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7eff4c09be10 con 0x7eff58102570 2024-09-19T04:14:23.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:23 smithi094 ceph-mon[30972]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:14:23.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:23 smithi094 ceph-mon[30972]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:23.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.375+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7eff5810a0d0 con 0x7eff58102570 2024-09-19T04:14:23.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.376+0000 7eff54ff9640 1 -- 172.21.15.94:0/249716075 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7eff4c09b710 con 0x7eff58102570 2024-09-19T04:14:23.377 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:23.377 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T04:14:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff400a32d0 msgr2=0x7eff400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff400a32d0 0x7eff400a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff480058c0 tx=0x7eff4800a2d0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102570 msgr2=0x7eff58198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102570 0x7eff58198930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b4a0 tx=0x7eff4c00b960 comp rx=0 tx=0).stop 2024-09-19T04:14:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 shutdown_connections 2024-09-19T04:14:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff400a32d0 0x7eff400a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff58102ef0 0x7eff5819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff58102570 0x7eff58198930 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 --2- 172.21.15.94:0/249716075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff58100e20 0x7eff581983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 >> 172.21.15.94:0/249716075 conn(0x7eff580fc7d0 msgr2=0x7eff5806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 shutdown_connections 2024-09-19T04:14:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:23.378+0000 7eff5e132640 1 -- 172.21.15.94:0/249716075 wait complete. 2024-09-19T04:14:23.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:23 smithi112 ceph-mon[35051]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:14:23.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:23 smithi112 ceph-mon[35051]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:23 smithi104 ceph-mon[34145]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:14:23.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:23 smithi104 ceph-mon[34145]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:24.058 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T04:14:24.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:14:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:24.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:24 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/249716075' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:14:24.556 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:24 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/249716075' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:14:24.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:24 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/249716075' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:14:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.795+0000 7eff6a453640 1 -- 172.21.15.94:0/378272646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64102da0 msgr2=0x7eff64103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.795+0000 7eff6a453640 1 --2- 172.21.15.94:0/378272646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64102da0 0x7eff64103180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff54009900 tx=0x7eff5402a480 comp rx=0 tx=0).stop 2024-09-19T04:14:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.795+0000 7eff6a453640 1 -- 172.21.15.94:0/378272646 shutdown_connections 2024-09-19T04:14:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.795+0000 7eff6a453640 1 --2- 172.21.15.94:0/378272646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff6410c180 0x7eff6410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.795+0000 7eff6a453640 1 --2- 172.21.15.94:0/378272646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff641036c0 0x7eff6410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.795+0000 7eff6a453640 1 --2- 172.21.15.94:0/378272646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64102da0 0x7eff64103180 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.795+0000 7eff6a453640 1 -- 172.21.15.94:0/378272646 >> 172.21.15.94:0/378272646 conn(0x7eff640fc7a0 msgr2=0x7eff640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.795+0000 7eff6a453640 1 -- 172.21.15.94:0/378272646 shutdown_connections 2024-09-19T04:14:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.795+0000 7eff6a453640 1 -- 172.21.15.94:0/378272646 wait complete. 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff6a453640 1 Processor -- start 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff6a453640 1 -- start start 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff6a453640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff64102da0 0x7eff6419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff6a453640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff641036c0 0x7eff6419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff6a453640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff6410c180 0x7eff6419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff6a453640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111e10 con 0x7eff6410c180 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff6a453640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111c90 con 0x7eff641036c0 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff6a453640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111f90 con 0x7eff64102da0 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff689f0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff6410c180 0x7eff6419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff689f0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff6410c180 0x7eff6419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37798/0 (socket says 172.21.15.94:37798) 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff689f0640 1 -- 172.21.15.94:0/2677352064 learned_addr learned my addr 172.21.15.94:0/2677352064 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff637fe640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff641036c0 0x7eff6419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:24.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.796+0000 7eff63fff640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff64102da0 0x7eff6419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:24.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff689f0640 1 -- 172.21.15.94:0/2677352064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff64102da0 msgr2=0x7eff6419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:24.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff689f0640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff64102da0 0x7eff6419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:24.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff689f0640 1 -- 172.21.15.94:0/2677352064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff641036c0 msgr2=0x7eff6419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:24.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff689f0640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff641036c0 0x7eff6419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:24.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff689f0640 1 -- 172.21.15.94:0/2677352064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6419f490 con 0x7eff6410c180 2024-09-19T04:14:24.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff689f0640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff6410c180 0x7eff6419edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00cad0 tx=0x7eff5c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:24.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff617fa640 1 -- 172.21.15.94:0/2677352064 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00ad60 con 0x7eff6410c180 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff617fa640 1 -- 172.21.15.94:0/2677352064 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff5c005020 con 0x7eff6410c180 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6419f780 con 0x7eff6410c180 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff617fa640 1 -- 172.21.15.94:0/2677352064 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00f6b0 con 0x7eff6410c180 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.797+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff641a7060 con 0x7eff6410c180 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.798+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff64103180 con 0x7eff6410c180 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.799+0000 7eff617fa640 1 -- 172.21.15.94:0/2677352064 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7eff5c0051d0 con 0x7eff6410c180 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.800+0000 7eff617fa640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff4c0a3350 0x7eff4c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.800+0000 7eff63fff640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff4c0a3350 0x7eff4c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.800+0000 7eff617fa640 1 -- 172.21.15.94:0/2677352064 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7eff5c0cf980 con 0x7eff6410c180 2024-09-19T04:14:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.800+0000 7eff63fff640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff4c0a3350 0x7eff4c0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff54031440 tx=0x7eff54005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:24.804+0000 7eff617fa640 1 -- 172.21.15.94:0/2677352064 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7eff5c014030 con 0x7eff6410c180 2024-09-19T04:14:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.005+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff641a72f0 con 0x7eff6410c180 2024-09-19T04:14:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.006+0000 7eff617fa640 1 -- 172.21.15.94:0/2677352064 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11915 (secure 0 0 0) 0x7eff5c09c540 con 0x7eff6410c180 2024-09-19T04:14:25.008 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:25.008 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"btime":"2024-09-19T04:13:28:480286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:13:28.166880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:14:25.008 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-09-19T04:14:25.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff4c0a3350 msgr2=0x7eff4c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:25.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff4c0a3350 0x7eff4c0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff54031440 tx=0x7eff54005e80 comp rx=0 tx=0).stop 2024-09-19T04:14:25.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff6410c180 msgr2=0x7eff6419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff6410c180 0x7eff6419edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00cad0 tx=0x7eff5c00ced0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 shutdown_connections 2024-09-19T04:14:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7eff4c0a3350 0x7eff4c0a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff6410c180 0x7eff6419edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff641036c0 0x7eff6419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 --2- 172.21.15.94:0/2677352064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff64102da0 0x7eff6419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 >> 172.21.15.94:0/2677352064 conn(0x7eff640fc7a0 msgr2=0x7eff64101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 shutdown_connections 2024-09-19T04:14:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:25.008+0000 7eff6a453640 1 -- 172.21.15.94:0/2677352064 wait complete. 2024-09-19T04:14:25.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:25 smithi112 ceph-mon[35051]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:25 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2677352064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:25.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:25 smithi094 ceph-mon[30972]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:25.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:25 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2677352064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:25 smithi104 ceph-mon[34145]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:25.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:25 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2677352064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:25.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:14:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.478+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/3938918109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 msgr2=0x7f2d9810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.478+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/3938918109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 0x7f2d9810d950 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009cd0 tx=0x7f2d9002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.478+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/3938918109 shutdown_connections 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.478+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/3938918109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 0x7f2d9810d950 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.478+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/3938918109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d98100a80 0x7f2d98100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.478+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/3938918109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d98106a80 0x7f2d98106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.478+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/3938918109 >> 172.21.15.94:0/3938918109 conn(0x7f2d980fc7a0 msgr2=0x7f2d980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.478+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/3938918109 shutdown_connections 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.478+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/3938918109 wait complete. 2024-09-19T04:14:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9fbfe640 1 Processor -- start 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9fbfe640 1 -- start start 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9fbfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d98100a80 0x7f2d98198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9fbfe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 0x7f2d981988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9fbfe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d98106a80 0x7f2d9819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9fbfe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98110d50 con 0x7f2d98101420 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9fbfe640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98110bd0 con 0x7f2d98106a80 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9fbfe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98110ed0 con 0x7f2d98100a80 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9d199640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 0x7f2d981988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9d199640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 0x7f2d981988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37816/0 (socket says 172.21.15.94:37816) 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9d199640 1 -- 172.21.15.94:0/4147840600 learned_addr learned my addr 172.21.15.94:0/4147840600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9e19b640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d98106a80 0x7f2d9819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9d99a640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d98100a80 0x7f2d98198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9d199640 1 -- 172.21.15.94:0/4147840600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d98100a80 msgr2=0x7f2d98198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9d199640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d98100a80 0x7f2d98198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9d199640 1 -- 172.21.15.94:0/4147840600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d98106a80 msgr2=0x7f2d9819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9d199640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d98106a80 0x7f2d9819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.479+0000 7f2d9d199640 1 -- 172.21.15.94:0/4147840600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9819d320 con 0x7f2d98101420 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.480+0000 7f2d9d199640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 0x7f2d981988b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800b4a0 tx=0x7f2d8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.480+0000 7f2d8effd640 1 -- 172.21.15.94:0/4147840600 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88014070 con 0x7f2d98101420 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.480+0000 7f2d8effd640 1 -- 172.21.15.94:0/4147840600 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d88010ac0 con 0x7f2d98101420 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.480+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9819d610 con 0x7f2d98101420 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.480+0000 7f2d8effd640 1 -- 172.21.15.94:0/4147840600 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88017c30 con 0x7f2d98101420 2024-09-19T04:14:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.480+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a4ef0 con 0x7f2d98101420 2024-09-19T04:14:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.481+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d98100ee0 con 0x7f2d98101420 2024-09-19T04:14:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.482+0000 7f2d8effd640 1 -- 172.21.15.94:0/4147840600 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2d88017070 con 0x7f2d98101420 2024-09-19T04:14:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.483+0000 7f2d8effd640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2d840a3350 0x7f2d840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.483+0000 7f2d9d99a640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2d840a3350 0x7f2d840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.483+0000 7f2d8effd640 1 -- 172.21.15.94:0/4147840600 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2d880cf110 con 0x7f2d98101420 2024-09-19T04:14:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.483+0000 7f2d9d99a640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2d840a3350 0x7f2d840a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d940058c0 tx=0x7f2d9400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.486+0000 7f2d8effd640 1 -- 172.21.15.94:0/4147840600 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2d8809bde0 con 0x7f2d98101420 2024-09-19T04:14:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.687+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d981996d0 con 0x7f2d98101420 2024-09-19T04:14:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.688+0000 7f2d8effd640 1 -- 172.21.15.94:0/4147840600 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11915 (secure 0 0 0) 0x7f2d981996d0 con 0x7f2d98101420 2024-09-19T04:14:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:26.690 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"btime":"2024-09-19T04:13:28:480286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:13:28.166880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:14:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-09-19T04:14:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2d840a3350 msgr2=0x7f2d840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2d840a3350 0x7f2d840a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d940058c0 tx=0x7f2d9400a4c0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 msgr2=0x7f2d981988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 0x7f2d981988b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800b4a0 tx=0x7f2d8800b960 comp rx=0 tx=0).stop 2024-09-19T04:14:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 shutdown_connections 2024-09-19T04:14:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f2d840a3350 0x7f2d840a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d98106a80 0x7f2d9819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d98101420 0x7f2d981988b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 --2- 172.21.15.94:0/4147840600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d98100a80 0x7f2d98198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 >> 172.21.15.94:0/4147840600 conn(0x7f2d980fc7a0 msgr2=0x7f2d981048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:26.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 shutdown_connections 2024-09-19T04:14:26.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:26.690+0000 7f2d9fbfe640 1 -- 172.21.15.94:0/4147840600 wait complete. 2024-09-19T04:14:27.290 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:14:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:27 smithi112 ceph-mon[35051]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:27 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4147840600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:27.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:27 smithi094 ceph-mon[30972]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:27.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:27 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4147840600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:27 smithi104 ceph-mon[34145]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:27.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:27 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4147840600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:28.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 -- 172.21.15.94:0/262125182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 msgr2=0x7ffbe810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 --2- 172.21.15.94:0/262125182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 0x7ffbe810e610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009ca0 tx=0x7ffbe002a950 comp rx=0 tx=0).stop 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 -- 172.21.15.94:0/262125182 shutdown_connections 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 --2- 172.21.15.94:0/262125182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 0x7ffbe810e610 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 --2- 172.21.15.94:0/262125182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8103720 0x7ffbe810bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 --2- 172.21.15.94:0/262125182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbe8102e00 0x7ffbe81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 -- 172.21.15.94:0/262125182 >> 172.21.15.94:0/262125182 conn(0x7ffbe80fc7a0 msgr2=0x7ffbe80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 -- 172.21.15.94:0/262125182 shutdown_connections 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 -- 172.21.15.94:0/262125182 wait complete. 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.069+0000 7ffbf003f640 1 Processor -- start 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbf003f640 1 -- start start 2024-09-19T04:14:28.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbf003f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8102e00 0x7ffbe8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:28.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbf003f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbe8103720 0x7ffbe81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:28.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbf003f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 0x7ffbe819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:28.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbf003f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8111e20 con 0x7ffbe810c1a0 2024-09-19T04:14:28.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbf003f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8111ca0 con 0x7ffbe8103720 2024-09-19T04:14:28.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbf003f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8111fa0 con 0x7ffbe8102e00 2024-09-19T04:14:28.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbee5dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 0x7ffbe819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:28.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbee5dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 0x7ffbe819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37826/0 (socket says 172.21.15.94:37826) 2024-09-19T04:14:28.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbee5dc640 1 -- 172.21.15.94:0/1718188419 learned_addr learned my addr 172.21.15.94:0/1718188419 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbedddb640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8102e00 0x7ffbe8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbed5da640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbe8103720 0x7ffbe81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbee5dc640 1 -- 172.21.15.94:0/1718188419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8102e00 msgr2=0x7ffbe8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbee5dc640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8102e00 0x7ffbe8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbee5dc640 1 -- 172.21.15.94:0/1718188419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbe8103720 msgr2=0x7ffbe81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbee5dc640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbe8103720 0x7ffbe81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbee5dc640 1 -- 172.21.15.94:0/1718188419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe819d330 con 0x7ffbe810c1a0 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbedddb640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8102e00 0x7ffbe8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbed5da640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbe8103720 0x7ffbe81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.070+0000 7ffbee5dc640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 0x7ffbe819cc50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0006010 tx=0x7ffbe002f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.071+0000 7ffbdaffd640 1 -- 172.21.15.94:0/1718188419 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0034040 con 0x7ffbe810c1a0 2024-09-19T04:14:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.071+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe819d5c0 con 0x7ffbe810c1a0 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.071+0000 7ffbdaffd640 1 -- 172.21.15.94:0/1718188419 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbe0003e00 con 0x7ffbe810c1a0 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.071+0000 7ffbdaffd640 1 -- 172.21.15.94:0/1718188419 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0045480 con 0x7ffbe810c1a0 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.071+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe81a4f00 con 0x7ffbe810c1a0 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.072+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe81031e0 con 0x7ffbe810c1a0 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.073+0000 7ffbdaffd640 1 -- 172.21.15.94:0/1718188419 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ffbe0037020 con 0x7ffbe810c1a0 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.073+0000 7ffbdaffd640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ffbd40a32d0 0x7ffbd40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.073+0000 7ffbedddb640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ffbd40a32d0 0x7ffbd40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.073+0000 7ffbdaffd640 1 -- 172.21.15.94:0/1718188419 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ffbe00f0b60 con 0x7ffbe810c1a0 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.074+0000 7ffbedddb640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ffbd40a32d0 0x7ffbd40a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0058c0 tx=0x7ffbdc005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.077+0000 7ffbdaffd640 1 -- 172.21.15.94:0/1718188419 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffbe00bd8c0 con 0x7ffbe810c1a0 2024-09-19T04:14:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.247+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbe8107280 con 0x7ffbe810c1a0 2024-09-19T04:14:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.248+0000 7ffbdaffd640 1 -- 172.21.15.94:0/1718188419 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22842 (secure 0 0 0) 0x7ffbe00bd1c0 con 0x7ffbe810c1a0 2024-09-19T04:14:28.250 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:28.250 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:13:31.495964+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:14:28.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ffbd40a32d0 msgr2=0x7ffbd40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ffbd40a32d0 0x7ffbd40a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0058c0 tx=0x7ffbdc005ec0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 msgr2=0x7ffbe819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 0x7ffbe819cc50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0006010 tx=0x7ffbe002f640 comp rx=0 tx=0).stop 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 shutdown_connections 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7ffbd40a32d0 0x7ffbd40a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe810c1a0 0x7ffbe819cc50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbe8103720 0x7ffbe81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 --2- 172.21.15.94:0/1718188419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8102e00 0x7ffbe8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 >> 172.21.15.94:0/1718188419 conn(0x7ffbe80fc7a0 msgr2=0x7ffbe80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 shutdown_connections 2024-09-19T04:14:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:28.251+0000 7ffbf003f640 1 -- 172.21.15.94:0/1718188419 wait complete. 2024-09-19T04:14:28.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:28 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1718188419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:14:28.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:28 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1718188419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:14:28.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:28 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1718188419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:14:28.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:14:29.098 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:29.539 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:29 smithi094 ceph-mon[30972]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:29 smithi104 ceph-mon[34145]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.616+0000 7f86f7f3a640 1 -- 172.21.15.94:0/93062179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0074510 msgr2=0x7f86f00748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.616+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/93062179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0074510 0x7f86f00748f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009900 tx=0x7f86e002a480 comp rx=0 tx=0).stop 2024-09-19T04:14:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.616+0000 7f86f7f3a640 1 -- 172.21.15.94:0/93062179 shutdown_connections 2024-09-19T04:14:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.616+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/93062179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86f010e440 0x7f86f0110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.616+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/93062179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f0074e30 0x7f86f010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.616+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/93062179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0074510 0x7f86f00748f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.616+0000 7f86f7f3a640 1 -- 172.21.15.94:0/93062179 >> 172.21.15.94:0/93062179 conn(0x7f86f00fd8c0 msgr2=0x7f86f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.616+0000 7f86f7f3a640 1 -- 172.21.15.94:0/93062179 shutdown_connections 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.616+0000 7f86f7f3a640 1 -- 172.21.15.94:0/93062179 wait complete. 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f7f3a640 1 Processor -- start 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f7f3a640 1 -- start start 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f7f3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f0074510 0x7f86f019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f7f3a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86f0074e30 0x7f86f019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f7f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f010e440 0x7f86f019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f7f3a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0114070 con 0x7f86f010e440 2024-09-19T04:14:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f7f3a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0113ef0 con 0x7f86f0074e30 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f7f3a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01141f0 con 0x7f86f0074510 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f64d7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f010e440 0x7f86f019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f64d7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f010e440 0x7f86f019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37840/0 (socket says 172.21.15.94:37840) 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f64d7640 1 -- 172.21.15.94:0/2654840002 learned_addr learned my addr 172.21.15.94:0/2654840002 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f5cd6640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f0074510 0x7f86f019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f54d5640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86f0074e30 0x7f86f019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f64d7640 1 -- 172.21.15.94:0/2654840002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f0074510 msgr2=0x7f86f019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f64d7640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f0074510 0x7f86f019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f64d7640 1 -- 172.21.15.94:0/2654840002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86f0074e30 msgr2=0x7f86f019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f64d7640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86f0074e30 0x7f86f019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f64d7640 1 -- 172.21.15.94:0/2654840002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f019f5d0 con 0x7f86f010e440 2024-09-19T04:14:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.617+0000 7f86f54d5640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86f0074e30 0x7f86f019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.618+0000 7f86f64d7640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f010e440 0x7f86f019eef0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f86e800d720 tx=0x7f86e800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.618+0000 7f86eeffd640 1 -- 172.21.15.94:0/2654840002 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8017070 con 0x7f86f010e440 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.618+0000 7f86eeffd640 1 -- 172.21.15.94:0/2654840002 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86e800de30 con 0x7f86f010e440 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.618+0000 7f86eeffd640 1 -- 172.21.15.94:0/2654840002 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8004cc0 con 0x7f86f010e440 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.618+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f019f8c0 con 0x7f86f010e440 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.618+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a71a0 con 0x7f86f010e440 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.619+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f00748f0 con 0x7f86f010e440 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.620+0000 7f86eeffd640 1 -- 172.21.15.94:0/2654840002 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f86e8004e60 con 0x7f86f010e440 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.620+0000 7f86eeffd640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f86d80a32d0 0x7f86d80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.620+0000 7f86f5cd6640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f86d80a32d0 0x7f86d80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.621+0000 7f86eeffd640 1 -- 172.21.15.94:0/2654840002 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f86e80cf410 con 0x7f86f010e440 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.621+0000 7f86f5cd6640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f86d80a32d0 0x7f86d80a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86e0031440 tx=0x7f86e0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.624+0000 7f86eeffd640 1 -- 172.21.15.94:0/2654840002 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86e809bfd0 con 0x7f86f010e440 2024-09-19T04:14:29.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:29 smithi112 ceph-mon[35051]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.823+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86f0109620 con 0x7f86f010e440 2024-09-19T04:14:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.824+0000 7f86eeffd640 1 -- 172.21.15.94:0/2654840002 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11915 (secure 0 0 0) 0x7f86e809b8d0 con 0x7f86f010e440 2024-09-19T04:14:29.826 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:29.826 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"btime":"2024-09-19T04:13:28:480286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:13:28.166880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:14:29.826 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-09-19T04:14:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f86d80a32d0 msgr2=0x7f86d80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f86d80a32d0 0x7f86d80a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86e0031440 tx=0x7f86e0005b50 comp rx=0 tx=0).stop 2024-09-19T04:14:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f010e440 msgr2=0x7f86f019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f010e440 0x7f86f019eef0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f86e800d720 tx=0x7f86e800dbe0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 shutdown_connections 2024-09-19T04:14:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f86d80a32d0 0x7f86d80a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f010e440 0x7f86f019eef0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86f0074e30 0x7f86f019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 --2- 172.21.15.94:0/2654840002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f0074510 0x7f86f019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:29.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 >> 172.21.15.94:0/2654840002 conn(0x7f86f00fd8c0 msgr2=0x7f86f00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:29.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 shutdown_connections 2024-09-19T04:14:29.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:29.826+0000 7f86f7f3a640 1 -- 172.21.15.94:0/2654840002 wait complete. 2024-09-19T04:14:30.418 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:30 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2654840002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:30.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T04:13:17.025380+0000', 'modified': '2024-09-19T04:13:28.166880+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24467, 'mds_1': 24466, 'mds_2': 24454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6835/2064839900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 2064839900}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 2064839900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6837/2682504501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2682504501}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2682504501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/1098195473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 1098195473}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 1098195473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.112:6837/134813546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 134813546}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 134813546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.112:6839/3588924475', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 3588924475}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 3588924475}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.104:6839/2749200235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 2749200235}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 2749200235}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24467, 'qdb_cluster': [24454, 24467, 24466]} 2024-09-19T04:14:30.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T04:14:30.446 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T04:14:30.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:30 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2654840002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:30.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:30 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2654840002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:31.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.182+0000 7f5dda101640 1 -- 172.21.15.94:0/3373304875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 msgr2=0x7f5dd41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.182+0000 7f5dda101640 1 --2- 172.21.15.94:0/3373304875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 0x7f5dd41067e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009900 tx=0x7f5dc802a410 comp rx=0 tx=0).stop 2024-09-19T04:14:31.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.182+0000 7f5dda101640 1 -- 172.21.15.94:0/3373304875 shutdown_connections 2024-09-19T04:14:31.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.182+0000 7f5dda101640 1 --2- 172.21.15.94:0/3373304875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dd4106f10 0x7f5dd410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.182+0000 7f5dda101640 1 --2- 172.21.15.94:0/3373304875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 0x7f5dd41067e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.182+0000 7f5dda101640 1 --2- 172.21.15.94:0/3373304875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd4101de0 0x7f5dd41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.182+0000 7f5dda101640 1 -- 172.21.15.94:0/3373304875 >> 172.21.15.94:0/3373304875 conn(0x7f5dd40fd8c0 msgr2=0x7f5dd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:31.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.182+0000 7f5dda101640 1 -- 172.21.15.94:0/3373304875 shutdown_connections 2024-09-19T04:14:31.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.182+0000 7f5dda101640 1 -- 172.21.15.94:0/3373304875 wait complete. 2024-09-19T04:14:31.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dda101640 1 Processor -- start 2024-09-19T04:14:31.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dda101640 1 -- start start 2024-09-19T04:14:31.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dda101640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dd4101de0 0x7f5dd4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dda101640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 0x7f5dd4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dda101640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd4106f10 0x7f5dd419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dda101640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd4111e20 con 0x7f5dd4102790 2024-09-19T04:14:31.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dda101640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd4111ca0 con 0x7f5dd4101de0 2024-09-19T04:14:31.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dda101640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd4111fa0 con 0x7f5dd4106f10 2024-09-19T04:14:31.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dd2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 0x7f5dd4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dd2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 0x7f5dd4198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37846/0 (socket says 172.21.15.94:37846) 2024-09-19T04:14:31.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.183+0000 7f5dd2ffd640 1 -- 172.21.15.94:0/3441066507 learned_addr learned my addr 172.21.15.94:0/3441066507 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:31.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd37fe640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dd4101de0 0x7f5dd4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd2ffd640 1 -- 172.21.15.94:0/3441066507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd4106f10 msgr2=0x7f5dd419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd4106f10 0x7f5dd419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd2ffd640 1 -- 172.21.15.94:0/3441066507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dd4101de0 msgr2=0x7f5dd4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dd4101de0 0x7f5dd4198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd2ffd640 1 -- 172.21.15.94:0/3441066507 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd419d2b0 con 0x7f5dd4102790 2024-09-19T04:14:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 0x7f5dd4198840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd0ff9640 1 -- 172.21.15.94:0/3441066507 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc803f030 con 0x7f5dd4102790 2024-09-19T04:14:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd0ff9640 1 -- 172.21.15.94:0/3441066507 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dc80035f0 con 0x7f5dd4102790 2024-09-19T04:14:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dda101640 1 -- 172.21.15.94:0/3441066507 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd419d540 con 0x7f5dd4102790 2024-09-19T04:14:31.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dd0ff9640 1 -- 172.21.15.94:0/3441066507 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8035070 con 0x7f5dd4102790 2024-09-19T04:14:31.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.184+0000 7f5dda101640 1 -- 172.21.15.94:0/3441066507 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd41a4e80 con 0x7f5dd4102790 2024-09-19T04:14:31.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.186+0000 7f5dd0ff9640 1 -- 172.21.15.94:0/3441066507 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5dc8003b10 con 0x7f5dd4102790 2024-09-19T04:14:31.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.186+0000 7f5ddc4da740 1 Processor -- start 2024-09-19T04:14:31.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5ddc4da740 1 -- start start 2024-09-19T04:14:31.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 0x558e12db0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 0x558e12dbb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12dbdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558e12d17090 con 0x558e12dbbbb0 2024-09-19T04:14:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558e12cbea70 con 0x558e12db12f0 2024-09-19T04:14:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558e12d13050 con 0x558e12db09d0 2024-09-19T04:14:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd0ff9640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5dbc0a3350 0x7f5dbc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12dbdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12dbdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37850/0 (socket says 172.21.15.94:37850) 2024-09-19T04:14:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd2ffd640 1 -- 172.21.15.94:0/1892396410 learned_addr learned my addr 172.21.15.94:0/1892396410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 0x558e12db0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5dbc0a3350 0x7f5dbc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd37fe640 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 0x558e12dbb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd0ff9640 1 -- 172.21.15.94:0/3441066507 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5dc80f0650 con 0x7f5dd4102790 2024-09-19T04:14:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd3fff640 1 -- 172.21.15.94:0/1892396410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 msgr2=0x558e12dbb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 0x558e12dbb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd3fff640 1 -- 172.21.15.94:0/1892396410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 msgr2=0x558e12dbdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12dbdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd3fff640 1 -- 172.21.15.94:0/1892396410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e12ccb130 con 0x558e12db09d0 2024-09-19T04:14:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.187+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12dbdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:14:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5dbc0a3350 0x7f5dbc0a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc011c50 tx=0x7f5dcc01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 0x558e12db0db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc013bb0 tx=0x7f5dcc011750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1129bd9ed2198038 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5db9ffb640 1 -- 172.21.15.94:0/1892396410 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc020070 con 0x558e12db09d0 2024-09-19T04:14:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5db9ffb640 1 -- 172.21.15.94:0/1892396410 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dcc009970 con 0x558e12db09d0 2024-09-19T04:14:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5db9ffb640 1 -- 172.21.15.94:0/1892396410 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc003bd0 con 0x558e12db09d0 2024-09-19T04:14:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5ddc4da740 1 -- 172.21.15.94:0/1892396410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 msgr2=0x558e12db0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 0x558e12db0db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc013bb0 tx=0x7f5dcc011750 comp rx=0 tx=0).stop 2024-09-19T04:14:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5ddc4da740 1 -- 172.21.15.94:0/1892396410 shutdown_connections 2024-09-19T04:14:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12dbdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 0x558e12dbb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/1892396410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 0x558e12db0db0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.188+0000 7f5ddc4da740 1 -- 172.21.15.94:0/1892396410 >> 172.21.15.94:0/1892396410 conn(0x558e12dacf60 msgr2=0x558e12dad340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 -- 172.21.15.94:0/1892396410 shutdown_connections 2024-09-19T04:14:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 -- 172.21.15.94:0/1892396410 wait complete. 2024-09-19T04:14:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 -- start start 2024-09-19T04:14:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 Processor -- start 2024-09-19T04:14:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dbb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 0x558e12db87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 0x558e12db8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558e12dc2b90 con 0x558e12db09d0 2024-09-19T04:14:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558e12cd1f20 con 0x558e12dbbbb0 2024-09-19T04:14:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558e12dc2d10 con 0x558e12db12f0 2024-09-19T04:14:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5ddc4da740 10 client.0 _ll_register_callbacks cb 0x558e12c9b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.189+0000 7f5dd37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dbb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dbb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37858/0 (socket says 172.21.15.94:37858) 2024-09-19T04:14:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd37fe640 1 -- 172.21.15.94:0/532153541 learned_addr learned my addr 172.21.15.94:0/532153541 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 0x558e12db87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 0x558e12db8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd37fe640 1 -- 172.21.15.94:0/532153541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 msgr2=0x558e12db87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd37fe640 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 0x558e12db87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd37fe640 1 -- 172.21.15.94:0/532153541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 msgr2=0x558e12db8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd37fe640 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 0x558e12db8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd37fe640 1 -- 172.21.15.94:0/532153541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e12a882b0 con 0x558e12db09d0 2024-09-19T04:14:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 0x558e12db87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:14:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5dd37fe640 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dbb460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4011760 tx=0x7f5dc4011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3ce8f75ed4cd58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5db2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5db2ffd640 1 -- 172.21.15.94:0/532153541 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc400fe10 con 0x558e12db09d0 2024-09-19T04:14:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5db2ffd640 1 -- 172.21.15.94:0/532153541 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5dc4005020 con 0x558e12db09d0 2024-09-19T04:14:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5ddc4da740 1 -- 172.21.15.94:0/532153541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dcc009970 con 0x558e12db09d0 2024-09-19T04:14:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5db2ffd640 1 -- 172.21.15.94:0/532153541 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5dc4005220 con 0x558e12db09d0 2024-09-19T04:14:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5ddc4da740 10 client.14979 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5ddc4da740 1 -- 172.21.15.94:0/532153541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558e12dce510 con 0x558e12db09d0 2024-09-19T04:14:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.190+0000 7f5db2ffd640 1 -- 172.21.15.94:0/532153541 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dcc009970 con 0x558e12db09d0 2024-09-19T04:14:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5db2ffd640 1 -- 172.21.15.94:0/532153541 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5dc4018030 con 0x558e12db09d0 2024-09-19T04:14:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5ddc4da740 10 client.14979 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5ddc4da740 10 client.14979 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:14:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5ddc4da740 20 client.14979 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558e12dd0b80 0x558e12dd1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5ddc4da740 4 client.14979 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5ddc4da740 1 -- 172.21.15.94:0/532153541 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e12d24600 con 0x558e12dd0b80 2024-09-19T04:14:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558e12dd0b80 0x558e12dd1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558e12dd0b80 0x558e12dd1ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d1824ce03d1d61b server_cookie=c641e4d707298700 in_seq=0 out_seq=0 2024-09-19T04:14:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.191+0000 7f5db2ffd640 10 client.14979 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:14:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5db2ffd640 1 -- 172.21.15.94:0/532153541 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e12d24600 con 0x558e12dd0b80 2024-09-19T04:14:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5db2ffd640 10 client.14979 handle_command_reply: tid=0 2024-09-19T04:14:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5ddc4da740 1 client.14979 shutdown 2024-09-19T04:14:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5ddc4da740 1 -- 172.21.15.94:0/532153541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 msgr2=0x558e12dbb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dbb460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4011760 tx=0x7f5dc4011c20 comp rx=0 tx=0).stop 2024-09-19T04:14:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5ddc4da740 1 -- 172.21.15.94:0/532153541 shutdown_connections 2024-09-19T04:14:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558e12dd0b80 0x558e12dd1ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 0x558e12db8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5db2ffd640 0 client.14979 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:14:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.192+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 0x558e12db87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.193+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/532153541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dbb460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.193+0000 7f5ddc4da740 1 -- 172.21.15.94:0/532153541 >> 172.21.15.94:0/532153541 conn(0x558e12dacf60 msgr2=0x558e12db2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.193+0000 7f5ddc4da740 1 -- 172.21.15.94:0/532153541 shutdown_connections 2024-09-19T04:14:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.193+0000 7f5ddc4da740 1 -- 172.21.15.94:0/532153541 wait complete. 2024-09-19T04:14:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.193+0000 7f5ddc4da740 20 client.14979 trim_cache size 0 max 16384 2024-09-19T04:14:31.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.210+0000 7f5ddc4da740 1 Processor -- start 2024-09-19T04:14:31.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.210+0000 7f5ddc4da740 1 -- start start 2024-09-19T04:14:31.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.210+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 0x558e12dd1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.210+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 0x558e12dabf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.210+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12db5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.210+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558e12dd0330 con 0x558e12dbbbb0 2024-09-19T04:14:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.210+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558e12dd0a00 con 0x558e12db12f0 2024-09-19T04:14:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.210+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558e12dd10d0 con 0x558e12db09d0 2024-09-19T04:14:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12db5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12db5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37874/0 (socket says 172.21.15.94:37874) 2024-09-19T04:14:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd3fff640 1 -- 172.21.15.94:0/2156255967 learned_addr learned my addr 172.21.15.94:0/2156255967 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 0x558e12dd1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/2156255967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 0x558e12dabf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd3fff640 1 -- 172.21.15.94:0/2156255967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 msgr2=0x558e12dd1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/2156255967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 0x558e12dd1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd3fff640 1 -- 172.21.15.94:0/2156255967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 msgr2=0x558e12dabf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/2156255967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 0x558e12dabf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd3fff640 1 -- 172.21.15.94:0/2156255967 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e12dbb080 con 0x558e12dbbbb0 2024-09-19T04:14:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/2156255967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12db5a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc004d10 tx=0x7f5dcc004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90171206f4e72a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5db97fa640 1 -- 172.21.15.94:0/2156255967 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc020040 con 0x558e12dbbbb0 2024-09-19T04:14:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5db97fa640 1 -- 172.21.15.94:0/2156255967 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dcc0135a0 con 0x558e12dbbbb0 2024-09-19T04:14:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.211+0000 7f5db97fa640 1 -- 172.21.15.94:0/2156255967 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc003cb0 con 0x558e12dbbbb0 2024-09-19T04:14:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 -- 172.21.15.94:0/2156255967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 msgr2=0x558e12db5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/2156255967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12db5a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc004d10 tx=0x7f5dcc004ca0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 -- 172.21.15.94:0/2156255967 shutdown_connections 2024-09-19T04:14:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/2156255967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12dbbbb0 0x558e12db5a10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/2156255967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12db12f0 0x558e12dabf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/2156255967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db09d0 0x558e12dd1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 -- 172.21.15.94:0/2156255967 >> 172.21.15.94:0/2156255967 conn(0x558e12dacf60 msgr2=0x558e12dab800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 -- 172.21.15.94:0/2156255967 shutdown_connections 2024-09-19T04:14:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 -- 172.21.15.94:0/2156255967 wait complete. 2024-09-19T04:14:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 -- start start 2024-09-19T04:14:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.212+0000 7f5ddc4da740 1 Processor -- start 2024-09-19T04:14:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dda5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 0x558e12ddab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5ddc4da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 0x558e12dd59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558e12dd1e70 con 0x558e12db09d0 2024-09-19T04:14:31.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558e12dd17a0 con 0x558e12dbbbb0 2024-09-19T04:14:31.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5ddc4da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558e12dd2540 con 0x558e12db12f0 2024-09-19T04:14:31.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5ddc4da740 10 client.0 _ll_register_callbacks cb 0x7f5dc400c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:31.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dda5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dda5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37882/0 (socket says 172.21.15.94:37882) 2024-09-19T04:14:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd37fe640 1 -- 172.21.15.94:0/4102165972 learned_addr learned my addr 172.21.15.94:0/4102165972 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd3fff640 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 0x558e12dd59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 0x558e12ddab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd37fe640 1 -- 172.21.15.94:0/4102165972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 msgr2=0x558e12ddab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd37fe640 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 0x558e12ddab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd37fe640 1 -- 172.21.15.94:0/4102165972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 msgr2=0x558e12dd59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd37fe640 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 0x558e12dd59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd37fe640 1 -- 172.21.15.94:0/4102165972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e12dd61e0 con 0x558e12db09d0 2024-09-19T04:14:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5dd37fe640 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dda5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4011760 tx=0x7f5dc400fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fe1a5b8af1d3db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.213+0000 7f5db2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5db2ffd640 1 -- 172.21.15.94:0/4102165972 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4032060 con 0x558e12db09d0 2024-09-19T04:14:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5db2ffd640 1 -- 172.21.15.94:0/4102165972 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5dc4011040 con 0x558e12db09d0 2024-09-19T04:14:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5ddc4da740 1 -- 172.21.15.94:0/4102165972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e12d1eb10 con 0x558e12db09d0 2024-09-19T04:14:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5db2ffd640 1 -- 172.21.15.94:0/4102165972 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5dc4028040 con 0x558e12db09d0 2024-09-19T04:14:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5ddc4da740 10 client.14985 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5ddc4da740 1 -- 172.21.15.94:0/4102165972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558e12e3ae70 con 0x558e12db09d0 2024-09-19T04:14:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5db2ffd640 1 -- 172.21.15.94:0/4102165972 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558e12d1eb10 con 0x558e12db09d0 2024-09-19T04:14:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5db2ffd640 1 -- 172.21.15.94:0/4102165972 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5dc4037070 con 0x558e12db09d0 2024-09-19T04:14:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5ddc4da740 10 client.14985 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5ddc4da740 10 client.14985 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:14:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5ddc4da740 20 client.14985 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558e12e3f9f0 0x558e12e3e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5ddc4da740 4 client.14985 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5ddc4da740 1 -- 172.21.15.94:0/4102165972 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "status"}) -- 0x558e12c9b0e0 con 0x558e12e3f9f0 2024-09-19T04:14:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.214+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558e12e3f9f0 0x558e12e3e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5db2ffd640 10 client.14985 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:14:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5dd2ffd640 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558e12e3f9f0 0x558e12e3e700 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45ae6b0f6534ed68 server_cookie=afcb7ebd72b05cca in_seq=0 out_seq=0 2024-09-19T04:14:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5db2ffd640 1 -- 172.21.15.94:0/4102165972 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558e12c9b0e0 con 0x558e12e3f9f0 2024-09-19T04:14:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5db2ffd640 10 client.14985 handle_command_reply: tid=0 2024-09-19T04:14:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 client.14985 shutdown 2024-09-19T04:14:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 -- 172.21.15.94:0/4102165972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 msgr2=0x558e12dda5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dda5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4011760 tx=0x7f5dc400fe40 comp rx=0 tx=0).stop 2024-09-19T04:14:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 -- 172.21.15.94:0/4102165972 shutdown_connections 2024-09-19T04:14:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558e12e3f9f0 0x558e12e3e700 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e12dbbbb0 0x558e12dd59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558e12db12f0 0x558e12ddab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 --2- 172.21.15.94:0/4102165972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558e12db09d0 0x558e12dda5d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 -- 172.21.15.94:0/4102165972 >> 172.21.15.94:0/4102165972 conn(0x558e12dacf60 msgr2=0x558e12dab7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5db2ffd640 0 client.14985 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:14:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 -- 172.21.15.94:0/4102165972 shutdown_connections 2024-09-19T04:14:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 1 -- 172.21.15.94:0/4102165972 wait complete. 2024-09-19T04:14:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.215+0000 7f5ddc4da740 20 client.14985 trim_cache size 0 max 16384 2024-09-19T04:14:31.244 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:14:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "503c4526-763c-11ef-bceb-c7b262605968", 2024-09-19T04:14:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 2, 2024-09-19T04:14:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24454, 2024-09-19T04:14:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-19T04:14:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-19T04:14:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-19T04:14:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 65.744496255000001, 2024-09-19T04:14:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-09-19T04:14:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-09-19T04:14:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T04:14:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 98.444100559999995 2024-09-19T04:14:31.245 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 -- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5dbc0a3350 msgr2=0x7f5dbc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5dbc0a3350 0x7f5dbc0a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc011c50 tx=0x7f5dcc01a000 comp rx=0 tx=0).stop 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 -- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 msgr2=0x7f5dd4198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 0x7f5dd4198840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc8004270 comp rx=0 tx=0).stop 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 -- 172.21.15.94:0/3441066507 shutdown_connections 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f5dbc0a3350 0x7f5dbc0a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd4106f10 0x7f5dd419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dd4102790 0x7f5dd4198840 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 --2- 172.21.15.94:0/3441066507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dd4101de0 0x7f5dd4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 -- 172.21.15.94:0/3441066507 >> 172.21.15.94:0/3441066507 conn(0x7f5dd40fd8c0 msgr2=0x7f5dd40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 -- 172.21.15.94:0/3441066507 shutdown_connections 2024-09-19T04:14:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:31.216+0000 7f5dda101640 1 -- 172.21.15.94:0/3441066507 wait complete. 2024-09-19T04:14:31.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:31 smithi094 ceph-mon[30972]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:31.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:31 smithi104 ceph-mon[34145]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:31.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:31 smithi112 ceph-mon[35051]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:32.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T04:14:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.773+0000 7fb59558e640 1 -- 172.21.15.94:0/4235403838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5901082a0 msgr2=0x7fb59010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.773+0000 7fb59558e640 1 --2- 172.21.15.94:0/4235403838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5901082a0 0x7fb59010ec40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb580009930 tx=0x7fb58002a440 comp rx=0 tx=0).stop 2024-09-19T04:14:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.773+0000 7fb59558e640 1 -- 172.21.15.94:0/4235403838 shutdown_connections 2024-09-19T04:14:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.773+0000 7fb59558e640 1 --2- 172.21.15.94:0/4235403838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5901082a0 0x7fb59010ec40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.773+0000 7fb59558e640 1 --2- 172.21.15.94:0/4235403838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5901036e0 0x7fb590107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.773+0000 7fb59558e640 1 --2- 172.21.15.94:0/4235403838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb590102d30 0x7fb590103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.773+0000 7fb59558e640 1 -- 172.21.15.94:0/4235403838 >> 172.21.15.94:0/4235403838 conn(0x7fb5900fc7d0 msgr2=0x7fb5900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:32.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.773+0000 7fb59558e640 1 -- 172.21.15.94:0/4235403838 shutdown_connections 2024-09-19T04:14:32.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.773+0000 7fb59558e640 1 -- 172.21.15.94:0/4235403838 wait complete. 2024-09-19T04:14:32.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.774+0000 7fb59558e640 1 Processor -- start 2024-09-19T04:14:32.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.774+0000 7fb59558e640 1 -- start start 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.774+0000 7fb59558e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb590102d30 0x7fb5901984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.774+0000 7fb59558e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5901036e0 0x7fb590198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.774+0000 7fb59558e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5901082a0 0x7fb59019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.774+0000 7fb59558e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb590111f90 con 0x7fb590102d30 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.774+0000 7fb59558e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb590111e10 con 0x7fb5901082a0 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.774+0000 7fb59558e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb590112110 con 0x7fb5901036e0 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb590102d30 0x7fb5901984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb590102d30 0x7fb5901984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46296/0 (socket says 172.21.15.94:46296) 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58effd640 1 -- 172.21.15.94:0/1177648165 learned_addr learned my addr 172.21.15.94:0/1177648165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58f7fe640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5901082a0 0x7fb59019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5901036e0 0x7fb590198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58effd640 1 -- 172.21.15.94:0/1177648165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5901036e0 msgr2=0x7fb590198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58effd640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5901036e0 0x7fb590198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58effd640 1 -- 172.21.15.94:0/1177648165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5901082a0 msgr2=0x7fb59019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58effd640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5901082a0 0x7fb59019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58effd640 1 -- 172.21.15.94:0/1177648165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59019d490 con 0x7fb590102d30 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb58effd640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb590102d30 0x7fb5901984e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb58800eab0 tx=0x7fb58800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb587fff640 1 -- 172.21.15.94:0/1177648165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58800ce40 con 0x7fb590102d30 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb59558e640 1 -- 172.21.15.94:0/1177648165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59019d780 con 0x7fb590102d30 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb587fff640 1 -- 172.21.15.94:0/1177648165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb588011920 con 0x7fb590102d30 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb587fff640 1 -- 172.21.15.94:0/1177648165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588010a00 con 0x7fb590102d30 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.775+0000 7fb59558e640 1 -- 172.21.15.94:0/1177648165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901a5060 con 0x7fb590102d30 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb597967740 1 Processor -- start 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb597967740 1 -- start start 2024-09-19T04:14:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb597967740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2069d0 0x55f06c206db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb597967740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c211670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb597967740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c211bb0 0x55f06c213fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb587fff640 1 -- 172.21.15.94:0/1177648165 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb588011ad0 con 0x7fb590102d30 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb597967740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f06c16d090 con 0x55f06c211bb0 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb597967740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f06c114a70 con 0x55f06c2072f0 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb597967740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f06c169050 con 0x55f06c2069d0 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb58e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c211bb0 0x55f06c213fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb58e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c211bb0 0x55f06c213fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46308/0 (socket says 172.21.15.94:46308) 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb58e7fc640 1 -- 172.21.15.94:0/3096895189 learned_addr learned my addr 172.21.15.94:0/3096895189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.778+0000 7fb58f7fe640 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2069d0 0x55f06c206db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58effd640 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c211670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58e7fc640 1 -- 172.21.15.94:0/3096895189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2069d0 msgr2=0x55f06c206db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2069d0 0x55f06c206db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58e7fc640 1 -- 172.21.15.94:0/3096895189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 msgr2=0x55f06c211670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c211670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58e7fc640 1 -- 172.21.15.94:0/3096895189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f06c121130 con 0x55f06c211bb0 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58f7fe640 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2069d0 0x55f06c206db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58effd640 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c211670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c211bb0 0x55f06c213fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb57800b790 tx=0x7fb57800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae43d9a11830d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb587fff640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb57c0a32d0 0x7fb57c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb5857fa640 1 -- 172.21.15.94:0/3096895189 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57800d950 con 0x55f06c211bb0 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb58f7fe640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb57c0a32d0 0x7fb57c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb587fff640 1 -- 172.21.15.94:0/1177648165 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb5880d01c0 con 0x7fb590102d30 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb5857fa640 1 -- 172.21.15.94:0/3096895189 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb578018070 con 0x55f06c211bb0 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.779+0000 7fb5857fa640 1 -- 172.21.15.94:0/3096895189 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5780123e0 con 0x55f06c211bb0 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 -- 172.21.15.94:0/3096895189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c211bb0 msgr2=0x55f06c213fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c211bb0 0x55f06c213fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb57800b790 tx=0x7fb57800bc50 comp rx=0 tx=0).stop 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb58f7fe640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb57c0a32d0 0x7fb57c0a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb580031440 tx=0x7fb580005a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 -- 172.21.15.94:0/3096895189 shutdown_connections 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c211bb0 0x55f06c213fa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c211670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 --2- 172.21.15.94:0/3096895189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2069d0 0x55f06c206db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 -- 172.21.15.94:0/3096895189 >> 172.21.15.94:0/3096895189 conn(0x55f06c202f60 msgr2=0x55f06c203340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 -- 172.21.15.94:0/3096895189 shutdown_connections 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 -- 172.21.15.94:0/3096895189 wait complete. 2024-09-19T04:14:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 -- start start 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.780+0000 7fb597967740 1 Processor -- start 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c2112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c20e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 0x55f06c20eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f06c214780 con 0x55f06c2069d0 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f06c127f20 con 0x55f06c2072f0 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f06c214900 con 0x55f06c211bb0 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 10 client.0 _ll_register_callbacks cb 0x55f06c0f1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c2112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c2112c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46316/0 (socket says 172.21.15.94:46316) 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58effd640 1 -- 172.21.15.94:0/595375412 learned_addr learned my addr 172.21.15.94:0/595375412 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c20e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58f7fe640 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 0x55f06c20eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58effd640 1 -- 172.21.15.94:0/595375412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 msgr2=0x55f06c20eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58effd640 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 0x55f06c20eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58effd640 1 -- 172.21.15.94:0/595375412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 msgr2=0x55f06c20e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58effd640 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c20e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58effd640 1 -- 172.21.15.94:0/595375412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f06bede2b0 con 0x55f06c2069d0 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb58effd640 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c2112c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb58800ea80 tx=0x7fb58809bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5383ac12051e4f77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb5767fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb5767fc640 1 -- 172.21.15.94:0/595375412 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58809cae0 con 0x55f06c2069d0 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb5767fc640 1 -- 172.21.15.94:0/595375412 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb5880a4e60 con 0x55f06c2069d0 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 1 -- 172.21.15.94:0/595375412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb578018070 con 0x55f06c2069d0 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb5767fc640 1 -- 172.21.15.94:0/595375412 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb58809c680 con 0x55f06c2069d0 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 10 client.14997 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.781+0000 7fb597967740 1 -- 172.21.15.94:0/595375412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f06c21fe70 con 0x55f06c2069d0 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb5767fc640 1 -- 172.21.15.94:0/595375412 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb578018070 con 0x55f06c2069d0 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb5767fc640 1 -- 172.21.15.94:0/595375412 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb58800f040 con 0x55f06c2069d0 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb597967740 10 client.14997 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb597967740 10 client.14997 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:14:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb597967740 20 client.14997 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb597967740 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55f06c2224a0 0x55f06c223910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb597967740 4 client.14997 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb597967740 1 -- 172.21.15.94:0/595375412 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f06c17a600 con 0x55f06c2224a0 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55f06c2224a0 0x55f06c223910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55f06c2224a0 0x55f06c223910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24713087748e0659 server_cookie=4b56f6954752d9f2 in_seq=0 out_seq=0 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.782+0000 7fb5767fc640 10 client.14997 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.783+0000 7fb5767fc640 1 -- 172.21.15.94:0/595375412 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f06c17a600 con 0x55f06c2224a0 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.783+0000 7fb5767fc640 10 client.14997 handle_command_reply: tid=0 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.783+0000 7fb597967740 1 client.14997 shutdown 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 -- 172.21.15.94:0/595375412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 msgr2=0x55f06c2112c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c2112c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb58800ea80 tx=0x7fb58809bfe0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 -- 172.21.15.94:0/595375412 shutdown_connections 2024-09-19T04:14:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55f06c2224a0 0x55f06c223910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 0x55f06c20eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c20e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 --2- 172.21.15.94:0/595375412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c2112c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 -- 172.21.15.94:0/595375412 >> 172.21.15.94:0/595375412 conn(0x55f06c202f60 msgr2=0x55f06c208c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb5767fc640 0 client.14997 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 -- 172.21.15.94:0/595375412 shutdown_connections 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 1 -- 172.21.15.94:0/595375412 wait complete. 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.784+0000 7fb597967740 20 client.14997 trim_cache size 0 max 16384 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.801+0000 7fb597967740 1 Processor -- start 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.801+0000 7fb597967740 1 -- start start 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb597967740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c20f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb597967740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2072f0 0x55f06c203340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb597967740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c211bb0 0x55f06c203880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb597967740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f06c221c50 con 0x55f06c2069d0 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb597967740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f06c222320 con 0x55f06c211bb0 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb597967740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f06c2229f0 con 0x55f06c2072f0 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c20f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c20f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46328/0 (socket says 172.21.15.94:46328) 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58effd640 1 -- 172.21.15.94:0/2648349946 learned_addr learned my addr 172.21.15.94:0/2648349946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58f7fe640 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c211bb0 0x55f06c203880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2072f0 0x55f06c203340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58effd640 1 -- 172.21.15.94:0/2648349946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2072f0 msgr2=0x55f06c203340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58effd640 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2072f0 0x55f06c203340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58effd640 1 -- 172.21.15.94:0/2648349946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c211bb0 msgr2=0x55f06c203880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58effd640 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c211bb0 0x55f06c203880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58effd640 1 -- 172.21.15.94:0/2648349946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f06c214880 con 0x55f06c2069d0 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58f7fe640 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c211bb0 0x55f06c203880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.802+0000 7fb58effd640 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c20f230 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb5880a4dc0 tx=0x7fb5880989f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c475789ae6c5125 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb584ff9640 1 -- 172.21.15.94:0/2648349946 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588011e00 con 0x55f06c2069d0 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb584ff9640 1 -- 172.21.15.94:0/2648349946 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5880cf070 con 0x55f06c2069d0 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb584ff9640 1 -- 172.21.15.94:0/2648349946 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58809c8e0 con 0x55f06c2069d0 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb597967740 1 -- 172.21.15.94:0/2648349946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 msgr2=0x55f06c20f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb597967740 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c20f230 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb5880a4dc0 tx=0x7fb5880989f0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb597967740 1 -- 172.21.15.94:0/2648349946 shutdown_connections 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb597967740 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c211bb0 0x55f06c203880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb597967740 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c2072f0 0x55f06c203340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb597967740 1 --2- 172.21.15.94:0/2648349946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c20f230 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb597967740 1 -- 172.21.15.94:0/2648349946 >> 172.21.15.94:0/2648349946 conn(0x55f06c202f60 msgr2=0x55f06c20e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb597967740 1 -- 172.21.15.94:0/2648349946 shutdown_connections 2024-09-19T04:14:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.803+0000 7fb597967740 1 -- 172.21.15.94:0/2648349946 wait complete. 2024-09-19T04:14:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb597967740 1 -- start start 2024-09-19T04:14:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb597967740 1 Processor -- start 2024-09-19T04:14:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb597967740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c22bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb597967740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c22c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb597967740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 0x55f06c2128d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb597967740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f06c223790 con 0x55f06c2069d0 2024-09-19T04:14:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb597967740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f06c2230c0 con 0x55f06c2072f0 2024-09-19T04:14:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb597967740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f06c223e60 con 0x55f06c211bb0 2024-09-19T04:14:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb597967740 10 client.0 _ll_register_callbacks cb 0x7fb58809c450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c22bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c22bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46338/0 (socket says 172.21.15.94:46338) 2024-09-19T04:14:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58effd640 1 -- 172.21.15.94:0/222082377 learned_addr learned my addr 172.21.15.94:0/222082377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c22c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58f7fe640 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 0x55f06c2128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58effd640 1 -- 172.21.15.94:0/222082377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 msgr2=0x55f06c2128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:32.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58effd640 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 0x55f06c2128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58effd640 1 -- 172.21.15.94:0/222082377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 msgr2=0x55f06c22c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:32.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58effd640 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c22c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.804+0000 7fb58effd640 1 -- 172.21.15.94:0/222082377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f06c213100 con 0x55f06c2069d0 2024-09-19T04:14:32.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb58effd640 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c22bf10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb58809c380 tx=0x7fb5880cdc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bdd68a21ac61bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:32.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb5767fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:32.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb5767fc640 1 -- 172.21.15.94:0/222082377 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588011e00 con 0x55f06c2069d0 2024-09-19T04:14:32.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb5767fc640 1 -- 172.21.15.94:0/222082377 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb588067340 con 0x55f06c2069d0 2024-09-19T04:14:32.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb597967740 1 -- 172.21.15.94:0/222082377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5880cf070 con 0x55f06c2069d0 2024-09-19T04:14:32.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb5767fc640 1 -- 172.21.15.94:0/222082377 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5880a0a30 con 0x55f06c2069d0 2024-09-19T04:14:32.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb597967740 10 client.15003 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:32.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb597967740 1 -- 172.21.15.94:0/222082377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f06c213100 con 0x55f06c2069d0 2024-09-19T04:14:32.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb5767fc640 1 -- 172.21.15.94:0/222082377 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5880cf070 con 0x55f06c2069d0 2024-09-19T04:14:32.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb5767fc640 1 -- 172.21.15.94:0/222082377 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb58809b540 con 0x55f06c2069d0 2024-09-19T04:14:32.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb597967740 10 client.15003 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:32.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb597967740 10 client.15003 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:14:32.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb597967740 20 client.15003 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:32.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb597967740 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55f06c290d90 0x55f06c28faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb597967740 4 client.15003 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:32.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.805+0000 7fb597967740 1 -- 172.21.15.94:0/222082377 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "status"}) -- 0x55f06c174b10 con 0x55f06c290d90 2024-09-19T04:14:32.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55f06c290d90 0x55f06c28faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb58e7fc640 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55f06c290d90 0x55f06c28faa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6daa8920f7793f46 server_cookie=a3799ff67e0615a1 in_seq=0 out_seq=0 2024-09-19T04:14:32.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb5767fc640 10 client.15003 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb5767fc640 1 -- 172.21.15.94:0/222082377 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f06c174b10 con 0x55f06c290d90 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb5767fc640 10 client.15003 handle_command_reply: tid=0 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 client.15003 shutdown 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 -- 172.21.15.94:0/222082377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 msgr2=0x55f06c22bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c22bf10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb58809c380 tx=0x7fb5880cdc40 comp rx=0 tx=0).stop 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 -- 172.21.15.94:0/222082377 shutdown_connections 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55f06c290d90 0x55f06c28faa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f06c211bb0 0x55f06c2128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f06c2072f0 0x55f06c22c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb5767fc640 0 client.15003 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:14:32.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 --2- 172.21.15.94:0/222082377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f06c2069d0 0x55f06c22bf10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 -- 172.21.15.94:0/222082377 >> 172.21.15.94:0/222082377 conn(0x55f06c202f60 msgr2=0x55f06c20e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:32.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 -- 172.21.15.94:0/222082377 shutdown_connections 2024-09-19T04:14:32.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 1 -- 172.21.15.94:0/222082377 wait complete. 2024-09-19T04:14:32.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.806+0000 7fb597967740 20 client.15003 trim_cache size 0 max 16384 2024-09-19T04:14:32.852 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:14:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "503c4526-763c-11ef-bceb-c7b262605968", 2024-09-19T04:14:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 1, 2024-09-19T04:14:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24466, 2024-09-19T04:14:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-19T04:14:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 68.342741736999997, 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 95.064794547000005 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 -- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb57c0a32d0 msgr2=0x7fb57c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb57c0a32d0 0x7fb57c0a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb580031440 tx=0x7fb580005a00 comp rx=0 tx=0).stop 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 -- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb590102d30 msgr2=0x7fb5901984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb590102d30 0x7fb5901984e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb58800eab0 tx=0x7fb58800ef70 comp rx=0 tx=0).stop 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 -- 172.21.15.94:0/1177648165 shutdown_connections 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb57c0a32d0 0x7fb57c0a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5901082a0 0x7fb59019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5901036e0 0x7fb590198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 --2- 172.21.15.94:0/1177648165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb590102d30 0x7fb5901984e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 -- 172.21.15.94:0/1177648165 >> 172.21.15.94:0/1177648165 conn(0x7fb5900fc7d0 msgr2=0x7fb590105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:32.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 -- 172.21.15.94:0/1177648165 shutdown_connections 2024-09-19T04:14:32.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:32.807+0000 7fb59558e640 1 -- 172.21.15.94:0/1177648165 wait complete. 2024-09-19T04:14:33.567 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:33 smithi094 ceph-mon[30972]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:33.594 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:14:33.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:33 smithi104 ceph-mon[34145]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:33.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:33 smithi112 ceph-mon[35051]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:33.837 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 -- 172.21.15.94:0/1115071165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102ef0 msgr2=0x7f17d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/1115071165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102ef0 0x7f17d010da40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009ca0 tx=0x7f17c802a950 comp rx=0 tx=0).stop 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 -- 172.21.15.94:0/1115071165 shutdown_connections 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/1115071165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102ef0 0x7f17d010da40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/1115071165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17d0102570 0x7f17d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/1115071165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17d0100e20 0x7f17d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 -- 172.21.15.94:0/1115071165 >> 172.21.15.94:0/1115071165 conn(0x7f17d00fc7d0 msgr2=0x7f17d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 -- 172.21.15.94:0/1115071165 shutdown_connections 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 -- 172.21.15.94:0/1115071165 wait complete. 2024-09-19T04:14:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.291+0000 7f17d4f0f640 1 Processor -- start 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17d4f0f640 1 -- start start 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17d4f0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17d0100e20 0x7f17d0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17d4f0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102570 0x7f17d0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17d4f0f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17d0102ef0 0x7f17d019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17d4f0f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0110e30 con 0x7f17d0102570 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17d4f0f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0110cb0 con 0x7f17d0102ef0 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17d4f0f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0110fb0 con 0x7f17d0100e20 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102570 0x7f17d0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102570 0x7f17d0198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46358/0 (socket says 172.21.15.94:46358) 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c7fff640 1 -- 172.21.15.94:0/3214176156 learned_addr learned my addr 172.21.15.94:0/3214176156 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17ce59c640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17d0100e20 0x7f17d0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17ced9d640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17d0102ef0 0x7f17d019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c7fff640 1 -- 172.21.15.94:0/3214176156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17d0100e20 msgr2=0x7f17d0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c7fff640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17d0100e20 0x7f17d0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c7fff640 1 -- 172.21.15.94:0/3214176156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17d0102ef0 msgr2=0x7f17d019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c7fff640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17d0102ef0 0x7f17d019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c7fff640 1 -- 172.21.15.94:0/3214176156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d019d3b0 con 0x7f17d0102570 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17ced9d640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17d0102ef0 0x7f17d019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c7fff640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102570 0x7f17d0198940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f17b800d950 tx=0x7f17b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c77fe640 1 -- 172.21.15.94:0/3214176156 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8017070 con 0x7f17d0102570 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17d4f0f640 1 -- 172.21.15.94:0/3214176156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d019d6a0 con 0x7f17d0102570 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c77fe640 1 -- 172.21.15.94:0/3214176156 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17b8009c30 con 0x7f17d0102570 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.292+0000 7f17c77fe640 1 -- 172.21.15.94:0/3214176156 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b800f620 con 0x7f17d0102570 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.293+0000 7f17d4f0f640 1 -- 172.21.15.94:0/3214176156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d01a4f80 con 0x7f17d0102570 2024-09-19T04:14:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.295+0000 7f17c77fe640 1 -- 172.21.15.94:0/3214176156 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f17b80109a0 con 0x7f17d0102570 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.295+0000 7f17d72e8740 1 Processor -- start 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.295+0000 7f17d72e8740 1 -- start start 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.295+0000 7f17c77fe640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f17bc0a32d0 0x7f17bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.295+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c7453c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c7462a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 0x55d0c74653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17d72e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d0c73ba090 con 0x55d0c74541d0 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17d72e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d0c7361a70 con 0x55d0c7462fc0 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ced9d640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f17bc0a32d0 0x7f17bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17d72e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d0c73b6050 con 0x55d0c74538b0 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17c77fe640 1 -- 172.21.15.94:0/3214176156 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f17b80cf730 con 0x7f17d0102570 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c7453c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c7453c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:51502/0 (socket says 172.21.15.94:51502) 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ce59c640 1 -- 172.21.15.94:0/2271959770 learned_addr learned my addr 172.21.15.94:0/2271959770 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17c7fff640 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c7462a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ced9d640 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 0x55d0c74653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ce59c640 1 -- 172.21.15.94:0/2271959770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 msgr2=0x55d0c74653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ce59c640 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 0x55d0c74653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ce59c640 1 -- 172.21.15.94:0/2271959770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 msgr2=0x55d0c7462a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ce59c640 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c7462a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ce59c640 1 -- 172.21.15.94:0/2271959770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c736e130 con 0x55d0c74538b0 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ced9d640 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 0x55d0c74653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ced9d640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f17bc0a32d0 0x7f17bc0a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f17c8005e30 tx=0x7f17c8009a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.296+0000 7f17ce59c640 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c7453c90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f17c000bd10 tx=0x7f17c000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a850dc570822effd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.297+0000 7f17c4ff9640 1 -- 172.21.15.94:0/2271959770 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0015070 con 0x55d0c74538b0 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.297+0000 7f17c4ff9640 1 -- 172.21.15.94:0/2271959770 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c001a910 con 0x55d0c74538b0 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.297+0000 7f17c4ff9640 1 -- 172.21.15.94:0/2271959770 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0011990 con 0x55d0c74538b0 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 -- 172.21.15.94:0/2271959770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 msgr2=0x55d0c7453c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c7453c90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f17c000bd10 tx=0x7f17c000f730 comp rx=0 tx=0).stop 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 -- 172.21.15.94:0/2271959770 shutdown_connections 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 0x55d0c74653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c7462a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 --2- 172.21.15.94:0/2271959770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c7453c90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 -- 172.21.15.94:0/2271959770 >> 172.21.15.94:0/2271959770 conn(0x55d0c744ff60 msgr2=0x55d0c7450340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 -- 172.21.15.94:0/2271959770 shutdown_connections 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 -- 172.21.15.94:0/2271959770 wait complete. 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 -- start start 2024-09-19T04:14:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.298+0000 7f17d72e8740 1 Processor -- start 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c74628d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c745fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 0x55d0c7460160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17d72e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d0c7465b90 con 0x55d0c74541d0 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17d72e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d0c7374f20 con 0x55d0c7462fc0 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17d72e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d0c7465d10 con 0x55d0c74538b0 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17d72e8740 10 client.0 _ll_register_callbacks cb 0x55d0c733e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c745fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c745fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46376/0 (socket says 172.21.15.94:46376) 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17c7fff640 1 -- 172.21.15.94:0/1580046174 learned_addr learned my addr 172.21.15.94:0/1580046174 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17ced9d640 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 0x55d0c7460160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17ce59c640 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c74628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17c7fff640 1 -- 172.21.15.94:0/1580046174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 msgr2=0x55d0c74628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17c7fff640 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c74628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17c7fff640 1 -- 172.21.15.94:0/1580046174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 msgr2=0x55d0c7460160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17c7fff640 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 0x55d0c7460160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17c7fff640 1 -- 172.21.15.94:0/1580046174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c712b2b0 con 0x55d0c74541d0 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17ce59c640 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c74628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17c7fff640 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c745fc20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f17b80985c0 tx=0x7f17b809c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5038775815a0583 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17b5ffb640 1 -- 172.21.15.94:0/1580046174 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8020040 con 0x55d0c74541d0 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17b5ffb640 1 -- 172.21.15.94:0/1580046174 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f17b809b040 con 0x55d0c74541d0 2024-09-19T04:14:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.299+0000 7f17d72e8740 1 -- 172.21.15.94:0/1580046174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c001a910 con 0x55d0c74541d0 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17b5ffb640 1 -- 172.21.15.94:0/1580046174 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f17b80a4860 con 0x55d0c74541d0 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17d72e8740 10 client.15015 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17d72e8740 1 -- 172.21.15.94:0/1580046174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d0c7471520 con 0x55d0c74541d0 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17b5ffb640 1 -- 172.21.15.94:0/1580046174 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c001a910 con 0x55d0c74541d0 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17b5ffb640 1 -- 172.21.15.94:0/1580046174 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f17b8097b40 con 0x55d0c74541d0 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17d72e8740 10 client.15015 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17d72e8740 10 client.15015 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17d72e8740 20 client.15015 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17d72e8740 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c7473b90 0x55d0c7475000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17d72e8740 4 client.15015 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17d72e8740 1 -- 172.21.15.94:0/1580046174 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0c73c7600 con 0x55d0c7473b90 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.300+0000 7f17ce59c640 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c7473b90 0x55d0c7475000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.301+0000 7f17b5ffb640 10 client.15015 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.301+0000 7f17ce59c640 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c7473b90 0x55d0c7475000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d3194c2af7dd0a server_cookie=b0c490a02ed670c2 in_seq=0 out_seq=0 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17b5ffb640 1 -- 172.21.15.94:0/1580046174 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0c73c7600 con 0x55d0c7473b90 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17b5ffb640 10 client.15015 handle_command_reply: tid=0 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 client.15015 shutdown 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 -- 172.21.15.94:0/1580046174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 msgr2=0x55d0c745fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c745fc20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f17b80985c0 tx=0x7f17b809c3b0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 -- 172.21.15.94:0/1580046174 shutdown_connections 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c7473b90 0x55d0c7475000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c7462fc0 0x55d0c7460160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c745fc20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17b5ffb640 0 client.15015 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:14:34.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 --2- 172.21.15.94:0/1580046174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c74538b0 0x55d0c74628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 -- 172.21.15.94:0/1580046174 >> 172.21.15.94:0/1580046174 conn(0x55d0c744ff60 msgr2=0x55d0c7455cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 -- 172.21.15.94:0/1580046174 shutdown_connections 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 1 -- 172.21.15.94:0/1580046174 wait complete. 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.302+0000 7f17d72e8740 20 client.15015 trim_cache size 0 max 16384 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17d72e8740 1 Processor -- start 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17d72e8740 1 -- start start 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74538b0 0x55d0c7475000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c747d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c7462fc0 0x55d0c747d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17d72e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d0c7473340 con 0x55d0c74541d0 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17d72e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d0c7473a10 con 0x55d0c74538b0 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17d72e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d0c74740e0 con 0x55d0c7462fc0 2024-09-19T04:14:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c747d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c747d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46386/0 (socket says 172.21.15.94:46386) 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17c7fff640 1 -- 172.21.15.94:0/164585271 learned_addr learned my addr 172.21.15.94:0/164585271 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17ce59c640 1 --2- 172.21.15.94:0/164585271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74538b0 0x55d0c7475000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17c7fff640 1 -- 172.21.15.94:0/164585271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c7462fc0 msgr2=0x55d0c747d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17c7fff640 1 --2- 172.21.15.94:0/164585271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c7462fc0 0x55d0c747d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17c7fff640 1 -- 172.21.15.94:0/164585271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74538b0 msgr2=0x55d0c7475000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17c7fff640 1 --2- 172.21.15.94:0/164585271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74538b0 0x55d0c7475000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17c7fff640 1 -- 172.21.15.94:0/164585271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c7454000 con 0x55d0c74541d0 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17c7fff640 1 --2- 172.21.15.94:0/164585271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c747d2e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f17b809bb80 tx=0x7f17b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c2e8415b3163ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.320+0000 7f17b7fff640 1 -- 172.21.15.94:0/164585271 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8014070 con 0x55d0c74541d0 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17b7fff640 1 -- 172.21.15.94:0/164585271 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17b809b040 con 0x55d0c74541d0 2024-09-19T04:14:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17b7fff640 1 -- 172.21.15.94:0/164585271 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8020050 con 0x55d0c74541d0 2024-09-19T04:14:34.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17d72e8740 1 -- 172.21.15.94:0/164585271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 msgr2=0x55d0c747d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17d72e8740 1 --2- 172.21.15.94:0/164585271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c747d2e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f17b809bb80 tx=0x7f17b8005880 comp rx=0 tx=0).stop 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17d72e8740 1 -- 172.21.15.94:0/164585271 shutdown_connections 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17d72e8740 1 --2- 172.21.15.94:0/164585271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c7462fc0 0x55d0c747d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17d72e8740 1 --2- 172.21.15.94:0/164585271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74541d0 0x55d0c747d2e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17d72e8740 1 --2- 172.21.15.94:0/164585271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74538b0 0x55d0c7475000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17d72e8740 1 -- 172.21.15.94:0/164585271 >> 172.21.15.94:0/164585271 conn(0x55d0c744ff60 msgr2=0x55d0c744ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17d72e8740 1 -- 172.21.15.94:0/164585271 shutdown_connections 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.321+0000 7f17d72e8740 1 -- 172.21.15.94:0/164585271 wait complete. 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17d72e8740 1 -- start start 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17d72e8740 1 Processor -- start 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74538b0 0x55d0c74767a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74541d0 0x55d0c7476ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17d72e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c7462fc0 0x55d0c74dc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17d72e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d0c7474e80 con 0x55d0c74538b0 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17d72e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d0c74747b0 con 0x55d0c74541d0 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17d72e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d0c7475550 con 0x55d0c7462fc0 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17d72e8740 10 client.0 _ll_register_callbacks cb 0x7f17b8009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17ce59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74538b0 0x55d0c74767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17ce59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74538b0 0x55d0c74767a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46400/0 (socket says 172.21.15.94:46400) 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17ce59c640 1 -- 172.21.15.94:0/180106738 learned_addr learned my addr 172.21.15.94:0/180106738 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17c7fff640 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74541d0 0x55d0c7476ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17ce59c640 1 -- 172.21.15.94:0/180106738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c7462fc0 msgr2=0x55d0c74dc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17ce59c640 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c7462fc0 0x55d0c74dc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17ce59c640 1 -- 172.21.15.94:0/180106738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74541d0 msgr2=0x55d0c7476ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17ce59c640 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74541d0 0x55d0c7476ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17ce59c640 1 -- 172.21.15.94:0/180106738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c74dc820 con 0x55d0c74538b0 2024-09-19T04:14:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17ce59c640 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74538b0 0x55d0c74767a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f17c0019000 tx=0x7f17c000bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a3415fcda715a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.322+0000 7f17b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17b5ffb640 1 -- 172.21.15.94:0/180106738 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0015040 con 0x55d0c74538b0 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17b5ffb640 1 -- 172.21.15.94:0/180106738 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f17c0004440 con 0x55d0c74538b0 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17d72e8740 1 -- 172.21.15.94:0/180106738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17b809b040 con 0x55d0c74538b0 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17b5ffb640 1 -- 172.21.15.94:0/180106738 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f17c0005040 con 0x55d0c74538b0 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17d72e8740 10 client.15021 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17d72e8740 1 -- 172.21.15.94:0/180106738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d0c74ddb20 con 0x55d0c74538b0 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17b5ffb640 1 -- 172.21.15.94:0/180106738 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17b809b040 con 0x55d0c74538b0 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17b5ffb640 1 -- 172.21.15.94:0/180106738 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f17c00035f0 con 0x55d0c74538b0 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17d72e8740 10 client.15021 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17d72e8740 10 client.15021 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17d72e8740 20 client.15021 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17d72e8740 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c7473b90 0x55d0c74e1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17d72e8740 4 client.15021 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.323+0000 7f17d72e8740 1 -- 172.21.15.94:0/180106738 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "status"}) -- 0x55d0c73c1b10 con 0x55d0c7473b90 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17c7fff640 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c7473b90 0x55d0c74e1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17b5ffb640 10 client.15021 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17c7fff640 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c7473b90 0x55d0c74e1330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1712f2f95e0ab9d server_cookie=d299df341db5898b in_seq=0 out_seq=0 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17b5ffb640 1 -- 172.21.15.94:0/180106738 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d0c73c1b10 con 0x55d0c7473b90 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17b5ffb640 10 client.15021 handle_command_reply: tid=0 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 client.15021 shutdown 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 -- 172.21.15.94:0/180106738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74538b0 msgr2=0x55d0c74767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74538b0 0x55d0c74767a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f17c0019000 tx=0x7f17c000bc30 comp rx=0 tx=0).stop 2024-09-19T04:14:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 -- 172.21.15.94:0/180106738 shutdown_connections 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c7473b90 0x55d0c74e1330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c7462fc0 0x55d0c74dc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c74541d0 0x55d0c7476ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17b5ffb640 0 client.15021 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 --2- 172.21.15.94:0/180106738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c74538b0 0x55d0c74767a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 -- 172.21.15.94:0/180106738 >> 172.21.15.94:0/180106738 conn(0x55d0c744ff60 msgr2=0x55d0c745aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 -- 172.21.15.94:0/180106738 shutdown_connections 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 1 -- 172.21.15.94:0/180106738 wait complete. 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.324+0000 7f17d72e8740 20 client.15021 trim_cache size 0 max 16384 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "503c4526-763c-11ef-bceb-c7b262605968", 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24467, 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 76.877161791000006, 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 87.521092805999999 2024-09-19T04:14:34.342 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:14:34.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 -- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f17bc0a32d0 msgr2=0x7f17bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f17bc0a32d0 0x7f17bc0a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f17c8005e30 tx=0x7f17c8009a20 comp rx=0 tx=0).stop 2024-09-19T04:14:34.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 -- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102570 msgr2=0x7f17d0198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102570 0x7f17d0198940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f17b800d950 tx=0x7f17b800de10 comp rx=0 tx=0).stop 2024-09-19T04:14:34.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 -- 172.21.15.94:0/3214176156 shutdown_connections 2024-09-19T04:14:34.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f17bc0a32d0 0x7f17bc0a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17d0102ef0 0x7f17d019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d0102570 0x7f17d0198940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 --2- 172.21.15.94:0/3214176156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17d0100e20 0x7f17d0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 -- 172.21.15.94:0/3214176156 >> 172.21.15.94:0/3214176156 conn(0x7f17d00fc7d0 msgr2=0x7f17d006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:34.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 -- 172.21.15.94:0/3214176156 shutdown_connections 2024-09-19T04:14:34.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:34.325+0000 7f17d4f0f640 1 -- 172.21.15.94:0/3214176156 wait complete. 2024-09-19T04:14:35.060 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T04:14:35.060 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T04:14:35.071 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T04:14:35.071 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T04:14:35.318 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:35.574 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:35 smithi094 ceph-mon[30972]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:35 smithi104 ceph-mon[34145]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:35.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:35 smithi112 ceph-mon[35051]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 -- 172.21.15.94:0/3015477801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 msgr2=0x7f856010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 --2- 172.21.15.94:0/3015477801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 0x7f856010a050 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8550009900 tx=0x7f855002a410 comp rx=0 tx=0).stop 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 -- 172.21.15.94:0/3015477801 shutdown_connections 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 --2- 172.21.15.94:0/3015477801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 0x7f856010a050 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 --2- 172.21.15.94:0/3015477801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8560102e00 0x7f8560103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 --2- 172.21.15.94:0/3015477801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101c00 0x7f8560102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 -- 172.21.15.94:0/3015477801 >> 172.21.15.94:0/3015477801 conn(0x7f85600fd390 msgr2=0x7f85600ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 -- 172.21.15.94:0/3015477801 shutdown_connections 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 -- 172.21.15.94:0/3015477801 wait complete. 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 Processor -- start 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.790+0000 7f856846f640 1 -- start start 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f856846f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101c00 0x7f8560197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f856846f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8560102e00 0x7f85601983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f856846f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 0x7f856019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f856846f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f856010d7b0 con 0x7f85601037c0 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f856846f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f856010d630 con 0x7f8560102e00 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f856846f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f856010d930 con 0x7f8560101c00 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8566a0c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 0x7f856019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8566a0c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 0x7f856019f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46420/0 (socket says 172.21.15.94:46420) 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8566a0c640 1 -- 172.21.15.94:0/707190905 learned_addr learned my addr 172.21.15.94:0/707190905 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f856620b640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101c00 0x7f8560197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8565a0a640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8560102e00 0x7f85601983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8566a0c640 1 -- 172.21.15.94:0/707190905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101c00 msgr2=0x7f8560197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8566a0c640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101c00 0x7f8560197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8566a0c640 1 -- 172.21.15.94:0/707190905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8560102e00 msgr2=0x7f85601983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8566a0c640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8560102e00 0x7f85601983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8566a0c640 1 -- 172.21.15.94:0/707190905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856019fa90 con 0x7f85601037c0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f8566a0c640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 0x7f856019f420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8550005940 tx=0x7f8550031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f855f7fe640 1 -- 172.21.15.94:0/707190905 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855003f030 con 0x7f85601037c0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856019fd20 con 0x7f85601037c0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.791+0000 7f855f7fe640 1 -- 172.21.15.94:0/707190905 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8550003aa0 con 0x7f85601037c0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.792+0000 7f855f7fe640 1 -- 172.21.15.94:0/707190905 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550035070 con 0x7f85601037c0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.792+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85601a0230 con 0x7f85601037c0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.793+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8560109050 con 0x7f85601037c0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.794+0000 7f855f7fe640 1 -- 172.21.15.94:0/707190905 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8550042030 con 0x7f85601037c0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.794+0000 7f855f7fe640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f85480a3300 0x7f85480a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.794+0000 7f856620b640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f85480a3300 0x7f85480a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.794+0000 7f855f7fe640 1 -- 172.21.15.94:0/707190905 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f85500effc0 con 0x7f85601037c0 2024-09-19T04:14:35.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.795+0000 7f856620b640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f85480a3300 0x7f85480a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8560102c60 tx=0x7f8558009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:35.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.798+0000 7f855f7fe640 1 -- 172.21.15.94:0/707190905 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f85500bcc40 con 0x7f85601037c0 2024-09-19T04:14:35.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.979+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f8560062630 con 0x7f85480a3300 2024-09-19T04:14:35.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.980+0000 7f855f7fe640 1 -- 172.21.15.94:0/707190905 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f8560062630 con 0x7f85480a3300 2024-09-19T04:14:35.982 INFO:teuthology.orchestra.run.smithi094.stdout:Backend: cephadm 2024-09-19T04:14:35.982 INFO:teuthology.orchestra.run.smithi094.stdout:Available: Yes 2024-09-19T04:14:35.982 INFO:teuthology.orchestra.run.smithi094.stdout:Paused: No 2024-09-19T04:14:35.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.982+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f85480a3300 msgr2=0x7f85480a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:35.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.982+0000 7f856846f640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f85480a3300 0x7f85480a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8560102c60 tx=0x7f8558009680 comp rx=0 tx=0).stop 2024-09-19T04:14:35.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.982+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 msgr2=0x7f856019f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:35.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.982+0000 7f856846f640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 0x7f856019f420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8550005940 tx=0x7f8550031a30 comp rx=0 tx=0).stop 2024-09-19T04:14:35.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.983+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 shutdown_connections 2024-09-19T04:14:35.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.983+0000 7f856846f640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f85480a3300 0x7f85480a57c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:35.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.983+0000 7f856846f640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85601037c0 0x7f856019f420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:35.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.983+0000 7f856846f640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8560102e00 0x7f85601983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:35.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.983+0000 7f856846f640 1 --2- 172.21.15.94:0/707190905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101c00 0x7f8560197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:35.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.983+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 >> 172.21.15.94:0/707190905 conn(0x7f85600fd390 msgr2=0x7f8560106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:35.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.983+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 shutdown_connections 2024-09-19T04:14:35.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:35.983+0000 7f856846f640 1 -- 172.21.15.94:0/707190905 wait complete. 2024-09-19T04:14:36.667 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T04:14:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:37.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.356+0000 7f081c390640 1 -- 172.21.15.94:0/2251925557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814102690 msgr2=0x7f0814108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:37.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.356+0000 7f081c390640 1 --2- 172.21.15.94:0/2251925557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814102690 0x7f0814108f20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f080c009ca0 tx=0x7f080c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.356+0000 7f081c390640 1 -- 172.21.15.94:0/2251925557 shutdown_connections 2024-09-19T04:14:37.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.356+0000 7f081c390640 1 --2- 172.21.15.94:0/2251925557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814102690 0x7f0814108f20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.356+0000 7f081c390640 1 --2- 172.21.15.94:0/2251925557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0814101cd0 0x7f0814102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.356+0000 7f081c390640 1 --2- 172.21.15.94:0/2251925557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814100ad0 0x7f0814100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.356+0000 7f081c390640 1 -- 172.21.15.94:0/2251925557 >> 172.21.15.94:0/2251925557 conn(0x7f08140fc280 msgr2=0x7f08140fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:37.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.356+0000 7f081c390640 1 -- 172.21.15.94:0/2251925557 shutdown_connections 2024-09-19T04:14:37.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.356+0000 7f081c390640 1 -- 172.21.15.94:0/2251925557 wait complete. 2024-09-19T04:14:37.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081c390640 1 Processor -- start 2024-09-19T04:14:37.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081c390640 1 -- start start 2024-09-19T04:14:37.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081c390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814100ad0 0x7f0814199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:37.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081c390640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814101cd0 0x7f081419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:37.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081c390640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0814102690 0x7f08141a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:37.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081c390640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f081410c600 con 0x7f0814100ad0 2024-09-19T04:14:37.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081c390640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f081410c480 con 0x7f0814101cd0 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081c390640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f081410c780 con 0x7f0814102690 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081a12c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814100ad0 0x7f0814199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081a12c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814100ad0 0x7f0814199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46434/0 (socket says 172.21.15.94:46434) 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081a12c640 1 -- 172.21.15.94:0/3855196872 learned_addr learned my addr 172.21.15.94:0/3855196872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081992b640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814101cd0 0x7f081419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081a92d640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0814102690 0x7f08141a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081a12c640 1 -- 172.21.15.94:0/3855196872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0814102690 msgr2=0x7f08141a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081a12c640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0814102690 0x7f08141a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081a12c640 1 -- 172.21.15.94:0/3855196872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814101cd0 msgr2=0x7f081419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081a12c640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814101cd0 0x7f081419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.357+0000 7f081a12c640 1 -- 172.21.15.94:0/3855196872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08141a1b80 con 0x7f0814100ad0 2024-09-19T04:14:37.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.358+0000 7f081a12c640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814100ad0 0x7f0814199f50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f081000c970 tx=0x7f081000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:37.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.358+0000 7f080b7fe640 1 -- 172.21.15.94:0/3855196872 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810007c10 con 0x7f0814100ad0 2024-09-19T04:14:37.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.358+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08141a1e70 con 0x7f0814100ad0 2024-09-19T04:14:37.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.358+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08141a2320 con 0x7f0814100ad0 2024-09-19T04:14:37.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.358+0000 7f080b7fe640 1 -- 172.21.15.94:0/3855196872 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0810007db0 con 0x7f0814100ad0 2024-09-19T04:14:37.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.358+0000 7f080b7fe640 1 -- 172.21.15.94:0/3855196872 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081000f870 con 0x7f0814100ad0 2024-09-19T04:14:37.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.359+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f8005290 con 0x7f0814100ad0 2024-09-19T04:14:37.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.360+0000 7f080b7fe640 1 -- 172.21.15.94:0/3855196872 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f081000fa70 con 0x7f0814100ad0 2024-09-19T04:14:37.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.361+0000 7f080b7fe640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08000a3380 0x7f08000a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:37.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.361+0000 7f081992b640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08000a3380 0x7f08000a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:37.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.361+0000 7f080b7fe640 1 -- 172.21.15.94:0/3855196872 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f08100cffc0 con 0x7f0814100ad0 2024-09-19T04:14:37.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.361+0000 7f081992b640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08000a3380 0x7f08000a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f081419b470 tx=0x7f080400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:37.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.364+0000 7f080b7fe640 1 -- 172.21.15.94:0/3855196872 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f081009cca0 con 0x7f0814100ad0 2024-09-19T04:14:37.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.542+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f07f8000bc0 con 0x7f08000a3380 2024-09-19T04:14:37.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.554+0000 7f080b7fe640 1 -- 172.21.15.94:0/3855196872 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f07f8000bc0 con 0x7f08000a3380 2024-09-19T04:14:37.556 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T04:14:37.556 INFO:teuthology.orchestra.run.smithi094.stdout:mds.a smithi094 running (93s) 16s ago 93s 14.0M - 19.1.1-361-g2f599a46 d2baa855c067 f2db2a61c56b 2024-09-19T04:14:37.556 INFO:teuthology.orchestra.run.smithi094.stdout:mds.b smithi104 running (91s) 23s ago 90s 25.6M - 19.1.1-361-g2f599a46 d2baa855c067 707f082b4b8f 2024-09-19T04:14:37.556 INFO:teuthology.orchestra.run.smithi094.stdout:mds.c smithi112 running (105s) 23s ago 104s 19.7M - 19.1.1-361-g2f599a46 d2baa855c067 fc3ebc87679e 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mds.d smithi094 running (112s) 16s ago 112s 16.8M - 19.1.1-361-g2f599a46 d2baa855c067 094be904acb4 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mds.e smithi104 running (110s) 23s ago 109s 19.2M - 19.1.1-361-g2f599a46 d2baa855c067 91311a4e8208 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mds.f smithi112 running (107s) 23s ago 107s 18.0M - 19.1.1-361-g2f599a46 d2baa855c067 8f1c7824f25b 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mds.g smithi094 running (97s) 16s ago 96s 15.7M - 19.1.1-361-g2f599a46 d2baa855c067 d640449f7961 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mds.h smithi104 running (102s) 23s ago 102s 18.5M - 19.1.1-361-g2f599a46 d2baa855c067 fd7a16416277 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mds.i smithi112 running (100s) 23s ago 99s 19.7M - 19.1.1-361-g2f599a46 d2baa855c067 00741c13c111 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mds.j smithi094 running (116s) 16s ago 115s 14.2M - 19.1.1-361-g2f599a46 d2baa855c067 8d883c184a16 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mds.k smithi104 running (119s) 23s ago 118s 16.0M - 19.1.1-361-g2f599a46 d2baa855c067 5bd23669d672 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mds.l smithi112 running (2m) 23s ago 2m 15.9M - 19.1.1-361-g2f599a46 d2baa855c067 c9740431c0df 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.x smithi094 *:9283,8765 running (8m) 16s ago 8m 508M - 19.1.1-361-g2f599a46 d2baa855c067 d829d7ba3425 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.y smithi104 *:8443,8765 running (7m) 23s ago 7m 442M - 19.1.1-361-g2f599a46 d2baa855c067 56905ac5daf2 2024-09-19T04:14:37.557 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.z smithi112 *:8443,8765 running (7m) 23s ago 7m 444M - 19.1.1-361-g2f599a46 d2baa855c067 6a914845d4b9 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a smithi094 running (8m) 16s ago 9m 51.0M 2048M 19.1.1-361-g2f599a46 d2baa855c067 cf9c5201f285 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b smithi104 running (7m) 23s ago 7m 38.7M 2048M 19.1.1-361-g2f599a46 d2baa855c067 c97384baf7fc 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c smithi112 running (7m) 23s ago 7m 40.0M 2048M 19.1.1-361-g2f599a46 d2baa855c067 5450f6bd98af 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (6m) 16s ago 6m 56.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 0b165cdf9f58 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi104 running (6m) 23s ago 6m 56.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 b2be2720d26a 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi112 running (6m) 23s ago 6m 49.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 60bb56813460 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi094 running (5m) 16s ago 5m 47.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 17bd7afc89b2 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi104 running (5m) 23s ago 5m 53.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 8eabfde5842e 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi112 running (4m) 23s ago 4m 54.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 ea4cf25ca7c8 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 smithi094 running (4m) 16s ago 4m 51.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 7b2feba17e08 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 smithi104 running (3m) 23s ago 3m 46.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 be2e02635f58 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 smithi112 running (3m) 23s ago 3m 48.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 f8fc7241ac75 2024-09-19T04:14:37.558 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 smithi094 running (3m) 16s ago 3m 51.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 f6265443b2db 2024-09-19T04:14:37.559 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 smithi104 running (2m) 23s ago 2m 52.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 0a908449f746 2024-09-19T04:14:37.559 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 smithi112 running (2m) 23s ago 2m 50.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c581fb000b95 2024-09-19T04:14:37.559 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:37 smithi094 ceph-mon[30972]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:37.559 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:37 smithi094 ceph-mon[30972]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08000a3380 msgr2=0x7f08000a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08000a3380 0x7f08000a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f081419b470 tx=0x7f080400b390 comp rx=0 tx=0).stop 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814100ad0 msgr2=0x7f0814199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814100ad0 0x7f0814199f50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f081000c970 tx=0x7f081000ce30 comp rx=0 tx=0).stop 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 shutdown_connections 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f08000a3380 0x7f08000a5840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0814102690 0x7f08141a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814101cd0 0x7f081419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 --2- 172.21.15.94:0/3855196872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0814100ad0 0x7f0814199f50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 >> 172.21.15.94:0/3855196872 conn(0x7f08140fc280 msgr2=0x7f08140fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:37.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 shutdown_connections 2024-09-19T04:14:37.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:37.557+0000 7f081c390640 1 -- 172.21.15.94:0/3855196872 wait complete. 2024-09-19T04:14:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:37 smithi104 ceph-mon[34145]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:37 smithi104 ceph-mon[34145]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:37.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:37 smithi112 ceph-mon[35051]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:37.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:37 smithi112 ceph-mon[35051]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:38.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T04:14:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.879+0000 7fbf6865d640 1 -- 172.21.15.94:0/4100670700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf60101c10 msgr2=0x7fbf60102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.879+0000 7fbf6865d640 1 --2- 172.21.15.94:0/4100670700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf60101c10 0x7fbf60102010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009900 tx=0x7fbf5c02a410 comp rx=0 tx=0).stop 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 -- 172.21.15.94:0/4100670700 shutdown_connections 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 --2- 172.21.15.94:0/4100670700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf601037d0 0x7fbf6010a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 --2- 172.21.15.94:0/4100670700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf60102e10 0x7fbf60103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 --2- 172.21.15.94:0/4100670700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf60101c10 0x7fbf60102010 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 -- 172.21.15.94:0/4100670700 >> 172.21.15.94:0/4100670700 conn(0x7fbf600fd3c0 msgr2=0x7fbf600ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 -- 172.21.15.94:0/4100670700 shutdown_connections 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 -- 172.21.15.94:0/4100670700 wait complete. 2024-09-19T04:14:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 Processor -- start 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 -- start start 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf60101c10 0x7fbf60197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf60102e10 0x7fbf601984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf601037d0 0x7fbf6019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6010d820 con 0x7fbf601037d0 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6010d6a0 con 0x7fbf60102e10 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.880+0000 7fbf6865d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6010d9a0 con 0x7fbf60101c10 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf66bfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf601037d0 0x7fbf6019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf66bfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf601037d0 0x7fbf6019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46448/0 (socket says 172.21.15.94:46448) 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf66bfa640 1 -- 172.21.15.94:0/2528820688 learned_addr learned my addr 172.21.15.94:0/2528820688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf663f9640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf60101c10 0x7fbf60197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf65bf8640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf60102e10 0x7fbf601984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf66bfa640 1 -- 172.21.15.94:0/2528820688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf60101c10 msgr2=0x7fbf60197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf66bfa640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf60101c10 0x7fbf60197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf66bfa640 1 -- 172.21.15.94:0/2528820688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf60102e10 msgr2=0x7fbf601984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf66bfa640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf60102e10 0x7fbf601984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf66bfa640 1 -- 172.21.15.94:0/2528820688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6019fbe0 con 0x7fbf601037d0 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf66bfa640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf601037d0 0x7fbf6019f570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b970 tx=0x7fbf5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf577fe640 1 -- 172.21.15.94:0/2528820688 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58019070 con 0x7fbf601037d0 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6019fed0 con 0x7fbf601037d0 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf577fe640 1 -- 172.21.15.94:0/2528820688 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf58014440 con 0x7fbf601037d0 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf577fe640 1 -- 172.21.15.94:0/2528820688 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf580075d0 con 0x7fbf601037d0 2024-09-19T04:14:38.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.881+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf601a03b0 con 0x7fbf601037d0 2024-09-19T04:14:38.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.882+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf60109150 con 0x7fbf601037d0 2024-09-19T04:14:38.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.883+0000 7fbf577fe640 1 -- 172.21.15.94:0/2528820688 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbf5801c070 con 0x7fbf601037d0 2024-09-19T04:14:38.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.884+0000 7fbf577fe640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbf4c0a3300 0x7fbf4c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:38.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.884+0000 7fbf663f9640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbf4c0a3300 0x7fbf4c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:38.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.884+0000 7fbf577fe640 1 -- 172.21.15.94:0/2528820688 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbf580cef00 con 0x7fbf601037d0 2024-09-19T04:14:38.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.884+0000 7fbf663f9640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbf4c0a3300 0x7fbf4c0a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c006010 tx=0x7fbf5c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:38.888+0000 7fbf577fe640 1 -- 172.21.15.94:0/2528820688 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbf5809b580 con 0x7fbf601037d0 2024-09-19T04:14:39.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.061+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fbf60062630 con 0x7fbf4c0a3300 2024-09-19T04:14:39.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.067+0000 7fbf577fe640 1 -- 172.21.15.94:0/2528820688 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fbf60062630 con 0x7fbf4c0a3300 2024-09-19T04:14:39.068 INFO:teuthology.orchestra.run.smithi094.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T04:14:39.068 INFO:teuthology.orchestra.run.smithi094.stdout:mds.all 12/12 25s ago 2m smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l;count:12 2024-09-19T04:14:39.069 INFO:teuthology.orchestra.run.smithi094.stdout:mgr 3/3 25s ago 7m smithi094=x;smithi104=y;smithi112=z;count:3 2024-09-19T04:14:39.069 INFO:teuthology.orchestra.run.smithi094.stdout:mon 3/3 25s ago 7m smithi094:172.21.15.94=a;smithi104:172.21.15.104=b;smithi112:172.21.15.112=c;count:3 2024-09-19T04:14:39.069 INFO:teuthology.orchestra.run.smithi094.stdout:osd 12 25s ago - 2024-09-19T04:14:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbf4c0a3300 msgr2=0x7fbf4c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbf4c0a3300 0x7fbf4c0a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c006010 tx=0x7fbf5c000bc0 comp rx=0 tx=0).stop 2024-09-19T04:14:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf601037d0 msgr2=0x7fbf6019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf601037d0 0x7fbf6019f570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b970 tx=0x7fbf5800be30 comp rx=0 tx=0).stop 2024-09-19T04:14:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 shutdown_connections 2024-09-19T04:14:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fbf4c0a3300 0x7fbf4c0a57c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf601037d0 0x7fbf6019f570 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf60102e10 0x7fbf601984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 --2- 172.21.15.94:0/2528820688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf60101c10 0x7fbf60197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 >> 172.21.15.94:0/2528820688 conn(0x7fbf600fd3c0 msgr2=0x7fbf60106320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 shutdown_connections 2024-09-19T04:14:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:39.069+0000 7fbf6865d640 1 -- 172.21.15.94:0/2528820688 wait complete. 2024-09-19T04:14:39.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:39 smithi094 ceph-mon[30972]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:39.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:39 smithi094 ceph-mon[30972]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:39 smithi104 ceph-mon[34145]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:39 smithi104 ceph-mon[34145]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:39.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T04:14:39.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:39 smithi112 ceph-mon[35051]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:39 smithi112 ceph-mon[35051]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:40.002 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:40.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:40 smithi094 ceph-mon[30972]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:40.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 -- 172.21.15.94:0/2122809015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0941037c0 msgr2=0x7fa09410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 --2- 172.21.15.94:0/2122809015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0941037c0 0x7fa09410a050 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009ca0 tx=0x7fa08c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 -- 172.21.15.94:0/2122809015 shutdown_connections 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 --2- 172.21.15.94:0/2122809015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0941037c0 0x7fa09410a050 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 --2- 172.21.15.94:0/2122809015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa094102e00 0x7fa094103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 --2- 172.21.15.94:0/2122809015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa094101c00 0x7fa094102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 -- 172.21.15.94:0/2122809015 >> 172.21.15.94:0/2122809015 conn(0x7fa0940fd390 msgr2=0x7fa0940ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 -- 172.21.15.94:0/2122809015 shutdown_connections 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 -- 172.21.15.94:0/2122809015 wait complete. 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 Processor -- start 2024-09-19T04:14:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.488+0000 7fa099942640 1 -- start start 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa099942640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa094101c00 0x7fa094075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa099942640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa094102e00 0x7fa0940745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa099942640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0941037c0 0x7fa094074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa099942640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa09410d730 con 0x7fa094102e00 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa099942640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa09410d5b0 con 0x7fa094101c00 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa099942640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa09410d8b0 con 0x7fa0941037c0 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa094102e00 0x7fa0940745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa094102e00 0x7fa0940745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46474/0 (socket says 172.21.15.94:46474) 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0927fc640 1 -- 172.21.15.94:0/2585493420 learned_addr learned my addr 172.21.15.94:0/2585493420 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa092ffd640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa094101c00 0x7fa094075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0937fe640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0941037c0 0x7fa094074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0927fc640 1 -- 172.21.15.94:0/2585493420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0941037c0 msgr2=0x7fa094074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0927fc640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0941037c0 0x7fa094074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0927fc640 1 -- 172.21.15.94:0/2585493420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa094101c00 msgr2=0x7fa094075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0927fc640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa094101c00 0x7fa094075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0927fc640 1 -- 172.21.15.94:0/2585493420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0940753e0 con 0x7fa094102e00 2024-09-19T04:14:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa0927fc640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa094102e00 0x7fa0940745e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa08800d950 tx=0x7fa08800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa098940640 1 -- 172.21.15.94:0/2585493420 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088017070 con 0x7fa094102e00 2024-09-19T04:14:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa098940640 1 -- 172.21.15.94:0/2585493420 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa088009c30 con 0x7fa094102e00 2024-09-19T04:14:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0941a1f70 con 0x7fa094102e00 2024-09-19T04:14:40.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa098940640 1 -- 172.21.15.94:0/2585493420 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08800f620 con 0x7fa094102e00 2024-09-19T04:14:40.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.489+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0941a23d0 con 0x7fa094102e00 2024-09-19T04:14:40.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.490+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa094109050 con 0x7fa094102e00 2024-09-19T04:14:40.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.492+0000 7fa098940640 1 -- 172.21.15.94:0/2585493420 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa0880109a0 con 0x7fa094102e00 2024-09-19T04:14:40.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.492+0000 7fa098940640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa07c0a32d0 0x7fa07c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:40.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.492+0000 7fa092ffd640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa07c0a32d0 0x7fa07c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:40.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.492+0000 7fa098940640 1 -- 172.21.15.94:0/2585493420 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa0880cf670 con 0x7fa094102e00 2024-09-19T04:14:40.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.493+0000 7fa092ffd640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa07c0a32d0 0x7fa07c0a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa0840058c0 tx=0x7fa08400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:40.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.495+0000 7fa098940640 1 -- 172.21.15.94:0/2585493420 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa088009de0 con 0x7fa094102e00 2024-09-19T04:14:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:40 smithi104 ceph-mon[34145]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.669+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fa094062630 con 0x7fa07c0a32d0 2024-09-19T04:14:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.671+0000 7fa098940640 1 -- 172.21.15.94:0/2585493420 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fa094062630 con 0x7fa07c0a32d0 2024-09-19T04:14:40.672 INFO:teuthology.orchestra.run.smithi094.stdout:HOST ADDR LABELS STATUS 2024-09-19T04:14:40.672 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 172.21.15.94 2024-09-19T04:14:40.673 INFO:teuthology.orchestra.run.smithi094.stdout:smithi104 172.21.15.104 2024-09-19T04:14:40.673 INFO:teuthology.orchestra.run.smithi094.stdout:smithi112 172.21.15.112 2024-09-19T04:14:40.673 INFO:teuthology.orchestra.run.smithi094.stdout:3 hosts in cluster 2024-09-19T04:14:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa07c0a32d0 msgr2=0x7fa07c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa07c0a32d0 0x7fa07c0a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa0840058c0 tx=0x7fa08400a2d0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa094102e00 msgr2=0x7fa0940745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa094102e00 0x7fa0940745e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa08800d950 tx=0x7fa08800de10 comp rx=0 tx=0).stop 2024-09-19T04:14:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 shutdown_connections 2024-09-19T04:14:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fa07c0a32d0 0x7fa07c0a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0941037c0 0x7fa094074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa094102e00 0x7fa0940745e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 --2- 172.21.15.94:0/2585493420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa094101c00 0x7fa094075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 >> 172.21.15.94:0/2585493420 conn(0x7fa0940fd390 msgr2=0x7fa094106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 shutdown_connections 2024-09-19T04:14:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:40.673+0000 7fa099942640 1 -- 172.21.15.94:0/2585493420 wait complete. 2024-09-19T04:14:40.686 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:40 smithi112 ceph-mon[35051]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:41.290 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T04:14:41.536 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:41.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:41 smithi094 ceph-mon[30972]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:41 smithi104 ceph-mon[34145]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:41.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:41 smithi112 ceph-mon[35051]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.978+0000 7f46186d5640 1 -- 172.21.15.94:0/534861654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610107960 msgr2=0x7f4610109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.978+0000 7f46186d5640 1 --2- 172.21.15.94:0/534861654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610107960 0x7f4610109d50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4608009ca0 tx=0x7f460802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.978+0000 7f46186d5640 1 -- 172.21.15.94:0/534861654 shutdown_connections 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.978+0000 7f46186d5640 1 --2- 172.21.15.94:0/534861654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610107960 0x7f4610109d50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.978+0000 7f46186d5640 1 --2- 172.21.15.94:0/534861654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610105030 0x7f4610107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.978+0000 7f46186d5640 1 --2- 172.21.15.94:0/534861654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4610102700 0x7f4610104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.978+0000 7f46186d5640 1 -- 172.21.15.94:0/534861654 >> 172.21.15.94:0/534861654 conn(0x7f46100fc280 msgr2=0x7f46100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.978+0000 7f46186d5640 1 -- 172.21.15.94:0/534861654 shutdown_connections 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.978+0000 7f46186d5640 1 -- 172.21.15.94:0/534861654 wait complete. 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f46186d5640 1 Processor -- start 2024-09-19T04:14:41.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f46186d5640 1 -- start start 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f46186d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610102700 0x7f4610102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f46186d5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4610105030 0x7f46101006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f46186d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610107960 0x7f4610100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f46186d5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f461010d730 con 0x7f4610102700 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f46186d5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f461010d5b0 con 0x7f4610105030 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f46186d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f461010d8b0 con 0x7f4610107960 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616471640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610102700 0x7f4610102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616471640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610102700 0x7f4610102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57274/0 (socket says 172.21.15.94:57274) 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616471640 1 -- 172.21.15.94:0/1813303327 learned_addr learned my addr 172.21.15.94:0/1813303327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4615c70640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4610105030 0x7f46101006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616c72640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610107960 0x7f4610100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616471640 1 -- 172.21.15.94:0/1813303327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610107960 msgr2=0x7f4610100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616471640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610107960 0x7f4610100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616471640 1 -- 172.21.15.94:0/1813303327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4610105030 msgr2=0x7f46101006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616471640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4610105030 0x7f46101006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616471640 1 -- 172.21.15.94:0/1813303327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46101014b0 con 0x7f4610102700 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616c72640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610107960 0x7f4610100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.979+0000 7f4616471640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610102700 0x7f4610102060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f460c00e9f0 tx=0x7f460c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.980+0000 7f46077fe640 1 -- 172.21.15.94:0/1813303327 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c00cd80 con 0x7f4610102700 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.980+0000 7f46077fe640 1 -- 172.21.15.94:0/1813303327 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f460c0108e0 con 0x7f4610102700 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.980+0000 7f46077fe640 1 -- 172.21.15.94:0/1813303327 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c018920 con 0x7f4610102700 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.980+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46101a1f70 con 0x7f4610102700 2024-09-19T04:14:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.980+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46101a2360 con 0x7f4610102700 2024-09-19T04:14:41.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.981+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46100fdd70 con 0x7f4610102700 2024-09-19T04:14:41.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.982+0000 7f46077fe640 1 -- 172.21.15.94:0/1813303327 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f460c010a90 con 0x7f4610102700 2024-09-19T04:14:41.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.983+0000 7f46077fe640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45fc0a32d0 0x7f45fc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:41.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.983+0000 7f4615c70640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45fc0a32d0 0x7f45fc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:41.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.983+0000 7f46077fe640 1 -- 172.21.15.94:0/1813303327 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f460c0cfed0 con 0x7f4610102700 2024-09-19T04:14:41.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.983+0000 7f4615c70640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45fc0a32d0 0x7f45fc0a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4600005fd0 tx=0x7f4600005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:41.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:41.987+0000 7f46077fe640 1 -- 172.21.15.94:0/1813303327 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f460c09ca90 con 0x7f4610102700 2024-09-19T04:14:42.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.158+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 --> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4610062630 con 0x7f45fc0a32d0 2024-09-19T04:14:42.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.160+0000 7f46077fe640 1 -- 172.21.15.94:0/1813303327 <== mgr.14150 v2:172.21.15.94:6800/2308569937 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4610062630 con 0x7f45fc0a32d0 2024-09-19T04:14:42.161 INFO:teuthology.orchestra.run.smithi094.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T04:14:42.161 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-19T04:14:42.161 INFO:teuthology.orchestra.run.smithi094.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T04:14:42.162 INFO:teuthology.orchestra.run.smithi094.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45fc0a32d0 msgr2=0x7f45fc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45fc0a32d0 0x7f45fc0a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4600005fd0 tx=0x7f4600005850 comp rx=0 tx=0).stop 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610102700 msgr2=0x7f4610102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610102700 0x7f4610102060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f460c00e9f0 tx=0x7f460c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 shutdown_connections 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45fc0a32d0 0x7f45fc0a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610107960 0x7f4610100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4610105030 0x7f46101006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 --2- 172.21.15.94:0/1813303327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4610102700 0x7f4610102060 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 >> 172.21.15.94:0/1813303327 conn(0x7f46100fc280 msgr2=0x7f4610103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 shutdown_connections 2024-09-19T04:14:42.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:42.162+0000 7f46186d5640 1 -- 172.21.15.94:0/1813303327 wait complete. 2024-09-19T04:14:42.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:42 smithi094 ceph-mon[30972]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:42.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:42 smithi104 ceph-mon[34145]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:42.761 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T04:14:42.770 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T04:14:42.770 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T04:14:42.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:42 smithi112 ceph-mon[35051]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:43.015 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:43.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:43 smithi094 ceph-mon[30972]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:43.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:43 smithi094 ceph-mon[30972]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.450+0000 7fb02dc40640 1 -- 172.21.15.94:0/686412470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb028101d10 msgr2=0x7fb028102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.450+0000 7fb02dc40640 1 --2- 172.21.15.94:0/686412470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb028101d10 0x7fb028102190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb018009900 tx=0x7fb01802a410 comp rx=0 tx=0).stop 2024-09-19T04:14:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.450+0000 7fb02dc40640 1 -- 172.21.15.94:0/686412470 shutdown_connections 2024-09-19T04:14:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.450+0000 7fb02dc40640 1 --2- 172.21.15.94:0/686412470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0281026d0 0x7fb028108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.450+0000 7fb02dc40640 1 --2- 172.21.15.94:0/686412470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb028101d10 0x7fb028102190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.450+0000 7fb02dc40640 1 --2- 172.21.15.94:0/686412470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028100b10 0x7fb028100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.450+0000 7fb02dc40640 1 -- 172.21.15.94:0/686412470 >> 172.21.15.94:0/686412470 conn(0x7fb0280fc2a0 msgr2=0x7fb0280fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.450+0000 7fb02dc40640 1 -- 172.21.15.94:0/686412470 shutdown_connections 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.450+0000 7fb02dc40640 1 -- 172.21.15.94:0/686412470 wait complete. 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.451+0000 7fb02dc40640 1 Processor -- start 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.451+0000 7fb02dc40640 1 -- start start 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.451+0000 7fb02dc40640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028100b10 0x7fb02806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.451+0000 7fb02dc40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb028101d10 0x7fb02819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.451+0000 7fb02dc40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0281026d0 0x7fb02819f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.451+0000 7fb02dc40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb02810c6a0 con 0x7fb0281026d0 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.451+0000 7fb02dc40640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb02810c520 con 0x7fb028100b10 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.451+0000 7fb02dc40640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb02810c820 con 0x7fb028101d10 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.451+0000 7fb027fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0281026d0 0x7fb02819f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb027fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0281026d0 0x7fb02819f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57292/0 (socket says 172.21.15.94:57292) 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb027fff640 1 -- 172.21.15.94:0/4288493405 learned_addr learned my addr 172.21.15.94:0/4288493405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb0277fe640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028100b10 0x7fb02806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb026ffd640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb028101d10 0x7fb02819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb027fff640 1 -- 172.21.15.94:0/4288493405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb028101d10 msgr2=0x7fb02819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb027fff640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb028101d10 0x7fb02819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb027fff640 1 -- 172.21.15.94:0/4288493405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028100b10 msgr2=0x7fb02806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb027fff640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028100b10 0x7fb02806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb027fff640 1 -- 172.21.15.94:0/4288493405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02819fb10 con 0x7fb0281026d0 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb027fff640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0281026d0 0x7fb02819f200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb02000b970 tx=0x7fb02000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb024ff9640 1 -- 172.21.15.94:0/4288493405 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020019070 con 0x7fb0281026d0 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02819fda0 con 0x7fb0281026d0 2024-09-19T04:14:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb024ff9640 1 -- 172.21.15.94:0/4288493405 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb020014440 con 0x7fb0281026d0 2024-09-19T04:14:43.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb024ff9640 1 -- 172.21.15.94:0/4288493405 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200075d0 con 0x7fb0281026d0 2024-09-19T04:14:43.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.452+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0281a02e0 con 0x7fb0281026d0 2024-09-19T04:14:43.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.453+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb028107f50 con 0x7fb0281026d0 2024-09-19T04:14:43.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.454+0000 7fb024ff9640 1 -- 172.21.15.94:0/4288493405 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb02001c070 con 0x7fb0281026d0 2024-09-19T04:14:43.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.455+0000 7fb024ff9640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb0100a32d0 0x7fb0100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:43.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.455+0000 7fb0277fe640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb0100a32d0 0x7fb0100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:43.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.455+0000 7fb024ff9640 1 -- 172.21.15.94:0/4288493405 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb0200cef20 con 0x7fb0281026d0 2024-09-19T04:14:43.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.456+0000 7fb0277fe640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb0100a32d0 0x7fb0100a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb028101b70 tx=0x7fb01c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:43.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.459+0000 7fb024ff9640 1 -- 172.21.15.94:0/4288493405 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb02009b5a0 con 0x7fb0281026d0 2024-09-19T04:14:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:43 smithi104 ceph-mon[34145]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:43 smithi104 ceph-mon[34145]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:43.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.664+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb02802da10 con 0x7fb0281026d0 2024-09-19T04:14:43.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.664+0000 7fb024ff9640 1 -- 172.21.15.94:0/4288493405 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3089 (secure 0 0 0) 0x7fb02009f8a0 con 0x7fb0281026d0 2024-09-19T04:14:43.666 INFO:teuthology.orchestra.run.smithi094.stdout:e24 2024-09-19T04:14:43.666 INFO:teuthology.orchestra.run.smithi094.stdout:btime 2024-09-19T04:13:28:480286+0000 2024-09-19T04:14:43.666 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 24 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-09-19T04:13:17.025380+0000 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-09-19T04:13:28.166880+0000 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-09-19T04:14:43.667 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 300 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 3 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:in 0,1,2 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24467,1=24466,2=24454} 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-09-19T04:14:43.668 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 0 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 24467 members: 24454,24467,24466 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b{0:24467} state up:active seq 4 addr [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.f{0:24448} state up:standby-replay seq 1 addr [v2:172.21.15.112:6834/1098195473,v1:172.21.15.112:6835/1098195473] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.i{1:24466} state up:active seq 8 addr [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.h{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.104:6836/2682504501,v1:172.21.15.104:6837/2682504501] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c{2:24454} state up:active seq 10 addr [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.e{2:24431} state up:standby-replay seq 1 addr [v2:172.21.15.104:6834/2064839900,v1:172.21.15.104:6835/2064839900] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.94:6834/422186833,v1:172.21.15.94:6835/422186833] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.94:6836/3764912957,v1:172.21.15.94:6837/3764912957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.g{-1:14607} state up:standby seq 1 addr [v2:172.21.15.94:6838/4157648434,v1:172.21.15.94:6839/4157648434] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.670 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a{-1:14619} state up:standby seq 1 addr [v2:172.21.15.94:6840/44219912,v1:172.21.15.94:6841/44219912] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.671 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.k{-1:24404} state up:standby seq 1 addr [v2:172.21.15.104:6832/1061074349,v1:172.21.15.104:6833/1061074349] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.671 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.112:6832/1795597710,v1:172.21.15.112:6833/1795597710] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:14:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-09-19T04:14:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb0100a32d0 msgr2=0x7fb0100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb0100a32d0 0x7fb0100a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb028101b70 tx=0x7fb01c00a2d0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0281026d0 msgr2=0x7fb02819f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0281026d0 0x7fb02819f200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb02000b970 tx=0x7fb02000be30 comp rx=0 tx=0).stop 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 shutdown_connections 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fb0100a32d0 0x7fb0100a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0281026d0 0x7fb02819f200 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb028101d10 0x7fb02819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 --2- 172.21.15.94:0/4288493405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028100b10 0x7fb02806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 >> 172.21.15.94:0/4288493405 conn(0x7fb0280fc2a0 msgr2=0x7fb028105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 shutdown_connections 2024-09-19T04:14:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:43.667+0000 7fb02dc40640 1 -- 172.21.15.94:0/4288493405 wait complete. 2024-09-19T04:14:43.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:43 smithi112 ceph-mon[35051]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:43.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:43 smithi112 ceph-mon[35051]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:44.284 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T04:14:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:44 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4288493405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:14:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:44 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4288493405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:14:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:44 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4288493405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:14:45.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 -- 172.21.15.94:0/1650089310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628101c00 msgr2=0x7f0628102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:45.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 --2- 172.21.15.94:0/1650089310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628101c00 0x7f0628102000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0618009900 tx=0x7f061802a410 comp rx=0 tx=0).stop 2024-09-19T04:14:45.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 -- 172.21.15.94:0/1650089310 shutdown_connections 2024-09-19T04:14:45.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 --2- 172.21.15.94:0/1650089310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06281037c0 0x7f062810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 --2- 172.21.15.94:0/1650089310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0628102e00 0x7f0628103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 --2- 172.21.15.94:0/1650089310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628101c00 0x7f0628102000 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 -- 172.21.15.94:0/1650089310 >> 172.21.15.94:0/1650089310 conn(0x7f06280fd390 msgr2=0x7f06280ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 -- 172.21.15.94:0/1650089310 shutdown_connections 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 -- 172.21.15.94:0/1650089310 wait complete. 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.991+0000 7f062db92640 1 Processor -- start 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f062db92640 1 -- start start 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f062db92640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0628101c00 0x7f0628197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f062db92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628102e00 0x7f06281983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f062db92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06281037c0 0x7f062819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f062db92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f062810d790 con 0x7f0628102e00 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f062db92640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f062810d610 con 0x7f0628101c00 2024-09-19T04:14:45.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f062db92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f062810d910 con 0x7f06281037c0 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0626ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628102e00 0x7f06281983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0626ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628102e00 0x7f06281983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57310/0 (socket says 172.21.15.94:57310) 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0626ffd640 1 -- 172.21.15.94:0/380269626 learned_addr learned my addr 172.21.15.94:0/380269626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0627fff640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06281037c0 0x7f062819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f06277fe640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0628101c00 0x7f0628197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0626ffd640 1 -- 172.21.15.94:0/380269626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06281037c0 msgr2=0x7f062819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0626ffd640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06281037c0 0x7f062819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0626ffd640 1 -- 172.21.15.94:0/380269626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0628101c00 msgr2=0x7f0628197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0626ffd640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0628101c00 0x7f0628197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0626ffd640 1 -- 172.21.15.94:0/380269626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062819faa0 con 0x7f0628102e00 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0626ffd640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628102e00 0x7f06281983b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b4a0 tx=0x7f061c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0624ff9640 1 -- 172.21.15.94:0/380269626 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c014070 con 0x7f0628102e00 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0624ff9640 1 -- 172.21.15.94:0/380269626 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f061c010ac0 con 0x7f0628102e00 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062819fd90 con 0x7f0628102e00 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.992+0000 7f0624ff9640 1 -- 172.21.15.94:0/380269626 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c017c30 con 0x7f0628102e00 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.993+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281a02f0 con 0x7f0628102e00 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.993+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06281090a0 con 0x7f0628102e00 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.995+0000 7f0624ff9640 1 -- 172.21.15.94:0/380269626 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f061c017070 con 0x7f0628102e00 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.995+0000 7f0624ff9640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06100a3300 0x7f06100a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.995+0000 7f06277fe640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06100a3300 0x7f06100a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.995+0000 7f0624ff9640 1 -- 172.21.15.94:0/380269626 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f061c0cefa0 con 0x7f0628102e00 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.996+0000 7f06277fe640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06100a3300 0x7f06100a57c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0618000c00 tx=0x7f06180314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:44.998+0000 7f0624ff9640 1 -- 172.21.15.94:0/380269626 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f061c09bd30 con 0x7f0628102e00 2024-09-19T04:14:45.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.166+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f062802d7c0 con 0x7f0628102e00 2024-09-19T04:14:45.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.167+0000 7f0624ff9640 1 -- 172.21.15.94:0/380269626 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5316 (secure 0 0 0) 0x7f061c09b630 con 0x7f0628102e00 2024-09-19T04:14:45.168 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 81 2024-09-19T04:14:45.168 INFO:teuthology.orchestra.run.smithi094.stdout:fsid 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:14:45.168 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-09-19T04:05:34.057583+0000 2024-09-19T04:14:45.168 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-09-19T04:13:31.495964+0000 2024-09-19T04:14:45.168 INFO:teuthology.orchestra.run.smithi094.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T04:14:45.168 INFO:teuthology.orchestra.run.smithi094.stdout:crush_version 26 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:full_ratio 0.95 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:backfillfull_ratio 0.9 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:nearfull_ratio 0.85 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:require_min_compat_client luminous 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:min_compat_client luminous 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:require_osd_release squid 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:stretch_mode_enabled false 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.69 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:max_osd 12 2024-09-19T04:14:45.169 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6802/2563797567,v1:172.21.15.94:6803/2563797567] [v2:172.21.15.94:6804/2563797567,v1:172.21.15.94:6805/2563797567] exists,up 4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 up in weight 1 up_from 12 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6800/1485019338,v1:172.21.15.104:6801/1485019338] [v2:172.21.15.104:6802/1485019338,v1:172.21.15.104:6803/1485019338] exists,up cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 up in weight 1 up_from 16 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6800/1893936170,v1:172.21.15.112:6801/1893936170] [v2:172.21.15.112:6802/1893936170,v1:172.21.15.112:6803/1893936170] exists,up 92ebda5b-3f73-46b3-b026-a9d41639aa61 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 up in weight 1 up_from 24 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6810/1707310526,v1:172.21.15.94:6811/1707310526] [v2:172.21.15.94:6812/1707310526,v1:172.21.15.94:6813/1707310526] exists,up dcb93a96-92d9-4cf2-a2f0-f547d49835cf 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 up in weight 1 up_from 31 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6808/571438521,v1:172.21.15.104:6809/571438521] [v2:172.21.15.104:6810/571438521,v1:172.21.15.104:6811/571438521] exists,up 0edd43c9-dec2-4df0-b10e-cf68a6e809b1 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 up in weight 1 up_from 36 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6808/3131983830,v1:172.21.15.112:6809/3131983830] [v2:172.21.15.112:6810/3131983830,v1:172.21.15.112:6811/3131983830] exists,up abe550dd-07b4-46e6-a55e-b65a9b15c373 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 up in weight 1 up_from 43 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6818/1178759966,v1:172.21.15.94:6819/1178759966] [v2:172.21.15.94:6820/1178759966,v1:172.21.15.94:6821/1178759966] exists,up 26a90049-fa63-414c-a4e2-6a779aa15473 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 up in weight 1 up_from 47 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6816/2625303953,v1:172.21.15.104:6817/2625303953] [v2:172.21.15.104:6818/2625303953,v1:172.21.15.104:6819/2625303953] exists,up 699d883d-3aa9-4430-80ef-5a3de8cd773f 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 up in weight 1 up_from 54 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6816/3390158844,v1:172.21.15.112:6817/3390158844] [v2:172.21.15.112:6818/3390158844,v1:172.21.15.112:6819/3390158844] exists,up a4ed7d18-c753-4ddc-b1cc-518b5c67923d 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 up in weight 1 up_from 61 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6826/704568533,v1:172.21.15.94:6827/704568533] [v2:172.21.15.94:6828/704568533,v1:172.21.15.94:6829/704568533] exists,up d4dd28b1-8e3e-4871-9391-4d74b4c49b47 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 up in weight 1 up_from 66 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6824/1626292599,v1:172.21.15.104:6825/1626292599] [v2:172.21.15.104:6826/1626292599,v1:172.21.15.104:6827/1626292599] exists,up ab8a22e7-d51e-499d-8154-301d97f3790f 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 up in weight 1 up_from 73 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6824/3391428889,v1:172.21.15.112:6825/3391428889] [v2:172.21.15.112:6826/3391428889,v1:172.21.15.112:6827/3391428889] exists,up 47a94e97-ed76-4aa0-918c-a813e88ca539 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.f [8,11] 2024-09-19T04:14:45.170 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.13 [2,11] 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.1b [5,9] 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.21 [4,8] 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.22 [3,9] 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.27 [4,6] 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.28 [5,7] 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.34 [4,9] 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.3a [4,8] 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/3871891152 expires 2024-09-20T04:06:30.053518+0000 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/2204846817 expires 2024-09-20T04:06:30.053518+0000 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/3831757952 expires 2024-09-20T04:06:30.053518+0000 2024-09-19T04:14:45.171 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/3136401482 expires 2024-09-20T04:06:30.053518+0000 2024-09-19T04:14:45.172 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/3136401482 expires 2024-09-20T04:06:30.053518+0000 2024-09-19T04:14:45.172 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/4018061819 expires 2024-09-20T04:06:06.814655+0000 2024-09-19T04:14:45.172 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/4021627817 expires 2024-09-20T04:06:06.814655+0000 2024-09-19T04:14:45.172 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/2229490078 expires 2024-09-20T04:06:06.814655+0000 2024-09-19T04:14:45.172 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/2229490078 expires 2024-09-20T04:06:06.814655+0000 2024-09-19T04:14:45.172 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/516140053 expires 2024-09-20T04:06:06.814655+0000 2024-09-19T04:14:45.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06100a3300 msgr2=0x7f06100a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:45.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06100a3300 0x7f06100a57c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0618000c00 tx=0x7f06180314a0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628102e00 msgr2=0x7f06281983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:45.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628102e00 0x7f06281983b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b4a0 tx=0x7f061c00b960 comp rx=0 tx=0).stop 2024-09-19T04:14:45.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 shutdown_connections 2024-09-19T04:14:45.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06100a3300 0x7f06100a57c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06281037c0 0x7f062819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0628102e00 0x7f06281983b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 --2- 172.21.15.94:0/380269626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0628101c00 0x7f0628197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:45.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 >> 172.21.15.94:0/380269626 conn(0x7f06280fd390 msgr2=0x7f0628106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:45.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 shutdown_connections 2024-09-19T04:14:45.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:45.169+0000 7f062db92640 1 -- 172.21.15.94:0/380269626 wait complete. 2024-09-19T04:14:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:45 smithi094 ceph-mon[30972]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:45.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:45 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/380269626' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:14:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:45 smithi104 ceph-mon[34145]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:45 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/380269626' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:14:45.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:45 smithi112 ceph-mon[35051]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:45.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:45 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/380269626' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:14:45.853 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T04:14:45.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:14:46.111 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:46.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 -- 172.21.15.94:0/2095886175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c102d30 msgr2=0x7f067c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:46.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 --2- 172.21.15.94:0/2095886175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c102d30 0x7f067c103110 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f066c009cd0 tx=0x7f066c02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 -- 172.21.15.94:0/2095886175 shutdown_connections 2024-09-19T04:14:46.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 --2- 172.21.15.94:0/2095886175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f067c1082a0 0x7f067c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 --2- 172.21.15.94:0/2095886175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f067c1036e0 0x7f067c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 --2- 172.21.15.94:0/2095886175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c102d30 0x7f067c103110 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 -- 172.21.15.94:0/2095886175 >> 172.21.15.94:0/2095886175 conn(0x7f067c0fc7d0 msgr2=0x7f067c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 -- 172.21.15.94:0/2095886175 shutdown_connections 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 -- 172.21.15.94:0/2095886175 wait complete. 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 Processor -- start 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.597+0000 7f0684233640 1 -- start start 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f0684233640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f067c102d30 0x7f067c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f0684233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c1036e0 0x7f067c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f0684233640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f067c1082a0 0x7f067c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f0684233640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f067c111f50 con 0x7f067c1036e0 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f0684233640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f067c111dd0 con 0x7f067c1082a0 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f0684233640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1120d0 con 0x7f067c102d30 2024-09-19T04:14:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06817ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c1036e0 0x7f067c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06817ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c1036e0 0x7f067c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57322/0 (socket says 172.21.15.94:57322) 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06817ce640 1 -- 172.21.15.94:0/1991105411 learned_addr learned my addr 172.21.15.94:0/1991105411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f0681fcf640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f067c102d30 0x7f067c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06827d0640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f067c1082a0 0x7f067c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06817ce640 1 -- 172.21.15.94:0/1991105411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f067c102d30 msgr2=0x7f067c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06817ce640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f067c102d30 0x7f067c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06817ce640 1 -- 172.21.15.94:0/1991105411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f067c1082a0 msgr2=0x7f067c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06817ce640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f067c1082a0 0x7f067c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06817ce640 1 -- 172.21.15.94:0/1991105411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c19d4a0 con 0x7f067c1036e0 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f06817ce640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c1036e0 0x7f067c198a30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f067000d740 tx=0x7f067000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f067affd640 1 -- 172.21.15.94:0/1991105411 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0670017070 con 0x7f067c1036e0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f067affd640 1 -- 172.21.15.94:0/1991105411 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f067000de50 con 0x7f067c1036e0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c19d790 con 0x7f067c1036e0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f067affd640 1 -- 172.21.15.94:0/1991105411 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067000f620 con 0x7f067c1036e0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.598+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c1a5070 con 0x7f067c1036e0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.599+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c06a850 con 0x7f067c1036e0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.601+0000 7f067affd640 1 -- 172.21.15.94:0/1991105411 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f067001a070 con 0x7f067c1036e0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.601+0000 7f067affd640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06640a32d0 0x7f06640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.601+0000 7f0681fcf640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06640a32d0 0x7f06640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.601+0000 7f067affd640 1 -- 172.21.15.94:0/1991105411 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f06700cf3e0 con 0x7f067c1036e0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.602+0000 7f0681fcf640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06640a32d0 0x7f06640a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f066c006010 tx=0x7f066c0099c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.604+0000 7f067affd640 1 -- 172.21.15.94:0/1991105411 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f067009b950 con 0x7f067c1036e0 2024-09-19T04:14:46.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.803+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f067c10a2b0 con 0x7f067c1036e0 2024-09-19T04:14:46.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.804+0000 7f067affd640 1 -- 172.21.15.94:0/1991105411 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11915 (secure 0 0 0) 0x7f067009fc50 con 0x7f067c1036e0 2024-09-19T04:14:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:46.806 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"btime":"2024-09-19T04:13:28:480286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:13:28.166880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:14:46.806 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06640a32d0 msgr2=0x7f06640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06640a32d0 0x7f06640a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f066c006010 tx=0x7f066c0099c0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c1036e0 msgr2=0x7f067c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c1036e0 0x7f067c198a30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f067000d740 tx=0x7f067000dc00 comp rx=0 tx=0).stop 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 shutdown_connections 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f06640a32d0 0x7f06640a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f067c1082a0 0x7f067c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f067c1036e0 0x7f067c198a30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 --2- 172.21.15.94:0/1991105411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f067c102d30 0x7f067c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 >> 172.21.15.94:0/1991105411 conn(0x7f067c0fc7d0 msgr2=0x7f067c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:46.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 shutdown_connections 2024-09-19T04:14:46.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:46.806+0000 7f0684233640 1 -- 172.21.15.94:0/1991105411 wait complete. 2024-09-19T04:14:47.509 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:47 smithi094 ceph-mon[30972]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:47.509 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:47 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1991105411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:47.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:14:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:47 smithi104 ceph-mon[34145]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:47 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1991105411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:47.784 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:47 smithi112 ceph-mon[35051]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T04:14:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:47 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1991105411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.289+0000 7fd09ec56640 1 -- 172.21.15.94:0/3270191383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 msgr2=0x7fd09810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.289+0000 7fd09ec56640 1 --2- 172.21.15.94:0/3270191383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 0x7fd09810da40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd090009ca0 tx=0x7fd09002a950 comp rx=0 tx=0).stop 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.289+0000 7fd09ec56640 1 -- 172.21.15.94:0/3270191383 shutdown_connections 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.289+0000 7fd09ec56640 1 --2- 172.21.15.94:0/3270191383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 0x7fd09810da40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.289+0000 7fd09ec56640 1 --2- 172.21.15.94:0/3270191383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd098102570 0x7fd0981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.289+0000 7fd09ec56640 1 --2- 172.21.15.94:0/3270191383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd098100e20 0x7fd098101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.289+0000 7fd09ec56640 1 -- 172.21.15.94:0/3270191383 >> 172.21.15.94:0/3270191383 conn(0x7fd0980fc7d0 msgr2=0x7fd0980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.289+0000 7fd09ec56640 1 -- 172.21.15.94:0/3270191383 shutdown_connections 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.289+0000 7fd09ec56640 1 -- 172.21.15.94:0/3270191383 wait complete. 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09ec56640 1 Processor -- start 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09ec56640 1 -- start start 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09ec56640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd098100e20 0x7fd09819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09ec56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd098102570 0x7fd09819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09ec56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 0x7fd09819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09ec56640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd098110e30 con 0x7fd098102ef0 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09ec56640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd098110cb0 con 0x7fd098100e20 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09ec56640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd098110fb0 con 0x7fd098102570 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09d1f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 0x7fd09819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09d1f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 0x7fd09819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57338/0 (socket says 172.21.15.94:57338) 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09d1f3640 1 -- 172.21.15.94:0/2843956277 learned_addr learned my addr 172.21.15.94:0/2843956277 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd09c9f2640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd098100e20 0x7fd09819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.290+0000 7fd097fff640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd098102570 0x7fd09819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd09d1f3640 1 -- 172.21.15.94:0/2843956277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd098102570 msgr2=0x7fd09819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd09d1f3640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd098102570 0x7fd09819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd09d1f3640 1 -- 172.21.15.94:0/2843956277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd098100e20 msgr2=0x7fd09819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd09d1f3640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd098100e20 0x7fd09819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd09d1f3640 1 -- 172.21.15.94:0/2843956277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09819f4b0 con 0x7fd098102ef0 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd097fff640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd098102570 0x7fd09819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd09d1f3640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 0x7fd09819edd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd09002ae70 tx=0x7fd09002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd095ffb640 1 -- 172.21.15.94:0/2843956277 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090034040 con 0x7fd098102ef0 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd095ffb640 1 -- 172.21.15.94:0/2843956277 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0900308f0 con 0x7fd098102ef0 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09819f740 con 0x7fd098102ef0 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd095ffb640 1 -- 172.21.15.94:0/2843956277 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090004750 con 0x7fd098102ef0 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.291+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0981a7080 con 0x7fd098102ef0 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.292+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd098101200 con 0x7fd098102ef0 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.293+0000 7fd095ffb640 1 -- 172.21.15.94:0/2843956277 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd090037020 con 0x7fd098102ef0 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.294+0000 7fd095ffb640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd0880a3350 0x7fd0880a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.294+0000 7fd09c9f2640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd0880a3350 0x7fd0880a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.294+0000 7fd095ffb640 1 -- 172.21.15.94:0/2843956277 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd0900f0940 con 0x7fd098102ef0 2024-09-19T04:14:48.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.294+0000 7fd09c9f2640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd0880a3350 0x7fd0880a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00abe0 tx=0x7fd08c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.298+0000 7fd095ffb640 1 -- 172.21.15.94:0/2843956277 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0900bd5d0 con 0x7fd098102ef0 2024-09-19T04:14:48.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.503+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd09819b860 con 0x7fd098102ef0 2024-09-19T04:14:48.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.504+0000 7fd095ffb640 1 -- 172.21.15.94:0/2843956277 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11915 (secure 0 0 0) 0x7fd09819b860 con 0x7fd098102ef0 2024-09-19T04:14:48.506 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:48.506 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"btime":"2024-09-19T04:13:28:480286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:13:28.166880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:14:48.506 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-09-19T04:14:48.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.506+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd0880a3350 msgr2=0x7fd0880a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.506+0000 7fd09ec56640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd0880a3350 0x7fd0880a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00abe0 tx=0x7fd08c00a2d0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.506+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 msgr2=0x7fd09819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.506+0000 7fd09ec56640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 0x7fd09819edd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd09002ae70 tx=0x7fd09002f5d0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.506+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 shutdown_connections 2024-09-19T04:14:48.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.506+0000 7fd09ec56640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7fd0880a3350 0x7fd0880a5810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.506+0000 7fd09ec56640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd098102ef0 0x7fd09819edd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.506+0000 7fd09ec56640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd098102570 0x7fd09819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.506+0000 7fd09ec56640 1 --2- 172.21.15.94:0/2843956277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd098100e20 0x7fd09819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.507+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 >> 172.21.15.94:0/2843956277 conn(0x7fd0980fc7d0 msgr2=0x7fd09806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.507+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 shutdown_connections 2024-09-19T04:14:48.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:48.507+0000 7fd09ec56640 1 -- 172.21.15.94:0/2843956277 wait complete. 2024-09-19T04:14:49.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:14:49.454 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:49.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:49 smithi094 ceph-mon[30972]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:49.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:49 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2843956277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:49 smithi104 ceph-mon[34145]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:49 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2843956277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:49.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:49 smithi112 ceph-mon[35051]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T04:14:49.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:49 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2843956277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:14:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/2490399585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58106f20 msgr2=0x7f4a5810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/2490399585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58106f20 0x7f4a5810eaa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009900 tx=0x7f4a4802a410 comp rx=0 tx=0).stop 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/2490399585 shutdown_connections 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/2490399585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58106f20 0x7f4a5810eaa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/2490399585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a581027a0 0x7f4a581067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/2490399585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a58101df0 0x7f4a581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/2490399585 >> 172.21.15.94:0/2490399585 conn(0x7f4a580fd8b0 msgr2=0x7f4a580ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/2490399585 shutdown_connections 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/2490399585 wait complete. 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.967+0000 7f4a5f7b6640 1 Processor -- start 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5f7b6640 1 -- start start 2024-09-19T04:14:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5f7b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58101df0 0x7f4a58198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5f7b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a581027a0 0x7f4a58198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5f7b6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a58106f20 0x7f4a5819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5f7b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58111f20 con 0x7f4a58101df0 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5f7b6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58111da0 con 0x7f4a58106f20 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5f7b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581120a0 con 0x7f4a581027a0 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5d552640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58101df0 0x7f4a58198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5d552640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58101df0 0x7f4a58198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57354/0 (socket says 172.21.15.94:57354) 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5d552640 1 -- 172.21.15.94:0/1039859281 learned_addr learned my addr 172.21.15.94:0/1039859281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5dd53640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a58106f20 0x7f4a5819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5d552640 1 -- 172.21.15.94:0/1039859281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a581027a0 msgr2=0x7f4a58198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5d552640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a581027a0 0x7f4a58198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5d552640 1 -- 172.21.15.94:0/1039859281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a58106f20 msgr2=0x7f4a5819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5d552640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a58106f20 0x7f4a5819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5d552640 1 -- 172.21.15.94:0/1039859281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5819d3f0 con 0x7f4a58101df0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.968+0000 7f4a5d552640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58101df0 0x7f4a58198440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000ba50 tx=0x7f4a5000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.969+0000 7f4a567fc640 1 -- 172.21.15.94:0/1039859281 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50019070 con 0x7f4a58101df0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.969+0000 7f4a567fc640 1 -- 172.21.15.94:0/1039859281 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a5000cbe0 con 0x7f4a58101df0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.969+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5819d6e0 con 0x7f4a58101df0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.969+0000 7f4a567fc640 1 -- 172.21.15.94:0/1039859281 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500075d0 con 0x7f4a58101df0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.969+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a4fc0 con 0x7f4a58101df0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.970+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a58102250 con 0x7f4a58101df0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.971+0000 7f4a567fc640 1 -- 172.21.15.94:0/1039859281 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4a5001c070 con 0x7f4a58101df0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.971+0000 7f4a567fc640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4a400a32d0 0x7f4a400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.971+0000 7f4a5cd51640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4a400a32d0 0x7f4a400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.971+0000 7f4a567fc640 1 -- 172.21.15.94:0/1039859281 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4a500cef80 con 0x7f4a58101df0 2024-09-19T04:14:49.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.972+0000 7f4a5cd51640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4a400a32d0 0x7f4a400a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4a580ffe40 tx=0x7f4a4c006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:49.975+0000 7f4a567fc640 1 -- 172.21.15.94:0/1039859281 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4a5009bd10 con 0x7f4a58101df0 2024-09-19T04:14:50.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.143+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a5810a100 con 0x7f4a58101df0 2024-09-19T04:14:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.145+0000 7f4a567fc640 1 -- 172.21.15.94:0/1039859281 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22842 (secure 0 0 0) 0x7f4a5009b610 con 0x7f4a58101df0 2024-09-19T04:14:50.146 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:14:50.146 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:13:31.495964+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:14:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4a400a32d0 msgr2=0x7f4a400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4a400a32d0 0x7f4a400a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4a580ffe40 tx=0x7f4a4c006c90 comp rx=0 tx=0).stop 2024-09-19T04:14:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58101df0 msgr2=0x7f4a58198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58101df0 0x7f4a58198440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000ba50 tx=0x7f4a5000bf10 comp rx=0 tx=0).stop 2024-09-19T04:14:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 shutdown_connections 2024-09-19T04:14:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f4a400a32d0 0x7f4a400a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a58106f20 0x7f4a5819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a581027a0 0x7f4a58198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 --2- 172.21.15.94:0/1039859281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58101df0 0x7f4a58198440 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 >> 172.21.15.94:0/1039859281 conn(0x7f4a580fd8b0 msgr2=0x7f4a580fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 shutdown_connections 2024-09-19T04:14:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:50.147+0000 7f4a5f7b6640 1 -- 172.21.15.94:0/1039859281 wait complete. 2024-09-19T04:14:50.431 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:50 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1039859281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:14:50.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:50 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1039859281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:14:50.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:50 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1039859281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:14:50.871 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T04:13:17.025380+0000', 'modified': '2024-09-19T04:13:28.166880+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24467, 'mds_1': 24466, 'mds_2': 24454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6835/2064839900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 2064839900}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 2064839900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6837/2682504501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2682504501}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2682504501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/1098195473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 1098195473}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 1098195473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.112:6837/134813546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 134813546}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 134813546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.112:6839/3588924475', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 3588924475}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 3588924475}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.104:6839/2749200235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 2749200235}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 2749200235}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24467, 'qdb_cluster': [24454, 24467, 24466]} 2024-09-19T04:14:50.871 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T04:14:50.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T04:14:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:51.578 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:51 smithi094 ceph-mon[30972]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:51 smithi104 ceph-mon[34145]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:51.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 -- 172.21.15.94:0/4132564458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0100a70 msgr2=0x7f04b010ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:51.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 --2- 172.21.15.94:0/4132564458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0100a70 0x7f04b010ea60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009ca0 tx=0x7f04a802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 -- 172.21.15.94:0/4132564458 shutdown_connections 2024-09-19T04:14:51.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 --2- 172.21.15.94:0/4132564458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0100a70 0x7f04b010ea60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 --2- 172.21.15.94:0/4132564458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04b01000d0 0x7f04b0100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 --2- 172.21.15.94:0/4132564458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b0106ba0 0x7f04b0106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 -- 172.21.15.94:0/4132564458 >> 172.21.15.94:0/4132564458 conn(0x7f04b0077210 msgr2=0x7f04b00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:51.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 -- 172.21.15.94:0/4132564458 shutdown_connections 2024-09-19T04:14:51.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 -- 172.21.15.94:0/4132564458 wait complete. 2024-09-19T04:14:51.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.657+0000 7f04b6c01640 1 Processor -- start 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b6c01640 1 -- start start 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b6c01640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04b01000d0 0x7f04b01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b6c01640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b0100a70 0x7f04b01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b6c01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0106ba0 0x7f04b019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b6c01640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0111f60 con 0x7f04b0106ba0 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b6c01640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0111de0 con 0x7f04b01000d0 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b6c01640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01120e0 con 0x7f04b0100a70 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b519e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0106ba0 0x7f04b019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b519e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0106ba0 0x7f04b019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57378/0 (socket says 172.21.15.94:57378) 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b519e640 1 -- 172.21.15.94:0/1528082790 learned_addr learned my addr 172.21.15.94:0/1528082790 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:51.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b499d640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04b01000d0 0x7f04b01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04affff640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b0100a70 0x7f04b01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b519e640 1 -- 172.21.15.94:0/1528082790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b0100a70 msgr2=0x7f04b01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b519e640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b0100a70 0x7f04b01989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b519e640 1 -- 172.21.15.94:0/1528082790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04b01000d0 msgr2=0x7f04b01984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b519e640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04b01000d0 0x7f04b01984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b519e640 1 -- 172.21.15.94:0/1528082790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b019d450 con 0x7f04b0106ba0 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.658+0000 7f04b519e640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0106ba0 0x7f04b019cd70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009dc0 tx=0x7f04a8032fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.659+0000 7f04adffb640 1 -- 172.21.15.94:0/1528082790 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8003cf0 con 0x7f04b0106ba0 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.659+0000 7f04adffb640 1 -- 172.21.15.94:0/1528082790 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04a80044f0 con 0x7f04b0106ba0 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.659+0000 7f04b6c01640 1 -- 172.21.15.94:0/1528082790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b019d6e0 con 0x7f04b0106ba0 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.659+0000 7f04adffb640 1 -- 172.21.15.94:0/1528082790 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8030b10 con 0x7f04b0106ba0 2024-09-19T04:14:51.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.659+0000 7f04b6c01640 1 -- 172.21.15.94:0/1528082790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b01a5020 con 0x7f04b0106ba0 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.661+0000 7f04adffb640 1 -- 172.21.15.94:0/1528082790 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f04a8030d10 con 0x7f04b0106ba0 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.661+0000 7f04b8fda740 1 Processor -- start 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.661+0000 7f04b8fda740 1 -- start start 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.661+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 0x56504b4cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cd210 0x56504b4dba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04b8fda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56504b433090 con 0x56504b4dbfc0 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04b8fda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56504b3daa70 con 0x56504b4cc8f0 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04b8fda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56504b42f050 con 0x56504b4cd210 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04adffb640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04a00a32d0 0x7f04a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04b519e640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04a00a32d0 0x7f04a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4de3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57386/0 (socket says 172.21.15.94:57386) 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 -- 172.21.15.94:0/1304280252 learned_addr learned my addr 172.21.15.94:0/1304280252 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04b499d640 1 --2- 172.21.15.94:0/1304280252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cd210 0x56504b4dba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 --2- 172.21.15.94:0/1304280252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 0x56504b4cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04adffb640 1 -- 172.21.15.94:0/1528082790 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f04a80f0fb0 con 0x7f04b0106ba0 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 -- 172.21.15.94:0/1304280252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cd210 msgr2=0x56504b4dba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 --2- 172.21.15.94:0/1304280252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cd210 0x56504b4dba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 -- 172.21.15.94:0/1304280252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 msgr2=0x56504b4cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 --2- 172.21.15.94:0/1304280252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 0x56504b4cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 -- 172.21.15.94:0/1304280252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504b3e7130 con 0x56504b4dbfc0 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04affff640 1 --2- 172.21.15.94:0/1304280252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4de3b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f049c005fd0 tx=0x7f049c00e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1254c3dbacba4467 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:51.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f04b519e640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04a00a32d0 0x7f04a00a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f04a80c5cc0 tx=0x7f04a80e8350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f049affd640 1 -- 172.21.15.94:0/1304280252 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c00b680 con 0x56504b4dbfc0 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f049affd640 1 -- 172.21.15.94:0/1304280252 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f049c00bcd0 con 0x56504b4dbfc0 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.662+0000 7f049affd640 1 -- 172.21.15.94:0/1304280252 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c011c00 con 0x56504b4dbfc0 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 -- 172.21.15.94:0/1304280252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 msgr2=0x56504b4de3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 --2- 172.21.15.94:0/1304280252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4de3b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f049c005fd0 tx=0x7f049c00e3d0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 -- 172.21.15.94:0/1304280252 shutdown_connections 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 --2- 172.21.15.94:0/1304280252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4de3b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 --2- 172.21.15.94:0/1304280252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cd210 0x56504b4dba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 --2- 172.21.15.94:0/1304280252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 0x56504b4cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 -- 172.21.15.94:0/1304280252 >> 172.21.15.94:0/1304280252 conn(0x56504b4c8f60 msgr2=0x56504b4c9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 -- 172.21.15.94:0/1304280252 shutdown_connections 2024-09-19T04:14:51.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 -- 172.21.15.94:0/1304280252 wait complete. 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.663+0000 7f04b8fda740 1 -- start start 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b8fda740 1 Processor -- start 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 0x56504b4db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4dbfc0 0x56504b4d9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b8fda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56504b4deb30 con 0x56504b4cd210 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b8fda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56504b3edf20 con 0x56504b4dbfc0 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b8fda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56504b4decb0 con 0x56504b4cc8f0 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b8fda740 10 client.0 _ll_register_callbacks cb 0x56504b3b7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b499d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b499d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57396/0 (socket says 172.21.15.94:57396) 2024-09-19T04:14:51.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b499d640 1 -- 172.21.15.94:0/3293957501 learned_addr learned my addr 172.21.15.94:0/3293957501 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04affff640 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 0x56504b4db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b499d640 1 -- 172.21.15.94:0/3293957501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 msgr2=0x56504b4db940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b499d640 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 0x56504b4db940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b499d640 1 -- 172.21.15.94:0/3293957501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4dbfc0 msgr2=0x56504b4d9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b499d640 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4dbfc0 0x56504b4d9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b499d640 1 -- 172.21.15.94:0/3293957501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504b1a42b0 con 0x56504b4cd210 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f04b499d640 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d8c90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f04a4011590 tx=0x7f04a4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e86f871245ab77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.664+0000 7f048ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f048ffff640 1 -- 172.21.15.94:0/3293957501 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a400f6d0 con 0x56504b4cd210 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f048ffff640 1 -- 172.21.15.94:0/3293957501 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f04a400f870 con 0x56504b4cd210 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f04b8fda740 1 -- 172.21.15.94:0/3293957501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f049c00bcd0 con 0x56504b4cd210 2024-09-19T04:14:51.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f048ffff640 1 -- 172.21.15.94:0/3293957501 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f04a400fa70 con 0x56504b4cd210 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f04b8fda740 10 client.15093 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f04b8fda740 1 -- 172.21.15.94:0/3293957501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56504b4ea5c0 con 0x56504b4cd210 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f048ffff640 1 -- 172.21.15.94:0/3293957501 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f049c00bcd0 con 0x56504b4cd210 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f048ffff640 1 -- 172.21.15.94:0/3293957501 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f04a401d400 con 0x56504b4cd210 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f04b8fda740 10 client.15093 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f04b8fda740 10 client.15093 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f04b8fda740 20 client.15093 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f04b8fda740 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x56504b4ecc30 0x56504b4ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f04b8fda740 4 client.15093 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.665+0000 7f04b8fda740 1 -- 172.21.15.94:0/3293957501 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56504b440600 con 0x56504b4ecc30 2024-09-19T04:14:51.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.666+0000 7f04affff640 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x56504b4ecc30 0x56504b4ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.666+0000 7f048ffff640 10 client.15093 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.666+0000 7f04affff640 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x56504b4ecc30 0x56504b4ee0a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd8fb64b75d6b7ab server_cookie=c3a49402455327bf in_seq=0 out_seq=0 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f048ffff640 1 -- 172.21.15.94:0/3293957501 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56504b440600 con 0x56504b4ecc30 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f048ffff640 10 client.15093 handle_command_reply: tid=0 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 client.15093 shutdown 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 -- 172.21.15.94:0/3293957501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 msgr2=0x56504b4d8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d8c90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f04a4011590 tx=0x7f04a4011a50 comp rx=0 tx=0).stop 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 -- 172.21.15.94:0/3293957501 shutdown_connections 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x56504b4ecc30 0x56504b4ee0a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4dbfc0 0x56504b4d9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d8c90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 --2- 172.21.15.94:0/3293957501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 0x56504b4db940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f048ffff640 0 client.15093 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 -- 172.21.15.94:0/3293957501 >> 172.21.15.94:0/3293957501 conn(0x56504b4c8f60 msgr2=0x56504b4cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 -- 172.21.15.94:0/3293957501 shutdown_connections 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 1 -- 172.21.15.94:0/3293957501 wait complete. 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.667+0000 7f04b8fda740 20 client.15093 trim_cache size 0 max 16384 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04b8fda740 1 Processor -- start 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04b8fda740 1 -- start start 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 0x56504b4ed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cd210 0x56504b4c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04b8fda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56504b4ec3e0 con 0x56504b4dbfc0 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04b8fda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56504b4ecab0 con 0x56504b4cd210 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04b8fda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56504b4ed180 con 0x56504b4cc8f0 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57410/0 (socket says 172.21.15.94:57410) 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.685+0000 7f04affff640 1 -- 172.21.15.94:0/2756935776 learned_addr learned my addr 172.21.15.94:0/2756935776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f04affff640 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 0x56504b4ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f04b499d640 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cd210 0x56504b4c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f04affff640 1 -- 172.21.15.94:0/2756935776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 msgr2=0x56504b4ed300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f04affff640 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 0x56504b4ed300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f04affff640 1 -- 172.21.15.94:0/2756935776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cd210 msgr2=0x56504b4c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f04affff640 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cd210 0x56504b4c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f04affff640 1 -- 172.21.15.94:0/2756935776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504b4d7c60 con 0x56504b4dbfc0 2024-09-19T04:14:51.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f04b499d640 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cd210 0x56504b4c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:51.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f04affff640 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4d4130 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x56504b4dbf90 tx=0x7f049c020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6c6dd8df8f918c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:51.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f049a7fc640 1 -- 172.21.15.94:0/2756935776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c011910 con 0x56504b4dbfc0 2024-09-19T04:14:51.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f049a7fc640 1 -- 172.21.15.94:0/2756935776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f049c005040 con 0x56504b4dbfc0 2024-09-19T04:14:51.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.686+0000 7f049a7fc640 1 -- 172.21.15.94:0/2756935776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c004880 con 0x56504b4dbfc0 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 -- 172.21.15.94:0/2756935776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 msgr2=0x56504b4d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4d4130 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x56504b4dbf90 tx=0x7f049c020040 comp rx=0 tx=0).stop 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 -- 172.21.15.94:0/2756935776 shutdown_connections 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4dbfc0 0x56504b4d4130 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cd210 0x56504b4c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 --2- 172.21.15.94:0/2756935776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4cc8f0 0x56504b4ed300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 -- 172.21.15.94:0/2756935776 >> 172.21.15.94:0/2756935776 conn(0x56504b4c8f60 msgr2=0x56504b4d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 -- 172.21.15.94:0/2756935776 shutdown_connections 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 -- 172.21.15.94:0/2756935776 wait complete. 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 -- start start 2024-09-19T04:14:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.687+0000 7f04b8fda740 1 Processor -- start 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 0x56504b4d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b8fda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4dbfc0 0x56504b4d6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b8fda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56504b4edf20 con 0x56504b4cd210 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b8fda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56504b4ed850 con 0x56504b4cc8f0 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b8fda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56504b4ee5f0 con 0x56504b4dbfc0 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b8fda740 10 client.0 _ll_register_callbacks cb 0x7f04a40095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b499d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b499d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57414/0 (socket says 172.21.15.94:57414) 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b499d640 1 -- 172.21.15.94:0/61585510 learned_addr learned my addr 172.21.15.94:0/61585510 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04affff640 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 0x56504b4d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04affff640 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4dbfc0 0x56504b4d6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b499d640 1 -- 172.21.15.94:0/61585510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4dbfc0 msgr2=0x56504b4d6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b499d640 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4dbfc0 0x56504b4d6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b499d640 1 -- 172.21.15.94:0/61585510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 msgr2=0x56504b4d5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b499d640 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 0x56504b4d5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b499d640 1 -- 172.21.15.94:0/61585510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504b4f4780 con 0x56504b4cd210 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b499d640 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d61e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56504b4da2d0 tx=0x7f04a40091f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c1b3839edb8c0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f048ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f048ffff640 1 -- 172.21.15.94:0/61585510 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a402c070 con 0x56504b4cd210 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f048ffff640 1 -- 172.21.15.94:0/61585510 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f04a400fe10 con 0x56504b4cd210 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.688+0000 7f04b8fda740 1 -- 172.21.15.94:0/61585510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f049c005040 con 0x56504b4cd210 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f048ffff640 1 -- 172.21.15.94:0/61585510 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f04a401da40 con 0x56504b4cd210 2024-09-19T04:14:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f04b8fda740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f04b8fda740 1 -- 172.21.15.94:0/61585510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56504b557550 con 0x56504b4cd210 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f048ffff640 1 -- 172.21.15.94:0/61585510 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f049c005040 con 0x56504b4cd210 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f048ffff640 1 -- 172.21.15.94:0/61585510 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f04a402f020 con 0x56504b4cd210 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f04b8fda740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f04b8fda740 10 client.15099 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f04b8fda740 20 client.15099 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f04b8fda740 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x56504b55c110 0x56504b55ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f04b8fda740 4 client.15099 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f04b8fda740 1 -- 172.21.15.94:0/61585510 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "status"}) -- 0x56504b43ab10 con 0x56504b55c110 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.689+0000 7f04affff640 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x56504b55c110 0x56504b55ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04affff640 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x56504b55c110 0x56504b55ae20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a3604d7d8189d09 server_cookie=d8f698a86769c93f in_seq=0 out_seq=0 2024-09-19T04:14:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f048ffff640 10 client.15099 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f048ffff640 1 -- 172.21.15.94:0/61585510 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56504b43ab10 con 0x56504b55c110 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f048ffff640 10 client.15099 handle_command_reply: tid=0 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 client.15099 shutdown 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 -- 172.21.15.94:0/61585510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 msgr2=0x56504b4d61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d61e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56504b4da2d0 tx=0x7f04a40091f0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 -- 172.21.15.94:0/61585510 shutdown_connections 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x56504b55c110 0x56504b55ae20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56504b4dbfc0 0x56504b4d6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56504b4cd210 0x56504b4d61e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f048ffff640 0 client.15099 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 --2- 172.21.15.94:0/61585510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56504b4cc8f0 0x56504b4d5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 -- 172.21.15.94:0/61585510 >> 172.21.15.94:0/61585510 conn(0x56504b4c8f60 msgr2=0x56504b4d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 -- 172.21.15.94:0/61585510 shutdown_connections 2024-09-19T04:14:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 1 -- 172.21.15.94:0/61585510 wait complete. 2024-09-19T04:14:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.690+0000 7f04b8fda740 20 client.15099 trim_cache size 0 max 16384 2024-09-19T04:14:51.709 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:14:51.709 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "503c4526-763c-11ef-bceb-c7b262605968", 2024-09-19T04:14:51.709 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 2, 2024-09-19T04:14:51.709 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24454, 2024-09-19T04:14:51.709 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-19T04:14:51.709 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-19T04:14:51.710 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-19T04:14:51.710 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 86.219350911999996, 2024-09-19T04:14:51.710 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-09-19T04:14:51.710 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-09-19T04:14:51.710 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T04:14:51.710 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 118.918955813 2024-09-19T04:14:51.710 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:14:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 -- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04a00a32d0 msgr2=0x7f04a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04a00a32d0 0x7f04a00a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f04a80c5cc0 tx=0x7f04a80e8350 comp rx=0 tx=0).stop 2024-09-19T04:14:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 -- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0106ba0 msgr2=0x7f04b019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0106ba0 0x7f04b019cd70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009dc0 tx=0x7f04a8032fe0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 -- 172.21.15.94:0/1528082790 shutdown_connections 2024-09-19T04:14:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f04a00a32d0 0x7f04a00a5790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04b0106ba0 0x7f04b019cd70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b0100a70 0x7f04b01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 --2- 172.21.15.94:0/1528082790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04b01000d0 0x7f04b01984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:51.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 -- 172.21.15.94:0/1528082790 >> 172.21.15.94:0/1528082790 conn(0x7f04b0077210 msgr2=0x7f04b0104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:51.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 -- 172.21.15.94:0/1528082790 shutdown_connections 2024-09-19T04:14:51.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:51.691+0000 7f04b6c01640 1 -- 172.21.15.94:0/1528082790 wait complete. 2024-09-19T04:14:51.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:51 smithi112 ceph-mon[35051]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T04:14:52.511 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T04:14:52.758 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:53.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.214+0000 7f2482a2d640 1 -- 172.21.15.94:0/3276819901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 msgr2=0x7f247c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:53.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.214+0000 7f2482a2d640 1 --2- 172.21.15.94:0/3276819901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 0x7f247c1067e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2470009900 tx=0x7f247002a410 comp rx=0 tx=0).stop 2024-09-19T04:14:53.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.214+0000 7f2482a2d640 1 -- 172.21.15.94:0/3276819901 shutdown_connections 2024-09-19T04:14:53.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.214+0000 7f2482a2d640 1 --2- 172.21.15.94:0/3276819901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247c106f10 0x7f247c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.214+0000 7f2482a2d640 1 --2- 172.21.15.94:0/3276819901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 0x7f247c1067e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.214+0000 7f2482a2d640 1 --2- 172.21.15.94:0/3276819901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247c101de0 0x7f247c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.214+0000 7f2482a2d640 1 -- 172.21.15.94:0/3276819901 >> 172.21.15.94:0/3276819901 conn(0x7f247c0fd8c0 msgr2=0x7f247c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:53.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.215+0000 7f2482a2d640 1 -- 172.21.15.94:0/3276819901 shutdown_connections 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.215+0000 7f2482a2d640 1 -- 172.21.15.94:0/3276819901 wait complete. 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.215+0000 7f2482a2d640 1 Processor -- start 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.215+0000 7f2482a2d640 1 -- start start 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f2482a2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247c101de0 0x7f247c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f2482a2d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 0x7f247c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f2482a2d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247c106f10 0x7f247c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f2482a2d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f247c111e20 con 0x7f247c102790 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f2482a2d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f247c111ca0 con 0x7f247c106f10 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f2482a2d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f247c111fa0 con 0x7f247c101de0 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 0x7f247c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 0x7f247c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59102/0 (socket says 172.21.15.94:59102) 2024-09-19T04:14:53.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247b7fe640 1 -- 172.21.15.94:0/47070468 learned_addr learned my addr 172.21.15.94:0/47070468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247bfff640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247c101de0 0x7f247c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247b7fe640 1 -- 172.21.15.94:0/47070468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247c101de0 msgr2=0x7f247c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247b7fe640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247c101de0 0x7f247c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f2480fca640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247c106f10 0x7f247c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247b7fe640 1 -- 172.21.15.94:0/47070468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247c106f10 msgr2=0x7f247c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247b7fe640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247c106f10 0x7f247c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247b7fe640 1 -- 172.21.15.94:0/47070468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c19d350 con 0x7f247c102790 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247bfff640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247c101de0 0x7f247c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f247b7fe640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 0x7f247c1988e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2470005940 tx=0x7f2470004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f24797fa640 1 -- 172.21.15.94:0/47070468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247003f030 con 0x7f247c102790 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f2482a2d640 1 -- 172.21.15.94:0/47070468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c19d5e0 con 0x7f247c102790 2024-09-19T04:14:53.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f24797fa640 1 -- 172.21.15.94:0/47070468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2470003aa0 con 0x7f247c102790 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f24797fa640 1 -- 172.21.15.94:0/47070468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470035070 con 0x7f247c102790 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.216+0000 7f2482a2d640 1 -- 172.21.15.94:0/47070468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c1a4f20 con 0x7f247c102790 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.219+0000 7f24797fa640 1 -- 172.21.15.94:0/47070468 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2470042030 con 0x7f247c102790 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.219+0000 7f2484e06740 1 Processor -- start 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.219+0000 7f2484e06740 1 -- start start 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2484e06740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a540acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2484e06740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 0x5623a5419a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2484e06740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a5419fc0 0x5623a541c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2484e06740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5623a5371090 con 0x5623a540a8d0 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2484e06740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623a5318a70 con 0x5623a540b1f0 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2484e06740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623a536d050 con 0x5623a5419fc0 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2480fca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a540acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2480fca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a540acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59114/0 (socket says 172.21.15.94:59114) 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2480fca640 1 -- 172.21.15.94:0/3790744814 learned_addr learned my addr 172.21.15.94:0/3790744814 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f247b7fe640 1 --2- 172.21.15.94:0/3790744814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a5419fc0 0x5623a541c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f247bfff640 1 --2- 172.21.15.94:0/3790744814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 0x5623a5419a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2480fca640 1 -- 172.21.15.94:0/3790744814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a5419fc0 msgr2=0x5623a541c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2480fca640 1 --2- 172.21.15.94:0/3790744814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a5419fc0 0x5623a541c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2480fca640 1 -- 172.21.15.94:0/3790744814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 msgr2=0x5623a5419a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2480fca640 1 --2- 172.21.15.94:0/3790744814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 0x5623a5419a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2480fca640 1 -- 172.21.15.94:0/3790744814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a5325130 con 0x5623a540a8d0 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f24797fa640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24640a32d0 0x7f24640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f2480fca640 1 --2- 172.21.15.94:0/3790744814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a540acb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2474008c80 tx=0x7f247400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95d9ba69bb890ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f247bfff640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24640a32d0 0x7f24640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f24797fa640 1 -- 172.21.15.94:0/47070468 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f24700f0300 con 0x7f247c102790 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f24627fc640 1 -- 172.21.15.94:0/3790744814 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474018680 con 0x5623a540a8d0 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f24627fc640 1 -- 172.21.15.94:0/3790744814 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2474018cd0 con 0x5623a540a8d0 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.220+0000 7f24627fc640 1 -- 172.21.15.94:0/3790744814 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247400fd70 con 0x5623a540a8d0 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 -- 172.21.15.94:0/3790744814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 msgr2=0x5623a540acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f247bfff640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24640a32d0 0x7f24640a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5623a540d5c0 tx=0x7f246c00b710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 --2- 172.21.15.94:0/3790744814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a540acb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2474008c80 tx=0x7f247400d710 comp rx=0 tx=0).stop 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 -- 172.21.15.94:0/3790744814 shutdown_connections 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 --2- 172.21.15.94:0/3790744814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a5419fc0 0x5623a541c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 --2- 172.21.15.94:0/3790744814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 0x5623a5419a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 --2- 172.21.15.94:0/3790744814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a540acb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 -- 172.21.15.94:0/3790744814 >> 172.21.15.94:0/3790744814 conn(0x5623a5406f60 msgr2=0x5623a5407340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 -- 172.21.15.94:0/3790744814 shutdown_connections 2024-09-19T04:14:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 -- 172.21.15.94:0/3790744814 wait complete. 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.221+0000 7f2484e06740 1 -- start start 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2484e06740 1 Processor -- start 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2484e06740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 0x5623a5419950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2484e06740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 0x5623a5416ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2484e06740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a5417220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2484e06740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5623a541cbf0 con 0x5623a5419fc0 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2484e06740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623a532bf20 con 0x5623a540a8d0 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2484e06740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623a541cd70 con 0x5623a540b1f0 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2484e06740 10 client.0 _ll_register_callbacks cb 0x5623a52f5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2480fca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a5417220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2480fca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a5417220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59118/0 (socket says 172.21.15.94:59118) 2024-09-19T04:14:53.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2480fca640 1 -- 172.21.15.94:0/3349895383 learned_addr learned my addr 172.21.15.94:0/3349895383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f247bfff640 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 0x5623a5419950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f247b7fe640 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 0x5623a5416ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2480fca640 1 -- 172.21.15.94:0/3349895383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 msgr2=0x5623a5416ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2480fca640 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 0x5623a5416ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2480fca640 1 -- 172.21.15.94:0/3349895383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 msgr2=0x5623a5419950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2480fca640 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 0x5623a5419950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.222+0000 7f2480fca640 1 -- 172.21.15.94:0/3349895383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a50e22b0 con 0x5623a5419fc0 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2480fca640 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a5417220 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2474008350 tx=0x7f247400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362b8e0272430f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f245b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f245b7fe640 1 -- 172.21.15.94:0/3349895383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474004e50 con 0x5623a5419fc0 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f245b7fe640 1 -- 172.21.15.94:0/3349895383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2474018480 con 0x5623a5419fc0 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2484e06740 1 -- 172.21.15.94:0/3349895383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2474018cd0 con 0x5623a5419fc0 2024-09-19T04:14:53.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f245b7fe640 1 -- 172.21.15.94:0/3349895383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2474008850 con 0x5623a5419fc0 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2484e06740 10 client.15111 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2484e06740 1 -- 172.21.15.94:0/3349895383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5623a5428620 con 0x5623a5419fc0 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f245b7fe640 1 -- 172.21.15.94:0/3349895383 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2474018cd0 con 0x5623a5419fc0 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f245b7fe640 1 -- 172.21.15.94:0/3349895383 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f247402c050 con 0x5623a5419fc0 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2484e06740 10 client.15111 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2484e06740 10 client.15111 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2484e06740 20 client.15111 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2484e06740 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5623a542a7b0 0x5623a542bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2484e06740 4 client.15111 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.223+0000 7f2484e06740 1 -- 172.21.15.94:0/3349895383 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623a537e600 con 0x5623a542a7b0 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.224+0000 7f247bfff640 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5623a542a7b0 0x5623a542bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.224+0000 7f245b7fe640 10 client.15111 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.224+0000 7f247bfff640 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5623a542a7b0 0x5623a542bc20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f344874e631013dc server_cookie=d12c35f01664ce6c in_seq=0 out_seq=0 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f245b7fe640 1 -- 172.21.15.94:0/3349895383 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623a537e600 con 0x5623a542a7b0 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f245b7fe640 10 client.15111 handle_command_reply: tid=0 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 client.15111 shutdown 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 -- 172.21.15.94:0/3349895383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 msgr2=0x5623a5417220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a5417220 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2474008350 tx=0x7f247400f730 comp rx=0 tx=0).stop 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 -- 172.21.15.94:0/3349895383 shutdown_connections 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5623a542a7b0 0x5623a542bc20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a5417220 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 0x5623a5416ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f245b7fe640 0 client.15111 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:14:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 --2- 172.21.15.94:0/3349895383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 0x5623a5419950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 -- 172.21.15.94:0/3349895383 >> 172.21.15.94:0/3349895383 conn(0x5623a5406f60 msgr2=0x5623a540ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 -- 172.21.15.94:0/3349895383 shutdown_connections 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 1 -- 172.21.15.94:0/3349895383 wait complete. 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.225+0000 7f2484e06740 20 client.15111 trim_cache size 0 max 16384 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2484e06740 1 Processor -- start 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2484e06740 1 -- start start 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2484e06740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a542bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2484e06740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 0x5623a5412240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2484e06740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a5419fc0 0x5623a5413f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2484e06740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5623a5429f60 con 0x5623a540a8d0 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2484e06740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623a542a630 con 0x5623a540b1f0 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2484e06740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623a542ad00 con 0x5623a5419fc0 2024-09-19T04:14:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a542bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a542bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59122/0 (socket says 172.21.15.94:59122) 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247bfff640 1 -- 172.21.15.94:0/266514251 learned_addr learned my addr 172.21.15.94:0/266514251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247bfff640 1 -- 172.21.15.94:0/266514251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a5419fc0 msgr2=0x5623a5413f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247bfff640 1 --2- 172.21.15.94:0/266514251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a5419fc0 0x5623a5413f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247bfff640 1 -- 172.21.15.94:0/266514251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 msgr2=0x5623a5412240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247b7fe640 1 --2- 172.21.15.94:0/266514251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 0x5623a5412240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247bfff640 1 --2- 172.21.15.94:0/266514251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 0x5623a5412240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247bfff640 1 -- 172.21.15.94:0/266514251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a5417570 con 0x5623a540a8d0 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247b7fe640 1 --2- 172.21.15.94:0/266514251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 0x5623a5412240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f247bfff640 1 --2- 172.21.15.94:0/266514251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a542bc20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f246c0125c0 tx=0x7f246c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2098fd5db61e044 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2461ffb640 1 -- 172.21.15.94:0/266514251 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c02d070 con 0x5623a540a8d0 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2461ffb640 1 -- 172.21.15.94:0/266514251 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f246c012e20 con 0x5623a540a8d0 2024-09-19T04:14:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.243+0000 7f2461ffb640 1 -- 172.21.15.94:0/266514251 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c028bc0 con 0x5623a540a8d0 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 -- 172.21.15.94:0/266514251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 msgr2=0x5623a542bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 --2- 172.21.15.94:0/266514251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a542bc20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f246c0125c0 tx=0x7f246c014000 comp rx=0 tx=0).stop 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 -- 172.21.15.94:0/266514251 shutdown_connections 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 --2- 172.21.15.94:0/266514251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a5419fc0 0x5623a5413f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 --2- 172.21.15.94:0/266514251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540b1f0 0x5623a5412240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 --2- 172.21.15.94:0/266514251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a540a8d0 0x5623a542bc20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 -- 172.21.15.94:0/266514251 >> 172.21.15.94:0/266514251 conn(0x5623a5406f60 msgr2=0x5623a5401ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 -- 172.21.15.94:0/266514251 shutdown_connections 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 -- 172.21.15.94:0/266514251 wait complete. 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.244+0000 7f2484e06740 1 -- start start 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2484e06740 1 Processor -- start 2024-09-19T04:14:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2484e06740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 0x5623a5434490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2484e06740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 0x5623a542d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2484e06740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a542dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2484e06740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5623a542baa0 con 0x5623a5419fc0 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2484e06740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623a542b3d0 con 0x5623a540a8d0 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2484e06740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623a542c170 con 0x5623a540b1f0 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2484e06740 10 client.0 _ll_register_callbacks cb 0x7f2474008ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2480fca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a542dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2480fca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a542dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59126/0 (socket says 172.21.15.94:59126) 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2480fca640 1 -- 172.21.15.94:0/3690228065 learned_addr learned my addr 172.21.15.94:0/3690228065 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f247b7fe640 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 0x5623a542d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2480fca640 1 -- 172.21.15.94:0/3690228065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 msgr2=0x5623a542d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2480fca640 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 0x5623a542d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2480fca640 1 -- 172.21.15.94:0/3690228065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 msgr2=0x5623a5434490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f247bfff640 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 0x5623a5434490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2480fca640 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 0x5623a5434490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2480fca640 1 -- 172.21.15.94:0/3690228065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a542e340 con 0x5623a5419fc0 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f247b7fe640 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 0x5623a542d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f2480fca640 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a542dd00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f24740058e0 tx=0x7f24740108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae75b694aa9e3d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f245b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.245+0000 7f245b7fe640 1 -- 172.21.15.94:0/3690228065 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247400dc10 con 0x5623a5419fc0 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f245b7fe640 1 -- 172.21.15.94:0/3690228065 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2474005d90 con 0x5623a5419fc0 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f2484e06740 1 -- 172.21.15.94:0/3690228065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f246c012e20 con 0x5623a5419fc0 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f245b7fe640 1 -- 172.21.15.94:0/3690228065 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f247403d050 con 0x5623a5419fc0 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f2484e06740 10 client.15117 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f2484e06740 1 -- 172.21.15.94:0/3690228065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5623a5432d90 con 0x5623a5419fc0 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f245b7fe640 1 -- 172.21.15.94:0/3690228065 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f246c012e20 con 0x5623a5419fc0 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f245b7fe640 1 -- 172.21.15.94:0/3690228065 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f247402c040 con 0x5623a5419fc0 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f2484e06740 10 client.15117 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f2484e06740 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f2484e06740 20 client.15117 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f2484e06740 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5623a5499860 0x5623a5498570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f2484e06740 4 client.15117 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f2484e06740 1 -- 172.21.15.94:0/3690228065 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "status"}) -- 0x5623a5378b10 con 0x5623a5499860 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.246+0000 7f247bfff640 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5623a5499860 0x5623a5498570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f245b7fe640 10 client.15117 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f247bfff640 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5623a5499860 0x5623a5498570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b38d6772c2d2859c server_cookie=ea5b44dbf2a23e1f in_seq=0 out_seq=0 2024-09-19T04:14:53.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f245b7fe640 1 -- 172.21.15.94:0/3690228065 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5623a5378b10 con 0x5623a5499860 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f245b7fe640 10 client.15117 handle_command_reply: tid=0 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 client.15117 shutdown 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 -- 172.21.15.94:0/3690228065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 msgr2=0x5623a542dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a542dd00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f24740058e0 tx=0x7f24740108f0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 -- 172.21.15.94:0/3690228065 shutdown_connections 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5623a5499860 0x5623a5498570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623a5419fc0 0x5623a542dd00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a540b1f0 0x5623a542d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f245b7fe640 0 client.15117 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 --2- 172.21.15.94:0/3690228065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623a540a8d0 0x5623a5434490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 -- 172.21.15.94:0/3690228065 >> 172.21.15.94:0/3690228065 conn(0x5623a5406f60 msgr2=0x5623a5416d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 -- 172.21.15.94:0/3690228065 shutdown_connections 2024-09-19T04:14:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 1 -- 172.21.15.94:0/3690228065 wait complete. 2024-09-19T04:14:53.268 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:14:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "503c4526-763c-11ef-bceb-c7b262605968", 2024-09-19T04:14:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 1, 2024-09-19T04:14:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24466, 2024-09-19T04:14:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-19T04:14:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-19T04:14:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-19T04:14:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 88.783641953, 2024-09-19T04:14:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 115.505688131 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.247+0000 7f2484e06740 20 client.15117 trim_cache size 0 max 16384 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 -- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24640a32d0 msgr2=0x7f24640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24640a32d0 0x7f24640a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5623a540d5c0 tx=0x7f246c00b710 comp rx=0 tx=0).stop 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 -- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 msgr2=0x7f247c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 0x7f247c1988e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2470005940 tx=0x7f2470004270 comp rx=0 tx=0).stop 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 -- 172.21.15.94:0/47070468 shutdown_connections 2024-09-19T04:14:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f24640a32d0 0x7f24640a5790 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247c106f10 0x7f247c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f247c102790 0x7f247c1988e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 --2- 172.21.15.94:0/47070468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247c101de0 0x7f247c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 -- 172.21.15.94:0/47070468 >> 172.21.15.94:0/47070468 conn(0x7f247c0fd8c0 msgr2=0x7f247c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 -- 172.21.15.94:0/47070468 shutdown_connections 2024-09-19T04:14:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:53.248+0000 7f2482a2d640 1 -- 172.21.15.94:0/47070468 wait complete. 2024-09-19T04:14:53.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:53 smithi094 ceph-mon[30972]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:53 smithi104 ceph-mon[34145]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:53.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:53 smithi112 ceph-mon[35051]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T04:14:54.036 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:14:54.281 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:54.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.764+0000 7f455d00c640 1 -- 172.21.15.94:0/2354670857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558101420 msgr2=0x7f455810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:54.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.764+0000 7f455d00c640 1 --2- 172.21.15.94:0/2354670857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558101420 0x7f455810d950 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4550009cd0 tx=0x7f455002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.764+0000 7f455d00c640 1 -- 172.21.15.94:0/2354670857 shutdown_connections 2024-09-19T04:14:54.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.764+0000 7f455d00c640 1 --2- 172.21.15.94:0/2354670857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558101420 0x7f455810d950 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.764+0000 7f455d00c640 1 --2- 172.21.15.94:0/2354670857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4558100a80 0x7f4558100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.764+0000 7f455d00c640 1 --2- 172.21.15.94:0/2354670857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4558106a80 0x7f4558106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.764+0000 7f455d00c640 1 -- 172.21.15.94:0/2354670857 >> 172.21.15.94:0/2354670857 conn(0x7f45580fc7a0 msgr2=0x7f45580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:54.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.764+0000 7f455d00c640 1 -- 172.21.15.94:0/2354670857 shutdown_connections 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.764+0000 7f455d00c640 1 -- 172.21.15.94:0/2354670857 wait complete. 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f455d00c640 1 Processor -- start 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f455d00c640 1 -- start start 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f455d00c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558100a80 0x7f45581972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f455d00c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4558101420 0x7f4558197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f455d00c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4558106a80 0x7f455819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f455d00c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4558110d50 con 0x7f4558100a80 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f455d00c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4558110bd0 con 0x7f4558101420 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f455d00c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4558110ed0 con 0x7f4558106a80 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f4556d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558100a80 0x7f45581972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f4556d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558100a80 0x7f45581972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59142/0 (socket says 172.21.15.94:59142) 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.765+0000 7f4556d9d640 1 -- 172.21.15.94:0/22949712 learned_addr learned my addr 172.21.15.94:0/22949712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:54.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f454ffff640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4558101420 0x7f4558197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f4556d9d640 1 -- 172.21.15.94:0/22949712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4558106a80 msgr2=0x7f455819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f455759e640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4558106a80 0x7f455819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f4556d9d640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4558106a80 0x7f455819bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f4556d9d640 1 -- 172.21.15.94:0/22949712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4558101420 msgr2=0x7f4558197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f4556d9d640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4558101420 0x7f4558197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f4556d9d640 1 -- 172.21.15.94:0/22949712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455819c280 con 0x7f4558100a80 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f454ffff640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4558101420 0x7f4558197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f455759e640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4558106a80 0x7f455819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f4556d9d640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558100a80 0x7f45581972d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f454800d950 tx=0x7f454800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f4554d99640 1 -- 172.21.15.94:0/22949712 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548017070 con 0x7f4558100a80 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f4554d99640 1 -- 172.21.15.94:0/22949712 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4548009c30 con 0x7f4558100a80 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f455d00c640 1 -- 172.21.15.94:0/22949712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455819c570 con 0x7f4558100a80 2024-09-19T04:14:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f4554d99640 1 -- 172.21.15.94:0/22949712 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454800f620 con 0x7f4558100a80 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.766+0000 7f455d00c640 1 -- 172.21.15.94:0/22949712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45581a3e50 con 0x7f4558100a80 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.768+0000 7f4554d99640 1 -- 172.21.15.94:0/22949712 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f45480109a0 con 0x7f4558100a80 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455f3e5740 1 Processor -- start 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455f3e5740 1 -- start start 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46948d0 0x55a9c4694cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 0x55a9c46a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46a1dc0 0x55a9c46a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455f3e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9c45fb090 con 0x55a9c46a1dc0 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455f3e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a9c45a2a70 con 0x55a9c46948d0 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455f3e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9c45f7050 con 0x55a9c46951f0 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f454ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46a1dc0 0x55a9c46a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f454ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46a1dc0 0x55a9c46a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59152/0 (socket says 172.21.15.94:59152) 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f454ffff640 1 -- 172.21.15.94:0/3702155055 learned_addr learned my addr 172.21.15.94:0/3702155055 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f4554d99640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45440a32d0 0x7f45440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455759e640 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46948d0 0x55a9c4694cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f4556d9d640 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 0x55a9c46a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f455759e640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45440a32d0 0x7f45440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f454ffff640 1 -- 172.21.15.94:0/3702155055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 msgr2=0x55a9c46a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f454ffff640 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 0x55a9c46a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f454ffff640 1 -- 172.21.15.94:0/3702155055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46948d0 msgr2=0x55a9c4694cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.769+0000 7f454ffff640 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46948d0 0x55a9c4694cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.770+0000 7f454ffff640 1 -- 172.21.15.94:0/3702155055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9c45af130 con 0x55a9c46a1dc0 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.770+0000 7f4556d9d640 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 0x55a9c46a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.770+0000 7f4554d99640 1 -- 172.21.15.94:0/22949712 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f45480cf7c0 con 0x7f4558100a80 2024-09-19T04:14:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.770+0000 7f454ffff640 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46a1dc0 0x55a9c46a41b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4540007800 tx=0x7f4540007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ec341f52f4f3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.770+0000 7f454d7fa640 1 -- 172.21.15.94:0/3702155055 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454000d950 con 0x55a9c46a1dc0 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.770+0000 7f454d7fa640 1 -- 172.21.15.94:0/3702155055 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4540018070 con 0x55a9c46a1dc0 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.770+0000 7f454d7fa640 1 -- 172.21.15.94:0/3702155055 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45400123e0 con 0x55a9c46a1dc0 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 -- 172.21.15.94:0/3702155055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46a1dc0 msgr2=0x55a9c46a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455759e640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45440a32d0 0x7f45440a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4550005e30 tx=0x7f45500095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46a1dc0 0x55a9c46a41b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4540007800 tx=0x7f4540007cc0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 -- 172.21.15.94:0/3702155055 shutdown_connections 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46a1dc0 0x55a9c46a41b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 0x55a9c46a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3702155055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46948d0 0x55a9c4694cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 -- 172.21.15.94:0/3702155055 >> 172.21.15.94:0/3702155055 conn(0x55a9c4690f60 msgr2=0x55a9c4691340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 -- 172.21.15.94:0/3702155055 shutdown_connections 2024-09-19T04:14:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 -- 172.21.15.94:0/3702155055 wait complete. 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 -- start start 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 Processor -- start 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46a1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 0x55a9c469e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 0x55a9c469ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9c46a49f0 con 0x55a9c46948d0 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a9c45b5f20 con 0x55a9c46951f0 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9c46a4b70 con 0x55a9c46a1dc0 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f455f3e5740 10 client.0 _ll_register_callbacks cb 0x55a9c457f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.771+0000 7f4556d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f4556d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59160/0 (socket says 172.21.15.94:59160) 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f4556d9d640 1 -- 172.21.15.94:0/1428304425 learned_addr learned my addr 172.21.15.94:0/1428304425 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f454ffff640 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 0x55a9c469e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f4556d9d640 1 -- 172.21.15.94:0/1428304425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 msgr2=0x55a9c469ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f455759e640 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 0x55a9c469ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f4556d9d640 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 0x55a9c469ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f4556d9d640 1 -- 172.21.15.94:0/1428304425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 msgr2=0x55a9c469e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f4556d9d640 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 0x55a9c469e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f4556d9d640 1 -- 172.21.15.94:0/1428304425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9c436c2b0 con 0x55a9c46948d0 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f4556d9d640 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46a1590 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f454806ea90 tx=0x7f454806da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317e12cad21005bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f453e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f453e7fc640 1 -- 172.21.15.94:0/1428304425 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548020070 con 0x55a9c46948d0 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f453e7fc640 1 -- 172.21.15.94:0/1428304425 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f45480982b0 con 0x55a9c46948d0 2024-09-19T04:14:54.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f455f3e5740 1 -- 172.21.15.94:0/1428304425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4540018070 con 0x55a9c46948d0 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f453e7fc640 1 -- 172.21.15.94:0/1428304425 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4548066ae0 con 0x55a9c46948d0 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f455f3e5740 10 client.15129 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f455f3e5740 1 -- 172.21.15.94:0/1428304425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a9c46b0250 con 0x55a9c46948d0 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.772+0000 7f453e7fc640 1 -- 172.21.15.94:0/1428304425 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4540018070 con 0x55a9c46948d0 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f453e7fc640 1 -- 172.21.15.94:0/1428304425 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f45480cd380 con 0x55a9c46948d0 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f455f3e5740 10 client.15129 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f455f3e5740 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f455f3e5740 20 client.15129 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a9c46b2830 0x55a9c46b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f455f3e5740 4 client.15129 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f455f3e5740 1 -- 172.21.15.94:0/1428304425 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9c4608600 con 0x55a9c46b2830 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f454ffff640 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a9c46b2830 0x55a9c46b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f453e7fc640 10 client.15129 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.773+0000 7f454ffff640 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a9c46b2830 0x55a9c46b3ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2833d3b8245307b9 server_cookie=b7491f18660f4fc7 in_seq=0 out_seq=0 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f453e7fc640 1 -- 172.21.15.94:0/1428304425 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9c4608600 con 0x55a9c46b2830 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f453e7fc640 10 client.15129 handle_command_reply: tid=0 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 client.15129 shutdown 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 -- 172.21.15.94:0/1428304425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 msgr2=0x55a9c46a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46a1590 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f454806ea90 tx=0x7f454806da10 comp rx=0 tx=0).stop 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 -- 172.21.15.94:0/1428304425 shutdown_connections 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a9c46b2830 0x55a9c46b3ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 0x55a9c469ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 0x55a9c469e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1428304425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46a1590 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f453e7fc640 0 client.15129 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:14:54.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 -- 172.21.15.94:0/1428304425 >> 172.21.15.94:0/1428304425 conn(0x55a9c4690f60 msgr2=0x55a9c4696cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:54.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.774+0000 7f455f3e5740 1 -- 172.21.15.94:0/1428304425 shutdown_connections 2024-09-19T04:14:54.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.775+0000 7f455f3e5740 1 -- 172.21.15.94:0/1428304425 wait complete. 2024-09-19T04:14:54.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.775+0000 7f455f3e5740 20 client.15129 trim_cache size 0 max 16384 2024-09-19T04:14:54.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.792+0000 7f455f3e5740 1 Processor -- start 2024-09-19T04:14:54.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f455f3e5740 1 -- start start 2024-09-19T04:14:54.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 0x55a9c468bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46a1dc0 0x55a9c46bc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f455f3e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9c46b1fe0 con 0x55a9c46948d0 2024-09-19T04:14:54.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f455f3e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a9c46b26b0 con 0x55a9c46a1dc0 2024-09-19T04:14:54.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f455f3e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9c46b2d80 con 0x55a9c46951f0 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f4556d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f4556d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59166/0 (socket says 172.21.15.94:59166) 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f4556d9d640 1 -- 172.21.15.94:0/1527042854 learned_addr learned my addr 172.21.15.94:0/1527042854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f455759e640 1 --2- 172.21.15.94:0/1527042854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46a1dc0 0x55a9c46bc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f454ffff640 1 --2- 172.21.15.94:0/1527042854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 0x55a9c468bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f4556d9d640 1 -- 172.21.15.94:0/1527042854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 msgr2=0x55a9c468bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f4556d9d640 1 --2- 172.21.15.94:0/1527042854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 0x55a9c468bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f4556d9d640 1 -- 172.21.15.94:0/1527042854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46a1dc0 msgr2=0x55a9c46bc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f4556d9d640 1 --2- 172.21.15.94:0/1527042854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46a1dc0 0x55a9c46bc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f4556d9d640 1 -- 172.21.15.94:0/1527042854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9c46a1900 con 0x55a9c46948d0 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.793+0000 7f4556d9d640 1 --2- 172.21.15.94:0/1527042854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b3ca0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4548086010 tx=0x7f45480cd0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921ea1f8e9bf9e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.794+0000 7f454cff9640 1 -- 172.21.15.94:0/1527042854 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548020030 con 0x55a9c46948d0 2024-09-19T04:14:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.794+0000 7f454cff9640 1 -- 172.21.15.94:0/1527042854 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f454809c340 con 0x55a9c46948d0 2024-09-19T04:14:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.794+0000 7f454cff9640 1 -- 172.21.15.94:0/1527042854 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454809bc40 con 0x55a9c46948d0 2024-09-19T04:14:54.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.794+0000 7f455f3e5740 1 -- 172.21.15.94:0/1527042854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 msgr2=0x55a9c46b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.794+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1527042854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b3ca0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4548086010 tx=0x7f45480cd0f0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 -- 172.21.15.94:0/1527042854 shutdown_connections 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1527042854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46a1dc0 0x55a9c46bc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1527042854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46951f0 0x55a9c468bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 --2- 172.21.15.94:0/1527042854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b3ca0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 -- 172.21.15.94:0/1527042854 >> 172.21.15.94:0/1527042854 conn(0x55a9c4690f60 msgr2=0x55a9c469e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 -- 172.21.15.94:0/1527042854 shutdown_connections 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 -- 172.21.15.94:0/1527042854 wait complete. 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 -- start start 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 Processor -- start 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 0x55a9c46b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 0x55a9c46b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9c46b3b20 con 0x55a9c46948d0 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a9c46b3450 con 0x55a9c46951f0 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9c46b41f0 con 0x55a9c46a1dc0 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f455f3e5740 10 client.0 _ll_register_callbacks cb 0x7f454800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f4556d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f4556d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59176/0 (socket says 172.21.15.94:59176) 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f4556d9d640 1 -- 172.21.15.94:0/3608183263 learned_addr learned my addr 172.21.15.94:0/3608183263 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.795+0000 7f454ffff640 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 0x55a9c46b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455759e640 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 0x55a9c46b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f4556d9d640 1 -- 172.21.15.94:0/3608183263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 msgr2=0x55a9c46b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f4556d9d640 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 0x55a9c46b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f4556d9d640 1 -- 172.21.15.94:0/3608183263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 msgr2=0x55a9c46b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f4556d9d640 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 0x55a9c46b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f4556d9d640 1 -- 172.21.15.94:0/3608183263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9c46b5390 con 0x55a9c46948d0 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f4556d9d640 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b63a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4548012040 tx=0x7f45480a00e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab27240e00c652cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f453e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f453e7fc640 1 -- 172.21.15.94:0/3608183263 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548020070 con 0x55a9c46948d0 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f453e7fc640 1 -- 172.21.15.94:0/3608183263 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f454809b290 con 0x55a9c46948d0 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455f3e5740 1 -- 172.21.15.94:0/3608183263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454809c340 con 0x55a9c46948d0 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f453e7fc640 1 -- 172.21.15.94:0/3608183263 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f454809bc40 con 0x55a9c46948d0 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455f3e5740 10 client.15135 fetch_fsmap learned FSMap version 24 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455f3e5740 1 -- 172.21.15.94:0/3608183263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a9c471ea90 con 0x55a9c46948d0 2024-09-19T04:14:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f453e7fc640 1 -- 172.21.15.94:0/3608183263 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f454809c340 con 0x55a9c46948d0 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f453e7fc640 1 -- 172.21.15.94:0/3608183263 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f454809ab90 con 0x55a9c46948d0 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455f3e5740 10 client.15135 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455f3e5740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455f3e5740 20 client.15135 populate_metadata read hostname 'smithi094' 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a9c4723260 0x55a9c4721f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455f3e5740 4 client.15135 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.796+0000 7f455f3e5740 1 -- 172.21.15.94:0/3608183263 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "status"}) -- 0x55a9c4602b10 con 0x55a9c4723260 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f454ffff640 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a9c4723260 0x55a9c4721f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f454ffff640 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a9c4723260 0x55a9c4721f70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba554a5981dd52f server_cookie=79b5436aeff081 in_seq=0 out_seq=0 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f453e7fc640 10 client.15135 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f453e7fc640 1 -- 172.21.15.94:0/3608183263 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a9c4602b10 con 0x55a9c4723260 2024-09-19T04:14:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f453e7fc640 10 client.15135 handle_command_reply: tid=0 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 client.15135 shutdown 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 -- 172.21.15.94:0/3608183263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 msgr2=0x55a9c46b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b63a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4548012040 tx=0x7f45480a00e0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 -- 172.21.15.94:0/3608183263 shutdown_connections 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a9c4723260 0x55a9c4721f70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9c46a1dc0 0x55a9c46b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a9c46951f0 0x55a9c46b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 --2- 172.21.15.94:0/3608183263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9c46948d0 0x55a9c46b63a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 -- 172.21.15.94:0/3608183263 >> 172.21.15.94:0/3608183263 conn(0x55a9c4690f60 msgr2=0x55a9c469d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f453e7fc640 0 client.15135 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.797+0000 7f455f3e5740 1 -- 172.21.15.94:0/3608183263 shutdown_connections 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455f3e5740 1 -- 172.21.15.94:0/3608183263 wait complete. 2024-09-19T04:14:54.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455f3e5740 20 client.15135 trim_cache size 0 max 16384 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "503c4526-763c-11ef-bceb-c7b262605968", 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24467, 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 97.350267275999997, 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-09-19T04:14:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T04:14:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 107.994198944 2024-09-19T04:14:54.807 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:14:54.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 -- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45440a32d0 msgr2=0x7f45440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:54.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45440a32d0 0x7f45440a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4550005e30 tx=0x7f45500095c0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 -- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558100a80 msgr2=0x7f45581972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:54.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558100a80 0x7f45581972d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f454800d950 tx=0x7f454800de10 comp rx=0 tx=0).stop 2024-09-19T04:14:54.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 -- 172.21.15.94:0/22949712 shutdown_connections 2024-09-19T04:14:54.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f45440a32d0 0x7f45440a5790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4558106a80 0x7f455819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4558101420 0x7f4558197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 --2- 172.21.15.94:0/22949712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4558100a80 0x7f45581972d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:54.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 -- 172.21.15.94:0/22949712 >> 172.21.15.94:0/22949712 conn(0x7f45580fc7a0 msgr2=0x7f45580fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:54.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 -- 172.21.15.94:0/22949712 shutdown_connections 2024-09-19T04:14:54.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:54.798+0000 7f455d00c640 1 -- 172.21.15.94:0/22949712 wait complete. 2024-09-19T04:14:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:55 smithi094 ceph-mon[30972]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:55 smithi104 ceph-mon[34145]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:55.681 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T04:14:55.691 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T04:14:55.691 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T04:14:55.692 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T04:14:55.692 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T04:14:55.693 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:14:55.693 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T04:14:55.693 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T04:14:55.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:55 smithi112 ceph-mon[35051]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T04:14:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.795+0000 7f7632edd640 1 -- 172.21.15.94:0/2640247464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c0ff8e0 msgr2=0x7f762c0ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.795+0000 7f7632edd640 1 --2- 172.21.15.94:0/2640247464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c0ff8e0 0x7f762c0ffce0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7620009900 tx=0x7f762002a410 comp rx=0 tx=0).stop 2024-09-19T04:14:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.795+0000 7f7632edd640 1 -- 172.21.15.94:0/2640247464 shutdown_connections 2024-09-19T04:14:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.795+0000 7f7632edd640 1 --2- 172.21.15.94:0/2640247464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762c1077d0 0x7f762c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.795+0000 7f7632edd640 1 --2- 172.21.15.94:0/2640247464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c100220 0x7f762c107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.795+0000 7f7632edd640 1 --2- 172.21.15.94:0/2640247464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c0ff8e0 0x7f762c0ffce0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.795+0000 7f7632edd640 1 -- 172.21.15.94:0/2640247464 >> 172.21.15.94:0/2640247464 conn(0x7f762c0fb500 msgr2=0x7f762c0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.795+0000 7f7632edd640 1 -- 172.21.15.94:0/2640247464 shutdown_connections 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.795+0000 7f7632edd640 1 -- 172.21.15.94:0/2640247464 wait complete. 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f7632edd640 1 Processor -- start 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f7632edd640 1 -- start start 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f7632edd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762c0ff8e0 0x7f762c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f7632edd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c100220 0x7f762c1011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f7632edd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c1077d0 0x7f762c101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f7632edd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f762c10d6b0 con 0x7f762c1077d0 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f7632edd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f762c10d530 con 0x7f762c0ff8e0 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f7632edd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f762c10d830 con 0x7f762c100220 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f763147a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c1077d0 0x7f762c101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f763147a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c1077d0 0x7f762c101710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59194/0 (socket says 172.21.15.94:59194) 2024-09-19T04:14:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f763147a640 1 -- 172.21.15.94:0/952459308 learned_addr learned my addr 172.21.15.94:0/952459308 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f7630c79640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762c0ff8e0 0x7f762c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f763147a640 1 -- 172.21.15.94:0/952459308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c100220 msgr2=0x7f762c1011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f763147a640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c100220 0x7f762c1011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f763147a640 1 -- 172.21.15.94:0/952459308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762c0ff8e0 msgr2=0x7f762c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f763147a640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762c0ff8e0 0x7f762c102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.796+0000 7f763147a640 1 -- 172.21.15.94:0/952459308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c101fa0 con 0x7f762c1077d0 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.797+0000 7f763147a640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c1077d0 0x7f762c101710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7624009fd0 tx=0x7f762400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.797+0000 7f7629ffb640 1 -- 172.21.15.94:0/952459308 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624017070 con 0x7f762c1077d0 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.797+0000 7f7629ffb640 1 -- 172.21.15.94:0/952459308 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7624003e20 con 0x7f762c1077d0 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.797+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762c1a1e50 con 0x7f762c1077d0 2024-09-19T04:14:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.797+0000 7f7629ffb640 1 -- 172.21.15.94:0/952459308 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240106c0 con 0x7f762c1077d0 2024-09-19T04:14:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.797+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762c1a22c0 con 0x7f762c1077d0 2024-09-19T04:14:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.798+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762c106050 con 0x7f762c1077d0 2024-09-19T04:14:55.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.799+0000 7f7629ffb640 1 -- 172.21.15.94:0/952459308 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7624003970 con 0x7f762c1077d0 2024-09-19T04:14:55.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.800+0000 7f7629ffb640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f761c0a3390 0x7f761c0a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:55.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.800+0000 7f7630c79640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f761c0a3390 0x7f761c0a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:55.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.800+0000 7f7629ffb640 1 -- 172.21.15.94:0/952459308 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f76240cf810 con 0x7f762c1077d0 2024-09-19T04:14:55.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.800+0000 7f7630c79640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f761c0a3390 0x7f761c0a5850 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f762002a930 tx=0x7f7620000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:55.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:55.804+0000 7f7629ffb640 1 -- 172.21.15.94:0/952459308 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f762409c390 con 0x7f762c1077d0 2024-09-19T04:14:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.016+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f762c02da10 con 0x7f762c1077d0 2024-09-19T04:14:56.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.345+0000 7f7629ffb640 1 -- 172.21.15.94:0/952459308 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7f76240047e0 con 0x7f762c1077d0 2024-09-19T04:14:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f761c0a3390 msgr2=0x7f761c0a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:56.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f761c0a3390 0x7f761c0a5850 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f762002a930 tx=0x7f7620000bc0 comp rx=0 tx=0).stop 2024-09-19T04:14:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c1077d0 msgr2=0x7f762c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c1077d0 0x7f762c101710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7624009fd0 tx=0x7f762400b260 comp rx=0 tx=0).stop 2024-09-19T04:14:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 shutdown_connections 2024-09-19T04:14:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f761c0a3390 0x7f761c0a5850 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f762c1077d0 0x7f762c101710 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c100220 0x7f762c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 --2- 172.21.15.94:0/952459308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762c0ff8e0 0x7f762c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.348+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 >> 172.21.15.94:0/952459308 conn(0x7f762c0fb500 msgr2=0x7f762c103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:56.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.349+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 shutdown_connections 2024-09-19T04:14:56.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:56.350+0000 7f7632edd640 1 -- 172.21.15.94:0/952459308 wait complete. 2024-09-19T04:14:56.367 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-19T04:14:56.394 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-19T04:14:56.394 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-19T04:14:56.394 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-09-19T04:14:56.395 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:14:56.395 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-19T04:14:56.395 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-19T04:14:56.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:56.395 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-19T04:14:56.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:56.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-19T04:14:56.482 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-09-19T04:14:56.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:14:56.483 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-19T04:14:56.483 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T04:14:56.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T04:14:56.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:56 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/952459308' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:14:56.591 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: ignoring --setuser ceph since I am not root 2024-09-19T04:14:56.591 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: ignoring --setgroup ceph since I am not root 2024-09-19T04:14:56.591 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:56.501+0000 7fa0177de040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:14:56.591 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:56.561+0000 7fa0177de040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:14:56.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:56 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/952459308' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:14:56.604 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: ignoring --setuser ceph since I am not root 2024-09-19T04:14:56.605 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: ignoring --setgroup ceph since I am not root 2024-09-19T04:14:56.605 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:56 smithi104 ceph-mgr[35518]: -- 172.21.15.104:0/2917074898 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7ff5210e0 con 0x55f7ff4ff400 2024-09-19T04:14:56.605 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:56.496+0000 7f9939790040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:14:56.605 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:56.554+0000 7f9939790040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:14:56.704 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:56 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/952459308' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:14:56.704 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: ignoring --setuser ceph since I am not root 2024-09-19T04:14:56.704 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: ignoring --setgroup ceph since I am not root 2024-09-19T04:14:56.704 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:56 smithi112 ceph-mgr[36863]: -- 172.21.15.112:0/3336088618 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560018570e0 con 0x556001834800 2024-09-19T04:14:56.704 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:56.499+0000 7fb9e11ca040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:14:56.705 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:56.556+0000 7fb9e11ca040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:14:56.705 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:56.702+0000 7fb9e11ca040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:14:56.782 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:57.034 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:56.768+0000 7fb9e11ca040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:14:57.035 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:56.913+0000 7fb9e11ca040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:14:57.036 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:56.714+0000 7fa0177de040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:14:57.036 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:56.792+0000 7fa0177de040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:14:57.036 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:56.945+0000 7fa0177de040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:14:57.052 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:56.702+0000 7f9939790040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:14:57.052 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:56.767+0000 7f9939790040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:14:57.052 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:56.912+0000 7f9939790040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:14:57.053 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:57 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:57.051+0000 7f9939790040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:14:57.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.274+0000 7f189fe14640 1 -- 172.21.15.94:0/3331873174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 msgr2=0x7f18981021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:57.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.274+0000 7f189fe14640 1 --2- 172.21.15.94:0/3331873174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 0x7f18981021d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1888009900 tx=0x7f188802a410 comp rx=0 tx=0).stop 2024-09-19T04:14:57.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.274+0000 7f189fe14640 1 -- 172.21.15.94:0/3331873174 shutdown_connections 2024-09-19T04:14:57.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.274+0000 7f189fe14640 1 --2- 172.21.15.94:0/3331873174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1898106f20 0x7f189810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.274+0000 7f189fe14640 1 --2- 172.21.15.94:0/3331873174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18981027a0 0x7f18981067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.274+0000 7f189fe14640 1 --2- 172.21.15.94:0/3331873174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 0x7f18981021d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.274+0000 7f189fe14640 1 -- 172.21.15.94:0/3331873174 >> 172.21.15.94:0/3331873174 conn(0x7f18980fd8f0 msgr2=0x7f18980ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.274+0000 7f189fe14640 1 -- 172.21.15.94:0/3331873174 shutdown_connections 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.275+0000 7f189fe14640 1 -- 172.21.15.94:0/3331873174 wait complete. 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.275+0000 7f189fe14640 1 Processor -- start 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.275+0000 7f189fe14640 1 -- start start 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.275+0000 7f189fe14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 0x7f1898198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.275+0000 7f189fe14640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18981027a0 0x7f1898198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.275+0000 7f189fe14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898106f20 0x7f189819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.275+0000 7f189fe14640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1898111e30 con 0x7f1898101df0 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.275+0000 7f189fe14640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1898111cb0 con 0x7f18981027a0 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.275+0000 7f189fe14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1898111fb0 con 0x7f1898106f20 2024-09-19T04:14:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189dbb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 0x7f1898198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189d3af640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18981027a0 0x7f1898198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189d3af640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18981027a0 0x7f1898198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:34970/0 (socket says 172.21.15.94:34970) 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189dbb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 0x7f1898198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59218/0 (socket says 172.21.15.94:59218) 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189d3af640 1 -- 172.21.15.94:0/4002954094 learned_addr learned my addr 172.21.15.94:0/4002954094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189e3b1640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898106f20 0x7f189819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189e3b1640 1 -- 172.21.15.94:0/4002954094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18981027a0 msgr2=0x7f1898198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189e3b1640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18981027a0 0x7f1898198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189e3b1640 1 -- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 msgr2=0x7f1898198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189e3b1640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 0x7f1898198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189e3b1640 1 -- 172.21.15.94:0/4002954094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189819d3f0 con 0x7f1898106f20 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189d3af640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18981027a0 0x7f1898198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:14:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189dbb0640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 0x7f1898198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:14:57.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189e3b1640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898106f20 0x7f189819cd10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f189000e940 tx=0x7f189000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:57.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f188effd640 1 -- 172.21.15.94:0/4002954094 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890004d20 con 0x7f1898106f20 2024-09-19T04:14:57.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189819d6e0 con 0x7f1898106f20 2024-09-19T04:14:57.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f188effd640 1 -- 172.21.15.94:0/4002954094 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1890005020 con 0x7f1898106f20 2024-09-19T04:14:57.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f188effd640 1 -- 172.21.15.94:0/4002954094 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18900116d0 con 0x7f1898106f20 2024-09-19T04:14:57.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.276+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18981a4fc0 con 0x7f1898106f20 2024-09-19T04:14:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.278+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18981021d0 con 0x7f1898106f20 2024-09-19T04:14:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.279+0000 7f188effd640 1 -- 172.21.15.94:0/4002954094 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138423+0+0 (secure 0 0 0) 0x7f189000cb20 con 0x7f1898106f20 2024-09-19T04:14:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.280+0000 7f188effd640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f18840a3320 0x7f18840a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.280+0000 7f189dbb0640 1 -- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f18840a3320 msgr2=0x7f18840a57e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2308569937 2024-09-19T04:14:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.280+0000 7f188effd640 1 -- 172.21.15.94:0/4002954094 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f18900cfb40 con 0x7f1898106f20 2024-09-19T04:14:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.280+0000 7f189dbb0640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f18840a3320 0x7f18840a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:14:57.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.284+0000 7f188effd640 1 -- 172.21.15.94:0/4002954094 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1890016030 con 0x7f1898106f20 2024-09-19T04:14:57.340 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:57.103+0000 7fa0177de040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:14:57.348 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:57.054+0000 7fb9e11ca040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:14:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.481+0000 7f189dbb0640 1 -- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f18840a3320 msgr2=0x7f18840a57e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2308569937 2024-09-19T04:14:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.481+0000 7f189dbb0640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f18840a3320 0x7f18840a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:14:57.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.496+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f189810a160 con 0x7f1898106f20 2024-09-19T04:14:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.496+0000 7f188effd640 1 -- 172.21.15.94:0/4002954094 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f189009c0b0 con 0x7f1898106f20 2024-09-19T04:14:57.498 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T04:14:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f18840a3320 msgr2=0x7f18840a57e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:14:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f18840a3320 0x7f18840a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898106f20 msgr2=0x7f189819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898106f20 0x7f189819cd10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f189000e940 tx=0x7f189000ee00 comp rx=0 tx=0).stop 2024-09-19T04:14:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 shutdown_connections 2024-09-19T04:14:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:6800/2308569937,v1:172.21.15.94:6801/2308569937] conn(0x7f18840a3320 0x7f18840a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898106f20 0x7f189819cd10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18981027a0 0x7f1898198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 --2- 172.21.15.94:0/4002954094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1898101df0 0x7f1898198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 >> 172.21.15.94:0/4002954094 conn(0x7f18980fd8f0 msgr2=0x7f18980fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 shutdown_connections 2024-09-19T04:14:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:57.500+0000 7f189fe14640 1 -- 172.21.15.94:0/4002954094 wait complete. 2024-09-19T04:14:57.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:57 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/952459308' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T04:14:57.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:57 smithi104 ceph-mon[34145]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T04:14:57.603 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:57 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:57.359+0000 7f9939790040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:14:57.603 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:57 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:57.418+0000 7f9939790040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:14:57.614 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:57 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/952459308' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T04:14:57.614 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:57 smithi112 ceph-mon[35051]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T04:14:57.614 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:57.367+0000 7fb9e11ca040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:14:57.614 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:57.427+0000 7fb9e11ca040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:14:57.614 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:57.612+0000 7fb9e11ca040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:14:57.690 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:57 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/952459308' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T04:14:57.690 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:57 smithi094 ceph-mon[30972]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T04:14:57.691 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:57.429+0000 7fa0177de040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:14:57.691 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:57.491+0000 7fa0177de040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:14:57.691 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:57.688+0000 7fa0177de040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:14:57.854 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:57 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:57.601+0000 7f9939790040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:14:58.302 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:14:58.302 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:14:58.302 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-19T04:14:58.302 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:14:58.303 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-19T04:14:58.303 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:14:58.303 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:14:58.303 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:14:58.303 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-19T04:14:58.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:58.303 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-19T04:14:58.320 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:14:58.320 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:14:58.320 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:14:58.320 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:58.320 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-19T04:14:58.320 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:58.320 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:14:58.320 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:58.320 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42188sec preferred_lft 42188sec 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link noprefixroute 2024-09-19T04:14:58.321 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:58.322 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T04:14:58.322 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:14:58.322 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-19T04:14:58.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-09-19T04:14:58.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-09-19T04:14:58.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-09-19T04:14:58.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T04:14:58.322 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-19T04:14:58.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:14:58.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:58.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:14:58 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4002954094' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:14:58.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:58.530 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:14:58.555 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-19T04:14:58.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:58.557 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-19T04:14:58.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:14:58 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4002954094' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:14:58.604 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:58 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:58.476+0000 7f9939790040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:14:58.604 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:58 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:58.543+0000 7f9939790040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:14:58.636 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-19T04:14:58.636 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:14:58.637 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:14:58.637 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:14:58.637 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:14:58.637 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:14:58.637 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-19T04:14:58.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:14:58.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:14:58.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:14:58.638 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-19T04:14:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:14:58.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:14:58 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4002954094' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:14:58.785 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:58.496+0000 7fb9e11ca040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:14:58.785 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:58.564+0000 7fb9e11ca040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:14:58.840 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:58 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:58.603+0000 7fa0177de040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:14:58.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:58 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:58.674+0000 7fa0177de040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:14:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:58.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:58.942 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-19T04:14:58.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:58.958 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-09-19T04:14:59.014 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:14:59.014 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-19T04:14:59.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-19T04:14:59.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-19T04:14:59.014 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-19T04:14:59.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:14:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:59.132 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-19T04:14:59.133 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:14:59.133 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-19T04:14:59.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-19T04:14:59.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T04:14:59.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-19T04:14:59.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-19T04:14:59.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-19T04:14:59.133 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-19T04:14:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:14:59.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:59.259 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:14:59.259 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-19T04:14:59.259 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-19T04:14:59.259 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T04:14:59.259 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-19T04:14:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:14:59.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:59.338 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-09-19T04:14:59.338 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:14:59.338 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /tmp/cephfs 2024-09-19T04:14:59.354 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-19T04:14:59.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:14:59.355 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /tmp/cephfs 2024-09-19T04:14:59.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-19T04:14:59.477 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:14:59.504 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:59.245+0000 7f9939790040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:14:59.504 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:59.359+0000 7f9939790040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:14:59.534 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:59.271+0000 7fb9e11ca040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:14:59.534 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:59.387+0000 7fb9e11ca040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-19T04:14:59.536 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-19T04:14:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/b9aaab4d975ae60d11d838802f7413c1d60e1d4f69126bbd41ead298d641a79f/merged 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/b4d6dc4bb6a0df7ead24fa051ae9b3b640a01d085a5b1dfb80d07fb1e5be696c/merged 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/0 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/fbb9f24b6328aa2f356075953a64c67e75c8d356c20403c9efcb188da07de762/merged 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/21ea56250d801208ad5093c1fe7df638e5049838c8679bac80acb45a397649bf/merged 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/7dae40fe4bc08ef62932d77796e7d458d2115f279e261e8dd5c7f6a10d4afcf7/merged 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/5f21cbabd784b7f3ebcdbcd0d2bf794ccf453296bca257a0937fcd887dbbc93d/merged 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6e173205bd8027033497089251d04f566afd2198fe460c1d99203bceeb8d069a/merged 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/a14ec91449a766c4ddf99b03c01dc9671dd9de07f958ab00a787427927719c57/merged 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/60c7f104f253b64a26ced1973a949c5c28cbc86cc0a7ffb173309b3864fddd3a/merged 2024-09-19T04:14:59.538 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/3c4685331bfa5983449e5b6d228a1e173e4d9b9e9bb8e106ee21060118dbc822/merged 2024-09-19T04:14:59.539 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-19T04:14:59.539 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-19T04:14:59.539 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-19T04:14:59.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:59.539 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-19T04:14:59.592 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:14:59.592 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/e3926d97-f545-4f3f-8efc-f496b72b2ed6 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-09-19T04:14:59.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-19T04:14:59.659 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:14:59.703 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:59.430+0000 7fa0177de040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:14:59.703 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:59.552+0000 7fa0177de040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:14:59.715 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-19T04:14:59.715 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-19T04:14:59.715 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-19T04:14:59.715 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-19T04:14:59.715 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-19T04:14:59.715 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T04:14:59.716 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/b9aaab4d975ae60d11d838802f7413c1d60e1d4f69126bbd41ead298d641a79f/merged 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/b4d6dc4bb6a0df7ead24fa051ae9b3b640a01d085a5b1dfb80d07fb1e5be696c/merged 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/0 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:14:59.717 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/fbb9f24b6328aa2f356075953a64c67e75c8d356c20403c9efcb188da07de762/merged 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/21ea56250d801208ad5093c1fe7df638e5049838c8679bac80acb45a397649bf/merged 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/7dae40fe4bc08ef62932d77796e7d458d2115f279e261e8dd5c7f6a10d4afcf7/merged 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/5f21cbabd784b7f3ebcdbcd0d2bf794ccf453296bca257a0937fcd887dbbc93d/merged 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6e173205bd8027033497089251d04f566afd2198fe460c1d99203bceeb8d069a/merged 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/a14ec91449a766c4ddf99b03c01dc9671dd9de07f958ab00a787427927719c57/merged 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/60c7f104f253b64a26ced1973a949c5c28cbc86cc0a7ffb173309b3864fddd3a/merged 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/3c4685331bfa5983449e5b6d228a1e173e4d9b9e9bb8e106ee21060118dbc822/merged 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-19T04:14:59.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:59.719 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-19T04:14:59.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-19T04:14:59.742+0000 7f063572f480 -1 init, newargv = 0x55b1b7b66f90 newargc=17 2024-09-19T04:14:59.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[82549]: starting ceph client 2024-09-19T04:14:59.782 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:59.502+0000 7f9939790040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:14:59.782 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:59.563+0000 7f9939790040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:14:59.782 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:59.711+0000 7f9939790040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:14:59.782 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:59.780+0000 7f9939790040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:14:59.784 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:59.532+0000 7fb9e11ca040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:14:59.784 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:59.593+0000 7fb9e11ca040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:14:59.785 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:59.741+0000 7fb9e11ca040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:14:59.987 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:59.702+0000 7fa0177de040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:14:59.987 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:59.768+0000 7fa0177de040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:14:59.987 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:59.917+0000 7fa0177de040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:15:00.063 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:59.840+0000 7f9939790040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:15:00.063 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:14:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:14:59.997+0000 7f9939790040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:15:00.063 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:15:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:15:00.061+0000 7f9939790040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:15:00.095 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:59.810+0000 7fb9e11ca040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:15:00.095 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:14:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:14:59.871+0000 7fb9e11ca040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:15:00.095 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:15:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:15:00.029+0000 7fb9e11ca040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:15:00.095 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:15:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:15:00.094+0000 7fb9e11ca040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:15:00.268 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:14:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:14:59.985+0000 7fa0177de040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:15:00.268 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:00.045+0000 7fa0177de040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:15:00.268 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:00.200+0000 7fa0177de040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:15:00.326 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:15:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:15:00.127+0000 7f9939790040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:15:00.326 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:15:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:15:00.207+0000 7f9939790040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:15:00.326 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:15:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:15:00.267+0000 7f9939790040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:15:00.326 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:15:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:15:00.324+0000 7f9939790040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:15:00.361 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:15:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:15:00.160+0000 7fb9e11ca040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:15:00.361 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:15:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:15:00.242+0000 7fb9e11ca040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:15:00.361 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:15:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:15:00.301+0000 7fb9e11ca040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:15:00.361 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:15:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:15:00.359+0000 7fb9e11ca040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:15:00.537 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:00.266+0000 7fa0177de040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:15:00.537 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:00.332+0000 7fa0177de040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:15:00.537 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:00.414+0000 7fa0177de040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:15:00.537 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:00.476+0000 7fa0177de040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:15:00.537 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:00.535+0000 7fa0177de040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:15:00.861 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-19T04:15:00.886 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:15:00.942 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:15:00.943 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay 2024-09-19T04:15:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/b9aaab4d975ae60d11d838802f7413c1d60e1d4f69126bbd41ead298d641a79f/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/b4d6dc4bb6a0df7ead24fa051ae9b3b640a01d085a5b1dfb80d07fb1e5be696c/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/0 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/fbb9f24b6328aa2f356075953a64c67e75c8d356c20403c9efcb188da07de762/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/21ea56250d801208ad5093c1fe7df638e5049838c8679bac80acb45a397649bf/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/7dae40fe4bc08ef62932d77796e7d458d2115f279e261e8dd5c7f6a10d4afcf7/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/5f21cbabd784b7f3ebcdbcd0d2bf794ccf453296bca257a0937fcd887dbbc93d/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6e173205bd8027033497089251d04f566afd2198fe460c1d99203bceeb8d069a/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/a14ec91449a766c4ddf99b03c01dc9671dd9de07f958ab00a787427927719c57/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/60c7f104f253b64a26ced1973a949c5c28cbc86cc0a7ffb173309b3864fddd3a/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/3c4685331bfa5983449e5b6d228a1e173e4d9b9e9bb8e106ee21060118dbc822/merged 2024-09-19T04:15:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-19T04:15:00.946 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-19T04:15:00.946 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-19T04:15:00.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:00.946 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-19T04:15:01.410 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[82549]: starting fuse 2024-09-19T04:15:01.534 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z[36859]: 2024-09-19T04:15:01.150+0000 7fb9e11ca040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:15:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: Standby manager daemon y restarted 2024-09-19T04:15:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: Standby manager daemon y started 2024-09-19T04:15:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:15:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:15:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:15:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:15:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: Standby manager daemon z restarted 2024-09-19T04:15:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: Standby manager daemon z started 2024-09-19T04:15:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:15:01.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:15:01.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:15:01.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:01 smithi112 ceph-mon[35051]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:15:01.590 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.336+0000 7fa0177de040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: Standby manager daemon y restarted 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: Standby manager daemon y started 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: Standby manager daemon z restarted 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: Standby manager daemon z started 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:15:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-mon[30972]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:15:01.604 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y[35514]: 2024-09-19T04:15:01.116+0000 7f9939790040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:15:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: Standby manager daemon y restarted 2024-09-19T04:15:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: Standby manager daemon y started 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.104:0/4039623837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: Standby manager daemon z restarted 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: Standby manager daemon z started 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:15:01.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:01 smithi104 ceph-mon[34145]: from='mgr.? 172.21.15.112:0/1687569683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:15:01.872 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.607+0000 7f9f81db6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:01.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.607+0000 7f9f81db6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:01.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.607+0000 7f9f81db6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:01.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.607+0000 7f9f81db6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:01.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.607+0000 7f9f81db6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:01.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.607+0000 7f9f7e5af640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:01.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.608+0000 7f9f7e5af640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:01.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.608+0000 7f9f7e5af640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:01.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.608+0000 7f9f7e5af640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:01.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:01.608+0000 7f9f7e5af640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:15:02.100 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-19T04:15:02.127 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:15:02.184 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-09-19T04:15:02.185 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:15:02.186 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-19T04:15:02.186 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-19T04:15:02.186 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-19T04:15:02.186 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-19T04:15:02.186 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T04:15:02.186 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-19T04:15:02.186 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-19T04:15:02.186 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/b9aaab4d975ae60d11d838802f7413c1d60e1d4f69126bbd41ead298d641a79f/merged 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/b4d6dc4bb6a0df7ead24fa051ae9b3b640a01d085a5b1dfb80d07fb1e5be696c/merged 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/0 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:15:02.188 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/fbb9f24b6328aa2f356075953a64c67e75c8d356c20403c9efcb188da07de762/merged 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/21ea56250d801208ad5093c1fe7df638e5049838c8679bac80acb45a397649bf/merged 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/7dae40fe4bc08ef62932d77796e7d458d2115f279e261e8dd5c7f6a10d4afcf7/merged 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/5f21cbabd784b7f3ebcdbcd0d2bf794ccf453296bca257a0937fcd887dbbc93d/merged 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6e173205bd8027033497089251d04f566afd2198fe460c1d99203bceeb8d069a/merged 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/a14ec91449a766c4ddf99b03c01dc9671dd9de07f958ab00a787427927719c57/merged 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/60c7f104f253b64a26ced1973a949c5c28cbc86cc0a7ffb173309b3864fddd3a/merged 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/3c4685331bfa5983449e5b6d228a1e173e4d9b9e9bb8e106ee21060118dbc822/merged 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-19T04:15:02.189 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs 2024-09-19T04:15:02.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:02.190 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-19T04:15:02.246 INFO:teuthology.orchestra.run.smithi094.stdout:120 2024-09-19T04:15:02.246 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-19T04:15:02.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:15:02.246 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-19T04:15:02.246 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-19T04:15:02.246 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/tmp/cephfs" 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-19T04:15:02.247 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:15:02.248 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:15:02.248 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:15:02.248 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-19T04:15:02.248 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-19T04:15:02.248 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-19T04:15:02.248 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:15:02.248 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-19T04:15:02.248 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-19T04:15:02.248 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-19T04:15:02.374 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.82549.asok 2024-09-19T04:15:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:15:02.384 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-19T04:15:02.384 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: Active manager daemon x restarted 2024-09-19T04:15:02.384 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: Activating manager daemon x 2024-09-19T04:15:02.384 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:15:02.384 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: mgrmap e20: x(active, starting, since 0.0120228s), standbys: y, z 2024-09-19T04:15:02.384 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:15:02.384 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:15:02.384 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:15:02.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:02.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:15:02.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:15:02.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:15:02.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:15:02.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:15:02.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:15:02.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: Manager daemon x is now available 2024-09-19T04:15:02.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:02.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:15:02.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:15:02.388 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.82549.asok 2024-09-19T04:15:02.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:02.388 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.82549.asok status 2024-09-19T04:15:02.502 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-19T04:15:02.502 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: Active manager daemon x restarted 2024-09-19T04:15:02.502 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: Activating manager daemon x 2024-09-19T04:15:02.502 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:15:02.502 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: mgrmap e20: x(active, starting, since 0.0120228s), standbys: y, z 2024-09-19T04:15:02.502 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:15:02.503 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:15:02.504 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: Manager daemon x is now available 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:15:02.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:02 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:15:02.523 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:15:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-19T04:15:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:15:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:15:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-19T04:15:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-19T04:15:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/tmp/cephfs", 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "82549", 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/volumes/qa/sv_0/e3926d97-f545-4f3f-8efc-f496b72b2ed6" 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 15171, 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 15171 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-19T04:15:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736942336 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736942336 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.15171 192.168.0.1:0/736942336", 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/736942336", 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 5, 2024-09-19T04:15:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 24, 2024-09-19T04:15:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 81, 2024-09-19T04:15:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:15:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-19T04:15:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-19T04:15:02.526 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:15:02.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:02.532 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-19T04:15:02.532 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /tmp/cephfs 2024-09-19T04:15:02.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-19T04:15:02.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: Active manager daemon x restarted 2024-09-19T04:15:02.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: Activating manager daemon x 2024-09-19T04:15:02.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:15:02.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: mgrmap e20: x(active, starting, since 0.0120228s), standbys: y, z 2024-09-19T04:15:02.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:15:02.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:15:02.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:15:02.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:15:02.543 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:02.544 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:15:02.545 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:15:02.545 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:15:02.545 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:15:02.545 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:15:02.545 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:15:02.545 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: Manager daemon x is now available 2024-09-19T04:15:02.545 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:02.545 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:15:02.545 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:02 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:15:02.548 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-19T04:15:02.548 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-09-19T04:15:02.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:02.548 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /tmp/cephfs 2024-09-19T04:15:02.617 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T04:15:02.627 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:15:02.628 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T04:15:02.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-19T04:15:02.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.757+0000 7f1ba6b86640 1 -- 172.21.15.94:0/273141647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 msgr2=0x7f1ba0101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:02.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.757+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/273141647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 0x7f1ba0101840 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009900 tx=0x7f1b9002a410 comp rx=0 tx=0).stop 2024-09-19T04:15:02.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.757+0000 7f1ba6b86640 1 -- 172.21.15.94:0/273141647 shutdown_connections 2024-09-19T04:15:02.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.757+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/273141647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0101d80 0x7f1ba0108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.757+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/273141647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 0x7f1ba0101840 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.757+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/273141647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba01001d0 0x7f1ba01005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.757+0000 7f1ba6b86640 1 -- 172.21.15.94:0/273141647 >> 172.21.15.94:0/273141647 conn(0x7f1ba00fb970 msgr2=0x7f1ba00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.757+0000 7f1ba6b86640 1 -- 172.21.15.94:0/273141647 shutdown_connections 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.757+0000 7f1ba6b86640 1 -- 172.21.15.94:0/273141647 wait complete. 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba6b86640 1 Processor -- start 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba6b86640 1 -- start start 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba6b86640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba01001d0 0x7f1ba0196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba6b86640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 0x7f1ba0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba6b86640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0101d80 0x7f1ba019e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba6b86640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba010c610 con 0x7f1ba01013c0 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba6b86640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba010c490 con 0x7f1ba0101d80 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba6b86640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba010c790 con 0x7f1ba01001d0 2024-09-19T04:15:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 0x7f1ba0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 0x7f1ba0197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46466/0 (socket says 172.21.15.94:46466) 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1b97fff640 1 -- 172.21.15.94:0/738149488 learned_addr learned my addr 172.21.15.94:0/738149488 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba5123640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0101d80 0x7f1ba019e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1ba4922640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba01001d0 0x7f1ba0196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1b97fff640 1 -- 172.21.15.94:0/738149488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba01001d0 msgr2=0x7f1ba0196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1b97fff640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba01001d0 0x7f1ba0196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1b97fff640 1 -- 172.21.15.94:0/738149488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0101d80 msgr2=0x7f1ba019e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.758+0000 7f1b97fff640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0101d80 0x7f1ba019e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.759+0000 7f1b97fff640 1 -- 172.21.15.94:0/738149488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba019e7f0 con 0x7f1ba01013c0 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.759+0000 7f1ba4922640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba01001d0 0x7f1ba0196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.759+0000 7f1b97fff640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 0x7f1ba0197110 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b900314e0 tx=0x7f1b90003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.759+0000 7f1b95ffb640 1 -- 172.21.15.94:0/738149488 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9003f070 con 0x7f1ba01013c0 2024-09-19T04:15:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.759+0000 7f1b95ffb640 1 -- 172.21.15.94:0/738149488 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b90031c40 con 0x7f1ba01013c0 2024-09-19T04:15:02.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.759+0000 7f1b95ffb640 1 -- 172.21.15.94:0/738149488 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90035030 con 0x7f1ba01013c0 2024-09-19T04:15:02.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.759+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba019ea80 con 0x7f1ba01013c0 2024-09-19T04:15:02.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.759+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba019efc0 con 0x7f1ba01013c0 2024-09-19T04:15:02.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.760+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba01074f0 con 0x7f1ba01013c0 2024-09-19T04:15:02.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.761+0000 7f1b95ffb640 1 -- 172.21.15.94:0/738149488 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7f1b90042070 con 0x7f1ba01013c0 2024-09-19T04:15:02.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.762+0000 7f1b95ffb640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b880a3180 0x7f1b880a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:02.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.762+0000 7f1ba4922640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b880a3180 0x7f1b880a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:02.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.762+0000 7f1b95ffb640 1 -- 172.21.15.94:0/738149488 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f1b900efda0 con 0x7f1ba01013c0 2024-09-19T04:15:02.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.762+0000 7f1ba4922640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b880a3180 0x7f1b880a5640 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0101220 tx=0x7f1b9c009500 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.766+0000 7f1b95ffb640 1 -- 172.21.15.94:0/738149488 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1b900bcc80 con 0x7f1ba01013c0 2024-09-19T04:15:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:02.970+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f1ba0062df0 con 0x7f1ba01013c0 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:02.661+0000 7f9f6b009640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:15:02] ENGINE Error in HTTPServer.serve 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: Traceback (most recent call last): 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self._connections.run(self.expiration_interval) 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self._run(expiration_interval) 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:15:03.091 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: s = self.context.wrap_socket( 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: return self.sslsocket_class._create( 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self.do_handshake() 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: self._sslobj.do_handshake() 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:15:03.092 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:15:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:15:03.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.349+0000 7f1b95ffb640 1 -- 172.21.15.94:0/738149488 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f1b900bc580 con 0x7f1ba01013c0 2024-09-19T04:15:03.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.351+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b880a3180 msgr2=0x7f1b880a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.351+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b880a3180 0x7f1b880a5640 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0101220 tx=0x7f1b9c009500 comp rx=0 tx=0).stop 2024-09-19T04:15:03.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.351+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 msgr2=0x7f1ba0197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.351+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 0x7f1ba0197110 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b900314e0 tx=0x7f1b90003c10 comp rx=0 tx=0).stop 2024-09-19T04:15:03.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.352+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 shutdown_connections 2024-09-19T04:15:03.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.352+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b880a3180 0x7f1b880a5640 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.352+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0101d80 0x7f1ba019e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.352+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba01013c0 0x7f1ba0197110 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.352+0000 7f1ba6b86640 1 --2- 172.21.15.94:0/738149488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba01001d0 0x7f1ba0196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.352+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 >> 172.21.15.94:0/738149488 conn(0x7f1ba00fb970 msgr2=0x7f1ba01046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:03.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.353+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 shutdown_connections 2024-09-19T04:15:03.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.353+0000 7f1ba6b86640 1 -- 172.21.15.94:0/738149488 wait complete. 2024-09-19T04:15:03.365 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T04:15:03.377 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T04:15:03.521 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:15:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:03 smithi104 ceph-mon[34145]: mgrmap e21: x(active, since 1.01754s), standbys: y, z 2024-09-19T04:15:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:03 smithi104 ceph-mon[34145]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:15:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:03 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/738149488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T04:15:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:03 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:03 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:03 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:03 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.623+0000 7efcb1b03640 1 -- 172.21.15.94:0/3071051439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac100d20 msgr2=0x7efcac101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.623+0000 7efcb1b03640 1 --2- 172.21.15.94:0/3071051439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac100d20 0x7efcac101180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efca0009900 tx=0x7efca002a410 comp rx=0 tx=0).stop 2024-09-19T04:15:03.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.623+0000 7efcb1b03640 1 -- 172.21.15.94:0/3071051439 shutdown_connections 2024-09-19T04:15:03.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.623+0000 7efcb1b03640 1 --2- 172.21.15.94:0/3071051439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcac1016c0 0x7efcac101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.623+0000 7efcb1b03640 1 --2- 172.21.15.94:0/3071051439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac100d20 0x7efcac101180 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.623+0000 7efcb1b03640 1 --2- 172.21.15.94:0/3071051439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcac1003e0 0x7efcac1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.623+0000 7efcb1b03640 1 -- 172.21.15.94:0/3071051439 >> 172.21.15.94:0/3071051439 conn(0x7efcac0fc170 msgr2=0x7efcac0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:03.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.623+0000 7efcb1b03640 1 -- 172.21.15.94:0/3071051439 shutdown_connections 2024-09-19T04:15:03.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.623+0000 7efcb1b03640 1 -- 172.21.15.94:0/3071051439 wait complete. 2024-09-19T04:15:03.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.624+0000 7efcb1b03640 1 Processor -- start 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.624+0000 7efcb1b03640 1 -- start start 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.624+0000 7efcb1b03640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac1003e0 0x7efcac199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.624+0000 7efcb1b03640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcac100d20 0x7efcac19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.624+0000 7efcb1b03640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcac1016c0 0x7efcac1a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.624+0000 7efcb1b03640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcac10f820 con 0x7efcac1003e0 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.624+0000 7efcb1b03640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efcac10f6a0 con 0x7efcac100d20 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.624+0000 7efcb1b03640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcac10f9a0 con 0x7efcac1016c0 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac1003e0 0x7efcac199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac1003e0 0x7efcac199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46504/0 (socket says 172.21.15.94:46504) 2024-09-19T04:15:03.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcab7fe640 1 -- 172.21.15.94:0/1045958143 learned_addr learned my addr 172.21.15.94:0/1045958143 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:03.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcabfff640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcac1016c0 0x7efcac1a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:03.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcaaffd640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcac100d20 0x7efcac19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:03.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcab7fe640 1 -- 172.21.15.94:0/1045958143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcac1016c0 msgr2=0x7efcac1a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcab7fe640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcac1016c0 0x7efcac1a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcab7fe640 1 -- 172.21.15.94:0/1045958143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcac100d20 msgr2=0x7efcac19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcab7fe640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcac100d20 0x7efcac19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcab7fe640 1 -- 172.21.15.94:0/1045958143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac1a1bb0 con 0x7efcac1003e0 2024-09-19T04:15:03.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcabfff640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcac1016c0 0x7efcac1a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcab7fe640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac1003e0 0x7efcac199f90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00e9f0 tx=0x7efc9c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efca8ff9640 1 -- 172.21.15.94:0/1045958143 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00cd80 con 0x7efcac1003e0 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efca8ff9640 1 -- 172.21.15.94:0/1045958143 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc9c0054d0 con 0x7efcac1003e0 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcac1a1ea0 con 0x7efcac1003e0 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efca8ff9640 1 -- 172.21.15.94:0/1045958143 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c010490 con 0x7efcac1003e0 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.625+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcac1a22b0 con 0x7efcac1003e0 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.626+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcac10b130 con 0x7efcac1003e0 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.628+0000 7efca8ff9640 1 -- 172.21.15.94:0/1045958143 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7efc9c005020 con 0x7efcac1003e0 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.628+0000 7efca8ff9640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc940a3050 0x7efc940a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:03.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.628+0000 7efcaaffd640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc940a3050 0x7efc940a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:03.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.628+0000 7efca8ff9640 1 -- 172.21.15.94:0/1045958143 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7efc9c0cf9a0 con 0x7efcac1003e0 2024-09-19T04:15:03.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.629+0000 7efcaaffd640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc940a3050 0x7efc940a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efca00314e0 tx=0x7efca0005a20 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:03.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.633+0000 7efca8ff9640 1 -- 172.21.15.94:0/1045958143 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc9c09c880 con 0x7efcac1003e0 2024-09-19T04:15:03.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:03 smithi112 ceph-mon[35051]: mgrmap e21: x(active, since 1.01754s), standbys: y, z 2024-09-19T04:15:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:03 smithi112 ceph-mon[35051]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:15:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:03 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/738149488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T04:15:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:03 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:03 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:03 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:03 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.811+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efcac062ec0 con 0x7efcac1003e0 2024-09-19T04:15:03.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.812+0000 7efca8ff9640 1 -- 172.21.15.94:0/1045958143 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+23281 (secure 0 0 0) 0x7efc9c09c180 con 0x7efcac1003e0 2024-09-19T04:15:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:03.815 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":82,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:15:01.339120+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3284064221":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/860358556":"2024-09-20T04:15:01.339086+0000","172.21.15.94:6801/2308569937":"2024-09-20T04:15:01.339086+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/3663117113":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3123077951":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/1265443753":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2308569937":"2024-09-20T04:15:01.339086+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:15:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc940a3050 msgr2=0x7efc940a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc940a3050 0x7efc940a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efca00314e0 tx=0x7efca0005a20 comp rx=0 tx=0).stop 2024-09-19T04:15:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac1003e0 msgr2=0x7efcac199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac1003e0 0x7efcac199f90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00e9f0 tx=0x7efc9c00edf0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 shutdown_connections 2024-09-19T04:15:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc940a3050 0x7efc940a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcac1016c0 0x7efcac1a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcac100d20 0x7efcac19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 --2- 172.21.15.94:0/1045958143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcac1003e0 0x7efcac199f90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.814+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 >> 172.21.15.94:0/1045958143 conn(0x7efcac0fc170 msgr2=0x7efcac0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:03.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.815+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 shutdown_connections 2024-09-19T04:15:03.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.815+0000 7efcb1b03640 1 -- 172.21.15.94:0/1045958143 wait complete. 2024-09-19T04:15:03.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:03 smithi094 ceph-mon[30972]: mgrmap e21: x(active, since 1.01754s), standbys: y, z 2024-09-19T04:15:03.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:03 smithi094 ceph-mon[30972]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:15:03.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:03 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/738149488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T04:15:03.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:03 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:03 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:03 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:03 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:03.828 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:08:46.366796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T04:13:12.965501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T04:13:15.212761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:15:03.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:15:03.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 -- 172.21.15.94:0/1819281683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 msgr2=0x7f96401072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 --2- 172.21.15.94:0/1819281683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 0x7f96401072a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f963c009900 tx=0x7f963c02a410 comp rx=0 tx=0).stop 2024-09-19T04:15:03.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 -- 172.21.15.94:0/1819281683 shutdown_connections 2024-09-19T04:15:03.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 --2- 172.21.15.94:0/1819281683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96401077e0 0x7f9640109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 --2- 172.21.15.94:0/1819281683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 0x7f96401072a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 --2- 172.21.15.94:0/1819281683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9640102580 0x7f9640104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 -- 172.21.15.94:0/1819281683 >> 172.21.15.94:0/1819281683 conn(0x7f96400fc140 msgr2=0x7f96400fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:03.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 -- 172.21.15.94:0/1819281683 shutdown_connections 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 -- 172.21.15.94:0/1819281683 wait complete. 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 Processor -- start 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.933+0000 7f96487c4640 1 -- start start 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96487c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9640102580 0x7f9640197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96487c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 0x7f9640198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96487c4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96401077e0 0x7f964019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96487c4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f964010d660 con 0x7f9640104eb0 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96487c4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f964010d4e0 con 0x7f96401077e0 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96487c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f964010d7e0 con 0x7f9640102580 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9645d5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 0x7f9640198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9645d5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 0x7f9640198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46528/0 (socket says 172.21.15.94:46528) 2024-09-19T04:15:03.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9645d5f640 1 -- 172.21.15.94:0/4203699358 learned_addr learned my addr 172.21.15.94:0/4203699358 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9646560640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9640102580 0x7f9640197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9646d61640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96401077e0 0x7f964019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9645d5f640 1 -- 172.21.15.94:0/4203699358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9640102580 msgr2=0x7f9640197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9645d5f640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9640102580 0x7f9640197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9645d5f640 1 -- 172.21.15.94:0/4203699358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96401077e0 msgr2=0x7f964019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9645d5f640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96401077e0 0x7f964019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9645d5f640 1 -- 172.21.15.94:0/4203699358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964019f9f0 con 0x7f9640104eb0 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f9645d5f640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 0x7f9640198310 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f963c0314e0 tx=0x7f963c031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96337fe640 1 -- 172.21.15.94:0/4203699358 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c03f030 con 0x7f9640104eb0 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96337fe640 1 -- 172.21.15.94:0/4203699358 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963c0044b0 con 0x7f9640104eb0 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964019fc80 con 0x7f9640104eb0 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96337fe640 1 -- 172.21.15.94:0/4203699358 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c035070 con 0x7f9640104eb0 2024-09-19T04:15:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.934+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96401a01c0 con 0x7f9640104eb0 2024-09-19T04:15:03.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.935+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96400fdc30 con 0x7f9640104eb0 2024-09-19T04:15:03.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.937+0000 7f96337fe640 1 -- 172.21.15.94:0/4203699358 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7f963c042030 con 0x7f9640104eb0 2024-09-19T04:15:03.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.937+0000 7f96337fe640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f962c0a30d0 0x7f962c0a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:03.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.937+0000 7f96337fe640 1 -- 172.21.15.94:0/4203699358 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f963c0f00d0 con 0x7f9640104eb0 2024-09-19T04:15:03.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.937+0000 7f9646560640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f962c0a30d0 0x7f962c0a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:03.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.938+0000 7f9646560640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f962c0a30d0 0x7f962c0a5590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9634009b60 tx=0x7f9634005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:03.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:03.940+0000 7f96337fe640 1 -- 172.21.15.94:0/4203699358 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f963c0bd0d0 con 0x7f9640104eb0 2024-09-19T04:15:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.120+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f964002d9c0 con 0x7f9640104eb0 2024-09-19T04:15:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.120+0000 7f96337fe640 1 -- 172.21.15.94:0/4203699358 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f963c0bc9d0 con 0x7f9640104eb0 2024-09-19T04:15:04.122 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f962c0a30d0 msgr2=0x7f962c0a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f962c0a30d0 0x7f962c0a5590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9634009b60 tx=0x7f9634005f90 comp rx=0 tx=0).stop 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 msgr2=0x7f9640198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 0x7f9640198310 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f963c0314e0 tx=0x7f963c031a30 comp rx=0 tx=0).stop 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 shutdown_connections 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f962c0a30d0 0x7f962c0a5590 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96401077e0 0x7f964019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640104eb0 0x7f9640198310 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 --2- 172.21.15.94:0/4203699358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9640102580 0x7f9640197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 >> 172.21.15.94:0/4203699358 conn(0x7f96400fc140 msgr2=0x7f9640103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 shutdown_connections 2024-09-19T04:15:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.122+0000 7f96487c4640 1 -- 172.21.15.94:0/4203699358 wait complete. 2024-09-19T04:15:04.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T04:15:04.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.239+0000 7f424cf33640 1 -- 172.21.15.94:0/3115249288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248100a00 msgr2=0x7f4248100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.239+0000 7f424cf33640 1 --2- 172.21.15.94:0/3115249288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248100a00 0x7f4248100e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4238009900 tx=0x7f423802a410 comp rx=0 tx=0).stop 2024-09-19T04:15:04.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.239+0000 7f424cf33640 1 -- 172.21.15.94:0/3115249288 shutdown_connections 2024-09-19T04:15:04.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.239+0000 7f424cf33640 1 --2- 172.21.15.94:0/3115249288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42481025b0 0x7f4248108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.239+0000 7f424cf33640 1 --2- 172.21.15.94:0/3115249288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4248101bf0 0x7f4248102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.239+0000 7f424cf33640 1 --2- 172.21.15.94:0/3115249288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248100a00 0x7f4248100e00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.239+0000 7f424cf33640 1 -- 172.21.15.94:0/3115249288 >> 172.21.15.94:0/3115249288 conn(0x7f42480fc140 msgr2=0x7f42480fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:04.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.239+0000 7f424cf33640 1 -- 172.21.15.94:0/3115249288 shutdown_connections 2024-09-19T04:15:04.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.239+0000 7f424cf33640 1 -- 172.21.15.94:0/3115249288 wait complete. 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f424cf33640 1 Processor -- start 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f424cf33640 1 -- start start 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f424cf33640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4248100a00 0x7f4248069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f424cf33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248101bf0 0x7f424819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f424cf33640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42481025b0 0x7f424819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f424cf33640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f424810c510 con 0x7f4248101bf0 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f424cf33640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f424810c390 con 0x7f4248100a00 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f424cf33640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f424810c690 con 0x7f42481025b0 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f423ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248101bf0 0x7f424819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f423ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248101bf0 0x7f424819caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46532/0 (socket says 172.21.15.94:46532) 2024-09-19T04:15:04.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f423ffff640 1 -- 172.21.15.94:0/4053166454 learned_addr learned my addr 172.21.15.94:0/4053166454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.240+0000 7f4246d9d640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42481025b0 0x7f424819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f424659c640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4248100a00 0x7f4248069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f423ffff640 1 -- 172.21.15.94:0/4053166454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42481025b0 msgr2=0x7f424819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f423ffff640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42481025b0 0x7f424819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f423ffff640 1 -- 172.21.15.94:0/4053166454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4248100a00 msgr2=0x7f4248069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f423ffff640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4248100a00 0x7f4248069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f423ffff640 1 -- 172.21.15.94:0/4053166454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424819fa20 con 0x7f4248101bf0 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f423ffff640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248101bf0 0x7f424819caf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f423000d950 tx=0x7f423000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f423f7fe640 1 -- 172.21.15.94:0/4053166454 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230017030 con 0x7f4248101bf0 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424819fcb0 con 0x7f4248101bf0 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f423f7fe640 1 -- 172.21.15.94:0/4053166454 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4230004020 con 0x7f4248101bf0 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f423f7fe640 1 -- 172.21.15.94:0/4053166454 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300047a0 con 0x7f4248101bf0 2024-09-19T04:15:04.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.241+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42481a01f0 con 0x7f4248101bf0 2024-09-19T04:15:04.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.242+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4248107f50 con 0x7f4248101bf0 2024-09-19T04:15:04.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.244+0000 7f423f7fe640 1 -- 172.21.15.94:0/4053166454 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7f42300041d0 con 0x7f4248101bf0 2024-09-19T04:15:04.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.244+0000 7f423f7fe640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42340a3050 0x7f42340a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:04.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.245+0000 7f424659c640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42340a3050 0x7f42340a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:04.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.245+0000 7f423f7fe640 1 -- 172.21.15.94:0/4053166454 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f42300cf060 con 0x7f4248101bf0 2024-09-19T04:15:04.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.245+0000 7f424659c640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42340a3050 0x7f42340a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4238004770 tx=0x7f4238000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:04.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.249+0000 7f423f7fe640 1 -- 172.21.15.94:0/4053166454 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f423009bf40 con 0x7f4248101bf0 2024-09-19T04:15:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.286+0000 7f423f7fe640 1 -- 172.21.15.94:0/4053166454 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 22) ==== 138475+0+0 (secure 0 0 0) 0x7f4230097eb0 con 0x7f4248101bf0 2024-09-19T04:15:04.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.433+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4248062df0 con 0x7f4248101bf0 2024-09-19T04:15:04.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.433+0000 7f423f7fe640 1 -- 172.21.15.94:0/4053166454 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f42300571d0 con 0x7f4248101bf0 2024-09-19T04:15:04.435 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42340a3050 msgr2=0x7f42340a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42340a3050 0x7f42340a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4238004770 tx=0x7f4238000bc0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248101bf0 msgr2=0x7f424819caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248101bf0 0x7f424819caf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f423000d950 tx=0x7f423000de10 comp rx=0 tx=0).stop 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 shutdown_connections 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42340a3050 0x7f42340a5510 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42481025b0 0x7f424819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4248101bf0 0x7f424819caf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 --2- 172.21.15.94:0/4053166454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4248100a00 0x7f4248069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 >> 172.21.15.94:0/4053166454 conn(0x7f42480fc140 msgr2=0x7f4248105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 shutdown_connections 2024-09-19T04:15:04.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.435+0000 7f424cf33640 1 -- 172.21.15.94:0/4053166454 wait complete. 2024-09-19T04:15:04.448 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: [19/Sep/2024:04:15:02] ENGINE Bus STARTING 2024-09-19T04:15:04.449 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: [19/Sep/2024:04:15:02] ENGINE Serving on https://172.21.15.94:7150 2024-09-19T04:15:04.449 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: [19/Sep/2024:04:15:02] ENGINE Error in HTTPServer.serve 2024-09-19T04:15:04.449 INFO:journalctl@ceph.mon.a.smithi094.stdout: Traceback (most recent call last): 2024-09-19T04:15:04.449 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:15:04.449 INFO:journalctl@ceph.mon.a.smithi094.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:15:04.449 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:15:04.449 INFO:journalctl@ceph.mon.a.smithi094.stdout: self._run(expiration_interval) 2024-09-19T04:15:04.449 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:15:04.449 INFO:journalctl@ceph.mon.a.smithi094.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: s = self.context.wrap_socket( 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: return self.sslsocket_class._create( 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: self.do_handshake() 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: self._sslobj.do_handshake() 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: [19/Sep/2024:04:15:02] ENGINE Serving on http://172.21.15.94:8765 2024-09-19T04:15:04.450 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: [19/Sep/2024:04:15:02] ENGINE Bus STARTED 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/738149488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1045958143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:04.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:04.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4203699358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:15:04.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:04 smithi094 ceph-mon[30972]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-19T04:15:04.454 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T04:15:04.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.557+0000 7fb576c2e640 1 -- 172.21.15.94:0/3247743227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 msgr2=0x7fb5701020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.557+0000 7fb576c2e640 1 --2- 172.21.15.94:0/3247743227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 0x7fb5701020a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb560009900 tx=0x7fb56002a410 comp rx=0 tx=0).stop 2024-09-19T04:15:04.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 -- 172.21.15.94:0/3247743227 shutdown_connections 2024-09-19T04:15:04.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 --2- 172.21.15.94:0/3247743227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5701025e0 0x7fb570108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 --2- 172.21.15.94:0/3247743227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 0x7fb5701020a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 --2- 172.21.15.94:0/3247743227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb570100a30 0x7fb570100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 -- 172.21.15.94:0/3247743227 >> 172.21.15.94:0/3247743227 conn(0x7fb5700fc1b0 msgr2=0x7fb5700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:04.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 -- 172.21.15.94:0/3247743227 shutdown_connections 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 -- 172.21.15.94:0/3247743227 wait complete. 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 Processor -- start 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 -- start start 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb570100a30 0x7fb570197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 0x7fb570198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5701025e0 0x7fb57019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb57010c5b0 con 0x7fb570101c20 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb57010c430 con 0x7fb570100a30 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.558+0000 7fb576c2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb57010c730 con 0x7fb5701025e0 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb567fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 0x7fb570198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb567fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 0x7fb570198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46542/0 (socket says 172.21.15.94:46542) 2024-09-19T04:15:04.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb567fff640 1 -- 172.21.15.94:0/4054489326 learned_addr learned my addr 172.21.15.94:0/4054489326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb5751cb640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5701025e0 0x7fb57019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb5749ca640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb570100a30 0x7fb570197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb5749ca640 1 -- 172.21.15.94:0/4054489326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5701025e0 msgr2=0x7fb57019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb5749ca640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5701025e0 0x7fb57019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb5749ca640 1 -- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 msgr2=0x7fb570198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb5749ca640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 0x7fb570198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb5749ca640 1 -- 172.21.15.94:0/4054489326 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57019f9f0 con 0x7fb570100a30 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb567fff640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 0x7fb570198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb5751cb640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5701025e0 0x7fb57019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb5749ca640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb570100a30 0x7fb570197dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00e9c0 tx=0x7fb56c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb565ffb640 1 -- 172.21.15.94:0/4054489326 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00ccc0 con 0x7fb570100a30 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57019fce0 con 0x7fb570100a30 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb565ffb640 1 -- 172.21.15.94:0/4054489326 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb56c0035f0 con 0x7fb570100a30 2024-09-19T04:15:04.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.559+0000 7fb565ffb640 1 -- 172.21.15.94:0/4054489326 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00ce60 con 0x7fb570100a30 2024-09-19T04:15:04.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.560+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701a0180 con 0x7fb570100a30 2024-09-19T04:15:04.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.561+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5701a03b0 con 0x7fb570100a30 2024-09-19T04:15:04.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.562+0000 7fb565ffb640 1 -- 172.21.15.94:0/4054489326 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 22) ==== 138475+0+0 (secure 0 0 0) 0x7fb56c0037a0 con 0x7fb570100a30 2024-09-19T04:15:04.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.562+0000 7fb565ffb640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5580a33c0 0x7fb5580a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:04.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.563+0000 7fb565ffb640 1 -- 172.21.15.94:0/4054489326 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb56c0d00f0 con 0x7fb570100a30 2024-09-19T04:15:04.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.563+0000 7fb567fff640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5580a33c0 0x7fb5580a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:04.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.563+0000 7fb567fff640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5580a33c0 0x7fb5580a5880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5701992e0 tx=0x7fb560005a20 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:04.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.567+0000 7fb565ffb640 1 -- 172.21.15.94:0/4054489326 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb56c09ca60 con 0x7fb570100a30 2024-09-19T04:15:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.750+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5701a06f0 con 0x7fb570100a30 2024-09-19T04:15:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.751+0000 7fb565ffb640 1 -- 172.21.15.94:0/4054489326 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7fb56c09c360 con 0x7fb570100a30 2024-09-19T04:15:04.753 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-09-19T04:15:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5580a33c0 msgr2=0x7fb5580a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5580a33c0 0x7fb5580a5880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5701992e0 tx=0x7fb560005a20 comp rx=0 tx=0).stop 2024-09-19T04:15:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb570100a30 msgr2=0x7fb570197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb570100a30 0x7fb570197dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00e9c0 tx=0x7fb56c00ee80 comp rx=0 tx=0).stop 2024-09-19T04:15:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 shutdown_connections 2024-09-19T04:15:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5580a33c0 0x7fb5580a5880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5701025e0 0x7fb57019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb570101c20 0x7fb570198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 --2- 172.21.15.94:0/4054489326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb570100a30 0x7fb570197dd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:04.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.753+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 >> 172.21.15.94:0/4054489326 conn(0x7fb5700fc1b0 msgr2=0x7fb570069fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:04.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.754+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 shutdown_connections 2024-09-19T04:15:04.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:04.754+0000 7fb576c2e640 1 -- 172.21.15.94:0/4054489326 wait complete. 2024-09-19T04:15:04.767 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T04:15:04.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: [19/Sep/2024:04:15:02] ENGINE Bus STARTING 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: [19/Sep/2024:04:15:02] ENGINE Serving on https://172.21.15.94:7150 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: [19/Sep/2024:04:15:02] ENGINE Error in HTTPServer.serve 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout: Traceback (most recent call last): 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout: self._run(expiration_interval) 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:15:04.785 INFO:journalctl@ceph.mon.c.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: s = self.context.wrap_socket( 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: return self.sslsocket_class._create( 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: self.do_handshake() 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: self._sslobj.do_handshake() 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:15:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: [19/Sep/2024:04:15:02] ENGINE Serving on http://172.21.15.94:8765 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: [19/Sep/2024:04:15:02] ENGINE Bus STARTED 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/738149488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1045958143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:04.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:04.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4203699358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:15:04.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:04 smithi112 ceph-mon[35051]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-19T04:15:04.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: [19/Sep/2024:04:15:02] ENGINE Bus STARTING 2024-09-19T04:15:04.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: [19/Sep/2024:04:15:02] ENGINE Serving on https://172.21.15.94:7150 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: [19/Sep/2024:04:15:02] ENGINE Error in HTTPServer.serve 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: Traceback (most recent call last): 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: self._run(expiration_interval) 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:15:04.855 INFO:journalctl@ceph.mon.b.smithi104.stdout: s = self.context.wrap_socket( 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout: return self.sslsocket_class._create( 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout: self.do_handshake() 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout: self._sslobj.do_handshake() 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: [19/Sep/2024:04:15:02] ENGINE Serving on http://172.21.15.94:8765 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: [19/Sep/2024:04:15:02] ENGINE Bus STARTED 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/738149488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:15:04.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1045958143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:04.858 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4203699358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:15:04.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:04 smithi104 ceph-mon[34145]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-19T04:15:05.033 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:05.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Unable to set osd_memory_target on smithi112 to 184553779: error parsing value: Value '184553779' is below minimum 939524096 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Unable to set osd_memory_target on smithi104 to 184576716: error parsing value: Value '184576716' is below minimum 939524096 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi094 to 176.0M 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Unable to set osd_memory_target on smithi094 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Updating smithi094:/etc/ceph/ceph.conf 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:15:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4053166454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4054489326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:05.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:05.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:05 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.532+0000 7f6e72535640 1 -- 172.21.15.94:0/522210734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 msgr2=0x7f6e6c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:05.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.532+0000 7f6e72535640 1 --2- 172.21.15.94:0/522210734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 0x7f6e6c106e60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009930 tx=0x7f6e5c02a420 comp rx=0 tx=0).stop 2024-09-19T04:15:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.533+0000 7f6e72535640 1 -- 172.21.15.94:0/522210734 shutdown_connections 2024-09-19T04:15:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.533+0000 7f6e72535640 1 --2- 172.21.15.94:0/522210734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c101420 0x7f6e6c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.533+0000 7f6e72535640 1 --2- 172.21.15.94:0/522210734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c100a80 0x7f6e6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.533+0000 7f6e72535640 1 --2- 172.21.15.94:0/522210734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 0x7f6e6c106e60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.533+0000 7f6e72535640 1 -- 172.21.15.94:0/522210734 >> 172.21.15.94:0/522210734 conn(0x7f6e6c0fc7a0 msgr2=0x7f6e6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.533+0000 7f6e72535640 1 -- 172.21.15.94:0/522210734 shutdown_connections 2024-09-19T04:15:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.533+0000 7f6e72535640 1 -- 172.21.15.94:0/522210734 wait complete. 2024-09-19T04:15:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.533+0000 7f6e72535640 1 Processor -- start 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.533+0000 7f6e72535640 1 -- start start 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e72535640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c100a80 0x7f6e6c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e72535640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c101420 0x7f6e6c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e72535640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 0x7f6e6c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e72535640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c110d30 con 0x7f6e6c106a80 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e72535640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c110bb0 con 0x7f6e6c100a80 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e72535640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c110eb0 con 0x7f6e6c101420 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e70ad2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 0x7f6e6c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e70ad2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 0x7f6e6c19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46556/0 (socket says 172.21.15.94:46556) 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e70ad2640 1 -- 172.21.15.94:0/3545407084 learned_addr learned my addr 172.21.15.94:0/3545407084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e6bfff640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c100a80 0x7f6e6c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e6b7fe640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c101420 0x7f6e6c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e70ad2640 1 -- 172.21.15.94:0/3545407084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c101420 msgr2=0x7f6e6c197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e70ad2640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c101420 0x7f6e6c197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e70ad2640 1 -- 172.21.15.94:0/3545407084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c100a80 msgr2=0x7f6e6c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e70ad2640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c100a80 0x7f6e6c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e70ad2640 1 -- 172.21.15.94:0/3545407084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c19c2a0 con 0x7f6e6c106a80 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e70ad2640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 0x7f6e6c19bbc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400eb10 tx=0x7f6e6400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e697fa640 1 -- 172.21.15.94:0/3545407084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64004b20 con 0x7f6e6c106a80 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c19c590 con 0x7f6e6c106a80 2024-09-19T04:15:05.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e697fa640 1 -- 172.21.15.94:0/3545407084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e64004cc0 con 0x7f6e6c106a80 2024-09-19T04:15:05.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e697fa640 1 -- 172.21.15.94:0/3545407084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64010690 con 0x7f6e6c106a80 2024-09-19T04:15:05.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.534+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c06a790 con 0x7f6e6c106a80 2024-09-19T04:15:05.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.536+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c100ee0 con 0x7f6e6c106a80 2024-09-19T04:15:05.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.537+0000 7f6e697fa640 1 -- 172.21.15.94:0/3545407084 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 138475+0+0 (secure 0 0 0) 0x7f6e64005020 con 0x7f6e6c106a80 2024-09-19T04:15:05.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.537+0000 7f6e697fa640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e540a33b0 0x7f6e540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:05.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.537+0000 7f6e6bfff640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e540a33b0 0x7f6e540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:05.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.537+0000 7f6e697fa640 1 -- 172.21.15.94:0/3545407084 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f6e640d0290 con 0x7f6e6c106a80 2024-09-19T04:15:05.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.538+0000 7f6e6bfff640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e540a33b0 0x7f6e540a5870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c03a040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:05.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.542+0000 7f6e697fa640 1 -- 172.21.15.94:0/3545407084 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6e6409cc00 con 0x7f6e6c106a80 2024-09-19T04:15:05.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.743+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e6c06b210 con 0x7f6e6c106a80 2024-09-19T04:15:05.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.744+0000 7f6e697fa640 1 -- 172.21.15.94:0/3545407084 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11915 (secure 0 0 0) 0x7f6e6409c500 con 0x7f6e6c106a80 2024-09-19T04:15:05.746 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:05.746 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"btime":"2024-09-19T04:15:03:346164+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:15:02.971431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24466,24467,24454]},"id":1}]} 2024-09-19T04:15:05.746 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-09-19T04:15:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e540a33b0 msgr2=0x7f6e540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e540a33b0 0x7f6e540a5870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c03a040 comp rx=0 tx=0).stop 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 msgr2=0x7f6e6c19bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 0x7f6e6c19bbc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400eb10 tx=0x7f6e6400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 shutdown_connections 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e540a33b0 0x7f6e540a5870 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e6c106a80 0x7f6e6c19bbc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c101420 0x7f6e6c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 --2- 172.21.15.94:0/3545407084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c100a80 0x7f6e6c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 >> 172.21.15.94:0/3545407084 conn(0x7f6e6c0fc7a0 msgr2=0x7f6e6c0fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 shutdown_connections 2024-09-19T04:15:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:05.746+0000 7f6e72535640 1 -- 172.21.15.94:0/3545407084 wait complete. 2024-09-19T04:15:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-19T04:15:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Unable to set osd_memory_target on smithi112 to 184553779: error parsing value: Value '184553779' is below minimum 939524096 2024-09-19T04:15:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-19T04:15:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Unable to set osd_memory_target on smithi104 to 184576716: error parsing value: Value '184576716' is below minimum 939524096 2024-09-19T04:15:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi094 to 176.0M 2024-09-19T04:15:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Unable to set osd_memory_target on smithi094 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-19T04:15:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Updating smithi094:/etc/ceph/ceph.conf 2024-09-19T04:15:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-19T04:15:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4053166454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4054489326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:05.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:05.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:05.787 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:05 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-19T04:15:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Unable to set osd_memory_target on smithi112 to 184553779: error parsing value: Value '184553779' is below minimum 939524096 2024-09-19T04:15:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-19T04:15:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Unable to set osd_memory_target on smithi104 to 184576716: error parsing value: Value '184576716' is below minimum 939524096 2024-09-19T04:15:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi094 to 176.0M 2024-09-19T04:15:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Unable to set osd_memory_target on smithi094 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Updating smithi094:/etc/ceph/ceph.conf 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4053166454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4054489326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:05.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:05.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:05.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:05.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:05 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:06 smithi094 ceph-mon[30972]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:15:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:06 smithi094 ceph-mon[30972]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:15:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:06 smithi094 ceph-mon[30972]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:15:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:06 smithi094 ceph-mon[30972]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:15:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:06 smithi094 ceph-mon[30972]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:15:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:06 smithi094 ceph-mon[30972]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:15:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:06 smithi094 ceph-mon[30972]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:15:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:06 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3545407084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:15:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:06 smithi094 ceph-mon[30972]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-19T04:15:06.465 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T04:15:06.465 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T04:15:06.465 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:15:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.566+0000 7fed4a731640 1 -- 172.21.15.94:0/3744759135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed44101360 msgr2=0x7fed4410d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.566+0000 7fed4a731640 1 --2- 172.21.15.94:0/3744759135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed44101360 0x7fed4410d890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009cd0 tx=0x7fed3c02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.566+0000 7fed4a731640 1 -- 172.21.15.94:0/3744759135 shutdown_connections 2024-09-19T04:15:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.566+0000 7fed4a731640 1 --2- 172.21.15.94:0/3744759135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed44101360 0x7fed4410d890 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.566+0000 7fed4a731640 1 --2- 172.21.15.94:0/3744759135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed441009c0 0x7fed44100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.566+0000 7fed4a731640 1 --2- 172.21.15.94:0/3744759135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed441069d0 0x7fed44106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.566+0000 7fed4a731640 1 -- 172.21.15.94:0/3744759135 >> 172.21.15.94:0/3744759135 conn(0x7fed440fc6a0 msgr2=0x7fed440feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.566+0000 7fed4a731640 1 -- 172.21.15.94:0/3744759135 shutdown_connections 2024-09-19T04:15:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.566+0000 7fed4a731640 1 -- 172.21.15.94:0/3744759135 wait complete. 2024-09-19T04:15:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed4a731640 1 Processor -- start 2024-09-19T04:15:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed4a731640 1 -- start start 2024-09-19T04:15:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed4a731640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed441009c0 0x7fed4410e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed4a731640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed44101360 0x7fed4410e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed4a731640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed441069d0 0x7fed4410eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed4a731640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed44111e30 con 0x7fed441069d0 2024-09-19T04:15:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed4a731640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed44111cb0 con 0x7fed441009c0 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed4a731640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed44111fb0 con 0x7fed44101360 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed48cce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed441069d0 0x7fed4410eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed48cce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed441069d0 0x7fed4410eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46576/0 (socket says 172.21.15.94:46576) 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.567+0000 7fed48cce640 1 -- 172.21.15.94:0/1550585502 learned_addr learned my addr 172.21.15.94:0/1550585502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed43fff640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed441009c0 0x7fed4410e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed437fe640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed44101360 0x7fed4410e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed48cce640 1 -- 172.21.15.94:0/1550585502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed44101360 msgr2=0x7fed4410e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed48cce640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed44101360 0x7fed4410e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed48cce640 1 -- 172.21.15.94:0/1550585502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed441009c0 msgr2=0x7fed4410e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed48cce640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed441009c0 0x7fed4410e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed48cce640 1 -- 172.21.15.94:0/1550585502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed441a6980 con 0x7fed441069d0 2024-09-19T04:15:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed437fe640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed44101360 0x7fed4410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed43fff640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed441009c0 0x7fed4410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed48cce640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed441069d0 0x7fed4410eda0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005cc0 tx=0x7fed3c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed417fa640 1 -- 172.21.15.94:0/1550585502 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c004240 con 0x7fed441069d0 2024-09-19T04:15:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed417fa640 1 -- 172.21.15.94:0/1550585502 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed3c0305a0 con 0x7fed441069d0 2024-09-19T04:15:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed441a6c10 con 0x7fed441069d0 2024-09-19T04:15:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed417fa640 1 -- 172.21.15.94:0/1550585502 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0466c0 con 0x7fed441069d0 2024-09-19T04:15:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.568+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a7160 con 0x7fed441069d0 2024-09-19T04:15:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.569+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed44100ea0 con 0x7fed441069d0 2024-09-19T04:15:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.570+0000 7fed417fa640 1 -- 172.21.15.94:0/1550585502 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7fed3c030ac0 con 0x7fed441069d0 2024-09-19T04:15:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.571+0000 7fed417fa640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed2c0a3450 0x7fed2c0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.571+0000 7fed43fff640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed2c0a3450 0x7fed2c0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.571+0000 7fed417fa640 1 -- 172.21.15.94:0/1550585502 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fed3c02e030 con 0x7fed441069d0 2024-09-19T04:15:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.571+0000 7fed43fff640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed2c0a3450 0x7fed2c0a5910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed34008040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.575+0000 7fed417fa640 1 -- 172.21.15.94:0/1550585502 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fed3c04b060 con 0x7fed441069d0 2024-09-19T04:15:06.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.738+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 --> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed44062630 con 0x7fed2c0a3450 2024-09-19T04:15:06.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.745+0000 7fed417fa640 1 -- 172.21.15.94:0/1550585502 <== mgr.15159 v2:172.21.15.94:6800/3395191155 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257863 (secure 0 0 0) 0x7fed44062630 con 0x7fed2c0a3450 2024-09-19T04:15:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-19T04:15:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed2c0a3450 msgr2=0x7fed2c0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed2c0a3450 0x7fed2c0a5910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed34008040 comp rx=0 tx=0).stop 2024-09-19T04:15:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed441069d0 msgr2=0x7fed4410eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed441069d0 0x7fed4410eda0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005cc0 tx=0x7fed3c003ba0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 shutdown_connections 2024-09-19T04:15:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed2c0a3450 0x7fed2c0a5910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed441069d0 0x7fed4410eda0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed44101360 0x7fed4410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 --2- 172.21.15.94:0/1550585502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed441009c0 0x7fed4410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 >> 172.21.15.94:0/1550585502 conn(0x7fed440fc6a0 msgr2=0x7fed440fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 shutdown_connections 2024-09-19T04:15:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.748+0000 7fed4a731640 1 -- 172.21.15.94:0/1550585502 wait complete. 2024-09-19T04:15:06.761 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T04:15:05.384111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183118,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":836,"num_read_kb":573,"num_write":472,"num_write_kb":4371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":401296,"kb_used_data":14280,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540528,"statfs":{"total":1151940427776,"available":1151529500672,"internally_reserved":0,"allocated":14622720,"data_stored":8825931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.017538"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388917+0000","last_change":"2024-09-19T04:13:16.451031+0000","last_active":"2024-09-19T04:15:01.388917+0000","last_peered":"2024-09-19T04:15:01.388917+0000","last_clean":"2024-09-19T04:15:01.388917+0000","last_became_active":"2024-09-19T04:13:16.450713+0000","last_became_peered":"2024-09-19T04:13:16.450713+0000","last_unstale":"2024-09-19T04:15:01.388917+0000","last_undegraded":"2024-09-19T04:15:01.388917+0000","last_fullsized":"2024-09-19T04:15:01.388917+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:59.485415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.381120+0000","last_change":"2024-09-19T04:13:16.455689+0000","last_active":"2024-09-19T04:15:01.381120+0000","last_peered":"2024-09-19T04:15:01.381120+0000","last_clean":"2024-09-19T04:15:01.381120+0000","last_became_active":"2024-09-19T04:13:16.455154+0000","last_became_peered":"2024-09-19T04:13:16.455154+0000","last_unstale":"2024-09-19T04:15:01.381120+0000","last_undegraded":"2024-09-19T04:15:01.381120+0000","last_fullsized":"2024-09-19T04:15:01.381120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:57.610619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.381282+0000","last_change":"2024-09-19T04:13:16.447392+0000","last_active":"2024-09-19T04:15:01.381282+0000","last_peered":"2024-09-19T04:15:01.381282+0000","last_clean":"2024-09-19T04:15:01.381282+0000","last_became_active":"2024-09-19T04:13:16.447088+0000","last_became_peered":"2024-09-19T04:13:16.447088+0000","last_unstale":"2024-09-19T04:15:01.381282+0000","last_undegraded":"2024-09-19T04:15:01.381282+0000","last_fullsized":"2024-09-19T04:15:01.381282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:16.591420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387155+0000","last_change":"2024-09-19T04:13:16.452642+0000","last_active":"2024-09-19T04:15:01.387155+0000","last_peered":"2024-09-19T04:15:01.387155+0000","last_clean":"2024-09-19T04:15:01.387155+0000","last_became_active":"2024-09-19T04:13:16.452329+0000","last_became_peered":"2024-09-19T04:13:16.452329+0000","last_unstale":"2024-09-19T04:15:01.387155+0000","last_undegraded":"2024-09-19T04:15:01.387155+0000","last_fullsized":"2024-09-19T04:15:01.387155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:54.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384913+0000","last_change":"2024-09-19T04:13:16.447171+0000","last_active":"2024-09-19T04:15:01.384913+0000","last_peered":"2024-09-19T04:15:01.384913+0000","last_clean":"2024-09-19T04:15:01.384913+0000","last_became_active":"2024-09-19T04:13:16.446835+0000","last_became_peered":"2024-09-19T04:13:16.446835+0000","last_unstale":"2024-09-19T04:15:01.384913+0000","last_undegraded":"2024-09-19T04:15:01.384913+0000","last_fullsized":"2024-09-19T04:15:01.384913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:41.892118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.377175+0000","last_change":"2024-09-19T04:13:31.507665+0000","last_active":"2024-09-19T04:15:01.377175+0000","last_peered":"2024-09-19T04:15:01.377175+0000","last_clean":"2024-09-19T04:15:01.377175+0000","last_became_active":"2024-09-19T04:13:31.507329+0000","last_became_peered":"2024-09-19T04:13:31.507329+0000","last_unstale":"2024-09-19T04:15:01.377175+0000","last_undegraded":"2024-09-19T04:15:01.377175+0000","last_fullsized":"2024-09-19T04:15:01.377175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:35.832776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388719+0000","last_change":"2024-09-19T04:13:16.454568+0000","last_active":"2024-09-19T04:15:01.388719+0000","last_peered":"2024-09-19T04:15:01.388719+0000","last_clean":"2024-09-19T04:15:01.388719+0000","last_became_active":"2024-09-19T04:13:16.454230+0000","last_became_peered":"2024-09-19T04:13:16.454230+0000","last_unstale":"2024-09-19T04:15:01.388719+0000","last_undegraded":"2024-09-19T04:15:01.388719+0000","last_fullsized":"2024-09-19T04:15:01.388719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:23.401316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402188+0000","last_change":"2024-09-19T04:13:16.449223+0000","last_active":"2024-09-19T04:15:01.402188+0000","last_peered":"2024-09-19T04:15:01.402188+0000","last_clean":"2024-09-19T04:15:01.402188+0000","last_became_active":"2024-09-19T04:13:16.448665+0000","last_became_peered":"2024-09-19T04:13:16.448665+0000","last_unstale":"2024-09-19T04:15:01.402188+0000","last_undegraded":"2024-09-19T04:15:01.402188+0000","last_fullsized":"2024-09-19T04:15:01.402188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:29.619847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402682+0000","last_change":"2024-09-19T04:13:16.458203+0000","last_active":"2024-09-19T04:15:01.402682+0000","last_peered":"2024-09-19T04:15:01.402682+0000","last_clean":"2024-09-19T04:15:01.402682+0000","last_became_active":"2024-09-19T04:13:16.457544+0000","last_became_peered":"2024-09-19T04:13:16.457544+0000","last_unstale":"2024-09-19T04:15:01.402682+0000","last_undegraded":"2024-09-19T04:15:01.402682+0000","last_fullsized":"2024-09-19T04:15:01.402682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:47.611695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.385340+0000","last_change":"2024-09-19T04:13:16.454213+0000","last_active":"2024-09-19T04:15:01.385340+0000","last_peered":"2024-09-19T04:15:01.385340+0000","last_clean":"2024-09-19T04:15:01.385340+0000","last_became_active":"2024-09-19T04:13:16.453047+0000","last_became_peered":"2024-09-19T04:13:16.453047+0000","last_unstale":"2024-09-19T04:15:01.385340+0000","last_undegraded":"2024-09-19T04:15:01.385340+0000","last_fullsized":"2024-09-19T04:15:01.385340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:27.425259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407089+0000","last_change":"2024-09-19T04:13:16.458195+0000","last_active":"2024-09-19T04:15:01.407089+0000","last_peered":"2024-09-19T04:15:01.407089+0000","last_clean":"2024-09-19T04:15:01.407089+0000","last_became_active":"2024-09-19T04:13:16.457891+0000","last_became_peered":"2024-09-19T04:13:16.457891+0000","last_unstale":"2024-09-19T04:15:01.407089+0000","last_undegraded":"2024-09-19T04:15:01.407089+0000","last_fullsized":"2024-09-19T04:15:01.407089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:53.833249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.379554+0000","last_change":"2024-09-19T04:13:31.508117+0000","last_active":"2024-09-19T04:15:01.379554+0000","last_peered":"2024-09-19T04:15:01.379554+0000","last_clean":"2024-09-19T04:15:01.379554+0000","last_became_active":"2024-09-19T04:13:31.507693+0000","last_became_peered":"2024-09-19T04:13:31.507693+0000","last_unstale":"2024-09-19T04:15:01.379554+0000","last_undegraded":"2024-09-19T04:15:01.379554+0000","last_fullsized":"2024-09-19T04:15:01.379554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:58.447965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392023+0000","last_change":"2024-09-19T04:13:16.440079+0000","last_active":"2024-09-19T04:15:01.392023+0000","last_peered":"2024-09-19T04:15:01.392023+0000","last_clean":"2024-09-19T04:15:01.392023+0000","last_became_active":"2024-09-19T04:13:16.439474+0000","last_became_peered":"2024-09-19T04:13:16.439474+0000","last_unstale":"2024-09-19T04:15:01.392023+0000","last_undegraded":"2024-09-19T04:15:01.392023+0000","last_fullsized":"2024-09-19T04:15:01.392023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:11.151412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384315+0000","last_change":"2024-09-19T04:13:16.449958+0000","last_active":"2024-09-19T04:15:01.384315+0000","last_peered":"2024-09-19T04:15:01.384315+0000","last_clean":"2024-09-19T04:15:01.384315+0000","last_became_active":"2024-09-19T04:13:16.449665+0000","last_became_peered":"2024-09-19T04:13:16.449665+0000","last_unstale":"2024-09-19T04:15:01.384315+0000","last_undegraded":"2024-09-19T04:15:01.384315+0000","last_fullsized":"2024-09-19T04:15:01.384315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:24.023779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.383771+0000","last_change":"2024-09-19T04:13:16.454333+0000","last_active":"2024-09-19T04:15:01.383771+0000","last_peered":"2024-09-19T04:15:01.383771+0000","last_clean":"2024-09-19T04:15:01.383771+0000","last_became_active":"2024-09-19T04:13:16.454066+0000","last_became_peered":"2024-09-19T04:13:16.454066+0000","last_unstale":"2024-09-19T04:15:01.383771+0000","last_undegraded":"2024-09-19T04:15:01.383771+0000","last_fullsized":"2024-09-19T04:15:01.383771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:16.815173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.386728+0000","last_change":"2024-09-19T04:13:16.451864+0000","last_active":"2024-09-19T04:15:01.386728+0000","last_peered":"2024-09-19T04:15:01.386728+0000","last_clean":"2024-09-19T04:15:01.386728+0000","last_became_active":"2024-09-19T04:13:16.450995+0000","last_became_peered":"2024-09-19T04:13:16.450995+0000","last_unstale":"2024-09-19T04:15:01.386728+0000","last_undegraded":"2024-09-19T04:15:01.386728+0000","last_fullsized":"2024-09-19T04:15:01.386728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:40.750221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.389034+0000","last_change":"2024-09-19T04:13:16.453176+0000","last_active":"2024-09-19T04:15:01.389034+0000","last_peered":"2024-09-19T04:15:01.389034+0000","last_clean":"2024-09-19T04:15:01.389034+0000","last_became_active":"2024-09-19T04:13:16.452788+0000","last_became_peered":"2024-09-19T04:13:16.452788+0000","last_unstale":"2024-09-19T04:15:01.389034+0000","last_undegraded":"2024-09-19T04:15:01.389034+0000","last_fullsized":"2024-09-19T04:15:01.389034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:47.099180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.377948+0000","last_change":"2024-09-19T04:13:16.449167+0000","last_active":"2024-09-19T04:15:01.377948+0000","last_peered":"2024-09-19T04:15:01.377948+0000","last_clean":"2024-09-19T04:15:01.377948+0000","last_became_active":"2024-09-19T04:13:16.448579+0000","last_became_peered":"2024-09-19T04:13:16.448579+0000","last_unstale":"2024-09-19T04:15:01.377948+0000","last_undegraded":"2024-09-19T04:15:01.377948+0000","last_fullsized":"2024-09-19T04:15:01.377948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:22:40.023000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387764+0000","last_change":"2024-09-19T04:13:16.445449+0000","last_active":"2024-09-19T04:15:01.387764+0000","last_peered":"2024-09-19T04:15:01.387764+0000","last_clean":"2024-09-19T04:15:01.387764+0000","last_became_active":"2024-09-19T04:13:16.445028+0000","last_became_peered":"2024-09-19T04:13:16.445028+0000","last_unstale":"2024-09-19T04:15:01.387764+0000","last_undegraded":"2024-09-19T04:15:01.387764+0000","last_fullsized":"2024-09-19T04:15:01.387764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:13.095348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.386639+0000","last_change":"2024-09-19T04:13:16.454033+0000","last_active":"2024-09-19T04:15:01.386639+0000","last_peered":"2024-09-19T04:15:01.386639+0000","last_clean":"2024-09-19T04:15:01.386639+0000","last_became_active":"2024-09-19T04:13:16.453592+0000","last_became_peered":"2024-09-19T04:13:16.453592+0000","last_unstale":"2024-09-19T04:15:01.386639+0000","last_undegraded":"2024-09-19T04:15:01.386639+0000","last_fullsized":"2024-09-19T04:15:01.386639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.304478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.408187+0000","last_change":"2024-09-19T04:13:16.448763+0000","last_active":"2024-09-19T04:15:01.408187+0000","last_peered":"2024-09-19T04:15:01.408187+0000","last_clean":"2024-09-19T04:15:01.408187+0000","last_became_active":"2024-09-19T04:13:16.448459+0000","last_became_peered":"2024-09-19T04:13:16.448459+0000","last_unstale":"2024-09-19T04:15:01.408187+0000","last_undegraded":"2024-09-19T04:15:01.408187+0000","last_fullsized":"2024-09-19T04:15:01.408187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.403838+0000","last_change":"2024-09-19T04:13:16.451489+0000","last_active":"2024-09-19T04:15:01.403838+0000","last_peered":"2024-09-19T04:15:01.403838+0000","last_clean":"2024-09-19T04:15:01.403838+0000","last_became_active":"2024-09-19T04:13:16.451176+0000","last_became_peered":"2024-09-19T04:13:16.451176+0000","last_unstale":"2024-09-19T04:15:01.403838+0000","last_undegraded":"2024-09-19T04:15:01.403838+0000","last_fullsized":"2024-09-19T04:15:01.403838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.921102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387603+0000","last_change":"2024-09-19T04:13:16.454551+0000","last_active":"2024-09-19T04:15:01.387603+0000","last_peered":"2024-09-19T04:15:01.387603+0000","last_clean":"2024-09-19T04:15:01.387603+0000","last_became_active":"2024-09-19T04:13:16.453976+0000","last_became_peered":"2024-09-19T04:13:16.453976+0000","last_unstale":"2024-09-19T04:15:01.387603+0000","last_undegraded":"2024-09-19T04:15:01.387603+0000","last_fullsized":"2024-09-19T04:15:01.387603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:06.504615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407629+0000","last_change":"2024-09-19T04:13:31.503222+0000","last_active":"2024-09-19T04:15:01.407629+0000","last_peered":"2024-09-19T04:15:01.407629+0000","last_clean":"2024-09-19T04:15:01.407629+0000","last_became_active":"2024-09-19T04:13:31.502873+0000","last_became_peered":"2024-09-19T04:13:31.502873+0000","last_unstale":"2024-09-19T04:15:01.407629+0000","last_undegraded":"2024-09-19T04:15:01.407629+0000","last_fullsized":"2024-09-19T04:15:01.407629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:54.559592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388512+0000","last_change":"2024-09-19T04:13:31.505834+0000","last_active":"2024-09-19T04:15:01.388512+0000","last_peered":"2024-09-19T04:15:01.388512+0000","last_clean":"2024-09-19T04:15:01.388512+0000","last_became_active":"2024-09-19T04:13:31.505501+0000","last_became_peered":"2024-09-19T04:13:31.505501+0000","last_unstale":"2024-09-19T04:15:01.388512+0000","last_undegraded":"2024-09-19T04:15:01.388512+0000","last_fullsized":"2024-09-19T04:15:01.388512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:45.136128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384870+0000","last_change":"2024-09-19T04:13:16.449237+0000","last_active":"2024-09-19T04:15:01.384870+0000","last_peered":"2024-09-19T04:15:01.384870+0000","last_clean":"2024-09-19T04:15:01.384870+0000","last_became_active":"2024-09-19T04:13:16.448896+0000","last_became_peered":"2024-09-19T04:13:16.448896+0000","last_unstale":"2024-09-19T04:15:01.384870+0000","last_undegraded":"2024-09-19T04:15:01.384870+0000","last_fullsized":"2024-09-19T04:15:01.384870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:38.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384284+0000","last_change":"2024-09-19T04:13:16.456053+0000","last_active":"2024-09-19T04:15:01.384284+0000","last_peered":"2024-09-19T04:15:01.384284+0000","last_clean":"2024-09-19T04:15:01.384284+0000","last_became_active":"2024-09-19T04:13:16.454775+0000","last_became_peered":"2024-09-19T04:13:16.454775+0000","last_unstale":"2024-09-19T04:15:01.384284+0000","last_undegraded":"2024-09-19T04:15:01.384284+0000","last_fullsized":"2024-09-19T04:15:01.384284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.902036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402500+0000","last_change":"2024-09-19T04:13:16.451473+0000","last_active":"2024-09-19T04:15:01.402500+0000","last_peered":"2024-09-19T04:15:01.402500+0000","last_clean":"2024-09-19T04:15:01.402500+0000","last_became_active":"2024-09-19T04:13:16.451150+0000","last_became_peered":"2024-09-19T04:13:16.451150+0000","last_unstale":"2024-09-19T04:15:01.402500+0000","last_undegraded":"2024-09-19T04:15:01.402500+0000","last_fullsized":"2024-09-19T04:15:01.402500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:59.332745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392354+0000","last_change":"2024-09-19T04:13:16.452017+0000","last_active":"2024-09-19T04:15:01.392354+0000","last_peered":"2024-09-19T04:15:01.392354+0000","last_clean":"2024-09-19T04:15:01.392354+0000","last_became_active":"2024-09-19T04:13:16.450715+0000","last_became_peered":"2024-09-19T04:13:16.450715+0000","last_unstale":"2024-09-19T04:15:01.392354+0000","last_undegraded":"2024-09-19T04:15:01.392354+0000","last_fullsized":"2024-09-19T04:15:01.392354+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:43.548770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.380440+0000","last_change":"2024-09-19T04:13:31.736597+0000","last_active":"2024-09-19T04:15:01.380440+0000","last_peered":"2024-09-19T04:15:01.380440+0000","last_clean":"2024-09-19T04:15:01.380440+0000","last_became_active":"2024-09-19T04:13:31.736301+0000","last_became_peered":"2024-09-19T04:13:31.736301+0000","last_unstale":"2024-09-19T04:15:01.380440+0000","last_undegraded":"2024-09-19T04:15:01.380440+0000","last_fullsized":"2024-09-19T04:15:01.380440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:06.614117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384673+0000","last_change":"2024-09-19T04:13:31.509742+0000","last_active":"2024-09-19T04:15:01.384673+0000","last_peered":"2024-09-19T04:15:01.384673+0000","last_clean":"2024-09-19T04:15:01.384673+0000","last_became_active":"2024-09-19T04:13:31.509363+0000","last_became_peered":"2024-09-19T04:13:31.509363+0000","last_unstale":"2024-09-19T04:15:01.384673+0000","last_undegraded":"2024-09-19T04:15:01.384673+0000","last_fullsized":"2024-09-19T04:15:01.384673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.094930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384995+0000","last_change":"2024-09-19T04:13:16.449459+0000","last_active":"2024-09-19T04:15:01.384995+0000","last_peered":"2024-09-19T04:15:01.384995+0000","last_clean":"2024-09-19T04:15:01.384995+0000","last_became_active":"2024-09-19T04:13:16.449119+0000","last_became_peered":"2024-09-19T04:13:16.449119+0000","last_unstale":"2024-09-19T04:15:01.384995+0000","last_undegraded":"2024-09-19T04:15:01.384995+0000","last_fullsized":"2024-09-19T04:15:01.384995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:26:26.941609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.375543+0000","last_change":"2024-09-19T04:13:16.448939+0000","last_active":"2024-09-19T04:15:01.375543+0000","last_peered":"2024-09-19T04:15:01.375543+0000","last_clean":"2024-09-19T04:15:01.375543+0000","last_became_active":"2024-09-19T04:13:16.448619+0000","last_became_peered":"2024-09-19T04:13:16.448619+0000","last_unstale":"2024-09-19T04:15:01.375543+0000","last_undegraded":"2024-09-19T04:15:01.375543+0000","last_fullsized":"2024-09-19T04:15:01.375543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:41.017404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'9","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.631265+0000","last_change":"2024-09-19T04:13:14.430467+0000","last_active":"2024-09-19T04:15:01.631265+0000","last_peered":"2024-09-19T04:15:01.631265+0000","last_clean":"2024-09-19T04:15:01.631265+0000","last_became_active":"2024-09-19T04:13:14.430095+0000","last_became_peered":"2024-09-19T04:13:14.430095+0000","last_unstale":"2024-09-19T04:15:01.631265+0000","last_undegraded":"2024-09-19T04:15:01.631265+0000","last_fullsized":"2024-09-19T04:15:01.631265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:50.471745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2028,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.385789+0000","last_change":"2024-09-19T04:13:31.505798+0000","last_active":"2024-09-19T04:15:01.385789+0000","last_peered":"2024-09-19T04:15:01.385789+0000","last_clean":"2024-09-19T04:15:01.385789+0000","last_became_active":"2024-09-19T04:13:31.505469+0000","last_became_peered":"2024-09-19T04:13:31.505469+0000","last_unstale":"2024-09-19T04:15:01.385789+0000","last_undegraded":"2024-09-19T04:15:01.385789+0000","last_fullsized":"2024-09-19T04:15:01.385789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:13:42.405075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.348065+0000","last_change":"2024-09-19T04:13:14.426443+0000","last_active":"2024-09-19T04:15:01.348065+0000","last_peered":"2024-09-19T04:15:01.348065+0000","last_clean":"2024-09-19T04:15:01.348065+0000","last_became_active":"2024-09-19T04:13:14.426103+0000","last_became_peered":"2024-09-19T04:13:14.426103+0000","last_unstale":"2024-09-19T04:15:01.348065+0000","last_undegraded":"2024-09-19T04:15:01.348065+0000","last_fullsized":"2024-09-19T04:15:01.348065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:28.461962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.343355+0000","last_change":"2024-09-19T04:13:14.424439+0000","last_active":"2024-09-19T04:15:01.343355+0000","last_peered":"2024-09-19T04:15:01.343355+0000","last_clean":"2024-09-19T04:15:01.343355+0000","last_became_active":"2024-09-19T04:13:14.423975+0000","last_became_peered":"2024-09-19T04:13:14.423975+0000","last_unstale":"2024-09-19T04:15:01.343355+0000","last_undegraded":"2024-09-19T04:15:01.343355+0000","last_fullsized":"2024-09-19T04:15:01.343355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:12.032125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392899+0000","last_change":"2024-09-19T04:13:16.444865+0000","last_active":"2024-09-19T04:15:01.392899+0000","last_peered":"2024-09-19T04:15:01.392899+0000","last_clean":"2024-09-19T04:15:01.392899+0000","last_became_active":"2024-09-19T04:13:16.444556+0000","last_became_peered":"2024-09-19T04:13:16.444556+0000","last_unstale":"2024-09-19T04:15:01.392899+0000","last_undegraded":"2024-09-19T04:15:01.392899+0000","last_fullsized":"2024-09-19T04:15:01.392899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:38.025604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354210+0000","last_change":"2024-09-19T04:13:14.429018+0000","last_active":"2024-09-19T04:15:01.354210+0000","last_peered":"2024-09-19T04:15:01.354210+0000","last_clean":"2024-09-19T04:15:01.354210+0000","last_became_active":"2024-09-19T04:13:14.428383+0000","last_became_peered":"2024-09-19T04:13:14.428383+0000","last_unstale":"2024-09-19T04:15:01.354210+0000","last_undegraded":"2024-09-19T04:15:01.354210+0000","last_fullsized":"2024-09-19T04:15:01.354210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:46.190421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.391102+0000","last_change":"2024-09-19T04:13:16.449013+0000","last_active":"2024-09-19T04:15:01.391102+0000","last_peered":"2024-09-19T04:15:01.391102+0000","last_clean":"2024-09-19T04:15:01.391102+0000","last_became_active":"2024-09-19T04:13:16.448703+0000","last_became_peered":"2024-09-19T04:13:16.448703+0000","last_unstale":"2024-09-19T04:15:01.391102+0000","last_undegraded":"2024-09-19T04:15:01.391102+0000","last_fullsized":"2024-09-19T04:15:01.391102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:41.231759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":124,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.348337+0000","last_change":"2024-09-19T04:13:14.430050+0000","last_active":"2024-09-19T04:15:01.348337+0000","last_peered":"2024-09-19T04:15:01.348337+0000","last_clean":"2024-09-19T04:15:01.348337+0000","last_became_active":"2024-09-19T04:13:14.429710+0000","last_became_peered":"2024-09-19T04:13:14.429710+0000","last_unstale":"2024-09-19T04:15:01.348337+0000","last_undegraded":"2024-09-19T04:15:01.348337+0000","last_fullsized":"2024-09-19T04:15:01.348337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:55.796080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388332+0000","last_change":"2024-09-19T04:13:16.454431+0000","last_active":"2024-09-19T04:15:01.388332+0000","last_peered":"2024-09-19T04:15:01.388332+0000","last_clean":"2024-09-19T04:15:01.388332+0000","last_became_active":"2024-09-19T04:13:16.453033+0000","last_became_peered":"2024-09-19T04:13:16.453033+0000","last_unstale":"2024-09-19T04:15:01.388332+0000","last_undegraded":"2024-09-19T04:15:01.388332+0000","last_fullsized":"2024-09-19T04:15:01.388332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:52.817740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.362264+0000","last_change":"2024-09-19T04:13:14.428949+0000","last_active":"2024-09-19T04:15:01.362264+0000","last_peered":"2024-09-19T04:15:01.362264+0000","last_clean":"2024-09-19T04:15:01.362264+0000","last_became_active":"2024-09-19T04:13:14.428525+0000","last_became_peered":"2024-09-19T04:13:14.428525+0000","last_unstale":"2024-09-19T04:15:01.362264+0000","last_undegraded":"2024-09-19T04:15:01.362264+0000","last_fullsized":"2024-09-19T04:15:01.362264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:30.296730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.381741+0000","last_change":"2024-09-19T04:13:16.454077+0000","last_active":"2024-09-19T04:15:01.381741+0000","last_peered":"2024-09-19T04:15:01.381741+0000","last_clean":"2024-09-19T04:15:01.381741+0000","last_became_active":"2024-09-19T04:13:16.453791+0000","last_became_peered":"2024-09-19T04:13:16.453791+0000","last_unstale":"2024-09-19T04:15:01.381741+0000","last_undegraded":"2024-09-19T04:15:01.381741+0000","last_fullsized":"2024-09-19T04:15:01.381741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:29.426475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.356675+0000","last_change":"2024-09-19T04:13:14.620179+0000","last_active":"2024-09-19T04:15:01.356675+0000","last_peered":"2024-09-19T04:15:01.356675+0000","last_clean":"2024-09-19T04:15:01.356675+0000","last_became_active":"2024-09-19T04:13:14.619758+0000","last_became_peered":"2024-09-19T04:13:14.619758+0000","last_unstale":"2024-09-19T04:15:01.356675+0000","last_undegraded":"2024-09-19T04:15:01.356675+0000","last_fullsized":"2024-09-19T04:15:01.356675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:11.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.404018+0000","last_change":"2024-09-19T04:13:16.453036+0000","last_active":"2024-09-19T04:15:01.404018+0000","last_peered":"2024-09-19T04:15:01.404018+0000","last_clean":"2024-09-19T04:15:01.404018+0000","last_became_active":"2024-09-19T04:13:16.452736+0000","last_became_peered":"2024-09-19T04:13:16.452736+0000","last_unstale":"2024-09-19T04:15:01.404018+0000","last_undegraded":"2024-09-19T04:15:01.404018+0000","last_fullsized":"2024-09-19T04:15:01.404018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:30.003652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":260,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.634781+0000","last_change":"2024-09-19T04:13:14.428715+0000","last_active":"2024-09-19T04:15:01.634781+0000","last_peered":"2024-09-19T04:15:01.634781+0000","last_clean":"2024-09-19T04:15:01.634781+0000","last_became_active":"2024-09-19T04:13:14.428409+0000","last_became_peered":"2024-09-19T04:13:14.428409+0000","last_unstale":"2024-09-19T04:15:01.634781+0000","last_undegraded":"2024-09-19T04:15:01.634781+0000","last_fullsized":"2024-09-19T04:15:01.634781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:23.130857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61278,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":202,"num_read_kb":161,"num_write":35,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407414+0000","last_change":"2024-09-19T04:13:16.454512+0000","last_active":"2024-09-19T04:15:01.407414+0000","last_peered":"2024-09-19T04:15:01.407414+0000","last_clean":"2024-09-19T04:15:01.407414+0000","last_became_active":"2024-09-19T04:13:16.454193+0000","last_became_peered":"2024-09-19T04:13:16.454193+0000","last_unstale":"2024-09-19T04:15:01.407414+0000","last_undegraded":"2024-09-19T04:15:01.407414+0000","last_fullsized":"2024-09-19T04:15:01.407414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:11.825938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.344927+0000","last_change":"2024-09-19T04:13:31.515448+0000","last_active":"2024-09-19T04:15:01.344927+0000","last_peered":"2024-09-19T04:15:01.344927+0000","last_clean":"2024-09-19T04:15:01.344927+0000","last_became_active":"2024-09-19T04:13:31.506099+0000","last_became_peered":"2024-09-19T04:13:31.506099+0000","last_unstale":"2024-09-19T04:15:01.344927+0000","last_undegraded":"2024-09-19T04:15:01.344927+0000","last_fullsized":"2024-09-19T04:15:01.344927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:21.462488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407870+0000","last_change":"2024-09-19T04:13:16.447607+0000","last_active":"2024-09-19T04:15:01.407870+0000","last_peered":"2024-09-19T04:15:01.407870+0000","last_clean":"2024-09-19T04:15:01.407870+0000","last_became_active":"2024-09-19T04:13:16.447029+0000","last_became_peered":"2024-09-19T04:13:16.447029+0000","last_unstale":"2024-09-19T04:15:01.407870+0000","last_undegraded":"2024-09-19T04:15:01.407870+0000","last_fullsized":"2024-09-19T04:15:01.407870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:52.418104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.355933+0000","last_change":"2024-09-19T04:13:14.428856+0000","last_active":"2024-09-19T04:15:01.355933+0000","last_peered":"2024-09-19T04:15:01.355933+0000","last_clean":"2024-09-19T04:15:01.355933+0000","last_became_active":"2024-09-19T04:13:14.428510+0000","last_became_peered":"2024-09-19T04:13:14.428510+0000","last_unstale":"2024-09-19T04:15:01.355933+0000","last_undegraded":"2024-09-19T04:15:01.355933+0000","last_fullsized":"2024-09-19T04:15:01.355933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:49.633325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.409050+0000","last_change":"2024-09-19T04:13:16.455628+0000","last_active":"2024-09-19T04:15:01.409050+0000","last_peered":"2024-09-19T04:15:01.409050+0000","last_clean":"2024-09-19T04:15:01.409050+0000","last_became_active":"2024-09-19T04:13:16.453817+0000","last_became_peered":"2024-09-19T04:13:16.453817+0000","last_unstale":"2024-09-19T04:15:01.409050+0000","last_undegraded":"2024-09-19T04:15:01.409050+0000","last_fullsized":"2024-09-19T04:15:01.409050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.832806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.630728+0000","last_change":"2024-09-19T04:13:14.424924+0000","last_active":"2024-09-19T04:15:01.630728+0000","last_peered":"2024-09-19T04:15:01.630728+0000","last_clean":"2024-09-19T04:15:01.630728+0000","last_became_active":"2024-09-19T04:13:14.424627+0000","last_became_peered":"2024-09-19T04:13:14.424627+0000","last_unstale":"2024-09-19T04:15:01.630728+0000","last_undegraded":"2024-09-19T04:15:01.630728+0000","last_fullsized":"2024-09-19T04:15:01.630728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:49.830205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1994,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":5,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.405035+0000","last_change":"2024-09-19T04:13:16.455365+0000","last_active":"2024-09-19T04:15:01.405035+0000","last_peered":"2024-09-19T04:15:01.405035+0000","last_clean":"2024-09-19T04:15:01.405035+0000","last_became_active":"2024-09-19T04:13:16.455029+0000","last_became_peered":"2024-09-19T04:13:16.455029+0000","last_unstale":"2024-09-19T04:15:01.405035+0000","last_undegraded":"2024-09-19T04:15:01.405035+0000","last_fullsized":"2024-09-19T04:15:01.405035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:20.947391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.353018+0000","last_change":"2024-09-19T04:13:14.620462+0000","last_active":"2024-09-19T04:15:01.353018+0000","last_peered":"2024-09-19T04:15:01.353018+0000","last_clean":"2024-09-19T04:15:01.353018+0000","last_became_active":"2024-09-19T04:13:14.620103+0000","last_became_peered":"2024-09-19T04:13:14.620103+0000","last_unstale":"2024-09-19T04:15:01.353018+0000","last_undegraded":"2024-09-19T04:15:01.353018+0000","last_fullsized":"2024-09-19T04:15:01.353018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:14.496075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.389192+0000","last_change":"2024-09-19T04:13:16.450222+0000","last_active":"2024-09-19T04:15:01.389192+0000","last_peered":"2024-09-19T04:15:01.389192+0000","last_clean":"2024-09-19T04:15:01.389192+0000","last_became_active":"2024-09-19T04:13:16.449916+0000","last_became_peered":"2024-09-19T04:13:16.449916+0000","last_unstale":"2024-09-19T04:15:01.389192+0000","last_undegraded":"2024-09-19T04:15:01.389192+0000","last_fullsized":"2024-09-19T04:15:01.389192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:31.886342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.345119+0000","last_change":"2024-09-19T04:13:31.519190+0000","last_active":"2024-09-19T04:15:01.345119+0000","last_peered":"2024-09-19T04:15:01.345119+0000","last_clean":"2024-09-19T04:15:01.345119+0000","last_became_active":"2024-09-19T04:13:31.508605+0000","last_became_peered":"2024-09-19T04:13:31.508605+0000","last_unstale":"2024-09-19T04:15:01.345119+0000","last_undegraded":"2024-09-19T04:15:01.345119+0000","last_fullsized":"2024-09-19T04:15:01.345119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:51.437135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.386116+0000","last_change":"2024-09-19T04:13:16.450848+0000","last_active":"2024-09-19T04:15:01.386116+0000","last_peered":"2024-09-19T04:15:01.386116+0000","last_clean":"2024-09-19T04:15:01.386116+0000","last_became_active":"2024-09-19T04:13:16.450496+0000","last_became_peered":"2024-09-19T04:13:16.450496+0000","last_unstale":"2024-09-19T04:15:01.386116+0000","last_undegraded":"2024-09-19T04:15:01.386116+0000","last_fullsized":"2024-09-19T04:15:01.386116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:49:00.346998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.343626+0000","last_change":"2024-09-19T04:13:14.428654+0000","last_active":"2024-09-19T04:15:01.343626+0000","last_peered":"2024-09-19T04:15:01.343626+0000","last_clean":"2024-09-19T04:15:01.343626+0000","last_became_active":"2024-09-19T04:13:14.428297+0000","last_became_peered":"2024-09-19T04:13:14.428297+0000","last_unstale":"2024-09-19T04:15:01.343626+0000","last_undegraded":"2024-09-19T04:15:01.343626+0000","last_fullsized":"2024-09-19T04:15:01.343626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:30.834262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407394+0000","last_change":"2024-09-19T04:13:16.456002+0000","last_active":"2024-09-19T04:15:01.407394+0000","last_peered":"2024-09-19T04:15:01.407394+0000","last_clean":"2024-09-19T04:15:01.407394+0000","last_became_active":"2024-09-19T04:13:16.455463+0000","last_became_peered":"2024-09-19T04:13:16.455463+0000","last_unstale":"2024-09-19T04:15:01.407394+0000","last_undegraded":"2024-09-19T04:15:01.407394+0000","last_fullsized":"2024-09-19T04:15:01.407394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:47.266364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'39","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.568584+0000","last_change":"2024-09-19T04:13:14.429555+0000","last_active":"2024-09-19T04:15:01.568584+0000","last_peered":"2024-09-19T04:15:01.568584+0000","last_clean":"2024-09-19T04:15:01.568584+0000","last_became_active":"2024-09-19T04:13:14.429224+0000","last_became_peered":"2024-09-19T04:13:14.429224+0000","last_unstale":"2024-09-19T04:15:01.568584+0000","last_undegraded":"2024-09-19T04:15:01.568584+0000","last_fullsized":"2024-09-19T04:15:01.568584+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:35.304750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407617+0000","last_change":"2024-09-19T04:13:16.454337+0000","last_active":"2024-09-19T04:15:01.407617+0000","last_peered":"2024-09-19T04:15:01.407617+0000","last_clean":"2024-09-19T04:15:01.407617+0000","last_became_active":"2024-09-19T04:13:16.453831+0000","last_became_peered":"2024-09-19T04:13:16.453831+0000","last_unstale":"2024-09-19T04:15:01.407617+0000","last_undegraded":"2024-09-19T04:15:01.407617+0000","last_fullsized":"2024-09-19T04:15:01.407617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:41.751933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.351335+0000","last_change":"2024-09-19T04:13:14.620406+0000","last_active":"2024-09-19T04:15:01.351335+0000","last_peered":"2024-09-19T04:15:01.351335+0000","last_clean":"2024-09-19T04:15:01.351335+0000","last_became_active":"2024-09-19T04:13:14.620050+0000","last_became_peered":"2024-09-19T04:13:14.620050+0000","last_unstale":"2024-09-19T04:15:01.351335+0000","last_undegraded":"2024-09-19T04:15:01.351335+0000","last_fullsized":"2024-09-19T04:15:01.351335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:35.828760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.408215+0000","last_change":"2024-09-19T04:13:16.459016+0000","last_active":"2024-09-19T04:15:01.408215+0000","last_peered":"2024-09-19T04:15:01.408215+0000","last_clean":"2024-09-19T04:15:01.408215+0000","last_became_active":"2024-09-19T04:13:16.458674+0000","last_became_peered":"2024-09-19T04:13:16.458674+0000","last_unstale":"2024-09-19T04:15:01.408215+0000","last_undegraded":"2024-09-19T04:15:01.408215+0000","last_fullsized":"2024-09-19T04:15:01.408215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:55.894456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.361251+0000","last_change":"2024-09-19T04:12:05.913704+0000","last_active":"2024-09-19T04:15:01.361251+0000","last_peered":"2024-09-19T04:15:01.361251+0000","last_clean":"2024-09-19T04:15:01.361251+0000","last_became_active":"2024-09-19T04:12:05.913299+0000","last_became_peered":"2024-09-19T04:12:05.913299+0000","last_unstale":"2024-09-19T04:15:01.361251+0000","last_undegraded":"2024-09-19T04:15:01.361251+0000","last_fullsized":"2024-09-19T04:15:01.361251+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_clean_scrub_stamp":"2024-09-19T04:08:46.646000+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.638160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.357722+0000","last_change":"2024-09-19T04:13:14.423962+0000","last_active":"2024-09-19T04:15:01.357722+0000","last_peered":"2024-09-19T04:15:01.357722+0000","last_clean":"2024-09-19T04:15:01.357722+0000","last_became_active":"2024-09-19T04:13:14.423591+0000","last_became_peered":"2024-09-19T04:13:14.423591+0000","last_unstale":"2024-09-19T04:15:01.357722+0000","last_undegraded":"2024-09-19T04:15:01.357722+0000","last_fullsized":"2024-09-19T04:15:01.357722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:02.239975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.394412+0000","last_change":"2024-09-19T04:13:16.447113+0000","last_active":"2024-09-19T04:15:01.394412+0000","last_peered":"2024-09-19T04:15:01.394412+0000","last_clean":"2024-09-19T04:15:01.394412+0000","last_became_active":"2024-09-19T04:13:16.446792+0000","last_became_peered":"2024-09-19T04:13:16.446792+0000","last_unstale":"2024-09-19T04:15:01.394412+0000","last_undegraded":"2024-09-19T04:15:01.394412+0000","last_fullsized":"2024-09-19T04:15:01.394412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:28:21.527010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.361188+0000","last_change":"2024-09-19T04:13:14.432663+0000","last_active":"2024-09-19T04:15:01.361188+0000","last_peered":"2024-09-19T04:15:01.361188+0000","last_clean":"2024-09-19T04:15:01.361188+0000","last_became_active":"2024-09-19T04:13:14.432318+0000","last_became_peered":"2024-09-19T04:13:14.432318+0000","last_unstale":"2024-09-19T04:15:01.361188+0000","last_undegraded":"2024-09-19T04:15:01.361188+0000","last_fullsized":"2024-09-19T04:15:01.361188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:35.787596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402670+0000","last_change":"2024-09-19T04:13:16.451010+0000","last_active":"2024-09-19T04:15:01.402670+0000","last_peered":"2024-09-19T04:15:01.402670+0000","last_clean":"2024-09-19T04:15:01.402670+0000","last_became_active":"2024-09-19T04:13:16.450708+0000","last_became_peered":"2024-09-19T04:13:16.450708+0000","last_unstale":"2024-09-19T04:15:01.402670+0000","last_undegraded":"2024-09-19T04:15:01.402670+0000","last_fullsized":"2024-09-19T04:15:01.402670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:30.108541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.347518+0000","last_change":"2024-09-19T04:13:14.620037+0000","last_active":"2024-09-19T04:15:01.347518+0000","last_peered":"2024-09-19T04:15:01.347518+0000","last_clean":"2024-09-19T04:15:01.347518+0000","last_became_active":"2024-09-19T04:13:14.619686+0000","last_became_peered":"2024-09-19T04:13:14.619686+0000","last_unstale":"2024-09-19T04:15:01.347518+0000","last_undegraded":"2024-09-19T04:15:01.347518+0000","last_fullsized":"2024-09-19T04:15:01.347518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:32.490708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387959+0000","last_change":"2024-09-19T04:13:16.446429+0000","last_active":"2024-09-19T04:15:01.387959+0000","last_peered":"2024-09-19T04:15:01.387959+0000","last_clean":"2024-09-19T04:15:01.387959+0000","last_became_active":"2024-09-19T04:13:16.446075+0000","last_became_peered":"2024-09-19T04:13:16.446075+0000","last_unstale":"2024-09-19T04:15:01.387959+0000","last_undegraded":"2024-09-19T04:15:01.387959+0000","last_fullsized":"2024-09-19T04:15:01.387959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:03.613926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.361912+0000","last_change":"2024-09-19T04:13:14.429899+0000","last_active":"2024-09-19T04:15:01.361912+0000","last_peered":"2024-09-19T04:15:01.361912+0000","last_clean":"2024-09-19T04:15:01.361912+0000","last_became_active":"2024-09-19T04:13:14.429571+0000","last_became_peered":"2024-09-19T04:13:14.429571+0000","last_unstale":"2024-09-19T04:15:01.361912+0000","last_undegraded":"2024-09-19T04:15:01.361912+0000","last_fullsized":"2024-09-19T04:15:01.361912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:37.017431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384017+0000","last_change":"2024-09-19T04:13:16.458267+0000","last_active":"2024-09-19T04:15:01.384017+0000","last_peered":"2024-09-19T04:15:01.384017+0000","last_clean":"2024-09-19T04:15:01.384017+0000","last_became_active":"2024-09-19T04:13:16.457911+0000","last_became_peered":"2024-09-19T04:13:16.457911+0000","last_unstale":"2024-09-19T04:15:01.384017+0000","last_undegraded":"2024-09-19T04:15:01.384017+0000","last_fullsized":"2024-09-19T04:15:01.384017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:32.465923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354719+0000","last_change":"2024-09-19T04:13:14.621552+0000","last_active":"2024-09-19T04:15:01.354719+0000","last_peered":"2024-09-19T04:15:01.354719+0000","last_clean":"2024-09-19T04:15:01.354719+0000","last_became_active":"2024-09-19T04:13:14.620592+0000","last_became_peered":"2024-09-19T04:13:14.620592+0000","last_unstale":"2024-09-19T04:15:01.354719+0000","last_undegraded":"2024-09-19T04:15:01.354719+0000","last_fullsized":"2024-09-19T04:15:01.354719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:10.055202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402127+0000","last_change":"2024-09-19T04:13:16.443117+0000","last_active":"2024-09-19T04:15:01.402127+0000","last_peered":"2024-09-19T04:15:01.402127+0000","last_clean":"2024-09-19T04:15:01.402127+0000","last_became_active":"2024-09-19T04:13:16.442808+0000","last_became_peered":"2024-09-19T04:13:16.442808+0000","last_unstale":"2024-09-19T04:15:01.402127+0000","last_undegraded":"2024-09-19T04:15:01.402127+0000","last_fullsized":"2024-09-19T04:15:01.402127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:40.274320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354893+0000","last_change":"2024-09-19T04:13:14.620618+0000","last_active":"2024-09-19T04:15:01.354893+0000","last_peered":"2024-09-19T04:15:01.354893+0000","last_clean":"2024-09-19T04:15:01.354893+0000","last_became_active":"2024-09-19T04:13:14.620290+0000","last_became_peered":"2024-09-19T04:13:14.620290+0000","last_unstale":"2024-09-19T04:15:01.354893+0000","last_undegraded":"2024-09-19T04:15:01.354893+0000","last_fullsized":"2024-09-19T04:15:01.354893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:51.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.390139+0000","last_change":"2024-09-19T04:13:16.443498+0000","last_active":"2024-09-19T04:15:01.390139+0000","last_peered":"2024-09-19T04:15:01.390139+0000","last_clean":"2024-09-19T04:15:01.390139+0000","last_became_active":"2024-09-19T04:13:16.443104+0000","last_became_peered":"2024-09-19T04:13:16.443104+0000","last_unstale":"2024-09-19T04:15:01.390139+0000","last_undegraded":"2024-09-19T04:15:01.390139+0000","last_fullsized":"2024-09-19T04:15:01.390139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:10.733570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'45","reported_seq":92,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.571386+0000","last_change":"2024-09-19T04:13:14.425881+0000","last_active":"2024-09-19T04:15:01.571386+0000","last_peered":"2024-09-19T04:15:01.571386+0000","last_clean":"2024-09-19T04:15:01.571386+0000","last_became_active":"2024-09-19T04:13:14.425542+0000","last_became_peered":"2024-09-19T04:13:14.425542+0000","last_unstale":"2024-09-19T04:15:01.571386+0000","last_undegraded":"2024-09-19T04:15:01.571386+0000","last_fullsized":"2024-09-19T04:15:01.571386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:44.250890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392711+0000","last_change":"2024-09-19T04:13:16.454672+0000","last_active":"2024-09-19T04:15:01.392711+0000","last_peered":"2024-09-19T04:15:01.392711+0000","last_clean":"2024-09-19T04:15:01.392711+0000","last_became_active":"2024-09-19T04:13:16.454003+0000","last_became_peered":"2024-09-19T04:13:16.454003+0000","last_unstale":"2024-09-19T04:15:01.392711+0000","last_undegraded":"2024-09-19T04:15:01.392711+0000","last_fullsized":"2024-09-19T04:15:01.392711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:26.849617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.357930+0000","last_change":"2024-09-19T04:13:14.432312+0000","last_active":"2024-09-19T04:15:01.357930+0000","last_peered":"2024-09-19T04:15:01.357930+0000","last_clean":"2024-09-19T04:15:01.357930+0000","last_became_active":"2024-09-19T04:13:14.431994+0000","last_became_peered":"2024-09-19T04:13:14.431994+0000","last_unstale":"2024-09-19T04:15:01.357930+0000","last_undegraded":"2024-09-19T04:15:01.357930+0000","last_fullsized":"2024-09-19T04:15:01.357930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:00.410400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.391357+0000","last_change":"2024-09-19T04:13:16.455312+0000","last_active":"2024-09-19T04:15:01.391357+0000","last_peered":"2024-09-19T04:15:01.391357+0000","last_clean":"2024-09-19T04:15:01.391357+0000","last_became_active":"2024-09-19T04:13:16.454990+0000","last_became_peered":"2024-09-19T04:13:16.454990+0000","last_unstale":"2024-09-19T04:15:01.391357+0000","last_undegraded":"2024-09-19T04:15:01.391357+0000","last_fullsized":"2024-09-19T04:15:01.391357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:09.207395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.349902+0000","last_change":"2024-09-19T04:13:14.620316+0000","last_active":"2024-09-19T04:15:01.349902+0000","last_peered":"2024-09-19T04:15:01.349902+0000","last_clean":"2024-09-19T04:15:01.349902+0000","last_became_active":"2024-09-19T04:13:14.619979+0000","last_became_peered":"2024-09-19T04:13:14.619979+0000","last_unstale":"2024-09-19T04:15:01.349902+0000","last_undegraded":"2024-09-19T04:15:01.349902+0000","last_fullsized":"2024-09-19T04:15:01.349902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:24.382144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.379294+0000","last_change":"2024-09-19T04:13:16.455684+0000","last_active":"2024-09-19T04:15:01.379294+0000","last_peered":"2024-09-19T04:15:01.379294+0000","last_clean":"2024-09-19T04:15:01.379294+0000","last_became_active":"2024-09-19T04:13:16.455355+0000","last_became_peered":"2024-09-19T04:13:16.455355+0000","last_unstale":"2024-09-19T04:15:01.379294+0000","last_undegraded":"2024-09-19T04:15:01.379294+0000","last_fullsized":"2024-09-19T04:15:01.379294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:09.657447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.346821+0000","last_change":"2024-09-19T04:13:14.423599+0000","last_active":"2024-09-19T04:15:01.346821+0000","last_peered":"2024-09-19T04:15:01.346821+0000","last_clean":"2024-09-19T04:15:01.346821+0000","last_became_active":"2024-09-19T04:13:14.423282+0000","last_became_peered":"2024-09-19T04:13:14.423282+0000","last_unstale":"2024-09-19T04:15:01.346821+0000","last_undegraded":"2024-09-19T04:15:01.346821+0000","last_fullsized":"2024-09-19T04:15:01.346821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.975368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.376076+0000","last_change":"2024-09-19T04:13:16.450435+0000","last_active":"2024-09-19T04:15:01.376076+0000","last_peered":"2024-09-19T04:15:01.376076+0000","last_clean":"2024-09-19T04:15:01.376076+0000","last_became_active":"2024-09-19T04:13:16.450087+0000","last_became_peered":"2024-09-19T04:13:16.450087+0000","last_unstale":"2024-09-19T04:15:01.376076+0000","last_undegraded":"2024-09-19T04:15:01.376076+0000","last_fullsized":"2024-09-19T04:15:01.376076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:34:51.764776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.351312+0000","last_change":"2024-09-19T04:13:14.432868+0000","last_active":"2024-09-19T04:15:01.351312+0000","last_peered":"2024-09-19T04:15:01.351312+0000","last_clean":"2024-09-19T04:15:01.351312+0000","last_became_active":"2024-09-19T04:13:14.432507+0000","last_became_peered":"2024-09-19T04:13:14.432507+0000","last_unstale":"2024-09-19T04:15:01.351312+0000","last_undegraded":"2024-09-19T04:15:01.351312+0000","last_fullsized":"2024-09-19T04:15:01.351312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:14.773711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.390493+0000","last_change":"2024-09-19T04:13:16.450061+0000","last_active":"2024-09-19T04:15:01.390493+0000","last_peered":"2024-09-19T04:15:01.390493+0000","last_clean":"2024-09-19T04:15:01.390493+0000","last_became_active":"2024-09-19T04:13:16.448829+0000","last_became_peered":"2024-09-19T04:13:16.448829+0000","last_unstale":"2024-09-19T04:15:01.390493+0000","last_undegraded":"2024-09-19T04:15:01.390493+0000","last_fullsized":"2024-09-19T04:15:01.390493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:31.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354057+0000","last_change":"2024-09-19T04:13:14.431574+0000","last_active":"2024-09-19T04:15:01.354057+0000","last_peered":"2024-09-19T04:15:01.354057+0000","last_clean":"2024-09-19T04:15:01.354057+0000","last_became_active":"2024-09-19T04:13:14.431202+0000","last_became_peered":"2024-09-19T04:13:14.431202+0000","last_unstale":"2024-09-19T04:15:01.354057+0000","last_undegraded":"2024-09-19T04:15:01.354057+0000","last_fullsized":"2024-09-19T04:15:01.354057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:58.167201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.408741+0000","last_change":"2024-09-19T04:13:16.452688+0000","last_active":"2024-09-19T04:15:01.408741+0000","last_peered":"2024-09-19T04:15:01.408741+0000","last_clean":"2024-09-19T04:15:01.408741+0000","last_became_active":"2024-09-19T04:13:16.452342+0000","last_became_peered":"2024-09-19T04:13:16.452342+0000","last_unstale":"2024-09-19T04:15:01.408741+0000","last_undegraded":"2024-09-19T04:15:01.408741+0000","last_fullsized":"2024-09-19T04:15:01.408741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:03.162688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.403276+0000","last_change":"2024-09-19T04:13:16.449402+0000","last_active":"2024-09-19T04:15:01.403276+0000","last_peered":"2024-09-19T04:15:01.403276+0000","last_clean":"2024-09-19T04:15:01.403276+0000","last_became_active":"2024-09-19T04:13:16.449108+0000","last_became_peered":"2024-09-19T04:13:16.449108+0000","last_unstale":"2024-09-19T04:15:01.403276+0000","last_undegraded":"2024-09-19T04:15:01.403276+0000","last_fullsized":"2024-09-19T04:15:01.403276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:58.874638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354429+0000","last_change":"2024-09-19T04:13:14.430684+0000","last_active":"2024-09-19T04:15:01.354429+0000","last_peered":"2024-09-19T04:15:01.354429+0000","last_clean":"2024-09-19T04:15:01.354429+0000","last_became_active":"2024-09-19T04:13:14.430137+0000","last_became_peered":"2024-09-19T04:13:14.430137+0000","last_unstale":"2024-09-19T04:15:01.354429+0000","last_undegraded":"2024-09-19T04:15:01.354429+0000","last_fullsized":"2024-09-19T04:15:01.354429+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:39.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.403606+0000","last_change":"2024-09-19T04:13:16.452491+0000","last_active":"2024-09-19T04:15:01.403606+0000","last_peered":"2024-09-19T04:15:01.403606+0000","last_clean":"2024-09-19T04:15:01.403606+0000","last_became_active":"2024-09-19T04:13:16.452204+0000","last_became_peered":"2024-09-19T04:13:16.452204+0000","last_unstale":"2024-09-19T04:15:01.403606+0000","last_undegraded":"2024-09-19T04:15:01.403606+0000","last_fullsized":"2024-09-19T04:15:01.403606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:10.113510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354761+0000","last_change":"2024-09-19T04:13:14.431608+0000","last_active":"2024-09-19T04:15:01.354761+0000","last_peered":"2024-09-19T04:15:01.354761+0000","last_clean":"2024-09-19T04:15:01.354761+0000","last_became_active":"2024-09-19T04:13:14.431111+0000","last_became_peered":"2024-09-19T04:13:14.431111+0000","last_unstale":"2024-09-19T04:15:01.354761+0000","last_undegraded":"2024-09-19T04:15:01.354761+0000","last_fullsized":"2024-09-19T04:15:01.354761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:03:41.125315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.393098+0000","last_change":"2024-09-19T04:13:16.457867+0000","last_active":"2024-09-19T04:15:01.393098+0000","last_peered":"2024-09-19T04:15:01.393098+0000","last_clean":"2024-09-19T04:15:01.393098+0000","last_became_active":"2024-09-19T04:13:16.457582+0000","last_became_peered":"2024-09-19T04:13:16.457582+0000","last_unstale":"2024-09-19T04:15:01.393098+0000","last_undegraded":"2024-09-19T04:15:01.393098+0000","last_fullsized":"2024-09-19T04:15:01.393098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:58.076250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.346539+0000","last_change":"2024-09-19T04:13:14.620070+0000","last_active":"2024-09-19T04:15:01.346539+0000","last_peered":"2024-09-19T04:15:01.346539+0000","last_clean":"2024-09-19T04:15:01.346539+0000","last_became_active":"2024-09-19T04:13:14.619734+0000","last_became_peered":"2024-09-19T04:13:14.619734+0000","last_unstale":"2024-09-19T04:15:01.346539+0000","last_undegraded":"2024-09-19T04:15:01.346539+0000","last_fullsized":"2024-09-19T04:15:01.346539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:44:02.818859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388185+0000","last_change":"2024-09-19T04:13:16.443832+0000","last_active":"2024-09-19T04:15:01.388185+0000","last_peered":"2024-09-19T04:15:01.388185+0000","last_clean":"2024-09-19T04:15:01.388185+0000","last_became_active":"2024-09-19T04:13:16.442690+0000","last_became_peered":"2024-09-19T04:13:16.442690+0000","last_unstale":"2024-09-19T04:15:01.388185+0000","last_undegraded":"2024-09-19T04:15:01.388185+0000","last_fullsized":"2024-09-19T04:15:01.388185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:39.535628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.355196+0000","last_change":"2024-09-19T04:13:14.427186+0000","last_active":"2024-09-19T04:15:01.355196+0000","last_peered":"2024-09-19T04:15:01.355196+0000","last_clean":"2024-09-19T04:15:01.355196+0000","last_became_active":"2024-09-19T04:13:14.426949+0000","last_became_peered":"2024-09-19T04:13:14.426949+0000","last_unstale":"2024-09-19T04:15:01.355196+0000","last_undegraded":"2024-09-19T04:15:01.355196+0000","last_fullsized":"2024-09-19T04:15:01.355196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:26.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":592516,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":700,"num_read_kb":454,"num_write":289,"num_write_kb":1431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2002944,"data_stored":1777548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612642,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":440451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841575,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33000,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712152,"statfs":{"total":95995035648,"available":95961243648,"internally_reserved":0,"allocated":1736704,"data_stored":1226160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005100,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40792,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704360,"statfs":{"total":95995035648,"available":95953264640,"internally_reserved":0,"allocated":1314816,"data_stored":833847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28304,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716848,"statfs":{"total":95995035648,"available":95966052352,"internally_reserved":0,"allocated":1310720,"data_stored":833745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1515520,"data_stored":1034902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":909312,"data_stored":440020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37028,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708124,"statfs":{"total":95995035648,"available":95957118976,"internally_reserved":0,"allocated":1662976,"data_stored":1164780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":441470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1454080,"data_stored":965147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476815,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":503813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607636,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":500305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":929792,"data_stored":441291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":195862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:15:06.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:15:06.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:06 smithi112 ceph-mon[35051]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:15:06.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:06 smithi112 ceph-mon[35051]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:15:06.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:06 smithi112 ceph-mon[35051]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:15:06.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:06 smithi112 ceph-mon[35051]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:15:06.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:06 smithi112 ceph-mon[35051]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:15:06.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:06 smithi112 ceph-mon[35051]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:15:06.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:06 smithi112 ceph-mon[35051]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:15:06.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:06 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3545407084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:15:06.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:06 smithi112 ceph-mon[35051]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-19T04:15:06.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:06 smithi104 ceph-mon[34145]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:15:06.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:06 smithi104 ceph-mon[34145]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:15:06.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:06 smithi104 ceph-mon[34145]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:15:06.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:06 smithi104 ceph-mon[34145]: Updating smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:15:06.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:06 smithi104 ceph-mon[34145]: Updating smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:15:06.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:06 smithi104 ceph-mon[34145]: Updating smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:15:06.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:06 smithi104 ceph-mon[34145]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:15:06.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:06 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3545407084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:15:06.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:06 smithi104 ceph-mon[34145]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-19T04:15:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.899+0000 7f4ef571a640 1 -- 172.21.15.94:0/2036794087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0107ab0 msgr2=0x7f4ef0107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.899+0000 7f4ef571a640 1 --2- 172.21.15.94:0/2036794087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0107ab0 0x7f4ef0107e90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009930 tx=0x7f4ed802a420 comp rx=0 tx=0).stop 2024-09-19T04:15:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.899+0000 7f4ef571a640 1 -- 172.21.15.94:0/2036794087 shutdown_connections 2024-09-19T04:15:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.899+0000 7f4ef571a640 1 --2- 172.21.15.94:0/2036794087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef0102440 0x7f4ef010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.899+0000 7f4ef571a640 1 --2- 172.21.15.94:0/2036794087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ef0101aa0 0x7f4ef0101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.899+0000 7f4ef571a640 1 --2- 172.21.15.94:0/2036794087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0107ab0 0x7f4ef0107e90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.899+0000 7f4ef571a640 1 -- 172.21.15.94:0/2036794087 >> 172.21.15.94:0/2036794087 conn(0x7f4ef00fd7c0 msgr2=0x7f4ef00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.899+0000 7f4ef571a640 1 -- 172.21.15.94:0/2036794087 shutdown_connections 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.899+0000 7f4ef571a640 1 -- 172.21.15.94:0/2036794087 wait complete. 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4ef571a640 1 Processor -- start 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4ef571a640 1 -- start start 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4ef571a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ef0101aa0 0x7f4ef01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4ef571a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0102440 0x7f4ef01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4ef571a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef0107ab0 0x7f4ef019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4ef571a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0111da0 con 0x7f4ef0102440 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4ef571a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0111c20 con 0x7f4ef0107ab0 2024-09-19T04:15:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4ef571a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0111f20 con 0x7f4ef0101aa0 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4eee7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0102440 0x7f4ef01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4eee7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0102440 0x7f4ef01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46594/0 (socket says 172.21.15.94:46594) 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4eee7fc640 1 -- 172.21.15.94:0/3857998007 learned_addr learned my addr 172.21.15.94:0/3857998007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4eeeffd640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ef0101aa0 0x7f4ef01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4eee7fc640 1 -- 172.21.15.94:0/3857998007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ef0101aa0 msgr2=0x7f4ef01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4eee7fc640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ef0101aa0 0x7f4ef01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4eee7fc640 1 -- 172.21.15.94:0/3857998007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef0107ab0 msgr2=0x7f4ef019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.900+0000 7f4eee7fc640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef0107ab0 0x7f4ef019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.901+0000 7f4eee7fc640 1 -- 172.21.15.94:0/3857998007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef019d3a0 con 0x7f4ef0102440 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.901+0000 7f4eee7fc640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0102440 0x7f4ef01988f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000d950 tx=0x7f4ee000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.901+0000 7f4ee7fff640 1 -- 172.21.15.94:0/3857998007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0017070 con 0x7f4ef0102440 2024-09-19T04:15:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.901+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef019d690 con 0x7f4ef0102440 2024-09-19T04:15:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.901+0000 7f4ee7fff640 1 -- 172.21.15.94:0/3857998007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee0009c30 con 0x7f4ef0102440 2024-09-19T04:15:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.901+0000 7f4ee7fff640 1 -- 172.21.15.94:0/3857998007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee000f620 con 0x7f4ef0102440 2024-09-19T04:15:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.901+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef01a4f20 con 0x7f4ef0102440 2024-09-19T04:15:06.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.902+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef0101f00 con 0x7f4ef0102440 2024-09-19T04:15:06.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.903+0000 7f4ee7fff640 1 -- 172.21.15.94:0/3857998007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f4ee00109a0 con 0x7f4ef0102440 2024-09-19T04:15:06.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.903+0000 7f4ee7fff640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4edc0a3450 0x7f4edc0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.904+0000 7f4eeeffd640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4edc0a3450 0x7f4edc0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.904+0000 7f4ee7fff640 1 -- 172.21.15.94:0/3857998007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f4ee00cfbb0 con 0x7f4ef0102440 2024-09-19T04:15:06.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.904+0000 7f4eeeffd640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4edc0a3450 0x7f4edc0a5910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8006010 tx=0x7f4ed8000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:06.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:06.908+0000 7f4ee7fff640 1 -- 172.21.15.94:0/3857998007 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4ee0009de0 con 0x7f4ef0102440 2024-09-19T04:15:07.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.073+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 --> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ef0062630 con 0x7f4edc0a3450 2024-09-19T04:15:07.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.079+0000 7f4ee7fff640 1 -- 172.21.15.94:0/3857998007 <== mgr.15159 v2:172.21.15.94:6800/3395191155 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257863 (secure 0 0 0) 0x7f4ef0062630 con 0x7f4edc0a3450 2024-09-19T04:15:07.084 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:07.085 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-19T04:15:07.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4edc0a3450 msgr2=0x7f4edc0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:07.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4edc0a3450 0x7f4edc0a5910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8006010 tx=0x7f4ed8000bc0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0102440 msgr2=0x7f4ef01988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:07.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0102440 0x7f4ef01988f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000d950 tx=0x7f4ee000de10 comp rx=0 tx=0).stop 2024-09-19T04:15:07.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 shutdown_connections 2024-09-19T04:15:07.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4edc0a3450 0x7f4edc0a5910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef0107ab0 0x7f4ef019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef0102440 0x7f4ef01988f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 --2- 172.21.15.94:0/3857998007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ef0101aa0 0x7f4ef01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 >> 172.21.15.94:0/3857998007 conn(0x7f4ef00fd7c0 msgr2=0x7f4ef00ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 shutdown_connections 2024-09-19T04:15:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.082+0000 7f4ef571a640 1 -- 172.21.15.94:0/3857998007 wait complete. 2024-09-19T04:15:07.095 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T04:15:05.384111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183118,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":836,"num_read_kb":573,"num_write":472,"num_write_kb":4371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":401296,"kb_used_data":14280,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540528,"statfs":{"total":1151940427776,"available":1151529500672,"internally_reserved":0,"allocated":14622720,"data_stored":8825931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.017538"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388917+0000","last_change":"2024-09-19T04:13:16.451031+0000","last_active":"2024-09-19T04:15:01.388917+0000","last_peered":"2024-09-19T04:15:01.388917+0000","last_clean":"2024-09-19T04:15:01.388917+0000","last_became_active":"2024-09-19T04:13:16.450713+0000","last_became_peered":"2024-09-19T04:13:16.450713+0000","last_unstale":"2024-09-19T04:15:01.388917+0000","last_undegraded":"2024-09-19T04:15:01.388917+0000","last_fullsized":"2024-09-19T04:15:01.388917+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:59.485415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.381120+0000","last_change":"2024-09-19T04:13:16.455689+0000","last_active":"2024-09-19T04:15:01.381120+0000","last_peered":"2024-09-19T04:15:01.381120+0000","last_clean":"2024-09-19T04:15:01.381120+0000","last_became_active":"2024-09-19T04:13:16.455154+0000","last_became_peered":"2024-09-19T04:13:16.455154+0000","last_unstale":"2024-09-19T04:15:01.381120+0000","last_undegraded":"2024-09-19T04:15:01.381120+0000","last_fullsized":"2024-09-19T04:15:01.381120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:57.610619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.381282+0000","last_change":"2024-09-19T04:13:16.447392+0000","last_active":"2024-09-19T04:15:01.381282+0000","last_peered":"2024-09-19T04:15:01.381282+0000","last_clean":"2024-09-19T04:15:01.381282+0000","last_became_active":"2024-09-19T04:13:16.447088+0000","last_became_peered":"2024-09-19T04:13:16.447088+0000","last_unstale":"2024-09-19T04:15:01.381282+0000","last_undegraded":"2024-09-19T04:15:01.381282+0000","last_fullsized":"2024-09-19T04:15:01.381282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:16.591420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387155+0000","last_change":"2024-09-19T04:13:16.452642+0000","last_active":"2024-09-19T04:15:01.387155+0000","last_peered":"2024-09-19T04:15:01.387155+0000","last_clean":"2024-09-19T04:15:01.387155+0000","last_became_active":"2024-09-19T04:13:16.452329+0000","last_became_peered":"2024-09-19T04:13:16.452329+0000","last_unstale":"2024-09-19T04:15:01.387155+0000","last_undegraded":"2024-09-19T04:15:01.387155+0000","last_fullsized":"2024-09-19T04:15:01.387155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:54.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384913+0000","last_change":"2024-09-19T04:13:16.447171+0000","last_active":"2024-09-19T04:15:01.384913+0000","last_peered":"2024-09-19T04:15:01.384913+0000","last_clean":"2024-09-19T04:15:01.384913+0000","last_became_active":"2024-09-19T04:13:16.446835+0000","last_became_peered":"2024-09-19T04:13:16.446835+0000","last_unstale":"2024-09-19T04:15:01.384913+0000","last_undegraded":"2024-09-19T04:15:01.384913+0000","last_fullsized":"2024-09-19T04:15:01.384913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:41.892118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.377175+0000","last_change":"2024-09-19T04:13:31.507665+0000","last_active":"2024-09-19T04:15:01.377175+0000","last_peered":"2024-09-19T04:15:01.377175+0000","last_clean":"2024-09-19T04:15:01.377175+0000","last_became_active":"2024-09-19T04:13:31.507329+0000","last_became_peered":"2024-09-19T04:13:31.507329+0000","last_unstale":"2024-09-19T04:15:01.377175+0000","last_undegraded":"2024-09-19T04:15:01.377175+0000","last_fullsized":"2024-09-19T04:15:01.377175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:35.832776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388719+0000","last_change":"2024-09-19T04:13:16.454568+0000","last_active":"2024-09-19T04:15:01.388719+0000","last_peered":"2024-09-19T04:15:01.388719+0000","last_clean":"2024-09-19T04:15:01.388719+0000","last_became_active":"2024-09-19T04:13:16.454230+0000","last_became_peered":"2024-09-19T04:13:16.454230+0000","last_unstale":"2024-09-19T04:15:01.388719+0000","last_undegraded":"2024-09-19T04:15:01.388719+0000","last_fullsized":"2024-09-19T04:15:01.388719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:23.401316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402188+0000","last_change":"2024-09-19T04:13:16.449223+0000","last_active":"2024-09-19T04:15:01.402188+0000","last_peered":"2024-09-19T04:15:01.402188+0000","last_clean":"2024-09-19T04:15:01.402188+0000","last_became_active":"2024-09-19T04:13:16.448665+0000","last_became_peered":"2024-09-19T04:13:16.448665+0000","last_unstale":"2024-09-19T04:15:01.402188+0000","last_undegraded":"2024-09-19T04:15:01.402188+0000","last_fullsized":"2024-09-19T04:15:01.402188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:29.619847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402682+0000","last_change":"2024-09-19T04:13:16.458203+0000","last_active":"2024-09-19T04:15:01.402682+0000","last_peered":"2024-09-19T04:15:01.402682+0000","last_clean":"2024-09-19T04:15:01.402682+0000","last_became_active":"2024-09-19T04:13:16.457544+0000","last_became_peered":"2024-09-19T04:13:16.457544+0000","last_unstale":"2024-09-19T04:15:01.402682+0000","last_undegraded":"2024-09-19T04:15:01.402682+0000","last_fullsized":"2024-09-19T04:15:01.402682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:47.611695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.385340+0000","last_change":"2024-09-19T04:13:16.454213+0000","last_active":"2024-09-19T04:15:01.385340+0000","last_peered":"2024-09-19T04:15:01.385340+0000","last_clean":"2024-09-19T04:15:01.385340+0000","last_became_active":"2024-09-19T04:13:16.453047+0000","last_became_peered":"2024-09-19T04:13:16.453047+0000","last_unstale":"2024-09-19T04:15:01.385340+0000","last_undegraded":"2024-09-19T04:15:01.385340+0000","last_fullsized":"2024-09-19T04:15:01.385340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:27.425259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407089+0000","last_change":"2024-09-19T04:13:16.458195+0000","last_active":"2024-09-19T04:15:01.407089+0000","last_peered":"2024-09-19T04:15:01.407089+0000","last_clean":"2024-09-19T04:15:01.407089+0000","last_became_active":"2024-09-19T04:13:16.457891+0000","last_became_peered":"2024-09-19T04:13:16.457891+0000","last_unstale":"2024-09-19T04:15:01.407089+0000","last_undegraded":"2024-09-19T04:15:01.407089+0000","last_fullsized":"2024-09-19T04:15:01.407089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:53.833249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.379554+0000","last_change":"2024-09-19T04:13:31.508117+0000","last_active":"2024-09-19T04:15:01.379554+0000","last_peered":"2024-09-19T04:15:01.379554+0000","last_clean":"2024-09-19T04:15:01.379554+0000","last_became_active":"2024-09-19T04:13:31.507693+0000","last_became_peered":"2024-09-19T04:13:31.507693+0000","last_unstale":"2024-09-19T04:15:01.379554+0000","last_undegraded":"2024-09-19T04:15:01.379554+0000","last_fullsized":"2024-09-19T04:15:01.379554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:58.447965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392023+0000","last_change":"2024-09-19T04:13:16.440079+0000","last_active":"2024-09-19T04:15:01.392023+0000","last_peered":"2024-09-19T04:15:01.392023+0000","last_clean":"2024-09-19T04:15:01.392023+0000","last_became_active":"2024-09-19T04:13:16.439474+0000","last_became_peered":"2024-09-19T04:13:16.439474+0000","last_unstale":"2024-09-19T04:15:01.392023+0000","last_undegraded":"2024-09-19T04:15:01.392023+0000","last_fullsized":"2024-09-19T04:15:01.392023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:11.151412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384315+0000","last_change":"2024-09-19T04:13:16.449958+0000","last_active":"2024-09-19T04:15:01.384315+0000","last_peered":"2024-09-19T04:15:01.384315+0000","last_clean":"2024-09-19T04:15:01.384315+0000","last_became_active":"2024-09-19T04:13:16.449665+0000","last_became_peered":"2024-09-19T04:13:16.449665+0000","last_unstale":"2024-09-19T04:15:01.384315+0000","last_undegraded":"2024-09-19T04:15:01.384315+0000","last_fullsized":"2024-09-19T04:15:01.384315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:24.023779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.383771+0000","last_change":"2024-09-19T04:13:16.454333+0000","last_active":"2024-09-19T04:15:01.383771+0000","last_peered":"2024-09-19T04:15:01.383771+0000","last_clean":"2024-09-19T04:15:01.383771+0000","last_became_active":"2024-09-19T04:13:16.454066+0000","last_became_peered":"2024-09-19T04:13:16.454066+0000","last_unstale":"2024-09-19T04:15:01.383771+0000","last_undegraded":"2024-09-19T04:15:01.383771+0000","last_fullsized":"2024-09-19T04:15:01.383771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:16.815173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.386728+0000","last_change":"2024-09-19T04:13:16.451864+0000","last_active":"2024-09-19T04:15:01.386728+0000","last_peered":"2024-09-19T04:15:01.386728+0000","last_clean":"2024-09-19T04:15:01.386728+0000","last_became_active":"2024-09-19T04:13:16.450995+0000","last_became_peered":"2024-09-19T04:13:16.450995+0000","last_unstale":"2024-09-19T04:15:01.386728+0000","last_undegraded":"2024-09-19T04:15:01.386728+0000","last_fullsized":"2024-09-19T04:15:01.386728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:40.750221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.389034+0000","last_change":"2024-09-19T04:13:16.453176+0000","last_active":"2024-09-19T04:15:01.389034+0000","last_peered":"2024-09-19T04:15:01.389034+0000","last_clean":"2024-09-19T04:15:01.389034+0000","last_became_active":"2024-09-19T04:13:16.452788+0000","last_became_peered":"2024-09-19T04:13:16.452788+0000","last_unstale":"2024-09-19T04:15:01.389034+0000","last_undegraded":"2024-09-19T04:15:01.389034+0000","last_fullsized":"2024-09-19T04:15:01.389034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:47.099180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.377948+0000","last_change":"2024-09-19T04:13:16.449167+0000","last_active":"2024-09-19T04:15:01.377948+0000","last_peered":"2024-09-19T04:15:01.377948+0000","last_clean":"2024-09-19T04:15:01.377948+0000","last_became_active":"2024-09-19T04:13:16.448579+0000","last_became_peered":"2024-09-19T04:13:16.448579+0000","last_unstale":"2024-09-19T04:15:01.377948+0000","last_undegraded":"2024-09-19T04:15:01.377948+0000","last_fullsized":"2024-09-19T04:15:01.377948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:22:40.023000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387764+0000","last_change":"2024-09-19T04:13:16.445449+0000","last_active":"2024-09-19T04:15:01.387764+0000","last_peered":"2024-09-19T04:15:01.387764+0000","last_clean":"2024-09-19T04:15:01.387764+0000","last_became_active":"2024-09-19T04:13:16.445028+0000","last_became_peered":"2024-09-19T04:13:16.445028+0000","last_unstale":"2024-09-19T04:15:01.387764+0000","last_undegraded":"2024-09-19T04:15:01.387764+0000","last_fullsized":"2024-09-19T04:15:01.387764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:13.095348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.386639+0000","last_change":"2024-09-19T04:13:16.454033+0000","last_active":"2024-09-19T04:15:01.386639+0000","last_peered":"2024-09-19T04:15:01.386639+0000","last_clean":"2024-09-19T04:15:01.386639+0000","last_became_active":"2024-09-19T04:13:16.453592+0000","last_became_peered":"2024-09-19T04:13:16.453592+0000","last_unstale":"2024-09-19T04:15:01.386639+0000","last_undegraded":"2024-09-19T04:15:01.386639+0000","last_fullsized":"2024-09-19T04:15:01.386639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.304478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.408187+0000","last_change":"2024-09-19T04:13:16.448763+0000","last_active":"2024-09-19T04:15:01.408187+0000","last_peered":"2024-09-19T04:15:01.408187+0000","last_clean":"2024-09-19T04:15:01.408187+0000","last_became_active":"2024-09-19T04:13:16.448459+0000","last_became_peered":"2024-09-19T04:13:16.448459+0000","last_unstale":"2024-09-19T04:15:01.408187+0000","last_undegraded":"2024-09-19T04:15:01.408187+0000","last_fullsized":"2024-09-19T04:15:01.408187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.403838+0000","last_change":"2024-09-19T04:13:16.451489+0000","last_active":"2024-09-19T04:15:01.403838+0000","last_peered":"2024-09-19T04:15:01.403838+0000","last_clean":"2024-09-19T04:15:01.403838+0000","last_became_active":"2024-09-19T04:13:16.451176+0000","last_became_peered":"2024-09-19T04:13:16.451176+0000","last_unstale":"2024-09-19T04:15:01.403838+0000","last_undegraded":"2024-09-19T04:15:01.403838+0000","last_fullsized":"2024-09-19T04:15:01.403838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.921102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387603+0000","last_change":"2024-09-19T04:13:16.454551+0000","last_active":"2024-09-19T04:15:01.387603+0000","last_peered":"2024-09-19T04:15:01.387603+0000","last_clean":"2024-09-19T04:15:01.387603+0000","last_became_active":"2024-09-19T04:13:16.453976+0000","last_became_peered":"2024-09-19T04:13:16.453976+0000","last_unstale":"2024-09-19T04:15:01.387603+0000","last_undegraded":"2024-09-19T04:15:01.387603+0000","last_fullsized":"2024-09-19T04:15:01.387603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:06.504615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407629+0000","last_change":"2024-09-19T04:13:31.503222+0000","last_active":"2024-09-19T04:15:01.407629+0000","last_peered":"2024-09-19T04:15:01.407629+0000","last_clean":"2024-09-19T04:15:01.407629+0000","last_became_active":"2024-09-19T04:13:31.502873+0000","last_became_peered":"2024-09-19T04:13:31.502873+0000","last_unstale":"2024-09-19T04:15:01.407629+0000","last_undegraded":"2024-09-19T04:15:01.407629+0000","last_fullsized":"2024-09-19T04:15:01.407629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:54.559592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388512+0000","last_change":"2024-09-19T04:13:31.505834+0000","last_active":"2024-09-19T04:15:01.388512+0000","last_peered":"2024-09-19T04:15:01.388512+0000","last_clean":"2024-09-19T04:15:01.388512+0000","last_became_active":"2024-09-19T04:13:31.505501+0000","last_became_peered":"2024-09-19T04:13:31.505501+0000","last_unstale":"2024-09-19T04:15:01.388512+0000","last_undegraded":"2024-09-19T04:15:01.388512+0000","last_fullsized":"2024-09-19T04:15:01.388512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:45.136128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384870+0000","last_change":"2024-09-19T04:13:16.449237+0000","last_active":"2024-09-19T04:15:01.384870+0000","last_peered":"2024-09-19T04:15:01.384870+0000","last_clean":"2024-09-19T04:15:01.384870+0000","last_became_active":"2024-09-19T04:13:16.448896+0000","last_became_peered":"2024-09-19T04:13:16.448896+0000","last_unstale":"2024-09-19T04:15:01.384870+0000","last_undegraded":"2024-09-19T04:15:01.384870+0000","last_fullsized":"2024-09-19T04:15:01.384870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:38.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384284+0000","last_change":"2024-09-19T04:13:16.456053+0000","last_active":"2024-09-19T04:15:01.384284+0000","last_peered":"2024-09-19T04:15:01.384284+0000","last_clean":"2024-09-19T04:15:01.384284+0000","last_became_active":"2024-09-19T04:13:16.454775+0000","last_became_peered":"2024-09-19T04:13:16.454775+0000","last_unstale":"2024-09-19T04:15:01.384284+0000","last_undegraded":"2024-09-19T04:15:01.384284+0000","last_fullsized":"2024-09-19T04:15:01.384284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.902036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402500+0000","last_change":"2024-09-19T04:13:16.451473+0000","last_active":"2024-09-19T04:15:01.402500+0000","last_peered":"2024-09-19T04:15:01.402500+0000","last_clean":"2024-09-19T04:15:01.402500+0000","last_became_active":"2024-09-19T04:13:16.451150+0000","last_became_peered":"2024-09-19T04:13:16.451150+0000","last_unstale":"2024-09-19T04:15:01.402500+0000","last_undegraded":"2024-09-19T04:15:01.402500+0000","last_fullsized":"2024-09-19T04:15:01.402500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:59.332745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392354+0000","last_change":"2024-09-19T04:13:16.452017+0000","last_active":"2024-09-19T04:15:01.392354+0000","last_peered":"2024-09-19T04:15:01.392354+0000","last_clean":"2024-09-19T04:15:01.392354+0000","last_became_active":"2024-09-19T04:13:16.450715+0000","last_became_peered":"2024-09-19T04:13:16.450715+0000","last_unstale":"2024-09-19T04:15:01.392354+0000","last_undegraded":"2024-09-19T04:15:01.392354+0000","last_fullsized":"2024-09-19T04:15:01.392354+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:43.548770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.380440+0000","last_change":"2024-09-19T04:13:31.736597+0000","last_active":"2024-09-19T04:15:01.380440+0000","last_peered":"2024-09-19T04:15:01.380440+0000","last_clean":"2024-09-19T04:15:01.380440+0000","last_became_active":"2024-09-19T04:13:31.736301+0000","last_became_peered":"2024-09-19T04:13:31.736301+0000","last_unstale":"2024-09-19T04:15:01.380440+0000","last_undegraded":"2024-09-19T04:15:01.380440+0000","last_fullsized":"2024-09-19T04:15:01.380440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:06.614117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384673+0000","last_change":"2024-09-19T04:13:31.509742+0000","last_active":"2024-09-19T04:15:01.384673+0000","last_peered":"2024-09-19T04:15:01.384673+0000","last_clean":"2024-09-19T04:15:01.384673+0000","last_became_active":"2024-09-19T04:13:31.509363+0000","last_became_peered":"2024-09-19T04:13:31.509363+0000","last_unstale":"2024-09-19T04:15:01.384673+0000","last_undegraded":"2024-09-19T04:15:01.384673+0000","last_fullsized":"2024-09-19T04:15:01.384673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.094930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384995+0000","last_change":"2024-09-19T04:13:16.449459+0000","last_active":"2024-09-19T04:15:01.384995+0000","last_peered":"2024-09-19T04:15:01.384995+0000","last_clean":"2024-09-19T04:15:01.384995+0000","last_became_active":"2024-09-19T04:13:16.449119+0000","last_became_peered":"2024-09-19T04:13:16.449119+0000","last_unstale":"2024-09-19T04:15:01.384995+0000","last_undegraded":"2024-09-19T04:15:01.384995+0000","last_fullsized":"2024-09-19T04:15:01.384995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:26:26.941609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.375543+0000","last_change":"2024-09-19T04:13:16.448939+0000","last_active":"2024-09-19T04:15:01.375543+0000","last_peered":"2024-09-19T04:15:01.375543+0000","last_clean":"2024-09-19T04:15:01.375543+0000","last_became_active":"2024-09-19T04:13:16.448619+0000","last_became_peered":"2024-09-19T04:13:16.448619+0000","last_unstale":"2024-09-19T04:15:01.375543+0000","last_undegraded":"2024-09-19T04:15:01.375543+0000","last_fullsized":"2024-09-19T04:15:01.375543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:41.017404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'9","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.631265+0000","last_change":"2024-09-19T04:13:14.430467+0000","last_active":"2024-09-19T04:15:01.631265+0000","last_peered":"2024-09-19T04:15:01.631265+0000","last_clean":"2024-09-19T04:15:01.631265+0000","last_became_active":"2024-09-19T04:13:14.430095+0000","last_became_peered":"2024-09-19T04:13:14.430095+0000","last_unstale":"2024-09-19T04:15:01.631265+0000","last_undegraded":"2024-09-19T04:15:01.631265+0000","last_fullsized":"2024-09-19T04:15:01.631265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:50.471745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2028,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.385789+0000","last_change":"2024-09-19T04:13:31.505798+0000","last_active":"2024-09-19T04:15:01.385789+0000","last_peered":"2024-09-19T04:15:01.385789+0000","last_clean":"2024-09-19T04:15:01.385789+0000","last_became_active":"2024-09-19T04:13:31.505469+0000","last_became_peered":"2024-09-19T04:13:31.505469+0000","last_unstale":"2024-09-19T04:15:01.385789+0000","last_undegraded":"2024-09-19T04:15:01.385789+0000","last_fullsized":"2024-09-19T04:15:01.385789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:13:42.405075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.348065+0000","last_change":"2024-09-19T04:13:14.426443+0000","last_active":"2024-09-19T04:15:01.348065+0000","last_peered":"2024-09-19T04:15:01.348065+0000","last_clean":"2024-09-19T04:15:01.348065+0000","last_became_active":"2024-09-19T04:13:14.426103+0000","last_became_peered":"2024-09-19T04:13:14.426103+0000","last_unstale":"2024-09-19T04:15:01.348065+0000","last_undegraded":"2024-09-19T04:15:01.348065+0000","last_fullsized":"2024-09-19T04:15:01.348065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:28.461962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.343355+0000","last_change":"2024-09-19T04:13:14.424439+0000","last_active":"2024-09-19T04:15:01.343355+0000","last_peered":"2024-09-19T04:15:01.343355+0000","last_clean":"2024-09-19T04:15:01.343355+0000","last_became_active":"2024-09-19T04:13:14.423975+0000","last_became_peered":"2024-09-19T04:13:14.423975+0000","last_unstale":"2024-09-19T04:15:01.343355+0000","last_undegraded":"2024-09-19T04:15:01.343355+0000","last_fullsized":"2024-09-19T04:15:01.343355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:12.032125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392899+0000","last_change":"2024-09-19T04:13:16.444865+0000","last_active":"2024-09-19T04:15:01.392899+0000","last_peered":"2024-09-19T04:15:01.392899+0000","last_clean":"2024-09-19T04:15:01.392899+0000","last_became_active":"2024-09-19T04:13:16.444556+0000","last_became_peered":"2024-09-19T04:13:16.444556+0000","last_unstale":"2024-09-19T04:15:01.392899+0000","last_undegraded":"2024-09-19T04:15:01.392899+0000","last_fullsized":"2024-09-19T04:15:01.392899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:38.025604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354210+0000","last_change":"2024-09-19T04:13:14.429018+0000","last_active":"2024-09-19T04:15:01.354210+0000","last_peered":"2024-09-19T04:15:01.354210+0000","last_clean":"2024-09-19T04:15:01.354210+0000","last_became_active":"2024-09-19T04:13:14.428383+0000","last_became_peered":"2024-09-19T04:13:14.428383+0000","last_unstale":"2024-09-19T04:15:01.354210+0000","last_undegraded":"2024-09-19T04:15:01.354210+0000","last_fullsized":"2024-09-19T04:15:01.354210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:46.190421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.391102+0000","last_change":"2024-09-19T04:13:16.449013+0000","last_active":"2024-09-19T04:15:01.391102+0000","last_peered":"2024-09-19T04:15:01.391102+0000","last_clean":"2024-09-19T04:15:01.391102+0000","last_became_active":"2024-09-19T04:13:16.448703+0000","last_became_peered":"2024-09-19T04:13:16.448703+0000","last_unstale":"2024-09-19T04:15:01.391102+0000","last_undegraded":"2024-09-19T04:15:01.391102+0000","last_fullsized":"2024-09-19T04:15:01.391102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:41.231759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":124,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.348337+0000","last_change":"2024-09-19T04:13:14.430050+0000","last_active":"2024-09-19T04:15:01.348337+0000","last_peered":"2024-09-19T04:15:01.348337+0000","last_clean":"2024-09-19T04:15:01.348337+0000","last_became_active":"2024-09-19T04:13:14.429710+0000","last_became_peered":"2024-09-19T04:13:14.429710+0000","last_unstale":"2024-09-19T04:15:01.348337+0000","last_undegraded":"2024-09-19T04:15:01.348337+0000","last_fullsized":"2024-09-19T04:15:01.348337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:55.796080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388332+0000","last_change":"2024-09-19T04:13:16.454431+0000","last_active":"2024-09-19T04:15:01.388332+0000","last_peered":"2024-09-19T04:15:01.388332+0000","last_clean":"2024-09-19T04:15:01.388332+0000","last_became_active":"2024-09-19T04:13:16.453033+0000","last_became_peered":"2024-09-19T04:13:16.453033+0000","last_unstale":"2024-09-19T04:15:01.388332+0000","last_undegraded":"2024-09-19T04:15:01.388332+0000","last_fullsized":"2024-09-19T04:15:01.388332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:52.817740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.362264+0000","last_change":"2024-09-19T04:13:14.428949+0000","last_active":"2024-09-19T04:15:01.362264+0000","last_peered":"2024-09-19T04:15:01.362264+0000","last_clean":"2024-09-19T04:15:01.362264+0000","last_became_active":"2024-09-19T04:13:14.428525+0000","last_became_peered":"2024-09-19T04:13:14.428525+0000","last_unstale":"2024-09-19T04:15:01.362264+0000","last_undegraded":"2024-09-19T04:15:01.362264+0000","last_fullsized":"2024-09-19T04:15:01.362264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:30.296730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.381741+0000","last_change":"2024-09-19T04:13:16.454077+0000","last_active":"2024-09-19T04:15:01.381741+0000","last_peered":"2024-09-19T04:15:01.381741+0000","last_clean":"2024-09-19T04:15:01.381741+0000","last_became_active":"2024-09-19T04:13:16.453791+0000","last_became_peered":"2024-09-19T04:13:16.453791+0000","last_unstale":"2024-09-19T04:15:01.381741+0000","last_undegraded":"2024-09-19T04:15:01.381741+0000","last_fullsized":"2024-09-19T04:15:01.381741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:29.426475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.356675+0000","last_change":"2024-09-19T04:13:14.620179+0000","last_active":"2024-09-19T04:15:01.356675+0000","last_peered":"2024-09-19T04:15:01.356675+0000","last_clean":"2024-09-19T04:15:01.356675+0000","last_became_active":"2024-09-19T04:13:14.619758+0000","last_became_peered":"2024-09-19T04:13:14.619758+0000","last_unstale":"2024-09-19T04:15:01.356675+0000","last_undegraded":"2024-09-19T04:15:01.356675+0000","last_fullsized":"2024-09-19T04:15:01.356675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:11.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.404018+0000","last_change":"2024-09-19T04:13:16.453036+0000","last_active":"2024-09-19T04:15:01.404018+0000","last_peered":"2024-09-19T04:15:01.404018+0000","last_clean":"2024-09-19T04:15:01.404018+0000","last_became_active":"2024-09-19T04:13:16.452736+0000","last_became_peered":"2024-09-19T04:13:16.452736+0000","last_unstale":"2024-09-19T04:15:01.404018+0000","last_undegraded":"2024-09-19T04:15:01.404018+0000","last_fullsized":"2024-09-19T04:15:01.404018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:30.003652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":260,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.634781+0000","last_change":"2024-09-19T04:13:14.428715+0000","last_active":"2024-09-19T04:15:01.634781+0000","last_peered":"2024-09-19T04:15:01.634781+0000","last_clean":"2024-09-19T04:15:01.634781+0000","last_became_active":"2024-09-19T04:13:14.428409+0000","last_became_peered":"2024-09-19T04:13:14.428409+0000","last_unstale":"2024-09-19T04:15:01.634781+0000","last_undegraded":"2024-09-19T04:15:01.634781+0000","last_fullsized":"2024-09-19T04:15:01.634781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:23.130857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61278,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":202,"num_read_kb":161,"num_write":35,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407414+0000","last_change":"2024-09-19T04:13:16.454512+0000","last_active":"2024-09-19T04:15:01.407414+0000","last_peered":"2024-09-19T04:15:01.407414+0000","last_clean":"2024-09-19T04:15:01.407414+0000","last_became_active":"2024-09-19T04:13:16.454193+0000","last_became_peered":"2024-09-19T04:13:16.454193+0000","last_unstale":"2024-09-19T04:15:01.407414+0000","last_undegraded":"2024-09-19T04:15:01.407414+0000","last_fullsized":"2024-09-19T04:15:01.407414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:11.825938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.344927+0000","last_change":"2024-09-19T04:13:31.515448+0000","last_active":"2024-09-19T04:15:01.344927+0000","last_peered":"2024-09-19T04:15:01.344927+0000","last_clean":"2024-09-19T04:15:01.344927+0000","last_became_active":"2024-09-19T04:13:31.506099+0000","last_became_peered":"2024-09-19T04:13:31.506099+0000","last_unstale":"2024-09-19T04:15:01.344927+0000","last_undegraded":"2024-09-19T04:15:01.344927+0000","last_fullsized":"2024-09-19T04:15:01.344927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:21.462488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407870+0000","last_change":"2024-09-19T04:13:16.447607+0000","last_active":"2024-09-19T04:15:01.407870+0000","last_peered":"2024-09-19T04:15:01.407870+0000","last_clean":"2024-09-19T04:15:01.407870+0000","last_became_active":"2024-09-19T04:13:16.447029+0000","last_became_peered":"2024-09-19T04:13:16.447029+0000","last_unstale":"2024-09-19T04:15:01.407870+0000","last_undegraded":"2024-09-19T04:15:01.407870+0000","last_fullsized":"2024-09-19T04:15:01.407870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:52.418104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.355933+0000","last_change":"2024-09-19T04:13:14.428856+0000","last_active":"2024-09-19T04:15:01.355933+0000","last_peered":"2024-09-19T04:15:01.355933+0000","last_clean":"2024-09-19T04:15:01.355933+0000","last_became_active":"2024-09-19T04:13:14.428510+0000","last_became_peered":"2024-09-19T04:13:14.428510+0000","last_unstale":"2024-09-19T04:15:01.355933+0000","last_undegraded":"2024-09-19T04:15:01.355933+0000","last_fullsized":"2024-09-19T04:15:01.355933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:49.633325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.409050+0000","last_change":"2024-09-19T04:13:16.455628+0000","last_active":"2024-09-19T04:15:01.409050+0000","last_peered":"2024-09-19T04:15:01.409050+0000","last_clean":"2024-09-19T04:15:01.409050+0000","last_became_active":"2024-09-19T04:13:16.453817+0000","last_became_peered":"2024-09-19T04:13:16.453817+0000","last_unstale":"2024-09-19T04:15:01.409050+0000","last_undegraded":"2024-09-19T04:15:01.409050+0000","last_fullsized":"2024-09-19T04:15:01.409050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.832806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.630728+0000","last_change":"2024-09-19T04:13:14.424924+0000","last_active":"2024-09-19T04:15:01.630728+0000","last_peered":"2024-09-19T04:15:01.630728+0000","last_clean":"2024-09-19T04:15:01.630728+0000","last_became_active":"2024-09-19T04:13:14.424627+0000","last_became_peered":"2024-09-19T04:13:14.424627+0000","last_unstale":"2024-09-19T04:15:01.630728+0000","last_undegraded":"2024-09-19T04:15:01.630728+0000","last_fullsized":"2024-09-19T04:15:01.630728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:49.830205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1994,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":5,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.405035+0000","last_change":"2024-09-19T04:13:16.455365+0000","last_active":"2024-09-19T04:15:01.405035+0000","last_peered":"2024-09-19T04:15:01.405035+0000","last_clean":"2024-09-19T04:15:01.405035+0000","last_became_active":"2024-09-19T04:13:16.455029+0000","last_became_peered":"2024-09-19T04:13:16.455029+0000","last_unstale":"2024-09-19T04:15:01.405035+0000","last_undegraded":"2024-09-19T04:15:01.405035+0000","last_fullsized":"2024-09-19T04:15:01.405035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:20.947391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.353018+0000","last_change":"2024-09-19T04:13:14.620462+0000","last_active":"2024-09-19T04:15:01.353018+0000","last_peered":"2024-09-19T04:15:01.353018+0000","last_clean":"2024-09-19T04:15:01.353018+0000","last_became_active":"2024-09-19T04:13:14.620103+0000","last_became_peered":"2024-09-19T04:13:14.620103+0000","last_unstale":"2024-09-19T04:15:01.353018+0000","last_undegraded":"2024-09-19T04:15:01.353018+0000","last_fullsized":"2024-09-19T04:15:01.353018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:14.496075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.389192+0000","last_change":"2024-09-19T04:13:16.450222+0000","last_active":"2024-09-19T04:15:01.389192+0000","last_peered":"2024-09-19T04:15:01.389192+0000","last_clean":"2024-09-19T04:15:01.389192+0000","last_became_active":"2024-09-19T04:13:16.449916+0000","last_became_peered":"2024-09-19T04:13:16.449916+0000","last_unstale":"2024-09-19T04:15:01.389192+0000","last_undegraded":"2024-09-19T04:15:01.389192+0000","last_fullsized":"2024-09-19T04:15:01.389192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:31.886342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.345119+0000","last_change":"2024-09-19T04:13:31.519190+0000","last_active":"2024-09-19T04:15:01.345119+0000","last_peered":"2024-09-19T04:15:01.345119+0000","last_clean":"2024-09-19T04:15:01.345119+0000","last_became_active":"2024-09-19T04:13:31.508605+0000","last_became_peered":"2024-09-19T04:13:31.508605+0000","last_unstale":"2024-09-19T04:15:01.345119+0000","last_undegraded":"2024-09-19T04:15:01.345119+0000","last_fullsized":"2024-09-19T04:15:01.345119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:51.437135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.386116+0000","last_change":"2024-09-19T04:13:16.450848+0000","last_active":"2024-09-19T04:15:01.386116+0000","last_peered":"2024-09-19T04:15:01.386116+0000","last_clean":"2024-09-19T04:15:01.386116+0000","last_became_active":"2024-09-19T04:13:16.450496+0000","last_became_peered":"2024-09-19T04:13:16.450496+0000","last_unstale":"2024-09-19T04:15:01.386116+0000","last_undegraded":"2024-09-19T04:15:01.386116+0000","last_fullsized":"2024-09-19T04:15:01.386116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:49:00.346998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.343626+0000","last_change":"2024-09-19T04:13:14.428654+0000","last_active":"2024-09-19T04:15:01.343626+0000","last_peered":"2024-09-19T04:15:01.343626+0000","last_clean":"2024-09-19T04:15:01.343626+0000","last_became_active":"2024-09-19T04:13:14.428297+0000","last_became_peered":"2024-09-19T04:13:14.428297+0000","last_unstale":"2024-09-19T04:15:01.343626+0000","last_undegraded":"2024-09-19T04:15:01.343626+0000","last_fullsized":"2024-09-19T04:15:01.343626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:30.834262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407394+0000","last_change":"2024-09-19T04:13:16.456002+0000","last_active":"2024-09-19T04:15:01.407394+0000","last_peered":"2024-09-19T04:15:01.407394+0000","last_clean":"2024-09-19T04:15:01.407394+0000","last_became_active":"2024-09-19T04:13:16.455463+0000","last_became_peered":"2024-09-19T04:13:16.455463+0000","last_unstale":"2024-09-19T04:15:01.407394+0000","last_undegraded":"2024-09-19T04:15:01.407394+0000","last_fullsized":"2024-09-19T04:15:01.407394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:47.266364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'39","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.568584+0000","last_change":"2024-09-19T04:13:14.429555+0000","last_active":"2024-09-19T04:15:01.568584+0000","last_peered":"2024-09-19T04:15:01.568584+0000","last_clean":"2024-09-19T04:15:01.568584+0000","last_became_active":"2024-09-19T04:13:14.429224+0000","last_became_peered":"2024-09-19T04:13:14.429224+0000","last_unstale":"2024-09-19T04:15:01.568584+0000","last_undegraded":"2024-09-19T04:15:01.568584+0000","last_fullsized":"2024-09-19T04:15:01.568584+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:35.304750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407617+0000","last_change":"2024-09-19T04:13:16.454337+0000","last_active":"2024-09-19T04:15:01.407617+0000","last_peered":"2024-09-19T04:15:01.407617+0000","last_clean":"2024-09-19T04:15:01.407617+0000","last_became_active":"2024-09-19T04:13:16.453831+0000","last_became_peered":"2024-09-19T04:13:16.453831+0000","last_unstale":"2024-09-19T04:15:01.407617+0000","last_undegraded":"2024-09-19T04:15:01.407617+0000","last_fullsized":"2024-09-19T04:15:01.407617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:41.751933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.351335+0000","last_change":"2024-09-19T04:13:14.620406+0000","last_active":"2024-09-19T04:15:01.351335+0000","last_peered":"2024-09-19T04:15:01.351335+0000","last_clean":"2024-09-19T04:15:01.351335+0000","last_became_active":"2024-09-19T04:13:14.620050+0000","last_became_peered":"2024-09-19T04:13:14.620050+0000","last_unstale":"2024-09-19T04:15:01.351335+0000","last_undegraded":"2024-09-19T04:15:01.351335+0000","last_fullsized":"2024-09-19T04:15:01.351335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:35.828760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.408215+0000","last_change":"2024-09-19T04:13:16.459016+0000","last_active":"2024-09-19T04:15:01.408215+0000","last_peered":"2024-09-19T04:15:01.408215+0000","last_clean":"2024-09-19T04:15:01.408215+0000","last_became_active":"2024-09-19T04:13:16.458674+0000","last_became_peered":"2024-09-19T04:13:16.458674+0000","last_unstale":"2024-09-19T04:15:01.408215+0000","last_undegraded":"2024-09-19T04:15:01.408215+0000","last_fullsized":"2024-09-19T04:15:01.408215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:55.894456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.361251+0000","last_change":"2024-09-19T04:12:05.913704+0000","last_active":"2024-09-19T04:15:01.361251+0000","last_peered":"2024-09-19T04:15:01.361251+0000","last_clean":"2024-09-19T04:15:01.361251+0000","last_became_active":"2024-09-19T04:12:05.913299+0000","last_became_peered":"2024-09-19T04:12:05.913299+0000","last_unstale":"2024-09-19T04:15:01.361251+0000","last_undegraded":"2024-09-19T04:15:01.361251+0000","last_fullsized":"2024-09-19T04:15:01.361251+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_clean_scrub_stamp":"2024-09-19T04:08:46.646000+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.638160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.357722+0000","last_change":"2024-09-19T04:13:14.423962+0000","last_active":"2024-09-19T04:15:01.357722+0000","last_peered":"2024-09-19T04:15:01.357722+0000","last_clean":"2024-09-19T04:15:01.357722+0000","last_became_active":"2024-09-19T04:13:14.423591+0000","last_became_peered":"2024-09-19T04:13:14.423591+0000","last_unstale":"2024-09-19T04:15:01.357722+0000","last_undegraded":"2024-09-19T04:15:01.357722+0000","last_fullsized":"2024-09-19T04:15:01.357722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:02.239975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.394412+0000","last_change":"2024-09-19T04:13:16.447113+0000","last_active":"2024-09-19T04:15:01.394412+0000","last_peered":"2024-09-19T04:15:01.394412+0000","last_clean":"2024-09-19T04:15:01.394412+0000","last_became_active":"2024-09-19T04:13:16.446792+0000","last_became_peered":"2024-09-19T04:13:16.446792+0000","last_unstale":"2024-09-19T04:15:01.394412+0000","last_undegraded":"2024-09-19T04:15:01.394412+0000","last_fullsized":"2024-09-19T04:15:01.394412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:28:21.527010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.361188+0000","last_change":"2024-09-19T04:13:14.432663+0000","last_active":"2024-09-19T04:15:01.361188+0000","last_peered":"2024-09-19T04:15:01.361188+0000","last_clean":"2024-09-19T04:15:01.361188+0000","last_became_active":"2024-09-19T04:13:14.432318+0000","last_became_peered":"2024-09-19T04:13:14.432318+0000","last_unstale":"2024-09-19T04:15:01.361188+0000","last_undegraded":"2024-09-19T04:15:01.361188+0000","last_fullsized":"2024-09-19T04:15:01.361188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:35.787596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402670+0000","last_change":"2024-09-19T04:13:16.451010+0000","last_active":"2024-09-19T04:15:01.402670+0000","last_peered":"2024-09-19T04:15:01.402670+0000","last_clean":"2024-09-19T04:15:01.402670+0000","last_became_active":"2024-09-19T04:13:16.450708+0000","last_became_peered":"2024-09-19T04:13:16.450708+0000","last_unstale":"2024-09-19T04:15:01.402670+0000","last_undegraded":"2024-09-19T04:15:01.402670+0000","last_fullsized":"2024-09-19T04:15:01.402670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:30.108541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.347518+0000","last_change":"2024-09-19T04:13:14.620037+0000","last_active":"2024-09-19T04:15:01.347518+0000","last_peered":"2024-09-19T04:15:01.347518+0000","last_clean":"2024-09-19T04:15:01.347518+0000","last_became_active":"2024-09-19T04:13:14.619686+0000","last_became_peered":"2024-09-19T04:13:14.619686+0000","last_unstale":"2024-09-19T04:15:01.347518+0000","last_undegraded":"2024-09-19T04:15:01.347518+0000","last_fullsized":"2024-09-19T04:15:01.347518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:32.490708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387959+0000","last_change":"2024-09-19T04:13:16.446429+0000","last_active":"2024-09-19T04:15:01.387959+0000","last_peered":"2024-09-19T04:15:01.387959+0000","last_clean":"2024-09-19T04:15:01.387959+0000","last_became_active":"2024-09-19T04:13:16.446075+0000","last_became_peered":"2024-09-19T04:13:16.446075+0000","last_unstale":"2024-09-19T04:15:01.387959+0000","last_undegraded":"2024-09-19T04:15:01.387959+0000","last_fullsized":"2024-09-19T04:15:01.387959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:03.613926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.361912+0000","last_change":"2024-09-19T04:13:14.429899+0000","last_active":"2024-09-19T04:15:01.361912+0000","last_peered":"2024-09-19T04:15:01.361912+0000","last_clean":"2024-09-19T04:15:01.361912+0000","last_became_active":"2024-09-19T04:13:14.429571+0000","last_became_peered":"2024-09-19T04:13:14.429571+0000","last_unstale":"2024-09-19T04:15:01.361912+0000","last_undegraded":"2024-09-19T04:15:01.361912+0000","last_fullsized":"2024-09-19T04:15:01.361912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:37.017431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384017+0000","last_change":"2024-09-19T04:13:16.458267+0000","last_active":"2024-09-19T04:15:01.384017+0000","last_peered":"2024-09-19T04:15:01.384017+0000","last_clean":"2024-09-19T04:15:01.384017+0000","last_became_active":"2024-09-19T04:13:16.457911+0000","last_became_peered":"2024-09-19T04:13:16.457911+0000","last_unstale":"2024-09-19T04:15:01.384017+0000","last_undegraded":"2024-09-19T04:15:01.384017+0000","last_fullsized":"2024-09-19T04:15:01.384017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:32.465923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354719+0000","last_change":"2024-09-19T04:13:14.621552+0000","last_active":"2024-09-19T04:15:01.354719+0000","last_peered":"2024-09-19T04:15:01.354719+0000","last_clean":"2024-09-19T04:15:01.354719+0000","last_became_active":"2024-09-19T04:13:14.620592+0000","last_became_peered":"2024-09-19T04:13:14.620592+0000","last_unstale":"2024-09-19T04:15:01.354719+0000","last_undegraded":"2024-09-19T04:15:01.354719+0000","last_fullsized":"2024-09-19T04:15:01.354719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:10.055202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402127+0000","last_change":"2024-09-19T04:13:16.443117+0000","last_active":"2024-09-19T04:15:01.402127+0000","last_peered":"2024-09-19T04:15:01.402127+0000","last_clean":"2024-09-19T04:15:01.402127+0000","last_became_active":"2024-09-19T04:13:16.442808+0000","last_became_peered":"2024-09-19T04:13:16.442808+0000","last_unstale":"2024-09-19T04:15:01.402127+0000","last_undegraded":"2024-09-19T04:15:01.402127+0000","last_fullsized":"2024-09-19T04:15:01.402127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:40.274320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354893+0000","last_change":"2024-09-19T04:13:14.620618+0000","last_active":"2024-09-19T04:15:01.354893+0000","last_peered":"2024-09-19T04:15:01.354893+0000","last_clean":"2024-09-19T04:15:01.354893+0000","last_became_active":"2024-09-19T04:13:14.620290+0000","last_became_peered":"2024-09-19T04:13:14.620290+0000","last_unstale":"2024-09-19T04:15:01.354893+0000","last_undegraded":"2024-09-19T04:15:01.354893+0000","last_fullsized":"2024-09-19T04:15:01.354893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:51.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.390139+0000","last_change":"2024-09-19T04:13:16.443498+0000","last_active":"2024-09-19T04:15:01.390139+0000","last_peered":"2024-09-19T04:15:01.390139+0000","last_clean":"2024-09-19T04:15:01.390139+0000","last_became_active":"2024-09-19T04:13:16.443104+0000","last_became_peered":"2024-09-19T04:13:16.443104+0000","last_unstale":"2024-09-19T04:15:01.390139+0000","last_undegraded":"2024-09-19T04:15:01.390139+0000","last_fullsized":"2024-09-19T04:15:01.390139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:10.733570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'45","reported_seq":92,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.571386+0000","last_change":"2024-09-19T04:13:14.425881+0000","last_active":"2024-09-19T04:15:01.571386+0000","last_peered":"2024-09-19T04:15:01.571386+0000","last_clean":"2024-09-19T04:15:01.571386+0000","last_became_active":"2024-09-19T04:13:14.425542+0000","last_became_peered":"2024-09-19T04:13:14.425542+0000","last_unstale":"2024-09-19T04:15:01.571386+0000","last_undegraded":"2024-09-19T04:15:01.571386+0000","last_fullsized":"2024-09-19T04:15:01.571386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:44.250890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392711+0000","last_change":"2024-09-19T04:13:16.454672+0000","last_active":"2024-09-19T04:15:01.392711+0000","last_peered":"2024-09-19T04:15:01.392711+0000","last_clean":"2024-09-19T04:15:01.392711+0000","last_became_active":"2024-09-19T04:13:16.454003+0000","last_became_peered":"2024-09-19T04:13:16.454003+0000","last_unstale":"2024-09-19T04:15:01.392711+0000","last_undegraded":"2024-09-19T04:15:01.392711+0000","last_fullsized":"2024-09-19T04:15:01.392711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:26.849617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.357930+0000","last_change":"2024-09-19T04:13:14.432312+0000","last_active":"2024-09-19T04:15:01.357930+0000","last_peered":"2024-09-19T04:15:01.357930+0000","last_clean":"2024-09-19T04:15:01.357930+0000","last_became_active":"2024-09-19T04:13:14.431994+0000","last_became_peered":"2024-09-19T04:13:14.431994+0000","last_unstale":"2024-09-19T04:15:01.357930+0000","last_undegraded":"2024-09-19T04:15:01.357930+0000","last_fullsized":"2024-09-19T04:15:01.357930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:00.410400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.391357+0000","last_change":"2024-09-19T04:13:16.455312+0000","last_active":"2024-09-19T04:15:01.391357+0000","last_peered":"2024-09-19T04:15:01.391357+0000","last_clean":"2024-09-19T04:15:01.391357+0000","last_became_active":"2024-09-19T04:13:16.454990+0000","last_became_peered":"2024-09-19T04:13:16.454990+0000","last_unstale":"2024-09-19T04:15:01.391357+0000","last_undegraded":"2024-09-19T04:15:01.391357+0000","last_fullsized":"2024-09-19T04:15:01.391357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:09.207395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.349902+0000","last_change":"2024-09-19T04:13:14.620316+0000","last_active":"2024-09-19T04:15:01.349902+0000","last_peered":"2024-09-19T04:15:01.349902+0000","last_clean":"2024-09-19T04:15:01.349902+0000","last_became_active":"2024-09-19T04:13:14.619979+0000","last_became_peered":"2024-09-19T04:13:14.619979+0000","last_unstale":"2024-09-19T04:15:01.349902+0000","last_undegraded":"2024-09-19T04:15:01.349902+0000","last_fullsized":"2024-09-19T04:15:01.349902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:24.382144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.379294+0000","last_change":"2024-09-19T04:13:16.455684+0000","last_active":"2024-09-19T04:15:01.379294+0000","last_peered":"2024-09-19T04:15:01.379294+0000","last_clean":"2024-09-19T04:15:01.379294+0000","last_became_active":"2024-09-19T04:13:16.455355+0000","last_became_peered":"2024-09-19T04:13:16.455355+0000","last_unstale":"2024-09-19T04:15:01.379294+0000","last_undegraded":"2024-09-19T04:15:01.379294+0000","last_fullsized":"2024-09-19T04:15:01.379294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:09.657447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.346821+0000","last_change":"2024-09-19T04:13:14.423599+0000","last_active":"2024-09-19T04:15:01.346821+0000","last_peered":"2024-09-19T04:15:01.346821+0000","last_clean":"2024-09-19T04:15:01.346821+0000","last_became_active":"2024-09-19T04:13:14.423282+0000","last_became_peered":"2024-09-19T04:13:14.423282+0000","last_unstale":"2024-09-19T04:15:01.346821+0000","last_undegraded":"2024-09-19T04:15:01.346821+0000","last_fullsized":"2024-09-19T04:15:01.346821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.975368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.376076+0000","last_change":"2024-09-19T04:13:16.450435+0000","last_active":"2024-09-19T04:15:01.376076+0000","last_peered":"2024-09-19T04:15:01.376076+0000","last_clean":"2024-09-19T04:15:01.376076+0000","last_became_active":"2024-09-19T04:13:16.450087+0000","last_became_peered":"2024-09-19T04:13:16.450087+0000","last_unstale":"2024-09-19T04:15:01.376076+0000","last_undegraded":"2024-09-19T04:15:01.376076+0000","last_fullsized":"2024-09-19T04:15:01.376076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:34:51.764776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.351312+0000","last_change":"2024-09-19T04:13:14.432868+0000","last_active":"2024-09-19T04:15:01.351312+0000","last_peered":"2024-09-19T04:15:01.351312+0000","last_clean":"2024-09-19T04:15:01.351312+0000","last_became_active":"2024-09-19T04:13:14.432507+0000","last_became_peered":"2024-09-19T04:13:14.432507+0000","last_unstale":"2024-09-19T04:15:01.351312+0000","last_undegraded":"2024-09-19T04:15:01.351312+0000","last_fullsized":"2024-09-19T04:15:01.351312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:14.773711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.390493+0000","last_change":"2024-09-19T04:13:16.450061+0000","last_active":"2024-09-19T04:15:01.390493+0000","last_peered":"2024-09-19T04:15:01.390493+0000","last_clean":"2024-09-19T04:15:01.390493+0000","last_became_active":"2024-09-19T04:13:16.448829+0000","last_became_peered":"2024-09-19T04:13:16.448829+0000","last_unstale":"2024-09-19T04:15:01.390493+0000","last_undegraded":"2024-09-19T04:15:01.390493+0000","last_fullsized":"2024-09-19T04:15:01.390493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:31.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354057+0000","last_change":"2024-09-19T04:13:14.431574+0000","last_active":"2024-09-19T04:15:01.354057+0000","last_peered":"2024-09-19T04:15:01.354057+0000","last_clean":"2024-09-19T04:15:01.354057+0000","last_became_active":"2024-09-19T04:13:14.431202+0000","last_became_peered":"2024-09-19T04:13:14.431202+0000","last_unstale":"2024-09-19T04:15:01.354057+0000","last_undegraded":"2024-09-19T04:15:01.354057+0000","last_fullsized":"2024-09-19T04:15:01.354057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:58.167201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.408741+0000","last_change":"2024-09-19T04:13:16.452688+0000","last_active":"2024-09-19T04:15:01.408741+0000","last_peered":"2024-09-19T04:15:01.408741+0000","last_clean":"2024-09-19T04:15:01.408741+0000","last_became_active":"2024-09-19T04:13:16.452342+0000","last_became_peered":"2024-09-19T04:13:16.452342+0000","last_unstale":"2024-09-19T04:15:01.408741+0000","last_undegraded":"2024-09-19T04:15:01.408741+0000","last_fullsized":"2024-09-19T04:15:01.408741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:03.162688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.403276+0000","last_change":"2024-09-19T04:13:16.449402+0000","last_active":"2024-09-19T04:15:01.403276+0000","last_peered":"2024-09-19T04:15:01.403276+0000","last_clean":"2024-09-19T04:15:01.403276+0000","last_became_active":"2024-09-19T04:13:16.449108+0000","last_became_peered":"2024-09-19T04:13:16.449108+0000","last_unstale":"2024-09-19T04:15:01.403276+0000","last_undegraded":"2024-09-19T04:15:01.403276+0000","last_fullsized":"2024-09-19T04:15:01.403276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:58.874638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354429+0000","last_change":"2024-09-19T04:13:14.430684+0000","last_active":"2024-09-19T04:15:01.354429+0000","last_peered":"2024-09-19T04:15:01.354429+0000","last_clean":"2024-09-19T04:15:01.354429+0000","last_became_active":"2024-09-19T04:13:14.430137+0000","last_became_peered":"2024-09-19T04:13:14.430137+0000","last_unstale":"2024-09-19T04:15:01.354429+0000","last_undegraded":"2024-09-19T04:15:01.354429+0000","last_fullsized":"2024-09-19T04:15:01.354429+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:39.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.403606+0000","last_change":"2024-09-19T04:13:16.452491+0000","last_active":"2024-09-19T04:15:01.403606+0000","last_peered":"2024-09-19T04:15:01.403606+0000","last_clean":"2024-09-19T04:15:01.403606+0000","last_became_active":"2024-09-19T04:13:16.452204+0000","last_became_peered":"2024-09-19T04:13:16.452204+0000","last_unstale":"2024-09-19T04:15:01.403606+0000","last_undegraded":"2024-09-19T04:15:01.403606+0000","last_fullsized":"2024-09-19T04:15:01.403606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:10.113510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354761+0000","last_change":"2024-09-19T04:13:14.431608+0000","last_active":"2024-09-19T04:15:01.354761+0000","last_peered":"2024-09-19T04:15:01.354761+0000","last_clean":"2024-09-19T04:15:01.354761+0000","last_became_active":"2024-09-19T04:13:14.431111+0000","last_became_peered":"2024-09-19T04:13:14.431111+0000","last_unstale":"2024-09-19T04:15:01.354761+0000","last_undegraded":"2024-09-19T04:15:01.354761+0000","last_fullsized":"2024-09-19T04:15:01.354761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:03:41.125315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.393098+0000","last_change":"2024-09-19T04:13:16.457867+0000","last_active":"2024-09-19T04:15:01.393098+0000","last_peered":"2024-09-19T04:15:01.393098+0000","last_clean":"2024-09-19T04:15:01.393098+0000","last_became_active":"2024-09-19T04:13:16.457582+0000","last_became_peered":"2024-09-19T04:13:16.457582+0000","last_unstale":"2024-09-19T04:15:01.393098+0000","last_undegraded":"2024-09-19T04:15:01.393098+0000","last_fullsized":"2024-09-19T04:15:01.393098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:58.076250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.346539+0000","last_change":"2024-09-19T04:13:14.620070+0000","last_active":"2024-09-19T04:15:01.346539+0000","last_peered":"2024-09-19T04:15:01.346539+0000","last_clean":"2024-09-19T04:15:01.346539+0000","last_became_active":"2024-09-19T04:13:14.619734+0000","last_became_peered":"2024-09-19T04:13:14.619734+0000","last_unstale":"2024-09-19T04:15:01.346539+0000","last_undegraded":"2024-09-19T04:15:01.346539+0000","last_fullsized":"2024-09-19T04:15:01.346539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:44:02.818859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388185+0000","last_change":"2024-09-19T04:13:16.443832+0000","last_active":"2024-09-19T04:15:01.388185+0000","last_peered":"2024-09-19T04:15:01.388185+0000","last_clean":"2024-09-19T04:15:01.388185+0000","last_became_active":"2024-09-19T04:13:16.442690+0000","last_became_peered":"2024-09-19T04:13:16.442690+0000","last_unstale":"2024-09-19T04:15:01.388185+0000","last_undegraded":"2024-09-19T04:15:01.388185+0000","last_fullsized":"2024-09-19T04:15:01.388185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:39.535628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.355196+0000","last_change":"2024-09-19T04:13:14.427186+0000","last_active":"2024-09-19T04:15:01.355196+0000","last_peered":"2024-09-19T04:15:01.355196+0000","last_clean":"2024-09-19T04:15:01.355196+0000","last_became_active":"2024-09-19T04:13:14.426949+0000","last_became_peered":"2024-09-19T04:13:14.426949+0000","last_unstale":"2024-09-19T04:15:01.355196+0000","last_undegraded":"2024-09-19T04:15:01.355196+0000","last_fullsized":"2024-09-19T04:15:01.355196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:26.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":592516,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":700,"num_read_kb":454,"num_write":289,"num_write_kb":1431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2002944,"data_stored":1777548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612642,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":440451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841575,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33000,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712152,"statfs":{"total":95995035648,"available":95961243648,"internally_reserved":0,"allocated":1736704,"data_stored":1226160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005100,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40792,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704360,"statfs":{"total":95995035648,"available":95953264640,"internally_reserved":0,"allocated":1314816,"data_stored":833847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28304,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716848,"statfs":{"total":95995035648,"available":95966052352,"internally_reserved":0,"allocated":1310720,"data_stored":833745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1515520,"data_stored":1034902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":909312,"data_stored":440020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37028,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708124,"statfs":{"total":95995035648,"available":95957118976,"internally_reserved":0,"allocated":1662976,"data_stored":1164780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":441470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1454080,"data_stored":965147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476815,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":503813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607636,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":500305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":929792,"data_stored":441291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":195862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:15:07.100 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T04:15:07.100 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:15:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 -- 172.21.15.94:0/2547252117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc74100990 msgr2=0x7fcc74100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 --2- 172.21.15.94:0/2547252117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc74100990 0x7fcc74100df0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009930 tx=0x7fcc6402a420 comp rx=0 tx=0).stop 2024-09-19T04:15:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 -- 172.21.15.94:0/2547252117 shutdown_connections 2024-09-19T04:15:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 --2- 172.21.15.94:0/2547252117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc74101330 0x7fcc7410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 --2- 172.21.15.94:0/2547252117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc74100990 0x7fcc74100df0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 --2- 172.21.15.94:0/2547252117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc741069a0 0x7fcc74106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 -- 172.21.15.94:0/2547252117 >> 172.21.15.94:0/2547252117 conn(0x7fcc740fc6d0 msgr2=0x7fcc740feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:07.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 -- 172.21.15.94:0/2547252117 shutdown_connections 2024-09-19T04:15:07.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 -- 172.21.15.94:0/2547252117 wait complete. 2024-09-19T04:15:07.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.233+0000 7fcc791d8640 1 Processor -- start 2024-09-19T04:15:07.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc791d8640 1 -- start start 2024-09-19T04:15:07.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc791d8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc74100990 0x7fcc741991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:07.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc791d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc74101330 0x7fcc74199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:07.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc791d8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc741069a0 0x7fcc741a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:07.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc791d8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74110c90 con 0x7fcc741069a0 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc791d8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74110b10 con 0x7fcc74100990 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc791d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74110e10 con 0x7fcc74101330 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc741069a0 0x7fcc741a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc741069a0 0x7fcc741a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46614/0 (socket says 172.21.15.94:46614) 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7359e640 1 -- 172.21.15.94:0/94776072 learned_addr learned my addr 172.21.15.94:0/94776072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7259c640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc74101330 0x7fcc74199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc72d9d640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc74100990 0x7fcc741991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7359e640 1 -- 172.21.15.94:0/94776072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc74101330 msgr2=0x7fcc74199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7359e640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc74101330 0x7fcc74199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7359e640 1 -- 172.21.15.94:0/94776072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc74100990 msgr2=0x7fcc741991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7359e640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc74100990 0x7fcc741991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7359e640 1 -- 172.21.15.94:0/94776072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc741a47a0 con 0x7fcc741069a0 2024-09-19T04:15:07.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7259c640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc74101330 0x7fcc74199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.234+0000 7fcc7359e640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc741069a0 0x7fcc741a4260 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00eaa0 tx=0x7fcc6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.235+0000 7fcc6bfff640 1 -- 172.21.15.94:0/94776072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00cdb0 con 0x7fcc741069a0 2024-09-19T04:15:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.235+0000 7fcc6bfff640 1 -- 172.21.15.94:0/94776072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc6c005020 con 0x7fcc741069a0 2024-09-19T04:15:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.235+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc741a4970 con 0x7fcc741069a0 2024-09-19T04:15:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.235+0000 7fcc6bfff640 1 -- 172.21.15.94:0/94776072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0116d0 con 0x7fcc741069a0 2024-09-19T04:15:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.235+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc741a4e90 con 0x7fcc741069a0 2024-09-19T04:15:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.236+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc74100df0 con 0x7fcc741069a0 2024-09-19T04:15:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.237+0000 7fcc6bfff640 1 -- 172.21.15.94:0/94776072 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7fcc6c0051d0 con 0x7fcc741069a0 2024-09-19T04:15:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.237+0000 7fcc6bfff640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc600a3450 0x7fcc600a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:07.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.237+0000 7fcc72d9d640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc600a3450 0x7fcc600a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:07.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.237+0000 7fcc6bfff640 1 -- 172.21.15.94:0/94776072 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fcc6c0d0110 con 0x7fcc741069a0 2024-09-19T04:15:07.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.238+0000 7fcc72d9d640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc600a3450 0x7fcc600a5910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c005fd0 tx=0x7fcc5c005850 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:07.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.242+0000 7fcc6bfff640 1 -- 172.21.15.94:0/94776072 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc6c016030 con 0x7fcc741069a0 2024-09-19T04:15:07.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.407+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 --> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc74062630 con 0x7fcc600a3450 2024-09-19T04:15:07.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.413+0000 7fcc6bfff640 1 -- 172.21.15.94:0/94776072 <== mgr.15159 v2:172.21.15.94:6800/3395191155 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257739 (secure 0 0 0) 0x7fcc74062630 con 0x7fcc600a3450 2024-09-19T04:15:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:07.418 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-19T04:15:07.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc600a3450 msgr2=0x7fcc600a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:07.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc600a3450 0x7fcc600a5910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c005fd0 tx=0x7fcc5c005850 comp rx=0 tx=0).stop 2024-09-19T04:15:07.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc741069a0 msgr2=0x7fcc741a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:07.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc741069a0 0x7fcc741a4260 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00eaa0 tx=0x7fcc6c00ef60 comp rx=0 tx=0).stop 2024-09-19T04:15:07.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 shutdown_connections 2024-09-19T04:15:07.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc600a3450 0x7fcc600a5910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc741069a0 0x7fcc741a4260 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc74101330 0x7fcc74199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 --2- 172.21.15.94:0/94776072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc74100990 0x7fcc741991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:07.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 >> 172.21.15.94:0/94776072 conn(0x7fcc740fc6d0 msgr2=0x7fcc740fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:07.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 shutdown_connections 2024-09-19T04:15:07.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:07.416+0000 7fcc791d8640 1 -- 172.21.15.94:0/94776072 wait complete. 2024-09-19T04:15:07.429 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-09-19T04:15:07.384389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187432,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":967,"num_read_kb":799,"num_write":497,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":879,"ondisk_log_size":879,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":401296,"kb_used_data":14280,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540528,"statfs":{"total":1151940427776,"available":1151529500672,"internally_reserved":0,"allocated":14622720,"data_stored":8827110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4314,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":226,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.017815"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388917+0000","last_change":"2024-09-19T04:13:16.451031+0000","last_active":"2024-09-19T04:15:01.388917+0000","last_peered":"2024-09-19T04:15:01.388917+0000","last_clean":"2024-09-19T04:15:01.388917+0000","last_became_active":"2024-09-19T04:13:16.450713+0000","last_became_peered":"2024-09-19T04:13:16.450713+0000","last_unstale":"2024-09-19T04:15:01.388917+0000","last_undegraded":"2024-09-19T04:15:01.388917+0000","last_fullsized":"2024-09-19T04:15:01.388917+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:59.485415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.381120+0000","last_change":"2024-09-19T04:13:16.455689+0000","last_active":"2024-09-19T04:15:01.381120+0000","last_peered":"2024-09-19T04:15:01.381120+0000","last_clean":"2024-09-19T04:15:01.381120+0000","last_became_active":"2024-09-19T04:13:16.455154+0000","last_became_peered":"2024-09-19T04:13:16.455154+0000","last_unstale":"2024-09-19T04:15:01.381120+0000","last_undegraded":"2024-09-19T04:15:01.381120+0000","last_fullsized":"2024-09-19T04:15:01.381120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:57.610619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.381282+0000","last_change":"2024-09-19T04:13:16.447392+0000","last_active":"2024-09-19T04:15:01.381282+0000","last_peered":"2024-09-19T04:15:01.381282+0000","last_clean":"2024-09-19T04:15:01.381282+0000","last_became_active":"2024-09-19T04:13:16.447088+0000","last_became_peered":"2024-09-19T04:13:16.447088+0000","last_unstale":"2024-09-19T04:15:01.381282+0000","last_undegraded":"2024-09-19T04:15:01.381282+0000","last_fullsized":"2024-09-19T04:15:01.381282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:16.591420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387155+0000","last_change":"2024-09-19T04:13:16.452642+0000","last_active":"2024-09-19T04:15:01.387155+0000","last_peered":"2024-09-19T04:15:01.387155+0000","last_clean":"2024-09-19T04:15:01.387155+0000","last_became_active":"2024-09-19T04:13:16.452329+0000","last_became_peered":"2024-09-19T04:13:16.452329+0000","last_unstale":"2024-09-19T04:15:01.387155+0000","last_undegraded":"2024-09-19T04:15:01.387155+0000","last_fullsized":"2024-09-19T04:15:01.387155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:54.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384913+0000","last_change":"2024-09-19T04:13:16.447171+0000","last_active":"2024-09-19T04:15:01.384913+0000","last_peered":"2024-09-19T04:15:01.384913+0000","last_clean":"2024-09-19T04:15:01.384913+0000","last_became_active":"2024-09-19T04:13:16.446835+0000","last_became_peered":"2024-09-19T04:13:16.446835+0000","last_unstale":"2024-09-19T04:15:01.384913+0000","last_undegraded":"2024-09-19T04:15:01.384913+0000","last_fullsized":"2024-09-19T04:15:01.384913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:41.892118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.377175+0000","last_change":"2024-09-19T04:13:31.507665+0000","last_active":"2024-09-19T04:15:01.377175+0000","last_peered":"2024-09-19T04:15:01.377175+0000","last_clean":"2024-09-19T04:15:01.377175+0000","last_became_active":"2024-09-19T04:13:31.507329+0000","last_became_peered":"2024-09-19T04:13:31.507329+0000","last_unstale":"2024-09-19T04:15:01.377175+0000","last_undegraded":"2024-09-19T04:15:01.377175+0000","last_fullsized":"2024-09-19T04:15:01.377175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:35.832776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388719+0000","last_change":"2024-09-19T04:13:16.454568+0000","last_active":"2024-09-19T04:15:01.388719+0000","last_peered":"2024-09-19T04:15:01.388719+0000","last_clean":"2024-09-19T04:15:01.388719+0000","last_became_active":"2024-09-19T04:13:16.454230+0000","last_became_peered":"2024-09-19T04:13:16.454230+0000","last_unstale":"2024-09-19T04:15:01.388719+0000","last_undegraded":"2024-09-19T04:15:01.388719+0000","last_fullsized":"2024-09-19T04:15:01.388719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:23.401316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402188+0000","last_change":"2024-09-19T04:13:16.449223+0000","last_active":"2024-09-19T04:15:01.402188+0000","last_peered":"2024-09-19T04:15:01.402188+0000","last_clean":"2024-09-19T04:15:01.402188+0000","last_became_active":"2024-09-19T04:13:16.448665+0000","last_became_peered":"2024-09-19T04:13:16.448665+0000","last_unstale":"2024-09-19T04:15:01.402188+0000","last_undegraded":"2024-09-19T04:15:01.402188+0000","last_fullsized":"2024-09-19T04:15:01.402188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:29.619847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402682+0000","last_change":"2024-09-19T04:13:16.458203+0000","last_active":"2024-09-19T04:15:01.402682+0000","last_peered":"2024-09-19T04:15:01.402682+0000","last_clean":"2024-09-19T04:15:01.402682+0000","last_became_active":"2024-09-19T04:13:16.457544+0000","last_became_peered":"2024-09-19T04:13:16.457544+0000","last_unstale":"2024-09-19T04:15:01.402682+0000","last_undegraded":"2024-09-19T04:15:01.402682+0000","last_fullsized":"2024-09-19T04:15:01.402682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:47.611695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.385340+0000","last_change":"2024-09-19T04:13:16.454213+0000","last_active":"2024-09-19T04:15:01.385340+0000","last_peered":"2024-09-19T04:15:01.385340+0000","last_clean":"2024-09-19T04:15:01.385340+0000","last_became_active":"2024-09-19T04:13:16.453047+0000","last_became_peered":"2024-09-19T04:13:16.453047+0000","last_unstale":"2024-09-19T04:15:01.385340+0000","last_undegraded":"2024-09-19T04:15:01.385340+0000","last_fullsized":"2024-09-19T04:15:01.385340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:27.425259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407089+0000","last_change":"2024-09-19T04:13:16.458195+0000","last_active":"2024-09-19T04:15:01.407089+0000","last_peered":"2024-09-19T04:15:01.407089+0000","last_clean":"2024-09-19T04:15:01.407089+0000","last_became_active":"2024-09-19T04:13:16.457891+0000","last_became_peered":"2024-09-19T04:13:16.457891+0000","last_unstale":"2024-09-19T04:15:01.407089+0000","last_undegraded":"2024-09-19T04:15:01.407089+0000","last_fullsized":"2024-09-19T04:15:01.407089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:53.833249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.379554+0000","last_change":"2024-09-19T04:13:31.508117+0000","last_active":"2024-09-19T04:15:01.379554+0000","last_peered":"2024-09-19T04:15:01.379554+0000","last_clean":"2024-09-19T04:15:01.379554+0000","last_became_active":"2024-09-19T04:13:31.507693+0000","last_became_peered":"2024-09-19T04:13:31.507693+0000","last_unstale":"2024-09-19T04:15:01.379554+0000","last_undegraded":"2024-09-19T04:15:01.379554+0000","last_fullsized":"2024-09-19T04:15:01.379554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:58.447965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392023+0000","last_change":"2024-09-19T04:13:16.440079+0000","last_active":"2024-09-19T04:15:01.392023+0000","last_peered":"2024-09-19T04:15:01.392023+0000","last_clean":"2024-09-19T04:15:01.392023+0000","last_became_active":"2024-09-19T04:13:16.439474+0000","last_became_peered":"2024-09-19T04:13:16.439474+0000","last_unstale":"2024-09-19T04:15:01.392023+0000","last_undegraded":"2024-09-19T04:15:01.392023+0000","last_fullsized":"2024-09-19T04:15:01.392023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:11.151412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384315+0000","last_change":"2024-09-19T04:13:16.449958+0000","last_active":"2024-09-19T04:15:01.384315+0000","last_peered":"2024-09-19T04:15:01.384315+0000","last_clean":"2024-09-19T04:15:01.384315+0000","last_became_active":"2024-09-19T04:13:16.449665+0000","last_became_peered":"2024-09-19T04:13:16.449665+0000","last_unstale":"2024-09-19T04:15:01.384315+0000","last_undegraded":"2024-09-19T04:15:01.384315+0000","last_fullsized":"2024-09-19T04:15:01.384315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:24.023779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.383771+0000","last_change":"2024-09-19T04:13:16.454333+0000","last_active":"2024-09-19T04:15:01.383771+0000","last_peered":"2024-09-19T04:15:01.383771+0000","last_clean":"2024-09-19T04:15:01.383771+0000","last_became_active":"2024-09-19T04:13:16.454066+0000","last_became_peered":"2024-09-19T04:13:16.454066+0000","last_unstale":"2024-09-19T04:15:01.383771+0000","last_undegraded":"2024-09-19T04:15:01.383771+0000","last_fullsized":"2024-09-19T04:15:01.383771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:16.815173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.386728+0000","last_change":"2024-09-19T04:13:16.451864+0000","last_active":"2024-09-19T04:15:01.386728+0000","last_peered":"2024-09-19T04:15:01.386728+0000","last_clean":"2024-09-19T04:15:01.386728+0000","last_became_active":"2024-09-19T04:13:16.450995+0000","last_became_peered":"2024-09-19T04:13:16.450995+0000","last_unstale":"2024-09-19T04:15:01.386728+0000","last_undegraded":"2024-09-19T04:15:01.386728+0000","last_fullsized":"2024-09-19T04:15:01.386728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:40.750221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.389034+0000","last_change":"2024-09-19T04:13:16.453176+0000","last_active":"2024-09-19T04:15:01.389034+0000","last_peered":"2024-09-19T04:15:01.389034+0000","last_clean":"2024-09-19T04:15:01.389034+0000","last_became_active":"2024-09-19T04:13:16.452788+0000","last_became_peered":"2024-09-19T04:13:16.452788+0000","last_unstale":"2024-09-19T04:15:01.389034+0000","last_undegraded":"2024-09-19T04:15:01.389034+0000","last_fullsized":"2024-09-19T04:15:01.389034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:47.099180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.377948+0000","last_change":"2024-09-19T04:13:16.449167+0000","last_active":"2024-09-19T04:15:01.377948+0000","last_peered":"2024-09-19T04:15:01.377948+0000","last_clean":"2024-09-19T04:15:01.377948+0000","last_became_active":"2024-09-19T04:13:16.448579+0000","last_became_peered":"2024-09-19T04:13:16.448579+0000","last_unstale":"2024-09-19T04:15:01.377948+0000","last_undegraded":"2024-09-19T04:15:01.377948+0000","last_fullsized":"2024-09-19T04:15:01.377948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:22:40.023000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387764+0000","last_change":"2024-09-19T04:13:16.445449+0000","last_active":"2024-09-19T04:15:01.387764+0000","last_peered":"2024-09-19T04:15:01.387764+0000","last_clean":"2024-09-19T04:15:01.387764+0000","last_became_active":"2024-09-19T04:13:16.445028+0000","last_became_peered":"2024-09-19T04:13:16.445028+0000","last_unstale":"2024-09-19T04:15:01.387764+0000","last_undegraded":"2024-09-19T04:15:01.387764+0000","last_fullsized":"2024-09-19T04:15:01.387764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:13.095348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.386639+0000","last_change":"2024-09-19T04:13:16.454033+0000","last_active":"2024-09-19T04:15:01.386639+0000","last_peered":"2024-09-19T04:15:01.386639+0000","last_clean":"2024-09-19T04:15:01.386639+0000","last_became_active":"2024-09-19T04:13:16.453592+0000","last_became_peered":"2024-09-19T04:13:16.453592+0000","last_unstale":"2024-09-19T04:15:01.386639+0000","last_undegraded":"2024-09-19T04:15:01.386639+0000","last_fullsized":"2024-09-19T04:15:01.386639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.304478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.408187+0000","last_change":"2024-09-19T04:13:16.448763+0000","last_active":"2024-09-19T04:15:01.408187+0000","last_peered":"2024-09-19T04:15:01.408187+0000","last_clean":"2024-09-19T04:15:01.408187+0000","last_became_active":"2024-09-19T04:13:16.448459+0000","last_became_peered":"2024-09-19T04:13:16.448459+0000","last_unstale":"2024-09-19T04:15:01.408187+0000","last_undegraded":"2024-09-19T04:15:01.408187+0000","last_fullsized":"2024-09-19T04:15:01.408187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.403838+0000","last_change":"2024-09-19T04:13:16.451489+0000","last_active":"2024-09-19T04:15:01.403838+0000","last_peered":"2024-09-19T04:15:01.403838+0000","last_clean":"2024-09-19T04:15:01.403838+0000","last_became_active":"2024-09-19T04:13:16.451176+0000","last_became_peered":"2024-09-19T04:13:16.451176+0000","last_unstale":"2024-09-19T04:15:01.403838+0000","last_undegraded":"2024-09-19T04:15:01.403838+0000","last_fullsized":"2024-09-19T04:15:01.403838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.921102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387603+0000","last_change":"2024-09-19T04:13:16.454551+0000","last_active":"2024-09-19T04:15:01.387603+0000","last_peered":"2024-09-19T04:15:01.387603+0000","last_clean":"2024-09-19T04:15:01.387603+0000","last_became_active":"2024-09-19T04:13:16.453976+0000","last_became_peered":"2024-09-19T04:13:16.453976+0000","last_unstale":"2024-09-19T04:15:01.387603+0000","last_undegraded":"2024-09-19T04:15:01.387603+0000","last_fullsized":"2024-09-19T04:15:01.387603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:06.504615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407629+0000","last_change":"2024-09-19T04:13:31.503222+0000","last_active":"2024-09-19T04:15:01.407629+0000","last_peered":"2024-09-19T04:15:01.407629+0000","last_clean":"2024-09-19T04:15:01.407629+0000","last_became_active":"2024-09-19T04:13:31.502873+0000","last_became_peered":"2024-09-19T04:13:31.502873+0000","last_unstale":"2024-09-19T04:15:01.407629+0000","last_undegraded":"2024-09-19T04:15:01.407629+0000","last_fullsized":"2024-09-19T04:15:01.407629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:54.559592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388512+0000","last_change":"2024-09-19T04:13:31.505834+0000","last_active":"2024-09-19T04:15:01.388512+0000","last_peered":"2024-09-19T04:15:01.388512+0000","last_clean":"2024-09-19T04:15:01.388512+0000","last_became_active":"2024-09-19T04:13:31.505501+0000","last_became_peered":"2024-09-19T04:13:31.505501+0000","last_unstale":"2024-09-19T04:15:01.388512+0000","last_undegraded":"2024-09-19T04:15:01.388512+0000","last_fullsized":"2024-09-19T04:15:01.388512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:45.136128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384870+0000","last_change":"2024-09-19T04:13:16.449237+0000","last_active":"2024-09-19T04:15:01.384870+0000","last_peered":"2024-09-19T04:15:01.384870+0000","last_clean":"2024-09-19T04:15:01.384870+0000","last_became_active":"2024-09-19T04:13:16.448896+0000","last_became_peered":"2024-09-19T04:13:16.448896+0000","last_unstale":"2024-09-19T04:15:01.384870+0000","last_undegraded":"2024-09-19T04:15:01.384870+0000","last_fullsized":"2024-09-19T04:15:01.384870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:38.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384284+0000","last_change":"2024-09-19T04:13:16.456053+0000","last_active":"2024-09-19T04:15:01.384284+0000","last_peered":"2024-09-19T04:15:01.384284+0000","last_clean":"2024-09-19T04:15:01.384284+0000","last_became_active":"2024-09-19T04:13:16.454775+0000","last_became_peered":"2024-09-19T04:13:16.454775+0000","last_unstale":"2024-09-19T04:15:01.384284+0000","last_undegraded":"2024-09-19T04:15:01.384284+0000","last_fullsized":"2024-09-19T04:15:01.384284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.902036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402500+0000","last_change":"2024-09-19T04:13:16.451473+0000","last_active":"2024-09-19T04:15:01.402500+0000","last_peered":"2024-09-19T04:15:01.402500+0000","last_clean":"2024-09-19T04:15:01.402500+0000","last_became_active":"2024-09-19T04:13:16.451150+0000","last_became_peered":"2024-09-19T04:13:16.451150+0000","last_unstale":"2024-09-19T04:15:01.402500+0000","last_undegraded":"2024-09-19T04:15:01.402500+0000","last_fullsized":"2024-09-19T04:15:01.402500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:59.332745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392354+0000","last_change":"2024-09-19T04:13:16.452017+0000","last_active":"2024-09-19T04:15:01.392354+0000","last_peered":"2024-09-19T04:15:01.392354+0000","last_clean":"2024-09-19T04:15:01.392354+0000","last_became_active":"2024-09-19T04:13:16.450715+0000","last_became_peered":"2024-09-19T04:13:16.450715+0000","last_unstale":"2024-09-19T04:15:01.392354+0000","last_undegraded":"2024-09-19T04:15:01.392354+0000","last_fullsized":"2024-09-19T04:15:01.392354+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:43.548770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.380440+0000","last_change":"2024-09-19T04:13:31.736597+0000","last_active":"2024-09-19T04:15:01.380440+0000","last_peered":"2024-09-19T04:15:01.380440+0000","last_clean":"2024-09-19T04:15:01.380440+0000","last_became_active":"2024-09-19T04:13:31.736301+0000","last_became_peered":"2024-09-19T04:13:31.736301+0000","last_unstale":"2024-09-19T04:15:01.380440+0000","last_undegraded":"2024-09-19T04:15:01.380440+0000","last_fullsized":"2024-09-19T04:15:01.380440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:06.614117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384673+0000","last_change":"2024-09-19T04:13:31.509742+0000","last_active":"2024-09-19T04:15:01.384673+0000","last_peered":"2024-09-19T04:15:01.384673+0000","last_clean":"2024-09-19T04:15:01.384673+0000","last_became_active":"2024-09-19T04:13:31.509363+0000","last_became_peered":"2024-09-19T04:13:31.509363+0000","last_unstale":"2024-09-19T04:15:01.384673+0000","last_undegraded":"2024-09-19T04:15:01.384673+0000","last_fullsized":"2024-09-19T04:15:01.384673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.094930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384995+0000","last_change":"2024-09-19T04:13:16.449459+0000","last_active":"2024-09-19T04:15:01.384995+0000","last_peered":"2024-09-19T04:15:01.384995+0000","last_clean":"2024-09-19T04:15:01.384995+0000","last_became_active":"2024-09-19T04:13:16.449119+0000","last_became_peered":"2024-09-19T04:13:16.449119+0000","last_unstale":"2024-09-19T04:15:01.384995+0000","last_undegraded":"2024-09-19T04:15:01.384995+0000","last_fullsized":"2024-09-19T04:15:01.384995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:26:26.941609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.375543+0000","last_change":"2024-09-19T04:13:16.448939+0000","last_active":"2024-09-19T04:15:01.375543+0000","last_peered":"2024-09-19T04:15:01.375543+0000","last_clean":"2024-09-19T04:15:01.375543+0000","last_became_active":"2024-09-19T04:13:16.448619+0000","last_became_peered":"2024-09-19T04:13:16.448619+0000","last_unstale":"2024-09-19T04:15:01.375543+0000","last_undegraded":"2024-09-19T04:15:01.375543+0000","last_fullsized":"2024-09-19T04:15:01.375543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:41.017404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'9","reported_seq":139,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:05.635944+0000","last_change":"2024-09-19T04:13:14.430467+0000","last_active":"2024-09-19T04:15:05.635944+0000","last_peered":"2024-09-19T04:15:05.635944+0000","last_clean":"2024-09-19T04:15:05.635944+0000","last_became_active":"2024-09-19T04:13:14.430095+0000","last_became_peered":"2024-09-19T04:13:14.430095+0000","last_unstale":"2024-09-19T04:15:05.635944+0000","last_undegraded":"2024-09-19T04:15:05.635944+0000","last_fullsized":"2024-09-19T04:15:05.635944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:50.471745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2028,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.385789+0000","last_change":"2024-09-19T04:13:31.505798+0000","last_active":"2024-09-19T04:15:01.385789+0000","last_peered":"2024-09-19T04:15:01.385789+0000","last_clean":"2024-09-19T04:15:01.385789+0000","last_became_active":"2024-09-19T04:13:31.505469+0000","last_became_peered":"2024-09-19T04:13:31.505469+0000","last_unstale":"2024-09-19T04:15:01.385789+0000","last_undegraded":"2024-09-19T04:15:01.385789+0000","last_fullsized":"2024-09-19T04:15:01.385789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:13:42.405075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.348065+0000","last_change":"2024-09-19T04:13:14.426443+0000","last_active":"2024-09-19T04:15:01.348065+0000","last_peered":"2024-09-19T04:15:01.348065+0000","last_clean":"2024-09-19T04:15:01.348065+0000","last_became_active":"2024-09-19T04:13:14.426103+0000","last_became_peered":"2024-09-19T04:13:14.426103+0000","last_unstale":"2024-09-19T04:15:01.348065+0000","last_undegraded":"2024-09-19T04:15:01.348065+0000","last_fullsized":"2024-09-19T04:15:01.348065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:28.461962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.343355+0000","last_change":"2024-09-19T04:13:14.424439+0000","last_active":"2024-09-19T04:15:01.343355+0000","last_peered":"2024-09-19T04:15:01.343355+0000","last_clean":"2024-09-19T04:15:01.343355+0000","last_became_active":"2024-09-19T04:13:14.423975+0000","last_became_peered":"2024-09-19T04:13:14.423975+0000","last_unstale":"2024-09-19T04:15:01.343355+0000","last_undegraded":"2024-09-19T04:15:01.343355+0000","last_fullsized":"2024-09-19T04:15:01.343355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:12.032125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392899+0000","last_change":"2024-09-19T04:13:16.444865+0000","last_active":"2024-09-19T04:15:01.392899+0000","last_peered":"2024-09-19T04:15:01.392899+0000","last_clean":"2024-09-19T04:15:01.392899+0000","last_became_active":"2024-09-19T04:13:16.444556+0000","last_became_peered":"2024-09-19T04:13:16.444556+0000","last_unstale":"2024-09-19T04:15:01.392899+0000","last_undegraded":"2024-09-19T04:15:01.392899+0000","last_fullsized":"2024-09-19T04:15:01.392899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:38.025604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354210+0000","last_change":"2024-09-19T04:13:14.429018+0000","last_active":"2024-09-19T04:15:01.354210+0000","last_peered":"2024-09-19T04:15:01.354210+0000","last_clean":"2024-09-19T04:15:01.354210+0000","last_became_active":"2024-09-19T04:13:14.428383+0000","last_became_peered":"2024-09-19T04:13:14.428383+0000","last_unstale":"2024-09-19T04:15:01.354210+0000","last_undegraded":"2024-09-19T04:15:01.354210+0000","last_fullsized":"2024-09-19T04:15:01.354210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:46.190421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.391102+0000","last_change":"2024-09-19T04:13:16.449013+0000","last_active":"2024-09-19T04:15:01.391102+0000","last_peered":"2024-09-19T04:15:01.391102+0000","last_clean":"2024-09-19T04:15:01.391102+0000","last_became_active":"2024-09-19T04:13:16.448703+0000","last_became_peered":"2024-09-19T04:13:16.448703+0000","last_unstale":"2024-09-19T04:15:01.391102+0000","last_undegraded":"2024-09-19T04:15:01.391102+0000","last_fullsized":"2024-09-19T04:15:01.391102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:41.231759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":129,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:05.635414+0000","last_change":"2024-09-19T04:13:14.430050+0000","last_active":"2024-09-19T04:15:05.635414+0000","last_peered":"2024-09-19T04:15:05.635414+0000","last_clean":"2024-09-19T04:15:05.635414+0000","last_became_active":"2024-09-19T04:13:14.429710+0000","last_became_peered":"2024-09-19T04:13:14.429710+0000","last_unstale":"2024-09-19T04:15:05.635414+0000","last_undegraded":"2024-09-19T04:15:05.635414+0000","last_fullsized":"2024-09-19T04:15:05.635414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:55.796080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388332+0000","last_change":"2024-09-19T04:13:16.454431+0000","last_active":"2024-09-19T04:15:01.388332+0000","last_peered":"2024-09-19T04:15:01.388332+0000","last_clean":"2024-09-19T04:15:01.388332+0000","last_became_active":"2024-09-19T04:13:16.453033+0000","last_became_peered":"2024-09-19T04:13:16.453033+0000","last_unstale":"2024-09-19T04:15:01.388332+0000","last_undegraded":"2024-09-19T04:15:01.388332+0000","last_fullsized":"2024-09-19T04:15:01.388332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:52.817740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.362264+0000","last_change":"2024-09-19T04:13:14.428949+0000","last_active":"2024-09-19T04:15:01.362264+0000","last_peered":"2024-09-19T04:15:01.362264+0000","last_clean":"2024-09-19T04:15:01.362264+0000","last_became_active":"2024-09-19T04:13:14.428525+0000","last_became_peered":"2024-09-19T04:13:14.428525+0000","last_unstale":"2024-09-19T04:15:01.362264+0000","last_undegraded":"2024-09-19T04:15:01.362264+0000","last_fullsized":"2024-09-19T04:15:01.362264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:30.296730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.381741+0000","last_change":"2024-09-19T04:13:16.454077+0000","last_active":"2024-09-19T04:15:01.381741+0000","last_peered":"2024-09-19T04:15:01.381741+0000","last_clean":"2024-09-19T04:15:01.381741+0000","last_became_active":"2024-09-19T04:13:16.453791+0000","last_became_peered":"2024-09-19T04:13:16.453791+0000","last_unstale":"2024-09-19T04:15:01.381741+0000","last_undegraded":"2024-09-19T04:15:01.381741+0000","last_fullsized":"2024-09-19T04:15:01.381741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:29.426475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.356675+0000","last_change":"2024-09-19T04:13:14.620179+0000","last_active":"2024-09-19T04:15:01.356675+0000","last_peered":"2024-09-19T04:15:01.356675+0000","last_clean":"2024-09-19T04:15:01.356675+0000","last_became_active":"2024-09-19T04:13:14.619758+0000","last_became_peered":"2024-09-19T04:13:14.619758+0000","last_unstale":"2024-09-19T04:15:01.356675+0000","last_undegraded":"2024-09-19T04:15:01.356675+0000","last_fullsized":"2024-09-19T04:15:01.356675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:11.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.404018+0000","last_change":"2024-09-19T04:13:16.453036+0000","last_active":"2024-09-19T04:15:01.404018+0000","last_peered":"2024-09-19T04:15:01.404018+0000","last_clean":"2024-09-19T04:15:01.404018+0000","last_became_active":"2024-09-19T04:13:16.452736+0000","last_became_peered":"2024-09-19T04:13:16.452736+0000","last_unstale":"2024-09-19T04:15:01.404018+0000","last_undegraded":"2024-09-19T04:15:01.404018+0000","last_fullsized":"2024-09-19T04:15:01.404018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:30.003652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'35","reported_seq":271,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:06.640155+0000","last_change":"2024-09-19T04:13:14.428715+0000","last_active":"2024-09-19T04:15:06.640155+0000","last_peered":"2024-09-19T04:15:06.640155+0000","last_clean":"2024-09-19T04:15:06.640155+0000","last_became_active":"2024-09-19T04:13:14.428409+0000","last_became_peered":"2024-09-19T04:13:14.428409+0000","last_unstale":"2024-09-19T04:15:06.640155+0000","last_undegraded":"2024-09-19T04:15:06.640155+0000","last_fullsized":"2024-09-19T04:15:06.640155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:23.130857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65592,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":212,"num_read_kb":166,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407414+0000","last_change":"2024-09-19T04:13:16.454512+0000","last_active":"2024-09-19T04:15:01.407414+0000","last_peered":"2024-09-19T04:15:01.407414+0000","last_clean":"2024-09-19T04:15:01.407414+0000","last_became_active":"2024-09-19T04:13:16.454193+0000","last_became_peered":"2024-09-19T04:13:16.454193+0000","last_unstale":"2024-09-19T04:15:01.407414+0000","last_undegraded":"2024-09-19T04:15:01.407414+0000","last_fullsized":"2024-09-19T04:15:01.407414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:11.825938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.344927+0000","last_change":"2024-09-19T04:13:31.515448+0000","last_active":"2024-09-19T04:15:01.344927+0000","last_peered":"2024-09-19T04:15:01.344927+0000","last_clean":"2024-09-19T04:15:01.344927+0000","last_became_active":"2024-09-19T04:13:31.506099+0000","last_became_peered":"2024-09-19T04:13:31.506099+0000","last_unstale":"2024-09-19T04:15:01.344927+0000","last_undegraded":"2024-09-19T04:15:01.344927+0000","last_fullsized":"2024-09-19T04:15:01.344927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:21.462488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407870+0000","last_change":"2024-09-19T04:13:16.447607+0000","last_active":"2024-09-19T04:15:01.407870+0000","last_peered":"2024-09-19T04:15:01.407870+0000","last_clean":"2024-09-19T04:15:01.407870+0000","last_became_active":"2024-09-19T04:13:16.447029+0000","last_became_peered":"2024-09-19T04:13:16.447029+0000","last_unstale":"2024-09-19T04:15:01.407870+0000","last_undegraded":"2024-09-19T04:15:01.407870+0000","last_fullsized":"2024-09-19T04:15:01.407870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:52.418104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.355933+0000","last_change":"2024-09-19T04:13:14.428856+0000","last_active":"2024-09-19T04:15:01.355933+0000","last_peered":"2024-09-19T04:15:01.355933+0000","last_clean":"2024-09-19T04:15:01.355933+0000","last_became_active":"2024-09-19T04:13:14.428510+0000","last_became_peered":"2024-09-19T04:13:14.428510+0000","last_unstale":"2024-09-19T04:15:01.355933+0000","last_undegraded":"2024-09-19T04:15:01.355933+0000","last_fullsized":"2024-09-19T04:15:01.355933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:49.633325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.409050+0000","last_change":"2024-09-19T04:13:16.455628+0000","last_active":"2024-09-19T04:15:01.409050+0000","last_peered":"2024-09-19T04:15:01.409050+0000","last_clean":"2024-09-19T04:15:01.409050+0000","last_became_active":"2024-09-19T04:13:16.453817+0000","last_became_peered":"2024-09-19T04:13:16.453817+0000","last_unstale":"2024-09-19T04:15:01.409050+0000","last_undegraded":"2024-09-19T04:15:01.409050+0000","last_fullsized":"2024-09-19T04:15:01.409050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.832806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":136,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:05.635669+0000","last_change":"2024-09-19T04:13:14.424924+0000","last_active":"2024-09-19T04:15:05.635669+0000","last_peered":"2024-09-19T04:15:05.635669+0000","last_clean":"2024-09-19T04:15:05.635669+0000","last_became_active":"2024-09-19T04:13:14.424627+0000","last_became_peered":"2024-09-19T04:13:14.424627+0000","last_unstale":"2024-09-19T04:15:05.635669+0000","last_undegraded":"2024-09-19T04:15:05.635669+0000","last_fullsized":"2024-09-19T04:15:05.635669+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:49.830205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1994,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.405035+0000","last_change":"2024-09-19T04:13:16.455365+0000","last_active":"2024-09-19T04:15:01.405035+0000","last_peered":"2024-09-19T04:15:01.405035+0000","last_clean":"2024-09-19T04:15:01.405035+0000","last_became_active":"2024-09-19T04:13:16.455029+0000","last_became_peered":"2024-09-19T04:13:16.455029+0000","last_unstale":"2024-09-19T04:15:01.405035+0000","last_undegraded":"2024-09-19T04:15:01.405035+0000","last_fullsized":"2024-09-19T04:15:01.405035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:20.947391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.353018+0000","last_change":"2024-09-19T04:13:14.620462+0000","last_active":"2024-09-19T04:15:01.353018+0000","last_peered":"2024-09-19T04:15:01.353018+0000","last_clean":"2024-09-19T04:15:01.353018+0000","last_became_active":"2024-09-19T04:13:14.620103+0000","last_became_peered":"2024-09-19T04:13:14.620103+0000","last_unstale":"2024-09-19T04:15:01.353018+0000","last_undegraded":"2024-09-19T04:15:01.353018+0000","last_fullsized":"2024-09-19T04:15:01.353018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:14.496075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.389192+0000","last_change":"2024-09-19T04:13:16.450222+0000","last_active":"2024-09-19T04:15:01.389192+0000","last_peered":"2024-09-19T04:15:01.389192+0000","last_clean":"2024-09-19T04:15:01.389192+0000","last_became_active":"2024-09-19T04:13:16.449916+0000","last_became_peered":"2024-09-19T04:13:16.449916+0000","last_unstale":"2024-09-19T04:15:01.389192+0000","last_undegraded":"2024-09-19T04:15:01.389192+0000","last_fullsized":"2024-09-19T04:15:01.389192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:31.886342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.345163+0000","last_change":"2024-09-19T04:13:31.519190+0000","last_active":"2024-09-19T04:15:01.345163+0000","last_peered":"2024-09-19T04:15:01.345163+0000","last_clean":"2024-09-19T04:15:01.345163+0000","last_became_active":"2024-09-19T04:13:31.508605+0000","last_became_peered":"2024-09-19T04:13:31.508605+0000","last_unstale":"2024-09-19T04:15:01.345163+0000","last_undegraded":"2024-09-19T04:15:01.345163+0000","last_fullsized":"2024-09-19T04:15:01.345163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:51.437135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.386116+0000","last_change":"2024-09-19T04:13:16.450848+0000","last_active":"2024-09-19T04:15:01.386116+0000","last_peered":"2024-09-19T04:15:01.386116+0000","last_clean":"2024-09-19T04:15:01.386116+0000","last_became_active":"2024-09-19T04:13:16.450496+0000","last_became_peered":"2024-09-19T04:13:16.450496+0000","last_unstale":"2024-09-19T04:15:01.386116+0000","last_undegraded":"2024-09-19T04:15:01.386116+0000","last_fullsized":"2024-09-19T04:15:01.386116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:49:00.346998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.343626+0000","last_change":"2024-09-19T04:13:14.428654+0000","last_active":"2024-09-19T04:15:01.343626+0000","last_peered":"2024-09-19T04:15:01.343626+0000","last_clean":"2024-09-19T04:15:01.343626+0000","last_became_active":"2024-09-19T04:13:14.428297+0000","last_became_peered":"2024-09-19T04:13:14.428297+0000","last_unstale":"2024-09-19T04:15:01.343626+0000","last_undegraded":"2024-09-19T04:15:01.343626+0000","last_fullsized":"2024-09-19T04:15:01.343626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:30.834262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407394+0000","last_change":"2024-09-19T04:13:16.456002+0000","last_active":"2024-09-19T04:15:01.407394+0000","last_peered":"2024-09-19T04:15:01.407394+0000","last_clean":"2024-09-19T04:15:01.407394+0000","last_became_active":"2024-09-19T04:13:16.455463+0000","last_became_peered":"2024-09-19T04:13:16.455463+0000","last_unstale":"2024-09-19T04:15:01.407394+0000","last_undegraded":"2024-09-19T04:15:01.407394+0000","last_fullsized":"2024-09-19T04:15:01.407394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:47.266364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'39","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.568584+0000","last_change":"2024-09-19T04:13:14.429555+0000","last_active":"2024-09-19T04:15:01.568584+0000","last_peered":"2024-09-19T04:15:01.568584+0000","last_clean":"2024-09-19T04:15:01.568584+0000","last_became_active":"2024-09-19T04:13:14.429224+0000","last_became_peered":"2024-09-19T04:13:14.429224+0000","last_unstale":"2024-09-19T04:15:01.568584+0000","last_undegraded":"2024-09-19T04:15:01.568584+0000","last_fullsized":"2024-09-19T04:15:01.568584+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:35.304750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.407617+0000","last_change":"2024-09-19T04:13:16.454337+0000","last_active":"2024-09-19T04:15:01.407617+0000","last_peered":"2024-09-19T04:15:01.407617+0000","last_clean":"2024-09-19T04:15:01.407617+0000","last_became_active":"2024-09-19T04:13:16.453831+0000","last_became_peered":"2024-09-19T04:13:16.453831+0000","last_unstale":"2024-09-19T04:15:01.407617+0000","last_undegraded":"2024-09-19T04:15:01.407617+0000","last_fullsized":"2024-09-19T04:15:01.407617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:41.751933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.351335+0000","last_change":"2024-09-19T04:13:14.620406+0000","last_active":"2024-09-19T04:15:01.351335+0000","last_peered":"2024-09-19T04:15:01.351335+0000","last_clean":"2024-09-19T04:15:01.351335+0000","last_became_active":"2024-09-19T04:13:14.620050+0000","last_became_peered":"2024-09-19T04:13:14.620050+0000","last_unstale":"2024-09-19T04:15:01.351335+0000","last_undegraded":"2024-09-19T04:15:01.351335+0000","last_fullsized":"2024-09-19T04:15:01.351335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:35.828760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.408215+0000","last_change":"2024-09-19T04:13:16.459016+0000","last_active":"2024-09-19T04:15:01.408215+0000","last_peered":"2024-09-19T04:15:01.408215+0000","last_clean":"2024-09-19T04:15:01.408215+0000","last_became_active":"2024-09-19T04:13:16.458674+0000","last_became_peered":"2024-09-19T04:13:16.458674+0000","last_unstale":"2024-09-19T04:15:01.408215+0000","last_undegraded":"2024-09-19T04:15:01.408215+0000","last_fullsized":"2024-09-19T04:15:01.408215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:55.894456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"82'148","reported_seq":393,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:03.472326+0000","last_change":"2024-09-19T04:12:05.913704+0000","last_active":"2024-09-19T04:15:03.472326+0000","last_peered":"2024-09-19T04:15:03.472326+0000","last_clean":"2024-09-19T04:15:03.472326+0000","last_became_active":"2024-09-19T04:12:05.913299+0000","last_became_peered":"2024-09-19T04:12:05.913299+0000","last_unstale":"2024-09-19T04:15:03.472326+0000","last_undegraded":"2024-09-19T04:15:03.472326+0000","last_fullsized":"2024-09-19T04:15:03.472326+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:08:46.646000+0000","last_clean_scrub_stamp":"2024-09-19T04:08:46.646000+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.638160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.357722+0000","last_change":"2024-09-19T04:13:14.423962+0000","last_active":"2024-09-19T04:15:01.357722+0000","last_peered":"2024-09-19T04:15:01.357722+0000","last_clean":"2024-09-19T04:15:01.357722+0000","last_became_active":"2024-09-19T04:13:14.423591+0000","last_became_peered":"2024-09-19T04:13:14.423591+0000","last_unstale":"2024-09-19T04:15:01.357722+0000","last_undegraded":"2024-09-19T04:15:01.357722+0000","last_fullsized":"2024-09-19T04:15:01.357722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:02.239975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.394412+0000","last_change":"2024-09-19T04:13:16.447113+0000","last_active":"2024-09-19T04:15:01.394412+0000","last_peered":"2024-09-19T04:15:01.394412+0000","last_clean":"2024-09-19T04:15:01.394412+0000","last_became_active":"2024-09-19T04:13:16.446792+0000","last_became_peered":"2024-09-19T04:13:16.446792+0000","last_unstale":"2024-09-19T04:15:01.394412+0000","last_undegraded":"2024-09-19T04:15:01.394412+0000","last_fullsized":"2024-09-19T04:15:01.394412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:28:21.527010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.361188+0000","last_change":"2024-09-19T04:13:14.432663+0000","last_active":"2024-09-19T04:15:01.361188+0000","last_peered":"2024-09-19T04:15:01.361188+0000","last_clean":"2024-09-19T04:15:01.361188+0000","last_became_active":"2024-09-19T04:13:14.432318+0000","last_became_peered":"2024-09-19T04:13:14.432318+0000","last_unstale":"2024-09-19T04:15:01.361188+0000","last_undegraded":"2024-09-19T04:15:01.361188+0000","last_fullsized":"2024-09-19T04:15:01.361188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:35.787596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402670+0000","last_change":"2024-09-19T04:13:16.451010+0000","last_active":"2024-09-19T04:15:01.402670+0000","last_peered":"2024-09-19T04:15:01.402670+0000","last_clean":"2024-09-19T04:15:01.402670+0000","last_became_active":"2024-09-19T04:13:16.450708+0000","last_became_peered":"2024-09-19T04:13:16.450708+0000","last_unstale":"2024-09-19T04:15:01.402670+0000","last_undegraded":"2024-09-19T04:15:01.402670+0000","last_fullsized":"2024-09-19T04:15:01.402670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:30.108541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:05.635275+0000","last_change":"2024-09-19T04:13:14.620037+0000","last_active":"2024-09-19T04:15:05.635275+0000","last_peered":"2024-09-19T04:15:05.635275+0000","last_clean":"2024-09-19T04:15:05.635275+0000","last_became_active":"2024-09-19T04:13:14.619686+0000","last_became_peered":"2024-09-19T04:13:14.619686+0000","last_unstale":"2024-09-19T04:15:05.635275+0000","last_undegraded":"2024-09-19T04:15:05.635275+0000","last_fullsized":"2024-09-19T04:15:05.635275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:32.490708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.387959+0000","last_change":"2024-09-19T04:13:16.446429+0000","last_active":"2024-09-19T04:15:01.387959+0000","last_peered":"2024-09-19T04:15:01.387959+0000","last_clean":"2024-09-19T04:15:01.387959+0000","last_became_active":"2024-09-19T04:13:16.446075+0000","last_became_peered":"2024-09-19T04:13:16.446075+0000","last_unstale":"2024-09-19T04:15:01.387959+0000","last_undegraded":"2024-09-19T04:15:01.387959+0000","last_fullsized":"2024-09-19T04:15:01.387959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:03.613926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.361912+0000","last_change":"2024-09-19T04:13:14.429899+0000","last_active":"2024-09-19T04:15:01.361912+0000","last_peered":"2024-09-19T04:15:01.361912+0000","last_clean":"2024-09-19T04:15:01.361912+0000","last_became_active":"2024-09-19T04:13:14.429571+0000","last_became_peered":"2024-09-19T04:13:14.429571+0000","last_unstale":"2024-09-19T04:15:01.361912+0000","last_undegraded":"2024-09-19T04:15:01.361912+0000","last_fullsized":"2024-09-19T04:15:01.361912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:37.017431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.384017+0000","last_change":"2024-09-19T04:13:16.458267+0000","last_active":"2024-09-19T04:15:01.384017+0000","last_peered":"2024-09-19T04:15:01.384017+0000","last_clean":"2024-09-19T04:15:01.384017+0000","last_became_active":"2024-09-19T04:13:16.457911+0000","last_became_peered":"2024-09-19T04:13:16.457911+0000","last_unstale":"2024-09-19T04:15:01.384017+0000","last_undegraded":"2024-09-19T04:15:01.384017+0000","last_fullsized":"2024-09-19T04:15:01.384017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:32.465923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354719+0000","last_change":"2024-09-19T04:13:14.621552+0000","last_active":"2024-09-19T04:15:01.354719+0000","last_peered":"2024-09-19T04:15:01.354719+0000","last_clean":"2024-09-19T04:15:01.354719+0000","last_became_active":"2024-09-19T04:13:14.620592+0000","last_became_peered":"2024-09-19T04:13:14.620592+0000","last_unstale":"2024-09-19T04:15:01.354719+0000","last_undegraded":"2024-09-19T04:15:01.354719+0000","last_fullsized":"2024-09-19T04:15:01.354719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:10.055202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.402127+0000","last_change":"2024-09-19T04:13:16.443117+0000","last_active":"2024-09-19T04:15:01.402127+0000","last_peered":"2024-09-19T04:15:01.402127+0000","last_clean":"2024-09-19T04:15:01.402127+0000","last_became_active":"2024-09-19T04:13:16.442808+0000","last_became_peered":"2024-09-19T04:13:16.442808+0000","last_unstale":"2024-09-19T04:15:01.402127+0000","last_undegraded":"2024-09-19T04:15:01.402127+0000","last_fullsized":"2024-09-19T04:15:01.402127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:40.274320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354893+0000","last_change":"2024-09-19T04:13:14.620618+0000","last_active":"2024-09-19T04:15:01.354893+0000","last_peered":"2024-09-19T04:15:01.354893+0000","last_clean":"2024-09-19T04:15:01.354893+0000","last_became_active":"2024-09-19T04:13:14.620290+0000","last_became_peered":"2024-09-19T04:13:14.620290+0000","last_unstale":"2024-09-19T04:15:01.354893+0000","last_undegraded":"2024-09-19T04:15:01.354893+0000","last_fullsized":"2024-09-19T04:15:01.354893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:51.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.390139+0000","last_change":"2024-09-19T04:13:16.443498+0000","last_active":"2024-09-19T04:15:01.390139+0000","last_peered":"2024-09-19T04:15:01.390139+0000","last_clean":"2024-09-19T04:15:01.390139+0000","last_became_active":"2024-09-19T04:13:16.443104+0000","last_became_peered":"2024-09-19T04:13:16.443104+0000","last_unstale":"2024-09-19T04:15:01.390139+0000","last_undegraded":"2024-09-19T04:15:01.390139+0000","last_fullsized":"2024-09-19T04:15:01.390139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:10.733570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'46","reported_seq":93,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.574116+0000","last_change":"2024-09-19T04:13:14.425881+0000","last_active":"2024-09-19T04:15:01.574116+0000","last_peered":"2024-09-19T04:15:01.574116+0000","last_clean":"2024-09-19T04:15:01.574116+0000","last_became_active":"2024-09-19T04:13:14.425542+0000","last_became_peered":"2024-09-19T04:13:14.425542+0000","last_unstale":"2024-09-19T04:15:01.574116+0000","last_undegraded":"2024-09-19T04:15:01.574116+0000","last_fullsized":"2024-09-19T04:15:01.574116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:44.250890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.392711+0000","last_change":"2024-09-19T04:13:16.454672+0000","last_active":"2024-09-19T04:15:01.392711+0000","last_peered":"2024-09-19T04:15:01.392711+0000","last_clean":"2024-09-19T04:15:01.392711+0000","last_became_active":"2024-09-19T04:13:16.454003+0000","last_became_peered":"2024-09-19T04:13:16.454003+0000","last_unstale":"2024-09-19T04:15:01.392711+0000","last_undegraded":"2024-09-19T04:15:01.392711+0000","last_fullsized":"2024-09-19T04:15:01.392711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:26.849617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.357930+0000","last_change":"2024-09-19T04:13:14.432312+0000","last_active":"2024-09-19T04:15:01.357930+0000","last_peered":"2024-09-19T04:15:01.357930+0000","last_clean":"2024-09-19T04:15:01.357930+0000","last_became_active":"2024-09-19T04:13:14.431994+0000","last_became_peered":"2024-09-19T04:13:14.431994+0000","last_unstale":"2024-09-19T04:15:01.357930+0000","last_undegraded":"2024-09-19T04:15:01.357930+0000","last_fullsized":"2024-09-19T04:15:01.357930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:00.410400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.391357+0000","last_change":"2024-09-19T04:13:16.455312+0000","last_active":"2024-09-19T04:15:01.391357+0000","last_peered":"2024-09-19T04:15:01.391357+0000","last_clean":"2024-09-19T04:15:01.391357+0000","last_became_active":"2024-09-19T04:13:16.454990+0000","last_became_peered":"2024-09-19T04:13:16.454990+0000","last_unstale":"2024-09-19T04:15:01.391357+0000","last_undegraded":"2024-09-19T04:15:01.391357+0000","last_fullsized":"2024-09-19T04:15:01.391357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:09.207395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.349902+0000","last_change":"2024-09-19T04:13:14.620316+0000","last_active":"2024-09-19T04:15:01.349902+0000","last_peered":"2024-09-19T04:15:01.349902+0000","last_clean":"2024-09-19T04:15:01.349902+0000","last_became_active":"2024-09-19T04:13:14.619979+0000","last_became_peered":"2024-09-19T04:13:14.619979+0000","last_unstale":"2024-09-19T04:15:01.349902+0000","last_undegraded":"2024-09-19T04:15:01.349902+0000","last_fullsized":"2024-09-19T04:15:01.349902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:24.382144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.379294+0000","last_change":"2024-09-19T04:13:16.455684+0000","last_active":"2024-09-19T04:15:01.379294+0000","last_peered":"2024-09-19T04:15:01.379294+0000","last_clean":"2024-09-19T04:15:01.379294+0000","last_became_active":"2024-09-19T04:13:16.455355+0000","last_became_peered":"2024-09-19T04:13:16.455355+0000","last_unstale":"2024-09-19T04:15:01.379294+0000","last_undegraded":"2024-09-19T04:15:01.379294+0000","last_fullsized":"2024-09-19T04:15:01.379294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:09.657447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.346821+0000","last_change":"2024-09-19T04:13:14.423599+0000","last_active":"2024-09-19T04:15:01.346821+0000","last_peered":"2024-09-19T04:15:01.346821+0000","last_clean":"2024-09-19T04:15:01.346821+0000","last_became_active":"2024-09-19T04:13:14.423282+0000","last_became_peered":"2024-09-19T04:13:14.423282+0000","last_unstale":"2024-09-19T04:15:01.346821+0000","last_undegraded":"2024-09-19T04:15:01.346821+0000","last_fullsized":"2024-09-19T04:15:01.346821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.975368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.376076+0000","last_change":"2024-09-19T04:13:16.450435+0000","last_active":"2024-09-19T04:15:01.376076+0000","last_peered":"2024-09-19T04:15:01.376076+0000","last_clean":"2024-09-19T04:15:01.376076+0000","last_became_active":"2024-09-19T04:13:16.450087+0000","last_became_peered":"2024-09-19T04:13:16.450087+0000","last_unstale":"2024-09-19T04:15:01.376076+0000","last_undegraded":"2024-09-19T04:15:01.376076+0000","last_fullsized":"2024-09-19T04:15:01.376076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:34:51.764776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.351312+0000","last_change":"2024-09-19T04:13:14.432868+0000","last_active":"2024-09-19T04:15:01.351312+0000","last_peered":"2024-09-19T04:15:01.351312+0000","last_clean":"2024-09-19T04:15:01.351312+0000","last_became_active":"2024-09-19T04:13:14.432507+0000","last_became_peered":"2024-09-19T04:13:14.432507+0000","last_unstale":"2024-09-19T04:15:01.351312+0000","last_undegraded":"2024-09-19T04:15:01.351312+0000","last_fullsized":"2024-09-19T04:15:01.351312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:14.773711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.390493+0000","last_change":"2024-09-19T04:13:16.450061+0000","last_active":"2024-09-19T04:15:01.390493+0000","last_peered":"2024-09-19T04:15:01.390493+0000","last_clean":"2024-09-19T04:15:01.390493+0000","last_became_active":"2024-09-19T04:13:16.448829+0000","last_became_peered":"2024-09-19T04:13:16.448829+0000","last_unstale":"2024-09-19T04:15:01.390493+0000","last_undegraded":"2024-09-19T04:15:01.390493+0000","last_fullsized":"2024-09-19T04:15:01.390493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:31.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354057+0000","last_change":"2024-09-19T04:13:14.431574+0000","last_active":"2024-09-19T04:15:01.354057+0000","last_peered":"2024-09-19T04:15:01.354057+0000","last_clean":"2024-09-19T04:15:01.354057+0000","last_became_active":"2024-09-19T04:13:14.431202+0000","last_became_peered":"2024-09-19T04:13:14.431202+0000","last_unstale":"2024-09-19T04:15:01.354057+0000","last_undegraded":"2024-09-19T04:15:01.354057+0000","last_fullsized":"2024-09-19T04:15:01.354057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:58.167201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.408741+0000","last_change":"2024-09-19T04:13:16.452688+0000","last_active":"2024-09-19T04:15:01.408741+0000","last_peered":"2024-09-19T04:15:01.408741+0000","last_clean":"2024-09-19T04:15:01.408741+0000","last_became_active":"2024-09-19T04:13:16.452342+0000","last_became_peered":"2024-09-19T04:13:16.452342+0000","last_unstale":"2024-09-19T04:15:01.408741+0000","last_undegraded":"2024-09-19T04:15:01.408741+0000","last_fullsized":"2024-09-19T04:15:01.408741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:03.162688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.403276+0000","last_change":"2024-09-19T04:13:16.449402+0000","last_active":"2024-09-19T04:15:01.403276+0000","last_peered":"2024-09-19T04:15:01.403276+0000","last_clean":"2024-09-19T04:15:01.403276+0000","last_became_active":"2024-09-19T04:13:16.449108+0000","last_became_peered":"2024-09-19T04:13:16.449108+0000","last_unstale":"2024-09-19T04:15:01.403276+0000","last_undegraded":"2024-09-19T04:15:01.403276+0000","last_fullsized":"2024-09-19T04:15:01.403276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:58.874638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354429+0000","last_change":"2024-09-19T04:13:14.430684+0000","last_active":"2024-09-19T04:15:01.354429+0000","last_peered":"2024-09-19T04:15:01.354429+0000","last_clean":"2024-09-19T04:15:01.354429+0000","last_became_active":"2024-09-19T04:13:14.430137+0000","last_became_peered":"2024-09-19T04:13:14.430137+0000","last_unstale":"2024-09-19T04:15:01.354429+0000","last_undegraded":"2024-09-19T04:15:01.354429+0000","last_fullsized":"2024-09-19T04:15:01.354429+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:39.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.403606+0000","last_change":"2024-09-19T04:13:16.452491+0000","last_active":"2024-09-19T04:15:01.403606+0000","last_peered":"2024-09-19T04:15:01.403606+0000","last_clean":"2024-09-19T04:15:01.403606+0000","last_became_active":"2024-09-19T04:13:16.452204+0000","last_became_peered":"2024-09-19T04:13:16.452204+0000","last_unstale":"2024-09-19T04:15:01.403606+0000","last_undegraded":"2024-09-19T04:15:01.403606+0000","last_fullsized":"2024-09-19T04:15:01.403606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:10.113510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.354761+0000","last_change":"2024-09-19T04:13:14.431608+0000","last_active":"2024-09-19T04:15:01.354761+0000","last_peered":"2024-09-19T04:15:01.354761+0000","last_clean":"2024-09-19T04:15:01.354761+0000","last_became_active":"2024-09-19T04:13:14.431111+0000","last_became_peered":"2024-09-19T04:13:14.431111+0000","last_unstale":"2024-09-19T04:15:01.354761+0000","last_undegraded":"2024-09-19T04:15:01.354761+0000","last_fullsized":"2024-09-19T04:15:01.354761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:03:41.125315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.393098+0000","last_change":"2024-09-19T04:13:16.457867+0000","last_active":"2024-09-19T04:15:01.393098+0000","last_peered":"2024-09-19T04:15:01.393098+0000","last_clean":"2024-09-19T04:15:01.393098+0000","last_became_active":"2024-09-19T04:13:16.457582+0000","last_became_peered":"2024-09-19T04:13:16.457582+0000","last_unstale":"2024-09-19T04:15:01.393098+0000","last_undegraded":"2024-09-19T04:15:01.393098+0000","last_fullsized":"2024-09-19T04:15:01.393098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:58.076250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.346539+0000","last_change":"2024-09-19T04:13:14.620070+0000","last_active":"2024-09-19T04:15:01.346539+0000","last_peered":"2024-09-19T04:15:01.346539+0000","last_clean":"2024-09-19T04:15:01.346539+0000","last_became_active":"2024-09-19T04:13:14.619734+0000","last_became_peered":"2024-09-19T04:13:14.619734+0000","last_unstale":"2024-09-19T04:15:01.346539+0000","last_undegraded":"2024-09-19T04:15:01.346539+0000","last_fullsized":"2024-09-19T04:15:01.346539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:44:02.818859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.388185+0000","last_change":"2024-09-19T04:13:16.443832+0000","last_active":"2024-09-19T04:15:01.388185+0000","last_peered":"2024-09-19T04:15:01.388185+0000","last_clean":"2024-09-19T04:15:01.388185+0000","last_became_active":"2024-09-19T04:13:16.442690+0000","last_became_peered":"2024-09-19T04:13:16.442690+0000","last_unstale":"2024-09-19T04:15:01.388185+0000","last_undegraded":"2024-09-19T04:15:01.388185+0000","last_fullsized":"2024-09-19T04:15:01.388185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:15.416285+0000","last_clean_scrub_stamp":"2024-09-19T04:13:15.416285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:39.535628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T04:15:01.355196+0000","last_change":"2024-09-19T04:13:14.427186+0000","last_active":"2024-09-19T04:15:01.355196+0000","last_peered":"2024-09-19T04:15:01.355196+0000","last_clean":"2024-09-19T04:15:01.355196+0000","last_became_active":"2024-09-19T04:13:14.426949+0000","last_became_peered":"2024-09-19T04:13:14.426949+0000","last_unstale":"2024-09-19T04:15:01.355196+0000","last_undegraded":"2024-09-19T04:15:01.355196+0000","last_fullsized":"2024-09-19T04:15:01.355196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.409009+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.409009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:26.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596830,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":730,"num_read_kb":469,"num_write":294,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2015232,"data_stored":1790490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612643,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":440451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841576,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33000,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712152,"statfs":{"total":95995035648,"available":95961243648,"internally_reserved":0,"allocated":1736704,"data_stored":1226160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005101,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40792,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704360,"statfs":{"total":95995035648,"available":95953264640,"internally_reserved":0,"allocated":1314816,"data_stored":833847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234034,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28304,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716848,"statfs":{"total":95995035648,"available":95966052352,"internally_reserved":0,"allocated":1310720,"data_stored":833745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1515520,"data_stored":1034902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":909312,"data_stored":440020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37028,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708124,"statfs":{"total":95995035648,"available":95957118976,"internally_reserved":0,"allocated":1662976,"data_stored":1164780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":441863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215179,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1454080,"data_stored":965147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476816,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":503813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607637,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":501091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":929792,"data_stored":441291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":200176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:15:07.435 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:15:07.724 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 -- 172.21.15.94:0/3150863673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8106a80 msgr2=0x7f94f8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 --2- 172.21.15.94:0/3150863673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8106a80 0x7f94f8106e60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009930 tx=0x7f94f002a420 comp rx=0 tx=0).stop 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 -- 172.21.15.94:0/3150863673 shutdown_connections 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 --2- 172.21.15.94:0/3150863673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f8101420 0x7f94f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 --2- 172.21.15.94:0/3150863673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94f8100a80 0x7f94f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 --2- 172.21.15.94:0/3150863673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8106a80 0x7f94f8106e60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 -- 172.21.15.94:0/3150863673 >> 172.21.15.94:0/3150863673 conn(0x7f94f80fc7a0 msgr2=0x7f94f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 -- 172.21.15.94:0/3150863673 shutdown_connections 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 -- 172.21.15.94:0/3150863673 wait complete. 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 Processor -- start 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.202+0000 7f94ff678640 1 -- start start 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94ff678640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94f8100a80 0x7f94f8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94ff678640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8101420 0x7f94f8102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94ff678640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f8106a80 0x7f94f81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94ff678640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94f8110da0 con 0x7f94f8101420 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94ff678640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94f8110c20 con 0x7f94f8100a80 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94ff678640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94f8110f20 con 0x7f94f8106a80 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fcc13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8101420 0x7f94f8102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fcc13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8101420 0x7f94f8102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46622/0 (socket says 172.21.15.94:46622) 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fcc13640 1 -- 172.21.15.94:0/263805779 learned_addr learned my addr 172.21.15.94:0/263805779 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fd414640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94f8100a80 0x7f94f8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fdc15640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f8106a80 0x7f94f81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fcc13640 1 -- 172.21.15.94:0/263805779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f8106a80 msgr2=0x7f94f81030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fcc13640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f8106a80 0x7f94f81030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fcc13640 1 -- 172.21.15.94:0/263805779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94f8100a80 msgr2=0x7f94f8102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fcc13640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94f8100a80 0x7f94f8102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fcc13640 1 -- 172.21.15.94:0/263805779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f81a6860 con 0x7f94f8101420 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94fcc13640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8101420 0x7f94f8102b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b4a0 tx=0x7f94ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.203+0000 7f94f67fc640 1 -- 172.21.15.94:0/263805779 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec014070 con 0x7f94f8101420 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.204+0000 7f94f67fc640 1 -- 172.21.15.94:0/263805779 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94ec010ac0 con 0x7f94f8101420 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.204+0000 7f94f67fc640 1 -- 172.21.15.94:0/263805779 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec017c30 con 0x7f94f8101420 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.204+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f81a6b50 con 0x7f94f8101420 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.204+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f81a7090 con 0x7f94f8101420 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.204+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f8100ee0 con 0x7f94f8101420 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.206+0000 7f94f67fc640 1 -- 172.21.15.94:0/263805779 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f94ec017070 con 0x7f94f8101420 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.207+0000 7f94f67fc640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94e00a3440 0x7f94e00a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.207+0000 7f94fd414640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94e00a3440 0x7f94e00a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.207+0000 7f94f67fc640 1 -- 172.21.15.94:0/263805779 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f94ec0cf530 con 0x7f94f8101420 2024-09-19T04:15:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.207+0000 7f94fd414640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94e00a3440 0x7f94e00a5900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94f0031440 tx=0x7f94f0005a20 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.210+0000 7f94f67fc640 1 -- 172.21.15.94:0/263805779 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f94ec09be40 con 0x7f94f8101420 2024-09-19T04:15:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.410+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94f8103b80 con 0x7f94f8101420 2024-09-19T04:15:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.410+0000 7f94f67fc640 1 -- 172.21.15.94:0/263805779 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11915 (secure 0 0 0) 0x7f94ec09b740 con 0x7f94f8101420 2024-09-19T04:15:08.412 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:08.412 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"btime":"2024-09-19T04:15:03:346164+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:15:02.971431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24466,24467,24454]},"id":1}]} 2024-09-19T04:15:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-09-19T04:15:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.412+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94e00a3440 msgr2=0x7f94e00a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.412+0000 7f94ff678640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94e00a3440 0x7f94e00a5900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94f0031440 tx=0x7f94f0005a20 comp rx=0 tx=0).stop 2024-09-19T04:15:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.412+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8101420 msgr2=0x7f94f8102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.412+0000 7f94ff678640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8101420 0x7f94f8102b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b4a0 tx=0x7f94ec00b960 comp rx=0 tx=0).stop 2024-09-19T04:15:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.413+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 shutdown_connections 2024-09-19T04:15:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.413+0000 7f94ff678640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94e00a3440 0x7f94e00a5900 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.413+0000 7f94ff678640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f8106a80 0x7f94f81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.413+0000 7f94ff678640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94f8101420 0x7f94f8102b90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.413+0000 7f94ff678640 1 --2- 172.21.15.94:0/263805779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94f8100a80 0x7f94f8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.413+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 >> 172.21.15.94:0/263805779 conn(0x7f94f80fc7a0 msgr2=0x7f94f81048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.413+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 shutdown_connections 2024-09-19T04:15:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:08.413+0000 7f94ff678640 1 -- 172.21.15.94:0/263805779 wait complete. 2024-09-19T04:15:08.756 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:08 smithi094 ceph-mon[30972]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:15:08.757 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:08 smithi094 ceph-mon[30972]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:15:08.757 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:08 smithi094 ceph-mon[30972]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1.4 KiB/s wr, 31 op/s 2024-09-19T04:15:08.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:08 smithi112 ceph-mon[35051]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:15:08.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:08 smithi112 ceph-mon[35051]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:15:08.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:08 smithi112 ceph-mon[35051]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1.4 KiB/s wr, 31 op/s 2024-09-19T04:15:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:08 smithi104 ceph-mon[34145]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:15:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:08 smithi104 ceph-mon[34145]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:15:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:08 smithi104 ceph-mon[34145]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1.4 KiB/s wr, 31 op/s 2024-09-19T04:15:09.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:15:09.415 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:09.676 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:09 smithi094 ceph-mon[30972]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:15:09.676 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:09 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/263805779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:15:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:09 smithi112 ceph-mon[35051]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:15:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:09 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/263805779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:15:09.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:09 smithi104 ceph-mon[34145]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:15:09.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:09 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/263805779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:15:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.881+0000 7f83e0f19640 1 -- 172.21.15.94:0/4111681029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc102790 msgr2=0x7f83dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.881+0000 7f83e0f19640 1 --2- 172.21.15.94:0/4111681029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc102790 0x7f83dc1067e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009900 tx=0x7f83c402a410 comp rx=0 tx=0).stop 2024-09-19T04:15:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.881+0000 7f83e0f19640 1 -- 172.21.15.94:0/4111681029 shutdown_connections 2024-09-19T04:15:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.881+0000 7f83e0f19640 1 --2- 172.21.15.94:0/4111681029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc106f10 0x7f83dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.881+0000 7f83e0f19640 1 --2- 172.21.15.94:0/4111681029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc102790 0x7f83dc1067e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.881+0000 7f83e0f19640 1 --2- 172.21.15.94:0/4111681029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83dc101de0 0x7f83dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.881+0000 7f83e0f19640 1 -- 172.21.15.94:0/4111681029 >> 172.21.15.94:0/4111681029 conn(0x7f83dc0fd8c0 msgr2=0x7f83dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.881+0000 7f83e0f19640 1 -- 172.21.15.94:0/4111681029 shutdown_connections 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.881+0000 7f83e0f19640 1 -- 172.21.15.94:0/4111681029 wait complete. 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83e0f19640 1 Processor -- start 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83e0f19640 1 -- start start 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83e0f19640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83dc101de0 0x7f83dc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83e0f19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc102790 0x7f83dc198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83e0f19640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc106f10 0x7f83dc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83e0f19640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc111e60 con 0x7f83dc106f10 2024-09-19T04:15:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83e0f19640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc111ce0 con 0x7f83dc101de0 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83e0f19640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc111fe0 con 0x7f83dc102790 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83dad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc106f10 0x7f83dc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83dad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc106f10 0x7f83dc19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46632/0 (socket says 172.21.15.94:46632) 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.882+0000 7f83dad9d640 1 -- 172.21.15.94:0/3041007442 learned_addr learned my addr 172.21.15.94:0/3041007442 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83da59c640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83dc101de0 0x7f83dc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83d1d9b640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc102790 0x7f83dc198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83dad9d640 1 -- 172.21.15.94:0/3041007442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc102790 msgr2=0x7f83dc198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83dad9d640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc102790 0x7f83dc198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83dad9d640 1 -- 172.21.15.94:0/3041007442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83dc101de0 msgr2=0x7f83dc1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83dad9d640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83dc101de0 0x7f83dc1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83dad9d640 1 -- 172.21.15.94:0/3041007442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc19d390 con 0x7f83dc106f10 2024-09-19T04:15:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83da59c640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83dc101de0 0x7f83dc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:09.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83dad9d640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc106f10 0x7f83dc19ccb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f83d400eaa0 tx=0x7f83d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:09.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83d3fff640 1 -- 172.21.15.94:0/3041007442 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d400cdb0 con 0x7f83dc106f10 2024-09-19T04:15:09.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83d3fff640 1 -- 172.21.15.94:0/3041007442 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83d4005020 con 0x7f83dc106f10 2024-09-19T04:15:09.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc19d680 con 0x7f83dc106f10 2024-09-19T04:15:09.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83d3fff640 1 -- 172.21.15.94:0/3041007442 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d40116d0 con 0x7f83dc106f10 2024-09-19T04:15:09.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.883+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc1a4f60 con 0x7f83dc106f10 2024-09-19T04:15:09.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.884+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83dc1021c0 con 0x7f83dc106f10 2024-09-19T04:15:09.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.885+0000 7f83d3fff640 1 -- 172.21.15.94:0/3041007442 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f83d40051d0 con 0x7f83dc106f10 2024-09-19T04:15:09.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.886+0000 7f83d3fff640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83c80a3440 0x7f83c80a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:09.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.886+0000 7f83da59c640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83c80a3440 0x7f83c80a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:09.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.886+0000 7f83d3fff640 1 -- 172.21.15.94:0/3041007442 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f83d40d0150 con 0x7f83dc106f10 2024-09-19T04:15:09.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.887+0000 7f83da59c640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83c80a3440 0x7f83c80a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f83cc006fd0 tx=0x7f83cc008040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:09.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:09.891+0000 7f83d3fff640 1 -- 172.21.15.94:0/3041007442 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f83d4016030 con 0x7f83dc106f10 2024-09-19T04:15:10.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.058+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83dc109b60 con 0x7f83dc106f10 2024-09-19T04:15:10.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.060+0000 7f83d3fff640 1 -- 172.21.15.94:0/3041007442 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+23281 (secure 0 0 0) 0x7f83d4096e20 con 0x7f83dc106f10 2024-09-19T04:15:10.061 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:10.061 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":82,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:15:01.339120+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3284064221":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/860358556":"2024-09-20T04:15:01.339086+0000","172.21.15.94:6801/2308569937":"2024-09-20T04:15:01.339086+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/3663117113":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3123077951":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/1265443753":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2308569937":"2024-09-20T04:15:01.339086+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:15:10.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83c80a3440 msgr2=0x7f83c80a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:10.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83c80a3440 0x7f83c80a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f83cc006fd0 tx=0x7f83cc008040 comp rx=0 tx=0).stop 2024-09-19T04:15:10.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc106f10 msgr2=0x7f83dc19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:10.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc106f10 0x7f83dc19ccb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f83d400eaa0 tx=0x7f83d400ef60 comp rx=0 tx=0).stop 2024-09-19T04:15:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 shutdown_connections 2024-09-19T04:15:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83c80a3440 0x7f83c80a5900 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83dc106f10 0x7f83dc19ccb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc102790 0x7f83dc198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 --2- 172.21.15.94:0/3041007442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83dc101de0 0x7f83dc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 >> 172.21.15.94:0/3041007442 conn(0x7f83dc0fd8c0 msgr2=0x7f83dc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 shutdown_connections 2024-09-19T04:15:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.062+0000 7f83e0f19640 1 -- 172.21.15.94:0/3041007442 wait complete. 2024-09-19T04:15:10.686 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:10 smithi112 ceph-mon[35051]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1021 B/s wr, 22 op/s 2024-09-19T04:15:10.686 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:10 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3041007442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:15:10.744 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:10 smithi094 ceph-mon[30972]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1021 B/s wr, 22 op/s 2024-09-19T04:15:10.744 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:10 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3041007442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:15:10.775 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T04:15:10.775 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T04:15:10.784 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:15:10.784 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T04:15:10.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T04:15:10.788 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T04:15:10.788 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:15:10.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:15:10.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:10 smithi104 ceph-mon[34145]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1021 B/s wr, 22 op/s 2024-09-19T04:15:10.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:10 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3041007442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:15:10.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.891+0000 7f46c86f1640 1 -- 172.21.15.94:0/541896554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 msgr2=0x7f46c0106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:10.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.891+0000 7f46c86f1640 1 --2- 172.21.15.94:0/541896554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 0x7f46c0106da0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009930 tx=0x7f46bc02a420 comp rx=0 tx=0).stop 2024-09-19T04:15:10.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.891+0000 7f46c86f1640 1 -- 172.21.15.94:0/541896554 shutdown_connections 2024-09-19T04:15:10.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.891+0000 7f46c86f1640 1 --2- 172.21.15.94:0/541896554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c0101350 0x7f46c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.891+0000 7f46c86f1640 1 --2- 172.21.15.94:0/541896554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c01009b0 0x7f46c0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.891+0000 7f46c86f1640 1 --2- 172.21.15.94:0/541896554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 0x7f46c0106da0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.891+0000 7f46c86f1640 1 -- 172.21.15.94:0/541896554 >> 172.21.15.94:0/541896554 conn(0x7f46c0077990 msgr2=0x7f46c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.891+0000 7f46c86f1640 1 -- 172.21.15.94:0/541896554 shutdown_connections 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.891+0000 7f46c86f1640 1 -- 172.21.15.94:0/541896554 wait complete. 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c86f1640 1 Processor -- start 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c86f1640 1 -- start start 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c86f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c01009b0 0x7f46c0199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c86f1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c0101350 0x7f46c0199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c86f1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 0x7f46c01a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:10.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c86f1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0110cb0 con 0x7f46c01069c0 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c86f1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0110b30 con 0x7f46c0101350 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c86f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0110e30 con 0x7f46c01009b0 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c6c8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 0x7f46c01a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c6c8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 0x7f46c01a40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46644/0 (socket says 172.21.15.94:46644) 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c6c8e640 1 -- 172.21.15.94:0/3840021221 learned_addr learned my addr 172.21.15.94:0/3840021221 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.892+0000 7f46c5c8c640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c0101350 0x7f46c0199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c648d640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c01009b0 0x7f46c0199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c6c8e640 1 -- 172.21.15.94:0/3840021221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c01009b0 msgr2=0x7f46c0199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c6c8e640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c01009b0 0x7f46c0199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c6c8e640 1 -- 172.21.15.94:0/3840021221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c0101350 msgr2=0x7f46c0199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c6c8e640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c0101350 0x7f46c0199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c6c8e640 1 -- 172.21.15.94:0/3840021221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c01a4600 con 0x7f46c01069c0 2024-09-19T04:15:10.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c5c8c640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c0101350 0x7f46c0199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:10.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c6c8e640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 0x7f46c01a40c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f46b800cad0 tx=0x7f46b800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:10.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46b77fe640 1 -- 172.21.15.94:0/3840021221 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800ae20 con 0x7f46c01069c0 2024-09-19T04:15:10.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46b77fe640 1 -- 172.21.15.94:0/3840021221 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46b8005020 con 0x7f46c01069c0 2024-09-19T04:15:10.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c01a47d0 con 0x7f46c01069c0 2024-09-19T04:15:10.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46b77fe640 1 -- 172.21.15.94:0/3840021221 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800f6b0 con 0x7f46c01069c0 2024-09-19T04:15:10.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.893+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c01a4d40 con 0x7f46c01069c0 2024-09-19T04:15:10.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.894+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c0100e10 con 0x7f46c01069c0 2024-09-19T04:15:10.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.895+0000 7f46b77fe640 1 -- 172.21.15.94:0/3840021221 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f46b80051d0 con 0x7f46c01069c0 2024-09-19T04:15:10.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.896+0000 7f46b77fe640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46ac0a3580 0x7f46ac0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:10.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.896+0000 7f46c648d640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46ac0a3580 0x7f46ac0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:10.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.896+0000 7f46b77fe640 1 -- 172.21.15.94:0/3840021221 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f46b80d0250 con 0x7f46c01069c0 2024-09-19T04:15:10.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.896+0000 7f46c648d640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46ac0a3580 0x7f46ac0a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46bc031400 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:10.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:10.899+0000 7f46b77fe640 1 -- 172.21.15.94:0/3840021221 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f46b8014030 con 0x7f46c01069c0 2024-09-19T04:15:11.049 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:11.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.087+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f46c01a50e0 con 0x7f46c01069c0 2024-09-19T04:15:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.091+0000 7f46b77fe640 1 -- 172.21.15.94:0/3840021221 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f46b809cb30 con 0x7f46c01069c0 2024-09-19T04:15:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46ac0a3580 msgr2=0x7f46ac0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46ac0a3580 0x7f46ac0a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46bc031400 comp rx=0 tx=0).stop 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 msgr2=0x7f46c01a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 0x7f46c01a40c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f46b800cad0 tx=0x7f46b800cf90 comp rx=0 tx=0).stop 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 shutdown_connections 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46ac0a3580 0x7f46ac0a5a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c01069c0 0x7f46c01a40c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c0101350 0x7f46c0199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 --2- 172.21.15.94:0/3840021221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c01009b0 0x7f46c0199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 >> 172.21.15.94:0/3840021221 conn(0x7f46c0077990 msgr2=0x7f46c01037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 shutdown_connections 2024-09-19T04:15:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.093+0000 7f46c86f1640 1 -- 172.21.15.94:0/3840021221 wait complete. 2024-09-19T04:15:11.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T04:15:11.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.206+0000 7f825cfe4640 1 -- 172.21.15.94:0/1293033826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258107b70 msgr2=0x7f8258107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.206+0000 7f825cfe4640 1 --2- 172.21.15.94:0/1293033826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258107b70 0x7f8258107f50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8250009cd0 tx=0x7f825002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 -- 172.21.15.94:0/1293033826 shutdown_connections 2024-09-19T04:15:11.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 --2- 172.21.15.94:0/1293033826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8258102470 0x7f825810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 --2- 172.21.15.94:0/1293033826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258101ad0 0x7f8258101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 --2- 172.21.15.94:0/1293033826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258107b70 0x7f8258107f50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 -- 172.21.15.94:0/1293033826 >> 172.21.15.94:0/1293033826 conn(0x7f82580fd7d0 msgr2=0x7f82580ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 -- 172.21.15.94:0/1293033826 shutdown_connections 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 -- 172.21.15.94:0/1293033826 wait complete. 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 Processor -- start 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 -- start start 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101ad0 0x7f82581982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.207+0000 7f825cfe4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8258102470 0x7f8258198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825cfe4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258107b70 0x7f825819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825cfe4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8258111dd0 con 0x7f8258101ad0 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825cfe4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8258111c50 con 0x7f8258102470 2024-09-19T04:15:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825cfe4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8258111f50 con 0x7f8258107b70 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101ad0 0x7f82581982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101ad0 0x7f82581982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46662/0 (socket says 172.21.15.94:46662) 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825659c640 1 -- 172.21.15.94:0/890280755 learned_addr learned my addr 172.21.15.94:0/890280755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f8255d9b640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8258102470 0x7f8258198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f8256d9d640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258107b70 0x7f825819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825659c640 1 -- 172.21.15.94:0/890280755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258107b70 msgr2=0x7f825819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825659c640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258107b70 0x7f825819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825659c640 1 -- 172.21.15.94:0/890280755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8258102470 msgr2=0x7f8258198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825659c640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8258102470 0x7f8258198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825659c640 1 -- 172.21.15.94:0/890280755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825819d2b0 con 0x7f8258101ad0 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825659c640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101ad0 0x7f82581982c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f825002ae00 tx=0x7f8250003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f824f7fe640 1 -- 172.21.15.94:0/890280755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250004240 con 0x7f8258101ad0 2024-09-19T04:15:11.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f824f7fe640 1 -- 172.21.15.94:0/890280755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82500305a0 con 0x7f8258101ad0 2024-09-19T04:15:11.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825819d540 con 0x7f8258101ad0 2024-09-19T04:15:11.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f824f7fe640 1 -- 172.21.15.94:0/890280755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82500466e0 con 0x7f8258101ad0 2024-09-19T04:15:11.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.208+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82581a4e30 con 0x7f8258101ad0 2024-09-19T04:15:11.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.209+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8258101f30 con 0x7f8258101ad0 2024-09-19T04:15:11.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.210+0000 7f824f7fe640 1 -- 172.21.15.94:0/890280755 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f8250030750 con 0x7f8258101ad0 2024-09-19T04:15:11.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.211+0000 7f824f7fe640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f82440a3500 0x7f82440a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.211+0000 7f8255d9b640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f82440a3500 0x7f82440a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.211+0000 7f824f7fe640 1 -- 172.21.15.94:0/890280755 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f825002e030 con 0x7f8258101ad0 2024-09-19T04:15:11.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.212+0000 7f8255d9b640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f82440a3500 0x7f82440a59c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82581071b0 tx=0x7f824000b390 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.215+0000 7f824f7fe640 1 -- 172.21.15.94:0/890280755 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f825004b060 con 0x7f8258101ad0 2024-09-19T04:15:11.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.392+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f825810a040 con 0x7f8258101ad0 2024-09-19T04:15:11.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.395+0000 7f824f7fe640 1 -- 172.21.15.94:0/890280755 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f82500bde00 con 0x7f8258101ad0 2024-09-19T04:15:11.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f82440a3500 msgr2=0x7f82440a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f82440a3500 0x7f82440a59c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82581071b0 tx=0x7f824000b390 comp rx=0 tx=0).stop 2024-09-19T04:15:11.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101ad0 msgr2=0x7f82581982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101ad0 0x7f82581982c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f825002ae00 tx=0x7f8250003ba0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 shutdown_connections 2024-09-19T04:15:11.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f82440a3500 0x7f82440a59c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258107b70 0x7f825819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8258102470 0x7f8258198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 --2- 172.21.15.94:0/890280755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101ad0 0x7f82581982c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 >> 172.21.15.94:0/890280755 conn(0x7f82580fd7d0 msgr2=0x7f82580ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 shutdown_connections 2024-09-19T04:15:11.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.398+0000 7f825cfe4640 1 -- 172.21.15.94:0/890280755 wait complete. 2024-09-19T04:15:11.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T04:15:11.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 -- 172.21.15.94:0/2134199618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d010c0d0 msgr2=0x7fd0d010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 --2- 172.21.15.94:0/2134199618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d010c0d0 0x7fd0d010e550 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009ca0 tx=0x7fd0c802a950 comp rx=0 tx=0).stop 2024-09-19T04:15:11.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 -- 172.21.15.94:0/2134199618 shutdown_connections 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 --2- 172.21.15.94:0/2134199618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d010c0d0 0x7fd0d010e550 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 --2- 172.21.15.94:0/2134199618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0d01035a0 0x7fd0d010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 --2- 172.21.15.94:0/2134199618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0d0102c80 0x7fd0d0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 -- 172.21.15.94:0/2134199618 >> 172.21.15.94:0/2134199618 conn(0x7fd0d0077990 msgr2=0x7fd0d00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 -- 172.21.15.94:0/2134199618 shutdown_connections 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 -- 172.21.15.94:0/2134199618 wait complete. 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 Processor -- start 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 -- start start 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0102c80 0x7fd0d0198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0d01035a0 0x7fd0d0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0d010c0d0 0x7fd0d019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d0111db0 con 0x7fd0d0102c80 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d0111c30 con 0x7fd0d01035a0 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.515+0000 7fd0d721e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d0111f30 con 0x7fd0d010c0d0 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d4fba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0102c80 0x7fd0d0198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d4fba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0102c80 0x7fd0d0198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46676/0 (socket says 172.21.15.94:46676) 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d4fba640 1 -- 172.21.15.94:0/878175489 learned_addr learned my addr 172.21.15.94:0/878175489 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d57bb640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0d010c0d0 0x7fd0d019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0cffff640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0d01035a0 0x7fd0d0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d4fba640 1 -- 172.21.15.94:0/878175489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0d010c0d0 msgr2=0x7fd0d019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d4fba640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0d010c0d0 0x7fd0d019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d4fba640 1 -- 172.21.15.94:0/878175489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0d01035a0 msgr2=0x7fd0d0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d4fba640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0d01035a0 0x7fd0d0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d4fba640 1 -- 172.21.15.94:0/878175489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d019d210 con 0x7fd0d0102c80 2024-09-19T04:15:11.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d4fba640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0102c80 0x7fd0d0198220 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e9f0 tx=0x7fd0c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0cdffb640 1 -- 172.21.15.94:0/878175489 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400cd80 con 0x7fd0d0102c80 2024-09-19T04:15:11.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d019d500 con 0x7fd0d0102c80 2024-09-19T04:15:11.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0cdffb640 1 -- 172.21.15.94:0/878175489 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0c40054d0 con 0x7fd0d0102c80 2024-09-19T04:15:11.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.516+0000 7fd0cdffb640 1 -- 172.21.15.94:0/878175489 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4010490 con 0x7fd0d0102c80 2024-09-19T04:15:11.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.517+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d01a4d90 con 0x7fd0d0102c80 2024-09-19T04:15:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.517+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d0103060 con 0x7fd0d0102c80 2024-09-19T04:15:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.518+0000 7fd0cdffb640 1 -- 172.21.15.94:0/878175489 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7fd0c4005020 con 0x7fd0d0102c80 2024-09-19T04:15:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.519+0000 7fd0cdffb640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0c00a3500 0x7fd0c00a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.519+0000 7fd0cdffb640 1 -- 172.21.15.94:0/878175489 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fd0c40d0300 con 0x7fd0d0102c80 2024-09-19T04:15:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.519+0000 7fd0cffff640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0c00a3500 0x7fd0c00a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.520+0000 7fd0cffff640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0c00a3500 0x7fd0c00a59c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc005e00 tx=0x7fd0bc005c90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.523+0000 7fd0cdffb640 1 -- 172.21.15.94:0/878175489 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0c409cb60 con 0x7fd0d0102c80 2024-09-19T04:15:11.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.572+0000 7f667c38a640 1 -- 172.21.15.94:0/1274466874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674100100 msgr2=0x7f6674100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.572+0000 7f667c38a640 1 --2- 172.21.15.94:0/1274466874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674100100 0x7f6674100560 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6664009900 tx=0x7f666402a410 comp rx=0 tx=0).stop 2024-09-19T04:15:11.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.572+0000 7f667c38a640 1 -- 172.21.15.94:0/1274466874 shutdown_connections 2024-09-19T04:15:11.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.572+0000 7f667c38a640 1 --2- 172.21.15.94:0/1274466874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6674100aa0 0x7f667410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.572+0000 7f667c38a640 1 --2- 172.21.15.94:0/1274466874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674100100 0x7f6674100560 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.572+0000 7f667c38a640 1 --2- 172.21.15.94:0/1274466874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6674106b80 0x7f6674106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.572+0000 7f667c38a640 1 -- 172.21.15.94:0/1274466874 >> 172.21.15.94:0/1274466874 conn(0x7f6674077230 msgr2=0x7f66740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.573+0000 7f667c38a640 1 -- 172.21.15.94:0/1274466874 shutdown_connections 2024-09-19T04:15:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.573+0000 7f667c38a640 1 -- 172.21.15.94:0/1274466874 wait complete. 2024-09-19T04:15:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667c38a640 1 Processor -- start 2024-09-19T04:15:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667c38a640 1 -- start start 2024-09-19T04:15:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667c38a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6674100100 0x7f6674198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667c38a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6674100aa0 0x7f6674198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667c38a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674106b80 0x7f667419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667c38a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6674111f00 con 0x7f6674106b80 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667c38a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6674111d80 con 0x7f6674100aa0 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667c38a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6674112080 con 0x7f6674100100 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a927640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674106b80 0x7f667419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a927640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674106b80 0x7f667419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46702/0 (socket says 172.21.15.94:46702) 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a927640 1 -- 172.21.15.94:0/858163659 learned_addr learned my addr 172.21.15.94:0/858163659 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a126640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6674100100 0x7f6674198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f6679925640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6674100aa0 0x7f6674198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a126640 1 -- 172.21.15.94:0/858163659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6674100aa0 msgr2=0x7f6674198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a126640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6674100aa0 0x7f6674198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a126640 1 -- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674106b80 msgr2=0x7f667419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a126640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674106b80 0x7f667419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a126640 1 -- 172.21.15.94:0/858163659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667419d3d0 con 0x7f6674100100 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.574+0000 7f667a927640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674106b80 0x7f667419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.575+0000 7f667a126640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6674100100 0x7f6674198420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f667000e9f0 tx=0x7f667000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.575+0000 7f666b7fe640 1 -- 172.21.15.94:0/858163659 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667000ccc0 con 0x7f6674100100 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.575+0000 7f667c38a640 1 -- 172.21.15.94:0/858163659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667419d6c0 con 0x7f6674100100 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.575+0000 7f666b7fe640 1 -- 172.21.15.94:0/858163659 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6670005020 con 0x7f6674100100 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.575+0000 7f667c38a640 1 -- 172.21.15.94:0/858163659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66741a4fa0 con 0x7f6674100100 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.576+0000 7f666b7fe640 1 -- 172.21.15.94:0/858163659 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670004a00 con 0x7f6674100100 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.577+0000 7f667e763740 1 Processor -- start 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667e763740 1 -- start start 2024-09-19T04:15:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f666b7fe640 1 -- 172.21.15.94:0/858163659 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f6670004ba0 con 0x7f6674100100 2024-09-19T04:15:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667e763740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a8a60 0x5578ee9a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667e763740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a9380 0x5578ee9bc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667e763740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9af7c0 0x5578ee9be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667e763740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5578ee90e280 con 0x5578ee9a9380 2024-09-19T04:15:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667e763740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5578ee8b5b10 con 0x5578ee9af7c0 2024-09-19T04:15:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667e763740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578ee90a240 con 0x5578ee9a8a60 2024-09-19T04:15:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a126640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a9380 0x5578ee9bc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a126640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a9380 0x5578ee9bc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46704/0 (socket says 172.21.15.94:46704) 2024-09-19T04:15:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a126640 1 -- 172.21.15.94:0/3129491464 learned_addr learned my addr 172.21.15.94:0/3129491464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a927640 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a8a60 0x5578ee9a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f6679925640 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9af7c0 0x5578ee9be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a126640 1 -- 172.21.15.94:0/3129491464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a8a60 msgr2=0x5578ee9a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a126640 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a8a60 0x5578ee9a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a126640 1 -- 172.21.15.94:0/3129491464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9af7c0 msgr2=0x5578ee9be950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a126640 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9af7c0 0x5578ee9be950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a126640 1 -- 172.21.15.94:0/3129491464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ee8c21f0 con 0x5578ee9a9380 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f667a927640 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a8a60 0x5578ee9a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.578+0000 7f6679925640 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9af7c0 0x5578ee9be950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.579+0000 7f667a126640 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a9380 0x5578ee9bc400 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f66700cd440 tx=0x7f66700d80e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abe7f09d6148ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.579+0000 7f6668dea640 1 -- 172.21.15.94:0/3129491464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66700e3070 con 0x5578ee9a9380 2024-09-19T04:15:11.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.579+0000 7f666b7fe640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f66600a3440 0x7f66600a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.579+0000 7f6668dea640 1 -- 172.21.15.94:0/3129491464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66700a0360 con 0x5578ee9a9380 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.579+0000 7f667a927640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f66600a3440 0x7f66600a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.579+0000 7f666b7fe640 1 -- 172.21.15.94:0/858163659 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f66700cfeb0 con 0x7f6674100100 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.579+0000 7f6668dea640 1 -- 172.21.15.94:0/3129491464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667009f5e0 con 0x5578ee9a9380 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 -- 172.21.15.94:0/3129491464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a9380 msgr2=0x5578ee9bc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667a927640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f66600a3440 0x7f66600a5900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f666c010b70 tx=0x7f666c009740 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a9380 0x5578ee9bc400 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f66700cd440 tx=0x7f66700d80e0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 -- 172.21.15.94:0/3129491464 shutdown_connections 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9af7c0 0x5578ee9be950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a9380 0x5578ee9bc400 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 --2- 172.21.15.94:0/3129491464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a8a60 0x5578ee9a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 -- 172.21.15.94:0/3129491464 >> 172.21.15.94:0/3129491464 conn(0x5578ee9a50f0 msgr2=0x5578ee9a54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 -- 172.21.15.94:0/3129491464 shutdown_connections 2024-09-19T04:15:11.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 -- 172.21.15.94:0/3129491464 wait complete. 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 -- start start 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.580+0000 7f667e763740 1 Processor -- start 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667e763740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667e763740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 0x5578ee9b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667e763740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667e763740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5578ee9bf1e0 con 0x5578ee9a8a60 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667e763740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5578ee8c8f10 con 0x5578ee9a9380 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667e763740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578ee9bf360 con 0x5578ee9b15b0 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667e763740 10 client.0 _ll_register_callbacks cb 0x5578ee891a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a126640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a927640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a126640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46712/0 (socket says 172.21.15.94:46712) 2024-09-19T04:15:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a126640 1 -- 172.21.15.94:0/4021466083 learned_addr learned my addr 172.21.15.94:0/4021466083 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f6679925640 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 0x5578ee9b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a927640 1 -- 172.21.15.94:0/4021466083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 msgr2=0x5578ee9b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a927640 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 0x5578ee9b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a927640 1 -- 172.21.15.94:0/4021466083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 msgr2=0x5578ee9b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a927640 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a927640 1 -- 172.21.15.94:0/4021466083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ee67f6a0 con 0x5578ee9b15b0 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f667a927640 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c8690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f666c011ca0 tx=0x7f666c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c128c6f6339cbbbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.581+0000 7f665dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f665dffb640 1 -- 172.21.15.94:0/4021466083 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c02c060 con 0x5578ee9b15b0 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f667e763740 1 -- 172.21.15.94:0/4021466083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66700a0360 con 0x5578ee9b15b0 2024-09-19T04:15:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f665dffb640 1 -- 172.21.15.94:0/4021466083 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f666c017030 con 0x5578ee9b15b0 2024-09-19T04:15:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f665dffb640 1 -- 172.21.15.94:0/4021466083 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f666c030420 con 0x5578ee9b15b0 2024-09-19T04:15:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f667e763740 10 client.24904 fetch_fsmap learned FSMap version 25 2024-09-19T04:15:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f667e763740 1 -- 172.21.15.94:0/4021466083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578ee9cb300 con 0x5578ee9b15b0 2024-09-19T04:15:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f665dffb640 1 -- 172.21.15.94:0/4021466083 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66700a0360 con 0x5578ee9b15b0 2024-09-19T04:15:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f665dffb640 1 -- 172.21.15.94:0/4021466083 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f666c01c3f0 con 0x5578ee9b15b0 2024-09-19T04:15:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f667e763740 10 client.24904 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T04:15:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f667e763740 10 client.24904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f667e763740 20 client.24904 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f667e763740 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5578ee9cc710 0x5578ee9ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f667e763740 4 client.24904 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.582+0000 7f667e763740 1 -- 172.21.15.94:0/4021466083 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578ee91b7f0 con 0x5578ee9cc710 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.583+0000 7f667a126640 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5578ee9cc710 0x5578ee9ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.583+0000 7f665dffb640 10 client.24904 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.583+0000 7f667a126640 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5578ee9cc710 0x5578ee9ceb00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d1efd6706a1b6c4 server_cookie=7487022b4524a2d9 in_seq=0 out_seq=0 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f665dffb640 1 -- 172.21.15.94:0/4021466083 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578ee91b7f0 con 0x5578ee9cc710 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f665dffb640 10 client.24904 handle_command_reply: tid=0 2024-09-19T04:15:11.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 client.24904 shutdown 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 -- 172.21.15.94:0/4021466083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 msgr2=0x5578ee9c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c8690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f666c011ca0 tx=0x7f666c004ab0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 -- 172.21.15.94:0/4021466083 shutdown_connections 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5578ee9cc710 0x5578ee9ceb00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c8690 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f665dffb640 0 client.24904 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 0x5578ee9b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 --2- 172.21.15.94:0/4021466083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 -- 172.21.15.94:0/4021466083 >> 172.21.15.94:0/4021466083 conn(0x5578ee9a50f0 msgr2=0x5578ee9aadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 -- 172.21.15.94:0/4021466083 shutdown_connections 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 1 -- 172.21.15.94:0/4021466083 wait complete. 2024-09-19T04:15:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.584+0000 7f667e763740 20 client.24904 trim_cache size 0 max 16384 2024-09-19T04:15:11.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667e763740 1 Processor -- start 2024-09-19T04:15:11.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667e763740 1 -- start start 2024-09-19T04:15:11.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667e763740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a8a60 0x5578ee9c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667e763740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a9380 0x5578ee9c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667e763740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9b15b0 0x5578ee9d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667e763740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66741adea0 con 0x5578ee9b15b0 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667e763740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ab770 con 0x5578ee9a8a60 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667e763740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66741a95c0 con 0x5578ee9a9380 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667a927640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9b15b0 0x5578ee9d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667a927640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9b15b0 0x5578ee9d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46726/0 (socket says 172.21.15.94:46726) 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667a927640 1 -- 172.21.15.94:0/2308109566 learned_addr learned my addr 172.21.15.94:0/2308109566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f6679925640 1 --2- 172.21.15.94:0/2308109566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a9380 0x5578ee9c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667a126640 1 --2- 172.21.15.94:0/2308109566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a8a60 0x5578ee9c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667a927640 1 -- 172.21.15.94:0/2308109566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a9380 msgr2=0x5578ee9c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667a927640 1 --2- 172.21.15.94:0/2308109566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a9380 0x5578ee9c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667a927640 1 -- 172.21.15.94:0/2308109566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a8a60 msgr2=0x5578ee9c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667a927640 1 --2- 172.21.15.94:0/2308109566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a8a60 0x5578ee9c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.602+0000 7f667a927640 1 -- 172.21.15.94:0/2308109566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ee9a0830 con 0x5578ee9b15b0 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.603+0000 7f667a927640 1 --2- 172.21.15.94:0/2308109566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9b15b0 0x5578ee9d4380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f666c000f80 tx=0x7f666c011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91c47bcdae5f33c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.603+0000 7f665ffff640 1 -- 172.21.15.94:0/2308109566 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c01c5f0 con 0x5578ee9b15b0 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.603+0000 7f665ffff640 1 -- 172.21.15.94:0/2308109566 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f666c01ce50 con 0x5578ee9b15b0 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.603+0000 7f665ffff640 1 -- 172.21.15.94:0/2308109566 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c027410 con 0x5578ee9b15b0 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 -- 172.21.15.94:0/2308109566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9b15b0 msgr2=0x5578ee9d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 --2- 172.21.15.94:0/2308109566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9b15b0 0x5578ee9d4380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f666c000f80 tx=0x7f666c011ee0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 -- 172.21.15.94:0/2308109566 shutdown_connections 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 --2- 172.21.15.94:0/2308109566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9b15b0 0x5578ee9d4380 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 --2- 172.21.15.94:0/2308109566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9a9380 0x5578ee9c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 --2- 172.21.15.94:0/2308109566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a8a60 0x5578ee9c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 -- 172.21.15.94:0/2308109566 >> 172.21.15.94:0/2308109566 conn(0x5578ee9a50f0 msgr2=0x5578ee9b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 -- 172.21.15.94:0/2308109566 shutdown_connections 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 -- 172.21.15.94:0/2308109566 wait complete. 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 -- start start 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.604+0000 7f667e763740 1 Processor -- start 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667e763740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667e763740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 0x5578ee9c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667e763740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667e763740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66741a9c30 con 0x5578ee9a8a60 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667e763740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ab0a0 con 0x5578ee9a9380 2024-09-19T04:15:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a126640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667e763740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66741aa300 con 0x5578ee9b15b0 2024-09-19T04:15:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a126640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9c0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46730/0 (socket says 172.21.15.94:46730) 2024-09-19T04:15:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a126640 1 -- 172.21.15.94:0/3699717229 learned_addr learned my addr 172.21.15.94:0/3699717229 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667e763740 10 client.0 _ll_register_callbacks cb 0x7f666c011b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a927640 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f6679925640 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 0x5578ee9c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a126640 1 -- 172.21.15.94:0/3699717229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 msgr2=0x5578ee9c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a126640 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a126640 1 -- 172.21.15.94:0/3699717229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 msgr2=0x5578ee9c0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a126640 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 0x5578ee9c0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a126640 1 -- 172.21.15.94:0/3699717229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578eea365a0 con 0x5578ee9a8a60 2024-09-19T04:15:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a927640 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f667a126640 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9c0590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6670098410 tx=0x7f6670009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d29694c9f0dbba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.605+0000 7f665dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f665dffb640 1 -- 172.21.15.94:0/3699717229 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667009e550 con 0x5578ee9a8a60 2024-09-19T04:15:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f665dffb640 1 -- 172.21.15.94:0/3699717229 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f6670005560 con 0x5578ee9a8a60 2024-09-19T04:15:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f667e763740 1 -- 172.21.15.94:0/3699717229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f666c01ce50 con 0x5578ee9a8a60 2024-09-19T04:15:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f665dffb640 1 -- 172.21.15.94:0/3699717229 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f667009e7c0 con 0x5578ee9a8a60 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f667e763740 10 client.15315 fetch_fsmap learned FSMap version 25 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f667e763740 1 -- 172.21.15.94:0/3699717229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578eea378c0 con 0x5578ee9a8a60 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f665dffb640 1 -- 172.21.15.94:0/3699717229 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f666c01ce50 con 0x5578ee9a8a60 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f665dffb640 1 -- 172.21.15.94:0/3699717229 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f66700cdce0 con 0x5578ee9a8a60 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f667e763740 10 client.15315 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f667e763740 10 client.15315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f667e763740 20 client.15315 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f667e763740 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5578eea37bb0 0x5578eea39fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f667e763740 4 client.15315 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.606+0000 7f667e763740 1 -- 172.21.15.94:0/3699717229 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5578ee915d00 con 0x5578eea37bb0 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.607+0000 7f6679925640 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5578eea37bb0 0x5578eea39fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.607+0000 7f6679925640 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5578eea37bb0 0x5578eea39fa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecfa64d621fe6d13 server_cookie=22d6390031182108 in_seq=0 out_seq=0 2024-09-19T04:15:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.607+0000 7f665dffb640 10 client.15315 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f665dffb640 1 -- 172.21.15.94:0/3699717229 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5578ee915d00 con 0x5578eea37bb0 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f665dffb640 10 client.15315 handle_command_reply: tid=0 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 client.15315 shutdown 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 -- 172.21.15.94:0/3699717229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 msgr2=0x5578ee9c0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9c0590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6670098410 tx=0x7f6670009f60 comp rx=0 tx=0).stop 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 -- 172.21.15.94:0/3699717229 shutdown_connections 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5578eea37bb0 0x5578eea39fa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f665dffb640 0 client.15315 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578ee9b15b0 0x5578ee9c1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578ee9a9380 0x5578ee9c0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 --2- 172.21.15.94:0/3699717229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578ee9a8a60 0x5578ee9c0590 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 -- 172.21.15.94:0/3699717229 >> 172.21.15.94:0/3699717229 conn(0x5578ee9a50f0 msgr2=0x5578ee9aa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.608+0000 7f667e763740 1 -- 172.21.15.94:0/3699717229 shutdown_connections 2024-09-19T04:15:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.609+0000 7f667e763740 1 -- 172.21.15.94:0/3699717229 wait complete. 2024-09-19T04:15:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.609+0000 7f667e763740 20 client.15315 trim_cache size 0 max 16384 2024-09-19T04:15:11.619 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:15:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:15:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "2059d731-06cf-4387-bc55-2e66f44dff78", 2024-09-19T04:15:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:15:11.620 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:15:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.609+0000 7f667c38a640 1 -- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f66600a3440 msgr2=0x7f66600a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.609+0000 7f667c38a640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f66600a3440 0x7f66600a5900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f666c010b70 tx=0x7f666c009740 comp rx=0 tx=0).stop 2024-09-19T04:15:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.609+0000 7f667c38a640 1 -- 172.21.15.94:0/858163659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6674100100 msgr2=0x7f6674198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.609+0000 7f667c38a640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6674100100 0x7f6674198420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f667000e9f0 tx=0x7f667000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.610+0000 7f667c38a640 1 -- 172.21.15.94:0/858163659 shutdown_connections 2024-09-19T04:15:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.610+0000 7f667c38a640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f66600a3440 0x7f66600a5900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.610+0000 7f667c38a640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6674106b80 0x7f667419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.610+0000 7f667c38a640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6674100aa0 0x7f6674198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.610+0000 7f667c38a640 1 --2- 172.21.15.94:0/858163659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6674100100 0x7f6674198420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.610+0000 7f667c38a640 1 -- 172.21.15.94:0/858163659 >> 172.21.15.94:0/858163659 conn(0x7f6674077230 msgr2=0x7f6674104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.610+0000 7f667c38a640 1 -- 172.21.15.94:0/858163659 shutdown_connections 2024-09-19T04:15:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.610+0000 7f667c38a640 1 -- 172.21.15.94:0/858163659 wait complete. 2024-09-19T04:15:11.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.720+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd0d01a5090 con 0x7fd0d0102c80 2024-09-19T04:15:11.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.725+0000 7fd0cdffb640 1 -- 172.21.15.94:0/878175489 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7fd0c409c460 con 0x7fd0d0102c80 2024-09-19T04:15:11.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0c00a3500 msgr2=0x7fd0c00a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0c00a3500 0x7fd0c00a59c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc005e00 tx=0x7fd0bc005c90 comp rx=0 tx=0).stop 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0102c80 msgr2=0x7fd0d0198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0102c80 0x7fd0d0198220 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e9f0 tx=0x7fd0c400edf0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 shutdown_connections 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0c00a3500 0x7fd0c00a59c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0d010c0d0 0x7fd0d019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0d01035a0 0x7fd0d0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 --2- 172.21.15.94:0/878175489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0102c80 0x7fd0d0198220 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 >> 172.21.15.94:0/878175489 conn(0x7fd0d0077990 msgr2=0x7fd0d00fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 shutdown_connections 2024-09-19T04:15:11.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.727+0000 7fd0d721e640 1 -- 172.21.15.94:0/878175489 wait complete. 2024-09-19T04:15:11.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T04:15:11.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 -- 172.21.15.94:0/1549351761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 msgr2=0x7f801010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 --2- 172.21.15.94:0/1549351761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 0x7f801010d880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8008009cd0 tx=0x7f800802a8e0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 -- 172.21.15.94:0/1549351761 shutdown_connections 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 --2- 172.21.15.94:0/1549351761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 0x7f801010d880 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 --2- 172.21.15.94:0/1549351761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80101009b0 0x7f8010100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 --2- 172.21.15.94:0/1549351761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80101069c0 0x7f8010106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 -- 172.21.15.94:0/1549351761 >> 172.21.15.94:0/1549351761 conn(0x7f8010077990 msgr2=0x7f80100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 -- 172.21.15.94:0/1549351761 shutdown_connections 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 -- 172.21.15.94:0/1549351761 wait complete. 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.845+0000 7f80170d0640 1 Processor -- start 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f80170d0640 1 -- start start 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f80170d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80101009b0 0x7f8010197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f80170d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 0x7f8010197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f80170d0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80101069c0 0x7f801019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f80170d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8010110cb0 con 0x7f8010101350 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f80170d0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8010110b30 con 0x7f80101069c0 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f80170d0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8010110e30 con 0x7f80101009b0 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 0x7f8010197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 0x7f8010197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46736/0 (socket says 172.21.15.94:46736) 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800ffff640 1 -- 172.21.15.94:0/2785353186 learned_addr learned my addr 172.21.15.94:0/2785353186 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f8014e6c640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80101009b0 0x7f8010197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f801566d640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80101069c0 0x7f801019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800ffff640 1 -- 172.21.15.94:0/2785353186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80101009b0 msgr2=0x7f8010197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800ffff640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80101009b0 0x7f8010197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800ffff640 1 -- 172.21.15.94:0/2785353186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80101069c0 msgr2=0x7f801019b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800ffff640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80101069c0 0x7f801019b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800ffff640 1 -- 172.21.15.94:0/2785353186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801019c110 con 0x7f8010101350 2024-09-19T04:15:11.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f801566d640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80101069c0 0x7f801019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800ffff640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 0x7f8010197660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f800400d950 tx=0x7f800400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.846+0000 7f800dffb640 1 -- 172.21.15.94:0/2785353186 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004017070 con 0x7f8010101350 2024-09-19T04:15:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.847+0000 7f800dffb640 1 -- 172.21.15.94:0/2785353186 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8004009c30 con 0x7f8010101350 2024-09-19T04:15:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.847+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801019c400 con 0x7f8010101350 2024-09-19T04:15:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.847+0000 7f800dffb640 1 -- 172.21.15.94:0/2785353186 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800400f620 con 0x7f8010101350 2024-09-19T04:15:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.847+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80100698a0 con 0x7f8010101350 2024-09-19T04:15:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.848+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8010100e10 con 0x7f8010101350 2024-09-19T04:15:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.849+0000 7f800dffb640 1 -- 172.21.15.94:0/2785353186 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f80040109a0 con 0x7f8010101350 2024-09-19T04:15:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.849+0000 7f800dffb640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7ff80a3500 0x7f7ff80a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.849+0000 7f800dffb640 1 -- 172.21.15.94:0/2785353186 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f80040cfb20 con 0x7f8010101350 2024-09-19T04:15:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.849+0000 7f8014e6c640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7ff80a3500 0x7f7ff80a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.850+0000 7f8014e6c640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7ff80a3500 0x7f7ff80a59c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8000009b50 tx=0x7f8000010040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:11.855+0000 7f800dffb640 1 -- 172.21.15.94:0/2785353186 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8004009de0 con 0x7f8010101350 2024-09-19T04:15:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.050+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 --> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8010103f10 con 0x7f7ff80a3500 2024-09-19T04:15:12.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.116+0000 7f800dffb640 1 -- 172.21.15.94:0/2785353186 <== mgr.15159 v2:172.21.15.94:6800/3395191155 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8010103f10 con 0x7f7ff80a3500 2024-09-19T04:15:12.117 INFO:teuthology.orchestra.run.smithi094.stdout:Schedule set for path / 2024-09-19T04:15:12.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7ff80a3500 msgr2=0x7f7ff80a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7ff80a3500 0x7f7ff80a59c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8000009b50 tx=0x7f8000010040 comp rx=0 tx=0).stop 2024-09-19T04:15:12.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 msgr2=0x7f8010197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 0x7f8010197660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f800400d950 tx=0x7f800400de10 comp rx=0 tx=0).stop 2024-09-19T04:15:12.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 shutdown_connections 2024-09-19T04:15:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7ff80a3500 0x7f7ff80a59c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80101069c0 0x7f801019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8010101350 0x7f8010197660 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 --2- 172.21.15.94:0/2785353186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80101009b0 0x7f8010197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 >> 172.21.15.94:0/2785353186 conn(0x7f8010077990 msgr2=0x7f8010103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 shutdown_connections 2024-09-19T04:15:12.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.118+0000 7f80170d0640 1 -- 172.21.15.94:0/2785353186 wait complete. 2024-09-19T04:15:12.130 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T04:15:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.234+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/1582894570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48101390 msgr2=0x7f1e4810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.234+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/1582894570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48101390 0x7f1e4810d8c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009cd0 tx=0x7f1e4002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.234+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/1582894570 shutdown_connections 2024-09-19T04:15:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.234+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/1582894570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48101390 0x7f1e4810d8c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.234+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/1582894570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481009f0 0x7f1e48100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.234+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/1582894570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e48106a00 0x7f1e48106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.234+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/1582894570 >> 172.21.15.94:0/1582894570 conn(0x7f1e480fc6b0 msgr2=0x7f1e480fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.234+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/1582894570 shutdown_connections 2024-09-19T04:15:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.235+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/1582894570 wait complete. 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.235+0000 7f1e4cdd6640 1 Processor -- start 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.235+0000 7f1e4cdd6640 1 -- start start 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.235+0000 7f1e4cdd6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481009f0 0x7f1e481990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.235+0000 7f1e4cdd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e48101390 0x7f1e481995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.235+0000 7f1e4cdd6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48106a00 0x7f1e481a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.235+0000 7f1e4cdd6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48110cf0 con 0x7f1e48106a00 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.235+0000 7f1e4cdd6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48110b70 con 0x7f1e481009f0 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.235+0000 7f1e4cdd6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48110e70 con 0x7f1e48101390 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e46d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48106a00 0x7f1e481a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e46d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48106a00 0x7f1e481a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46404/0 (socket says 172.21.15.94:46404) 2024-09-19T04:15:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e46d9d640 1 -- 172.21.15.94:0/2216174443 learned_addr learned my addr 172.21.15.94:0/2216174443 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e4659c640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481009f0 0x7f1e481990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e45d9b640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e48101390 0x7f1e481995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e46d9d640 1 -- 172.21.15.94:0/2216174443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e48101390 msgr2=0x7f1e481995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e46d9d640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e48101390 0x7f1e481995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e46d9d640 1 -- 172.21.15.94:0/2216174443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481009f0 msgr2=0x7f1e481990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e46d9d640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481009f0 0x7f1e481990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e46d9d640 1 -- 172.21.15.94:0/2216174443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e481a4660 con 0x7f1e48106a00 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e4659c640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481009f0 0x7f1e481990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e46d9d640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48106a00 0x7f1e481a4120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005cc0 tx=0x7f1e40003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e3f7fe640 1 -- 172.21.15.94:0/2216174443 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40004240 con 0x7f1e48106a00 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e481a4830 con 0x7f1e48106a00 2024-09-19T04:15:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e3f7fe640 1 -- 172.21.15.94:0/2216174443 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e400305a0 con 0x7f1e48106a00 2024-09-19T04:15:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e3f7fe640 1 -- 172.21.15.94:0/2216174443 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400466c0 con 0x7f1e48106a00 2024-09-19T04:15:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.236+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481a4cf0 con 0x7f1e48106a00 2024-09-19T04:15:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.237+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e48100e50 con 0x7f1e48106a00 2024-09-19T04:15:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.238+0000 7f1e3f7fe640 1 -- 172.21.15.94:0/2216174443 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f1e40030ac0 con 0x7f1e48106a00 2024-09-19T04:15:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.239+0000 7f1e3f7fe640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e340a3500 0x7f1e340a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.239+0000 7f1e4659c640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e340a3500 0x7f1e340a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.239+0000 7f1e3f7fe640 1 -- 172.21.15.94:0/2216174443 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f1e4002e030 con 0x7f1e48106a00 2024-09-19T04:15:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.240+0000 7f1e4659c640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e340a3500 0x7f1e340a59c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009930 tx=0x7f1e38008040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.242+0000 7f1e3f7fe640 1 -- 172.21.15.94:0/2216174443 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e4004b060 con 0x7f1e48106a00 2024-09-19T04:15:12.272 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3840021221' entity='client.admin' 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 794 B/s wr, 17 op/s 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/890280755' entity='client.admin' 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:15:12.273 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:12.274 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:15:12.274 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:12.274 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/878175489' entity='client.admin' 2024-09-19T04:15:12.274 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:12.274 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:12.274 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:12.274 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:12.299 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:15:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3840021221' entity='client.admin' 2024-09-19T04:15:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 794 B/s wr, 17 op/s 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/890280755' entity='client.admin' 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:12.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/878175489' entity='client.admin' 2024-09-19T04:15:12.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:12.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:12.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:12.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:12.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.440+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 --> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f1e480fd400 con 0x7f1e340a3500 2024-09-19T04:15:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.502+0000 7f1e3f7fe640 1 -- 172.21.15.94:0/2216174443 <== mgr.15159 v2:172.21.15.94:6800/3395191155 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f1e480fd400 con 0x7f1e340a3500 2024-09-19T04:15:12.503 INFO:teuthology.orchestra.run.smithi094.stdout:Retention added to path / 2024-09-19T04:15:12.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e340a3500 msgr2=0x7f1e340a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e340a3500 0x7f1e340a59c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009930 tx=0x7f1e38008040 comp rx=0 tx=0).stop 2024-09-19T04:15:12.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48106a00 msgr2=0x7f1e481a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48106a00 0x7f1e481a4120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005cc0 tx=0x7f1e40003ba0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 shutdown_connections 2024-09-19T04:15:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e340a3500 0x7f1e340a59c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e48106a00 0x7f1e481a4120 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e48101390 0x7f1e481995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 --2- 172.21.15.94:0/2216174443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481009f0 0x7f1e481990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 >> 172.21.15.94:0/2216174443 conn(0x7f1e480fc6b0 msgr2=0x7f1e480fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 shutdown_connections 2024-09-19T04:15:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.504+0000 7f1e4cdd6640 1 -- 172.21.15.94:0/2216174443 wait complete. 2024-09-19T04:15:12.520 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T04:15:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3840021221' entity='client.admin' 2024-09-19T04:15:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 794 B/s wr, 17 op/s 2024-09-19T04:15:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/890280755' entity='client.admin' 2024-09-19T04:15:12.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/878175489' entity='client.admin' 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:12.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:15:12.573 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:12.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 -- 172.21.15.94:0/2771048432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1069c0 msgr2=0x7f64ac106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 --2- 172.21.15.94:0/2771048432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1069c0 0x7f64ac106da0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f649c009930 tx=0x7f649c02a420 comp rx=0 tx=0).stop 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 -- 172.21.15.94:0/2771048432 shutdown_connections 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 --2- 172.21.15.94:0/2771048432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64ac101350 0x7f64ac10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 --2- 172.21.15.94:0/2771048432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ac1009b0 0x7f64ac100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 --2- 172.21.15.94:0/2771048432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1069c0 0x7f64ac106da0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 -- 172.21.15.94:0/2771048432 >> 172.21.15.94:0/2771048432 conn(0x7f64ac077990 msgr2=0x7f64ac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 -- 172.21.15.94:0/2771048432 shutdown_connections 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 -- 172.21.15.94:0/2771048432 wait complete. 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.632+0000 7f64b3482640 1 Processor -- start 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b3482640 1 -- start start 2024-09-19T04:15:12.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b3482640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1009b0 0x7f64ac199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b3482640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ac101350 0x7f64ac1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b3482640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64ac1069c0 0x7f64ac1a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b3482640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac110cb0 con 0x7f64ac1009b0 2024-09-19T04:15:12.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b3482640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac110b30 con 0x7f64ac1069c0 2024-09-19T04:15:12.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b3482640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac110e30 con 0x7f64ac101350 2024-09-19T04:15:12.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b121e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1009b0 0x7f64ac199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b121e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1009b0 0x7f64ac199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46424/0 (socket says 172.21.15.94:46424) 2024-09-19T04:15:12.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b121e640 1 -- 172.21.15.94:0/2126361792 learned_addr learned my addr 172.21.15.94:0/2126361792 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:12.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b1a1f640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64ac1069c0 0x7f64ac1a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b0a1d640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ac101350 0x7f64ac1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b121e640 1 -- 172.21.15.94:0/2126361792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ac101350 msgr2=0x7f64ac1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b121e640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ac101350 0x7f64ac1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b121e640 1 -- 172.21.15.94:0/2126361792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64ac1069c0 msgr2=0x7f64ac1a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b121e640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64ac1069c0 0x7f64ac1a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b121e640 1 -- 172.21.15.94:0/2126361792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac1a4730 con 0x7f64ac1009b0 2024-09-19T04:15:12.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b1a1f640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64ac1069c0 0x7f64ac1a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:12.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64b121e640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1009b0 0x7f64ac199170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f649c004770 tx=0x7f649c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:12.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.633+0000 7f64a27fc640 1 -- 172.21.15.94:0/2126361792 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c03f070 con 0x7f64ac1009b0 2024-09-19T04:15:12.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.634+0000 7f64a27fc640 1 -- 172.21.15.94:0/2126361792 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f649c03b460 con 0x7f64ac1009b0 2024-09-19T04:15:12.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.634+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac1a4900 con 0x7f64ac1009b0 2024-09-19T04:15:12.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.634+0000 7f64a27fc640 1 -- 172.21.15.94:0/2126361792 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c035070 con 0x7f64ac1009b0 2024-09-19T04:15:12.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.634+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac1a4d60 con 0x7f64ac1009b0 2024-09-19T04:15:12.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.635+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac100e10 con 0x7f64ac1009b0 2024-09-19T04:15:12.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.636+0000 7f64a27fc640 1 -- 172.21.15.94:0/2126361792 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f649c03b610 con 0x7f64ac1009b0 2024-09-19T04:15:12.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.636+0000 7f64a27fc640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64980a3500 0x7f64980a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.636+0000 7f64b0a1d640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64980a3500 0x7f64980a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.636+0000 7f64a27fc640 1 -- 172.21.15.94:0/2126361792 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f649c0f02f0 con 0x7f64ac1009b0 2024-09-19T04:15:12.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.637+0000 7f64b0a1d640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64980a3500 0x7f64980a59c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f64ac06a3f0 tx=0x7f64a8005c60 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.640+0000 7f64a27fc640 1 -- 172.21.15.94:0/2126361792 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f649c0bcb50 con 0x7f64ac1009b0 2024-09-19T04:15:12.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.831+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 --> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f64ac062630 con 0x7f64980a3500 2024-09-19T04:15:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.838+0000 7f64a27fc640 1 -- 172.21.15.94:0/2126361792 <== mgr.15159 v2:172.21.15.94:6800/3395191155 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f64ac062630 con 0x7f64980a3500 2024-09-19T04:15:12.840 INFO:teuthology.orchestra.run.smithi094.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T04:15:12", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T04:15:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64980a3500 msgr2=0x7f64980a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64980a3500 0x7f64980a59c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f64ac06a3f0 tx=0x7f64a8005c60 comp rx=0 tx=0).stop 2024-09-19T04:15:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1009b0 msgr2=0x7f64ac199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1009b0 0x7f64ac199170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f649c004770 tx=0x7f649c0042f0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 shutdown_connections 2024-09-19T04:15:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64980a3500 0x7f64980a59c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64ac1069c0 0x7f64ac1a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ac101350 0x7f64ac1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 --2- 172.21.15.94:0/2126361792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ac1009b0 0x7f64ac199170 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 >> 172.21.15.94:0/2126361792 conn(0x7f64ac077990 msgr2=0x7f64ac1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 shutdown_connections 2024-09-19T04:15:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.840+0000 7f64b3482640 1 -- 172.21.15.94:0/2126361792 wait complete. 2024-09-19T04:15:12.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T04:15:12.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.953+0000 7f7a6739f640 1 -- 172.21.15.94:0/3239738759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60100060 msgr2=0x7f7a60100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.953+0000 7f7a6739f640 1 --2- 172.21.15.94:0/3239738759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60100060 0x7f7a60100440 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009900 tx=0x7f7a5c02a410 comp rx=0 tx=0).stop 2024-09-19T04:15:12.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.953+0000 7f7a6739f640 1 -- 172.21.15.94:0/3239738759 shutdown_connections 2024-09-19T04:15:12.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.953+0000 7f7a6739f640 1 --2- 172.21.15.94:0/3239738759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a60106ca0 0x7f7a6010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.953+0000 7f7a6739f640 1 --2- 172.21.15.94:0/3239738759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a60100a10 0x7f7a60106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.953+0000 7f7a6739f640 1 --2- 172.21.15.94:0/3239738759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60100060 0x7f7a60100440 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.953+0000 7f7a6739f640 1 -- 172.21.15.94:0/3239738759 >> 172.21.15.94:0/3239738759 conn(0x7f7a60076ff0 msgr2=0x7f7a600fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:12.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.953+0000 7f7a6739f640 1 -- 172.21.15.94:0/3239738759 shutdown_connections 2024-09-19T04:15:12.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.953+0000 7f7a6739f640 1 -- 172.21.15.94:0/3239738759 wait complete. 2024-09-19T04:15:12.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a6739f640 1 Processor -- start 2024-09-19T04:15:12.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a6739f640 1 -- start start 2024-09-19T04:15:12.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a6739f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a60100060 0x7f7a60100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a6739f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60106ca0 0x7f7a60101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a6739f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a60101a50 0x7f7a601a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a6739f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60111bb0 con 0x7f7a60106ca0 2024-09-19T04:15:12.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a6739f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60111a30 con 0x7f7a60101a50 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a6739f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60111d30 con 0x7f7a60100060 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a65b9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60106ca0 0x7f7a60101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a65b9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60106ca0 0x7f7a60101510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46436/0 (socket says 172.21.15.94:46436) 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a65b9c640 1 -- 172.21.15.94:0/667269259 learned_addr learned my addr 172.21.15.94:0/667269259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a6639d640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a60100060 0x7f7a60100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.954+0000 7f7a66b9e640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a60101a50 0x7f7a601a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a65b9c640 1 -- 172.21.15.94:0/667269259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a60100060 msgr2=0x7f7a60100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a65b9c640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a60100060 0x7f7a60100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a65b9c640 1 -- 172.21.15.94:0/667269259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a60101a50 msgr2=0x7f7a601a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a65b9c640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a60101a50 0x7f7a601a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a65b9c640 1 -- 172.21.15.94:0/667269259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a601a6a30 con 0x7f7a60106ca0 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a65b9c640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60106ca0 0x7f7a60101510 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d770 tx=0x7f7a5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:12.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a577fe640 1 -- 172.21.15.94:0/667269259 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50017070 con 0x7f7a60106ca0 2024-09-19T04:15:12.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a577fe640 1 -- 172.21.15.94:0/667269259 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a50009c30 con 0x7f7a60106ca0 2024-09-19T04:15:12.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a601a6d20 con 0x7f7a60106ca0 2024-09-19T04:15:12.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a577fe640 1 -- 172.21.15.94:0/667269259 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000f620 con 0x7f7a60106ca0 2024-09-19T04:15:12.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.955+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a601a7210 con 0x7f7a60106ca0 2024-09-19T04:15:12.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.956+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a60100440 con 0x7f7a60106ca0 2024-09-19T04:15:12.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.957+0000 7f7a577fe640 1 -- 172.21.15.94:0/667269259 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f7a5001a070 con 0x7f7a60106ca0 2024-09-19T04:15:12.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.958+0000 7f7a577fe640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7a4c0a3470 0x7f7a4c0a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:12.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.958+0000 7f7a577fe640 1 -- 172.21.15.94:0/667269259 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f7a500cf910 con 0x7f7a60106ca0 2024-09-19T04:15:12.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.958+0000 7f7a6639d640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7a4c0a3470 0x7f7a4c0a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:12.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.958+0000 7f7a6639d640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7a4c0a3470 0x7f7a4c0a5930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c031440 tx=0x7f7a5c005b50 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:12.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:12.962+0000 7f7a577fe640 1 -- 172.21.15.94:0/667269259 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7a5009bb20 con 0x7f7a60106ca0 2024-09-19T04:15:13.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.050+0000 7ff6aef01640 1 -- 172.21.15.94:0/638122373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a8103760 msgr2=0x7ff6a8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.050+0000 7ff6aef01640 1 --2- 172.21.15.94:0/638122373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a8103760 0x7ff6a8107ba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff694009930 tx=0x7ff69402a420 comp rx=0 tx=0).stop 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.050+0000 7ff6aef01640 1 -- 172.21.15.94:0/638122373 shutdown_connections 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.050+0000 7ff6aef01640 1 --2- 172.21.15.94:0/638122373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a81082e0 0x7ff6a810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.050+0000 7ff6aef01640 1 --2- 172.21.15.94:0/638122373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a8103760 0x7ff6a8107ba0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.050+0000 7ff6aef01640 1 --2- 172.21.15.94:0/638122373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a8102db0 0x7ff6a8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.050+0000 7ff6aef01640 1 -- 172.21.15.94:0/638122373 >> 172.21.15.94:0/638122373 conn(0x7ff6a80fc7d0 msgr2=0x7ff6a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.050+0000 7ff6aef01640 1 -- 172.21.15.94:0/638122373 shutdown_connections 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.050+0000 7ff6aef01640 1 -- 172.21.15.94:0/638122373 wait complete. 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6aef01640 1 Processor -- start 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6aef01640 1 -- start start 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6aef01640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a8102db0 0x7ff6a8100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6aef01640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a8103760 0x7ff6a8100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6aef01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a81082e0 0x7ff6a8101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6aef01640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8111f80 con 0x7ff6a81082e0 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6aef01640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8111e00 con 0x7ff6a8102db0 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6aef01640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8112100 con 0x7ff6a8103760 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6ad49e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a81082e0 0x7ff6a8101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6ad49e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a81082e0 0x7ff6a8101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46454/0 (socket says 172.21.15.94:46454) 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6ad49e640 1 -- 172.21.15.94:0/1491365022 learned_addr learned my addr 172.21.15.94:0/1491365022 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.051+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a8102db0 0x7ff6a8100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6a7fff640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a8103760 0x7ff6a8100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6ad49e640 1 -- 172.21.15.94:0/1491365022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a8103760 msgr2=0x7ff6a8100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a8103760 0x7ff6a8100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6ad49e640 1 -- 172.21.15.94:0/1491365022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a8102db0 msgr2=0x7ff6a8100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a8102db0 0x7ff6a8100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6ad49e640 1 -- 172.21.15.94:0/1491365022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a81a6970 con 0x7ff6a81082e0 2024-09-19T04:15:13.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6a7fff640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a8103760 0x7ff6a8100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:13.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a81082e0 0x7ff6a8101490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b970 tx=0x7ff6a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:13.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6a5ffb640 1 -- 172.21.15.94:0/1491365022 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0019070 con 0x7ff6a81082e0 2024-09-19T04:15:13.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6a5ffb640 1 -- 172.21.15.94:0/1491365022 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6a0014440 con 0x7ff6a81082e0 2024-09-19T04:15:13.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.052+0000 7ff6aef01640 1 -- 172.21.15.94:0/1491365022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a81a6c60 con 0x7ff6a81082e0 2024-09-19T04:15:13.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.053+0000 7ff6a5ffb640 1 -- 172.21.15.94:0/1491365022 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00075d0 con 0x7ff6a81082e0 2024-09-19T04:15:13.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.053+0000 7ff6aef01640 1 -- 172.21.15.94:0/1491365022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a81a7170 con 0x7ff6a81082e0 2024-09-19T04:15:13.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.055+0000 7ff6a5ffb640 1 -- 172.21.15.94:0/1491365022 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7ff6a001c070 con 0x7ff6a81082e0 2024-09-19T04:15:13.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6a5ffb640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff6980a34c0 0x7ff6980a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6b12da740 1 Processor -- start 2024-09-19T04:15:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6b12da740 1 -- start start 2024-09-19T04:15:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff6980a34c0 0x7ff6980a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6a5ffb640 1 -- 172.21.15.94:0/1491365022 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff6a00cf300 con 0x7ff6a81082e0 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5e3c0 0x558d8cc5e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc71950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc71e90 0x558d8cc74280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6b12da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558d8cbc42e0 con 0x558d8cc5e3c0 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6b12da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558d8cb6bce0 con 0x558d8cc71e90 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6b12da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d8cbc02a0 con 0x558d8cc5ece0 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6acc9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5e3c0 0x558d8cc5e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6acc9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5e3c0 0x558d8cc5e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46462/0 (socket says 172.21.15.94:46462) 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6acc9d640 1 -- 172.21.15.94:0/2623054161 learned_addr learned my addr 172.21.15.94:0/2623054161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6a7fff640 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc71950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.056+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc71e90 0x558d8cc74280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff6acc9d640 1 -- 172.21.15.94:0/2623054161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 msgr2=0x558d8cc71950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc71950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff6acc9d640 1 -- 172.21.15.94:0/2623054161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc71e90 msgr2=0x558d8cc74280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc71e90 0x558d8cc74280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff6acc9d640 1 -- 172.21.15.94:0/2623054161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d8cb50440 con 0x558d8cc5e3c0 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff6a7fff640 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc71950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff6980a34c0 0x7ff6980a5980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff69c014a90 tx=0x7ff69c00ad00 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5e3c0 0x558d8cc5e7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff69c011d60 tx=0x7ff69c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a9ca5743a314ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff692ffd640 1 -- 172.21.15.94:0/2623054161 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011230 con 0x558d8cc5e3c0 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff692ffd640 1 -- 172.21.15.94:0/2623054161 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff69c014070 con 0x558d8cc5e3c0 2024-09-19T04:15:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.057+0000 7ff692ffd640 1 -- 172.21.15.94:0/2623054161 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00fb00 con 0x558d8cc5e3c0 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 -- 172.21.15.94:0/2623054161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5e3c0 msgr2=0x558d8cc5e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5e3c0 0x558d8cc5e7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff69c011d60 tx=0x7ff69c01b000 comp rx=0 tx=0).stop 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 -- 172.21.15.94:0/2623054161 shutdown_connections 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc71e90 0x558d8cc74280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc71950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2623054161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5e3c0 0x558d8cc5e7a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 -- 172.21.15.94:0/2623054161 >> 172.21.15.94:0/2623054161 conn(0x558d8cc5aa30 msgr2=0x558d8cc5ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 -- 172.21.15.94:0/2623054161 shutdown_connections 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 -- 172.21.15.94:0/2623054161 wait complete. 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 -- start start 2024-09-19T04:15:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.058+0000 7ff6b12da740 1 Processor -- start 2024-09-19T04:15:13.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 0x558d8cc71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc6eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cc6f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6b12da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558d8cc74940 con 0x558d8cc71e90 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6b12da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558d8cb7f050 con 0x558d8cc5e3c0 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6b12da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d8cc74ac0 con 0x558d8cc5ece0 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6ad49e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cc6f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6b12da740 10 client.0 _ll_register_callbacks cb 0x558d8cbbf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6ad49e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cc6f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46468/0 (socket says 172.21.15.94:46468) 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6ad49e640 1 -- 172.21.15.94:0/2556818942 learned_addr learned my addr 172.21.15.94:0/2556818942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 0x558d8cc71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6a7fff640 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc6eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6ad49e640 1 -- 172.21.15.94:0/2556818942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 msgr2=0x558d8cc6eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc6eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6ad49e640 1 -- 172.21.15.94:0/2556818942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 msgr2=0x558d8cc71890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 0x558d8cc71890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6ad49e640 1 -- 172.21.15.94:0/2556818942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d8c9353c0 con 0x558d8cc71e90 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cc6f130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00ccb20 tx=0x7ff6a0064520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789fdb32ab6239b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.059+0000 7ff687fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff687fff640 1 -- 172.21.15.94:0/2556818942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00b3020 con 0x558d8cc71e90 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff687fff640 1 -- 172.21.15.94:0/2556818942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff6a00ccc30 con 0x558d8cc71e90 2024-09-19T04:15:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6b12da740 1 -- 172.21.15.94:0/2556818942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff69c014070 con 0x558d8cc71e90 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff687fff640 1 -- 172.21.15.94:0/2556818942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6a00cce30 con 0x558d8cc71e90 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6b12da740 10 client.15351 fetch_fsmap learned FSMap version 25 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6b12da740 1 -- 172.21.15.94:0/2556818942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d8cc80440 con 0x558d8cc71e90 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff687fff640 1 -- 172.21.15.94:0/2556818942 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff69c014070 con 0x558d8cc71e90 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff687fff640 1 -- 172.21.15.94:0/2556818942 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff6a00a3d50 con 0x558d8cc71e90 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6b12da740 10 client.15351 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6b12da740 10 client.15351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6b12da740 20 client.15351 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d8cc818a0 0x558d8cc83c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6b12da740 4 client.15351 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6b12da740 1 -- 172.21.15.94:0/2556818942 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d8cbd1850 con 0x558d8cc818a0 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.060+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d8cc818a0 0x558d8cc83c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.061+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d8cc818a0 0x558d8cc83c90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb99bfe267a52c98 server_cookie=6feca5e6f34ee4d3 in_seq=0 out_seq=0 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.061+0000 7ff687fff640 10 client.15351 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff687fff640 1 -- 172.21.15.94:0/2556818942 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d8cbd1850 con 0x558d8cc818a0 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff687fff640 10 client.15351 handle_command_reply: tid=0 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 client.15351 shutdown 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 -- 172.21.15.94:0/2556818942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 msgr2=0x558d8cc6f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cc6f130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00ccb20 tx=0x7ff6a0064520 comp rx=0 tx=0).stop 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 -- 172.21.15.94:0/2556818942 shutdown_connections 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d8cc818a0 0x558d8cc83c90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cc6f130 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff687fff640 0 client.15351 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:13.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc6eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2556818942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 0x558d8cc71890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 -- 172.21.15.94:0/2556818942 >> 172.21.15.94:0/2556818942 conn(0x558d8cc5aa30 msgr2=0x558d8cc606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:13.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 -- 172.21.15.94:0/2556818942 shutdown_connections 2024-09-19T04:15:13.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 1 -- 172.21.15.94:0/2556818942 wait complete. 2024-09-19T04:15:13.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.062+0000 7ff6b12da740 20 client.15351 trim_cache size 0 max 16384 2024-09-19T04:15:13.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6b12da740 1 Processor -- start 2024-09-19T04:15:13.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6b12da740 1 -- start start 2024-09-19T04:15:13.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5e3c0 0x558d8cc7dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5ece0 0x558d8cc89560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc71e90 0x558d8cc89aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6b12da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81add40 con 0x558d8cc5ece0 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6b12da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81ab610 con 0x558d8cc71e90 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6b12da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81a9460 con 0x558d8cc5e3c0 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5ece0 0x558d8cc89560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5ece0 0x558d8cc89560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46478/0 (socket says 172.21.15.94:46478) 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6a7fff640 1 -- 172.21.15.94:0/2427073438 learned_addr learned my addr 172.21.15.94:0/2427073438 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6a7fff640 1 -- 172.21.15.94:0/2427073438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5e3c0 msgr2=0x558d8cc7dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6a7fff640 1 --2- 172.21.15.94:0/2427073438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5e3c0 0x558d8cc7dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6a7fff640 1 -- 172.21.15.94:0/2427073438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc71e90 msgr2=0x558d8cc89aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6a7fff640 1 --2- 172.21.15.94:0/2427073438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc71e90 0x558d8cc89aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.079+0000 7ff6a7fff640 1 -- 172.21.15.94:0/2427073438 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d8cc7e180 con 0x558d8cc5ece0 2024-09-19T04:15:13.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6a7fff640 1 --2- 172.21.15.94:0/2427073438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5ece0 0x558d8cc89560 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff6940096c0 tx=0x7ff694031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f1c7b625167878 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:13.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6927fc640 1 -- 172.21.15.94:0/2427073438 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694035040 con 0x558d8cc5ece0 2024-09-19T04:15:13.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6927fc640 1 -- 172.21.15.94:0/2427073438 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff694005040 con 0x558d8cc5ece0 2024-09-19T04:15:13.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6927fc640 1 -- 172.21.15.94:0/2427073438 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694004920 con 0x558d8cc5ece0 2024-09-19T04:15:13.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6b12da740 1 -- 172.21.15.94:0/2427073438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5ece0 msgr2=0x558d8cc89560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2427073438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5ece0 0x558d8cc89560 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff6940096c0 tx=0x7ff694031910 comp rx=0 tx=0).stop 2024-09-19T04:15:13.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6b12da740 1 -- 172.21.15.94:0/2427073438 shutdown_connections 2024-09-19T04:15:13.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2427073438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc71e90 0x558d8cc89aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2427073438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc5ece0 0x558d8cc89560 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6b12da740 1 --2- 172.21.15.94:0/2427073438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5e3c0 0x558d8cc7dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6b12da740 1 -- 172.21.15.94:0/2427073438 >> 172.21.15.94:0/2427073438 conn(0x558d8cc5aa30 msgr2=0x558d8cc6a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.080+0000 7ff6b12da740 1 -- 172.21.15.94:0/2427073438 shutdown_connections 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 1 -- 172.21.15.94:0/2427073438 wait complete. 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 1 -- start start 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 1 Processor -- start 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 0x558d8cc761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc76700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cceb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81a9ad0 con 0x558d8cc71e90 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81aaf40 con 0x558d8cc5e3c0 2024-09-19T04:15:13.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81aa1a0 con 0x558d8cc5ece0 2024-09-19T04:15:13.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6b12da740 10 client.0 _ll_register_callbacks cb 0x7ff6a009b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:13.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6ad49e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cceb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6ad49e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cceb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46480/0 (socket says 172.21.15.94:46480) 2024-09-19T04:15:13.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.081+0000 7ff6ad49e640 1 -- 172.21.15.94:0/3960348625 learned_addr learned my addr 172.21.15.94:0/3960348625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:13.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6a7fff640 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc76700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 0x558d8cc761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6ad49e640 1 -- 172.21.15.94:0/3960348625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 msgr2=0x558d8cc76700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:13.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc76700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6ad49e640 1 -- 172.21.15.94:0/3960348625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 msgr2=0x558d8cc761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 0x558d8cc761c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6ad49e640 1 -- 172.21.15.94:0/3960348625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d8cceba70 con 0x558d8cc71e90 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 0x558d8cc761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6ad49e640 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cceb530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000bfd0 tx=0x7ff6a0014cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5393e632d7cc0f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff687fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff687fff640 1 -- 172.21.15.94:0/3960348625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00b3070 con 0x558d8cc71e90 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff687fff640 1 -- 172.21.15.94:0/3960348625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff6a006d3e0 con 0x558d8cc71e90 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6b12da740 1 -- 172.21.15.94:0/3960348625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff694005040 con 0x558d8cc71e90 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff687fff640 1 -- 172.21.15.94:0/3960348625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6a00a41e0 con 0x558d8cc71e90 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6b12da740 10 client.15357 fetch_fsmap learned FSMap version 25 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff6b12da740 1 -- 172.21.15.94:0/3960348625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d8cceba70 con 0x558d8cc71e90 2024-09-19T04:15:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.082+0000 7ff687fff640 1 -- 172.21.15.94:0/3960348625 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff694005040 con 0x558d8cc71e90 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff687fff640 1 -- 172.21.15.94:0/3960348625 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff6a006c070 con 0x558d8cc71e90 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff6b12da740 10 client.15357 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff6b12da740 10 client.15357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff6b12da740 20 client.15357 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff6b12da740 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d8cc818a0 0x558d8ccef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff6b12da740 4 client.15357 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff6b12da740 1 -- 172.21.15.94:0/3960348625 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d8cbcbd60 con 0x558d8cc818a0 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d8cc818a0 0x558d8ccef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff687fff640 10 client.15357 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.083+0000 7ff6acc9d640 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d8cc818a0 0x558d8ccef0a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdc88b6ffb556e3c server_cookie=5d8e1cd343d5f633 in_seq=0 out_seq=0 2024-09-19T04:15:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff687fff640 1 -- 172.21.15.94:0/3960348625 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558d8cbcbd60 con 0x558d8cc818a0 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff687fff640 10 client.15357 handle_command_reply: tid=0 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 client.15357 shutdown 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 -- 172.21.15.94:0/3960348625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 msgr2=0x558d8cceb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cceb530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000bfd0 tx=0x7ff6a0014cd0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 -- 172.21.15.94:0/3960348625 shutdown_connections 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d8cc818a0 0x558d8ccef0a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d8cc71e90 0x558d8cceb530 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d8cc5ece0 0x558d8cc76700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 --2- 172.21.15.94:0/3960348625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d8cc5e3c0 0x558d8cc761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 -- 172.21.15.94:0/3960348625 >> 172.21.15.94:0/3960348625 conn(0x558d8cc5aa30 msgr2=0x558d8cc703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:13.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff687fff640 0 client.15357 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:13.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 -- 172.21.15.94:0/3960348625 shutdown_connections 2024-09-19T04:15:13.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 1 -- 172.21.15.94:0/3960348625 wait complete. 2024-09-19T04:15:13.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.084+0000 7ff6b12da740 20 client.15357 trim_cache size 0 max 16384 2024-09-19T04:15:13.093 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:15:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:15:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:15:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "2059d731-06cf-4387-bc55-2e66f44dff78": { 2024-09-19T04:15:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:15:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "2059d731-06cf-4387-bc55-2e66f44dff78", 2024-09-19T04:15:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:15:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:15:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:15:13.094 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 -- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff6980a34c0 msgr2=0x7ff6980a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff6980a34c0 0x7ff6980a5980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff69c014a90 tx=0x7ff69c00ad00 comp rx=0 tx=0).stop 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 -- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a81082e0 msgr2=0x7ff6a8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a81082e0 0x7ff6a8101490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b970 tx=0x7ff6a000bd70 comp rx=0 tx=0).stop 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 -- 172.21.15.94:0/1491365022 shutdown_connections 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff6980a34c0 0x7ff6980a5980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a81082e0 0x7ff6a8101490 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a8103760 0x7ff6a8100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 --2- 172.21.15.94:0/1491365022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a8102db0 0x7ff6a8100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 -- 172.21.15.94:0/1491365022 >> 172.21.15.94:0/1491365022 conn(0x7ff6a80fc7d0 msgr2=0x7ff6a8105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:13.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 -- 172.21.15.94:0/1491365022 shutdown_connections 2024-09-19T04:15:13.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.085+0000 7ff6aef01640 1 -- 172.21.15.94:0/1491365022 wait complete. 2024-09-19T04:15:13.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.165+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 --> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7a60100a10 con 0x7f7a4c0a3470 2024-09-19T04:15:13.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.172+0000 7f7a577fe640 1 -- 172.21.15.94:0/667269259 <== mgr.15159 v2:172.21.15.94:6800/3395191155 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7a60100a10 con 0x7f7a4c0a3470 2024-09-19T04:15:13.174 INFO:teuthology.orchestra.run.smithi094.stdout:/ 1m 6m3h 2024-09-19T04:15:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.174+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7a4c0a3470 msgr2=0x7f7a4c0a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.174+0000 7f7a6739f640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7a4c0a3470 0x7f7a4c0a5930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c031440 tx=0x7f7a5c005b50 comp rx=0 tx=0).stop 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60106ca0 msgr2=0x7f7a60101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60106ca0 0x7f7a60101510 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d770 tx=0x7f7a5000dc30 comp rx=0 tx=0).stop 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 shutdown_connections 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7a4c0a3470 0x7f7a4c0a5930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a60101a50 0x7f7a601a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a60106ca0 0x7f7a60101510 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 --2- 172.21.15.94:0/667269259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a60100060 0x7f7a60100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 >> 172.21.15.94:0/667269259 conn(0x7f7a60076ff0 msgr2=0x7f7a60105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 shutdown_connections 2024-09-19T04:15:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:13.175+0000 7f7a6739f640 1 -- 172.21.15.94:0/667269259 wait complete. 2024-09-19T04:15:13.187 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T04:15:13.214 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T04:15:13.225 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T04:15:13.282 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:15:13.282 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T04:15:13.282 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-19T04:15:13.317 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-09-19T04:15:13.328 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-19T04:15:13.328 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-fuse 340G 0 340G 0% /tmp/cephfs 2024-09-19T04:15:13.330 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-19T04:15:13.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:13 smithi094 ceph-mon[30972]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:13.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:13 smithi104 ceph-mon[34145]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:13.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-19T04:15:13.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-19T04:15:13.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:13 smithi112 ceph-mon[35051]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:13.958 INFO:tasks.cephfs.filesystem:scrub status for tag:2059d731-06cf-4387-bc55-2e66f44dff78 - {'path': '/', 'tag': '2059d731-06cf-4387-bc55-2e66f44dff78', 'options': 'recursive,force'} 2024-09-19T04:15:14.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:14 smithi094 ceph-mon[30972]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:14.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:14 smithi094 ceph-mon[30972]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:14.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:14 smithi094 ceph-mon[30972]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:14.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:14 smithi094 ceph-mon[30972]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 KiB/s wr, 32 op/s 2024-09-19T04:15:14.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:14 smithi112 ceph-mon[35051]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:14.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:14 smithi112 ceph-mon[35051]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:14.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:14 smithi112 ceph-mon[35051]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:14.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:14 smithi112 ceph-mon[35051]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 KiB/s wr, 32 op/s 2024-09-19T04:15:14.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:14 smithi104 ceph-mon[34145]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:14.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:14 smithi104 ceph-mon[34145]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:14.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:14 smithi104 ceph-mon[34145]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:14.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:14 smithi104 ceph-mon[34145]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 KiB/s wr, 32 op/s 2024-09-19T04:15:15.388 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-19T04:15:15.424 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-19T04:15:15.427 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-19T04:15:15.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-19T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:15 smithi094 ceph-mon[30972]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-09-19T04:15:15.588 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/: 2024-09-19T04:15:15.588 INFO:teuthology.orchestra.run.smithi094.stdout:total 2 2024-09-19T04:15:15.588 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:15.588 INFO:teuthology.orchestra.run.smithi094.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 .. 2024-09-19T04:15:15.588 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 data 2024-09-19T04:15:15.588 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:15.588 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data: 2024-09-19T04:15:15.588 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:15.588 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:15.589 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 .. 2024-09-19T04:15:15.590 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-19T04:15:15.672 INFO:teuthology.orchestra.run.smithi094.stdout:[Service] 2024-09-19T04:15:15.672 INFO:teuthology.orchestra.run.smithi094.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-19T04:15:15.672 INFO:teuthology.orchestra.run.smithi094.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-19T04:15:15.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-19T04:15:15.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:15 smithi112 ceph-mon[35051]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-09-19T04:15:15.796 INFO:teuthology.orchestra.run.smithi094.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-19T04:15:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:15 smithi104 ceph-mon[34145]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-09-19T04:15:17.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:17 smithi112 ceph-mon[35051]: pgmap v11: 97 pgs: 97 active+clean; 20 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 59 op/s 2024-09-19T04:15:17.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:17 smithi094 ceph-mon[30972]: pgmap v11: 97 pgs: 97 active+clean; 20 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 59 op/s 2024-09-19T04:15:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:17 smithi104 ceph-mon[34145]: pgmap v11: 97 pgs: 97 active+clean; 20 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 59 op/s 2024-09-19T04:15:19.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:19 smithi112 ceph-mon[35051]: pgmap v12: 97 pgs: 97 active+clean; 20 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 46 op/s 2024-09-19T04:15:19.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:19 smithi094 ceph-mon[30972]: pgmap v12: 97 pgs: 97 active+clean; 20 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 46 op/s 2024-09-19T04:15:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:19 smithi104 ceph-mon[34145]: pgmap v12: 97 pgs: 97 active+clean; 20 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 46 op/s 2024-09-19T04:15:21.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:21 smithi112 ceph-mon[35051]: pgmap v13: 97 pgs: 97 active+clean; 20 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 46 op/s 2024-09-19T04:15:21.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:21 smithi094 ceph-mon[30972]: pgmap v13: 97 pgs: 97 active+clean; 20 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 46 op/s 2024-09-19T04:15:21.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:21 smithi104 ceph-mon[34145]: pgmap v13: 97 pgs: 97 active+clean; 20 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 46 op/s 2024-09-19T04:15:22.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:22 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:15:22.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:22 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:15:22.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:22 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:15:23.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:23 smithi112 ceph-mon[35051]: pgmap v14: 97 pgs: 97 active+clean; 36 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 3.7 MiB/s wr, 118 op/s 2024-09-19T04:15:23.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:23 smithi094 ceph-mon[30972]: pgmap v14: 97 pgs: 97 active+clean; 36 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 3.7 MiB/s wr, 118 op/s 2024-09-19T04:15:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:23 smithi104 ceph-mon[34145]: pgmap v14: 97 pgs: 97 active+clean; 36 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 3.7 MiB/s wr, 118 op/s 2024-09-19T04:15:25.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:25 smithi112 ceph-mon[35051]: pgmap v15: 97 pgs: 97 active+clean; 36 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 3.7 MiB/s wr, 101 op/s 2024-09-19T04:15:25.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:25 smithi094 ceph-mon[30972]: pgmap v15: 97 pgs: 97 active+clean; 36 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 3.7 MiB/s wr, 101 op/s 2024-09-19T04:15:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:25 smithi104 ceph-mon[34145]: pgmap v15: 97 pgs: 97 active+clean; 36 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 3.7 MiB/s wr, 101 op/s 2024-09-19T04:15:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:27 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:15:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:27 smithi112 ceph-mon[35051]: pgmap v16: 97 pgs: 97 active+clean; 44 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 4.7 MiB/s wr, 129 op/s 2024-09-19T04:15:28.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:27 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:15:28.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:27 smithi094 ceph-mon[30972]: pgmap v16: 97 pgs: 97 active+clean; 44 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 4.7 MiB/s wr, 129 op/s 2024-09-19T04:15:28.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:27 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:15:28.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:27 smithi104 ceph-mon[34145]: pgmap v16: 97 pgs: 97 active+clean; 44 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 4.7 MiB/s wr, 129 op/s 2024-09-19T04:15:29.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:29 smithi112 ceph-mon[35051]: pgmap v17: 97 pgs: 97 active+clean; 44 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 3.1 MiB/s wr, 99 op/s 2024-09-19T04:15:29.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:29 smithi094 ceph-mon[30972]: pgmap v17: 97 pgs: 97 active+clean; 44 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 3.1 MiB/s wr, 99 op/s 2024-09-19T04:15:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:29 smithi104 ceph-mon[34145]: pgmap v17: 97 pgs: 97 active+clean; 44 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 3.1 MiB/s wr, 99 op/s 2024-09-19T04:15:30.214 INFO:teuthology.orchestra.run.smithi094.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-19T04:15:30.216 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-19T04:15:30.490 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/: 2024-09-19T04:15:30.490 INFO:teuthology.orchestra.run.smithi094.stdout:total 2 2024-09-19T04:15:30.490 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 . 2024-09-19T04:15:30.490 INFO:teuthology.orchestra.run.smithi094.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.490 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 data 2024-09-19T04:15:30.490 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.490 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data: 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:total 45 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 . 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 19 04:15 base 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 Sep 19 04:15 global 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 log 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_commit_ts 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_dynshmem 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 19 04:15 pg_hba.conf 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 19 04:15 pg_ident.conf 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:15 pg_logical 2024-09-19T04:15:30.491 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 pg_multixact 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_notify 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_replslot 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_serial 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_snapshots 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_stat 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_stat_tmp 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 pg_subtrans 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_tblspc 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_twophase 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 04:15 PG_VERSION 2024-09-19T04:15:30.492 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:15 pg_wal 2024-09-19T04:15:30.493 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 pg_xact 2024-09-19T04:15:30.493 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 19 04:15 postgresql.auto.conf 2024-09-19T04:15:30.493 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 19 04:15 postgresql.conf 2024-09-19T04:15:30.493 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.493 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base: 2024-09-19T04:15:30.493 INFO:teuthology.orchestra.run.smithi094.stdout:total 3 2024-09-19T04:15:30.493 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 19 04:15 . 2024-09-19T04:15:30.493 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.493 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 1 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 13493 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 13494 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:total 7778 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 . 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 19 04:15 .. 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 112 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 113 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 1247 2024-09-19T04:15:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1247_fsm 2024-09-19T04:15:30.495 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1247_vm 2024-09-19T04:15:30.495 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 04:15 1249 2024-09-19T04:15:30.495 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1249_fsm 2024-09-19T04:15:30.495 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1249_vm 2024-09-19T04:15:30.495 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 04:15 1255 2024-09-19T04:15:30.495 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1255_fsm 2024-09-19T04:15:30.495 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1255_vm 2024-09-19T04:15:30.495 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 04:15 1259 2024-09-19T04:15:30.496 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1259_fsm 2024-09-19T04:15:30.496 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1259_vm 2024-09-19T04:15:30.496 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 04:15 13344 2024-09-19T04:15:30.496 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13344_fsm 2024-09-19T04:15:30.496 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13344_vm 2024-09-19T04:15:30.496 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13346 2024-09-19T04:15:30.496 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13348 2024-09-19T04:15:30.496 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349 2024-09-19T04:15:30.497 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13349_fsm 2024-09-19T04:15:30.497 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349_vm 2024-09-19T04:15:30.497 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13351 2024-09-19T04:15:30.497 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13353 2024-09-19T04:15:30.497 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354 2024-09-19T04:15:30.497 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13354_fsm 2024-09-19T04:15:30.497 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354_vm 2024-09-19T04:15:30.497 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13356 2024-09-19T04:15:30.498 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13358 2024-09-19T04:15:30.498 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359 2024-09-19T04:15:30.498 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13359_fsm 2024-09-19T04:15:30.498 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359_vm 2024-09-19T04:15:30.498 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13361 2024-09-19T04:15:30.498 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13363 2024-09-19T04:15:30.498 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1417 2024-09-19T04:15:30.499 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1418 2024-09-19T04:15:30.499 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 174 2024-09-19T04:15:30.499 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 175 2024-09-19T04:15:30.499 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2187 2024-09-19T04:15:30.499 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2224 2024-09-19T04:15:30.499 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2328 2024-09-19T04:15:30.499 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2336 2024-09-19T04:15:30.499 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2337 2024-09-19T04:15:30.500 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2579 2024-09-19T04:15:30.500 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2600 2024-09-19T04:15:30.500 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2600_fsm 2024-09-19T04:15:30.500 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2600_vm 2024-09-19T04:15:30.500 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601 2024-09-19T04:15:30.500 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2601_fsm 2024-09-19T04:15:30.500 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601_vm 2024-09-19T04:15:30.500 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2602 2024-09-19T04:15:30.501 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2602_fsm 2024-09-19T04:15:30.501 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2602_vm 2024-09-19T04:15:30.501 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2603 2024-09-19T04:15:30.501 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2603_fsm 2024-09-19T04:15:30.501 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2603_vm 2024-09-19T04:15:30.501 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2604 2024-09-19T04:15:30.501 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2605 2024-09-19T04:15:30.501 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2605_fsm 2024-09-19T04:15:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2605_vm 2024-09-19T04:15:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606 2024-09-19T04:15:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2606_fsm 2024-09-19T04:15:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606_vm 2024-09-19T04:15:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2607 2024-09-19T04:15:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2607_fsm 2024-09-19T04:15:30.502 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2607_vm 2024-09-19T04:15:30.503 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 04:15 2608 2024-09-19T04:15:30.503 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2608_fsm 2024-09-19T04:15:30.503 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2608_vm 2024-09-19T04:15:30.503 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 04:15 2609 2024-09-19T04:15:30.503 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2609_fsm 2024-09-19T04:15:30.503 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2609_vm 2024-09-19T04:15:30.503 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2610 2024-09-19T04:15:30.503 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2610_fsm 2024-09-19T04:15:30.503 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2610_vm 2024-09-19T04:15:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2611 2024-09-19T04:15:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612 2024-09-19T04:15:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2612_fsm 2024-09-19T04:15:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612_vm 2024-09-19T04:15:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2613 2024-09-19T04:15:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615 2024-09-19T04:15:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2615_fsm 2024-09-19T04:15:30.505 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615_vm 2024-09-19T04:15:30.505 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2616 2024-09-19T04:15:30.505 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2616_fsm 2024-09-19T04:15:30.505 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2616_vm 2024-09-19T04:15:30.505 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 04:15 2617 2024-09-19T04:15:30.505 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2617_fsm 2024-09-19T04:15:30.505 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2617_vm 2024-09-19T04:15:30.505 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 04:15 2618 2024-09-19T04:15:30.506 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2618_fsm 2024-09-19T04:15:30.506 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2618_vm 2024-09-19T04:15:30.506 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 04:15 2619 2024-09-19T04:15:30.506 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2619_fsm 2024-09-19T04:15:30.506 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2619_vm 2024-09-19T04:15:30.506 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2620 2024-09-19T04:15:30.506 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2650 2024-09-19T04:15:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2651 2024-09-19T04:15:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2652 2024-09-19T04:15:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2653 2024-09-19T04:15:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2654 2024-09-19T04:15:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2655 2024-09-19T04:15:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2656 2024-09-19T04:15:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2657 2024-09-19T04:15:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 04:15 2658 2024-09-19T04:15:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 2659 2024-09-19T04:15:30.508 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2660 2024-09-19T04:15:30.508 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2661 2024-09-19T04:15:30.508 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2662 2024-09-19T04:15:30.508 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2663 2024-09-19T04:15:30.508 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2664 2024-09-19T04:15:30.508 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2665 2024-09-19T04:15:30.508 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2666 2024-09-19T04:15:30.508 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2667 2024-09-19T04:15:30.509 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2668 2024-09-19T04:15:30.509 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2669 2024-09-19T04:15:30.509 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2670 2024-09-19T04:15:30.509 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 04:15 2673 2024-09-19T04:15:30.509 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 04:15 2674 2024-09-19T04:15:30.509 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 04:15 2675 2024-09-19T04:15:30.509 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2678 2024-09-19T04:15:30.509 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2679 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2680 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2681 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2682 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2683 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2684 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2685 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2686 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2687 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2688 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2689 2024-09-19T04:15:30.510 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 04:15 2690 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 04:15 2691 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2692 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2693 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2696 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2699 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2701 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2702 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2703 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2704 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2753 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2753_fsm 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2753_vm 2024-09-19T04:15:30.511 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2754 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2755 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2756 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2757 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2830 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2831 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2832 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2833 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2834 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2835 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2836 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2837 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 04:15 2838 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2838_fsm 2024-09-19T04:15:30.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2838_vm 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2839 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840_fsm 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2840_vm 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2841 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2995 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2996 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3079_fsm 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079_vm 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3080 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3081 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3085 2024-09-19T04:15:30.513 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3118 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3119 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 04:15 3164 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3256 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3257 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3258 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3350 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3351 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3379 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3380 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3381 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3394 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3394_fsm 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3394_vm 2024-09-19T04:15:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3395 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3429 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3430 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3431 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3433 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3439 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3440 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3455 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 04:15 3456 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3456_fsm 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3456_vm 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3466 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3467 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3468 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3501 2024-09-19T04:15:30.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3502 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3503 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3534 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3541_fsm 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541_vm 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3542 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3574 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3575 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3576 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3596 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3597 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3598 2024-09-19T04:15:30.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3599 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3600_fsm 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600_vm 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3601_fsm 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601_vm 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3602_fsm 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602_vm 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603 2024-09-19T04:15:30.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603_fsm 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3603_vm 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3604 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3605 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3606 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3607 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3608 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 3609 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3712 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3764_fsm 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764_vm 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3766 2024-09-19T04:15:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3767 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3997 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4143 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4144 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4145 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4146 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4147 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4148 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4149 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4150 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4151 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4152 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4153 2024-09-19T04:15:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4154 2024-09-19T04:15:30.520 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4155 2024-09-19T04:15:30.520 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4156 2024-09-19T04:15:30.520 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4157 2024-09-19T04:15:30.520 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4158 2024-09-19T04:15:30.696 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4159 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4160 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4161 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4162 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4163 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4164 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4165 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4166 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4167 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4168 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4169 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4170 2024-09-19T04:15:30.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4171 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4172 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4173 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4174 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 5002 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 548 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 549 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6102 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6104 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6106 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6110 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6111 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6112 2024-09-19T04:15:30.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6113 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6117 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 826 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 827 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 828 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 04:15 pg_filenode.map 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 04:15 PG_VERSION 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:total 7778 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 . 2024-09-19T04:15:30.699 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 19 04:15 .. 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 112 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 113 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 1247 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1247_fsm 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1247_vm 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 04:15 1249 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1249_fsm 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1249_vm 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 04:15 1255 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1255_fsm 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1255_vm 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 04:15 1259 2024-09-19T04:15:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1259_fsm 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1259_vm 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 04:15 13344 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13344_fsm 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13344_vm 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13346 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13348 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13349_fsm 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349_vm 2024-09-19T04:15:30.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13351 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13353 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13354_fsm 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354_vm 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13356 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13358 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13359_fsm 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359_vm 2024-09-19T04:15:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13361 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13363 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1417 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1418 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 174 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 175 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2187 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2224 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2328 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2336 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2337 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2579 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2600 2024-09-19T04:15:30.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2600_fsm 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2600_vm 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2601_fsm 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601_vm 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2602 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2602_fsm 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2602_vm 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2603 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2603_fsm 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2603_vm 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2604 2024-09-19T04:15:30.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2605 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2605_fsm 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2605_vm 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2606_fsm 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606_vm 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2607 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2607_fsm 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2607_vm 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 04:15 2608 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2608_fsm 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2608_vm 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 04:15 2609 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2609_fsm 2024-09-19T04:15:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2609_vm 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2610 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2610_fsm 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2610_vm 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2611 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2612_fsm 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612_vm 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2613 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2615_fsm 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615_vm 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2616 2024-09-19T04:15:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2616_fsm 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2616_vm 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 04:15 2617 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2617_fsm 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2617_vm 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 04:15 2618 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2618_fsm 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2618_vm 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 04:15 2619 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2619_fsm 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2619_vm 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2620 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2650 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2651 2024-09-19T04:15:30.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2652 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2653 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2654 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2655 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2656 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2657 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 04:15 2658 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 2659 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2660 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2661 2024-09-19T04:15:30.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2662 2024-09-19T04:15:30.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2663 2024-09-19T04:15:30.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2664 2024-09-19T04:15:30.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2665 2024-09-19T04:15:30.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2666 2024-09-19T04:15:30.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2667 2024-09-19T04:15:30.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2668 2024-09-19T04:15:30.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2669 2024-09-19T04:15:30.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2670 2024-09-19T04:15:30.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 04:15 2673 2024-09-19T04:15:30.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 04:15 2674 2024-09-19T04:15:30.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 04:15 2675 2024-09-19T04:15:30.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2678 2024-09-19T04:15:30.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2679 2024-09-19T04:15:30.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2680 2024-09-19T04:15:30.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2681 2024-09-19T04:15:30.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2682 2024-09-19T04:15:30.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2683 2024-09-19T04:15:30.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2684 2024-09-19T04:15:30.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2685 2024-09-19T04:15:30.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2686 2024-09-19T04:15:30.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2687 2024-09-19T04:15:30.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2688 2024-09-19T04:15:30.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2689 2024-09-19T04:15:30.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 04:15 2690 2024-09-19T04:15:30.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 04:15 2691 2024-09-19T04:15:30.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2692 2024-09-19T04:15:30.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2693 2024-09-19T04:15:30.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2696 2024-09-19T04:15:30.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2699 2024-09-19T04:15:30.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2701 2024-09-19T04:15:30.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2702 2024-09-19T04:15:30.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2703 2024-09-19T04:15:30.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2704 2024-09-19T04:15:30.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2753 2024-09-19T04:15:30.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2753_fsm 2024-09-19T04:15:30.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2753_vm 2024-09-19T04:15:30.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2754 2024-09-19T04:15:30.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2755 2024-09-19T04:15:30.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2756 2024-09-19T04:15:30.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2757 2024-09-19T04:15:30.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2830 2024-09-19T04:15:30.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2831 2024-09-19T04:15:30.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2832 2024-09-19T04:15:30.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2833 2024-09-19T04:15:30.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2834 2024-09-19T04:15:30.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2835 2024-09-19T04:15:30.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2836 2024-09-19T04:15:30.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2837 2024-09-19T04:15:30.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 04:15 2838 2024-09-19T04:15:30.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2838_fsm 2024-09-19T04:15:30.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2838_vm 2024-09-19T04:15:30.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2839 2024-09-19T04:15:30.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840 2024-09-19T04:15:30.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840_fsm 2024-09-19T04:15:30.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2840_vm 2024-09-19T04:15:30.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2841 2024-09-19T04:15:30.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2995 2024-09-19T04:15:30.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2996 2024-09-19T04:15:30.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079 2024-09-19T04:15:30.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3079_fsm 2024-09-19T04:15:30.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079_vm 2024-09-19T04:15:30.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3080 2024-09-19T04:15:30.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3081 2024-09-19T04:15:30.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3085 2024-09-19T04:15:30.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3118 2024-09-19T04:15:30.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3119 2024-09-19T04:15:30.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 04:15 3164 2024-09-19T04:15:30.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3256 2024-09-19T04:15:30.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3257 2024-09-19T04:15:30.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3258 2024-09-19T04:15:30.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3350 2024-09-19T04:15:30.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3351 2024-09-19T04:15:30.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3379 2024-09-19T04:15:30.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3380 2024-09-19T04:15:30.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3381 2024-09-19T04:15:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3394 2024-09-19T04:15:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3394_fsm 2024-09-19T04:15:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3394_vm 2024-09-19T04:15:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3395 2024-09-19T04:15:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3429 2024-09-19T04:15:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3430 2024-09-19T04:15:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3431 2024-09-19T04:15:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3433 2024-09-19T04:15:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3439 2024-09-19T04:15:30.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3440 2024-09-19T04:15:30.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3455 2024-09-19T04:15:30.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 04:15 3456 2024-09-19T04:15:30.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3456_fsm 2024-09-19T04:15:30.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3456_vm 2024-09-19T04:15:30.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3466 2024-09-19T04:15:30.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3467 2024-09-19T04:15:30.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3468 2024-09-19T04:15:30.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3501 2024-09-19T04:15:30.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3502 2024-09-19T04:15:30.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3503 2024-09-19T04:15:30.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3534 2024-09-19T04:15:30.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541 2024-09-19T04:15:30.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3541_fsm 2024-09-19T04:15:30.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541_vm 2024-09-19T04:15:30.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3542 2024-09-19T04:15:30.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3574 2024-09-19T04:15:30.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3575 2024-09-19T04:15:30.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3576 2024-09-19T04:15:30.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3596 2024-09-19T04:15:30.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3597 2024-09-19T04:15:30.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3598 2024-09-19T04:15:30.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3599 2024-09-19T04:15:30.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600 2024-09-19T04:15:30.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3600_fsm 2024-09-19T04:15:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600_vm 2024-09-19T04:15:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601 2024-09-19T04:15:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3601_fsm 2024-09-19T04:15:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601_vm 2024-09-19T04:15:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602 2024-09-19T04:15:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3602_fsm 2024-09-19T04:15:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602_vm 2024-09-19T04:15:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603 2024-09-19T04:15:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603_fsm 2024-09-19T04:15:30.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3603_vm 2024-09-19T04:15:30.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3604 2024-09-19T04:15:30.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3605 2024-09-19T04:15:30.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3606 2024-09-19T04:15:30.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3607 2024-09-19T04:15:30.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3608 2024-09-19T04:15:30.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 3609 2024-09-19T04:15:30.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3712 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3764_fsm 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764_vm 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3766 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3767 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3997 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4143 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4144 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4145 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4146 2024-09-19T04:15:30.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4147 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4148 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4149 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4150 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4151 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4152 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4153 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4154 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4155 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4156 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4157 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4158 2024-09-19T04:15:30.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4159 2024-09-19T04:15:30.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4160 2024-09-19T04:15:30.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4161 2024-09-19T04:15:30.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4162 2024-09-19T04:15:30.905 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4163 2024-09-19T04:15:30.905 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4164 2024-09-19T04:15:30.905 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4165 2024-09-19T04:15:30.905 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4166 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4167 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4168 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4169 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4170 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4171 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4172 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4173 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4174 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 5002 2024-09-19T04:15:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 548 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 549 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6102 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6104 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6106 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6110 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6111 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6112 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6113 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6117 2024-09-19T04:15:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 826 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 827 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 828 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 04:15 pg_filenode.map 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 04:15 PG_VERSION 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:total 7778 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 . 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 19 04:15 .. 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 112 2024-09-19T04:15:30.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 113 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 1247 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1247_fsm 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1247_vm 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 04:15 1249 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1249_fsm 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1249_vm 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 04:15 1255 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1255_fsm 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1255_vm 2024-09-19T04:15:30.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 04:15 1259 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1259_fsm 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1259_vm 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 04:15 13344 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13344_fsm 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13344_vm 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13346 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13348 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13349_fsm 2024-09-19T04:15:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349_vm 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13351 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13353 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13354_fsm 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354_vm 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13356 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13358 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13359_fsm 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359_vm 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13361 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13363 2024-09-19T04:15:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1417 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1418 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 174 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 175 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2187 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2224 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2328 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2336 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2337 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2579 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2600 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2600_fsm 2024-09-19T04:15:30.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2600_vm 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2601_fsm 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601_vm 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2602 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2602_fsm 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2602_vm 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2603 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2603_fsm 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2603_vm 2024-09-19T04:15:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2604 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2605 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2605_fsm 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2605_vm 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2606_fsm 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606_vm 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2607 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2607_fsm 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2607_vm 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 04:15 2608 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2608_fsm 2024-09-19T04:15:30.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2608_vm 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 04:15 2609 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2609_fsm 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2609_vm 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2610 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2610_fsm 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2610_vm 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2611 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2612_fsm 2024-09-19T04:15:30.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612_vm 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2613 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2615_fsm 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615_vm 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2616 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2616_fsm 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2616_vm 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 04:15 2617 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2617_fsm 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2617_vm 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 04:15 2618 2024-09-19T04:15:30.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2618_fsm 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2618_vm 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 04:15 2619 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2619_fsm 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2619_vm 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2620 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2650 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2651 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2652 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2653 2024-09-19T04:15:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2654 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2655 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2656 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2657 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 04:15 2658 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 2659 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2660 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2661 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2662 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2663 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2664 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2665 2024-09-19T04:15:30.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2666 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2667 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2668 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2669 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2670 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 04:15 2673 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 04:15 2674 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 04:15 2675 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2678 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2679 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2680 2024-09-19T04:15:30.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2681 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2682 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2683 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2684 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2685 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2686 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2687 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2688 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2689 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 04:15 2690 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 04:15 2691 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2692 2024-09-19T04:15:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2693 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2696 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2699 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2701 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2702 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2703 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2704 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2753 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2753_fsm 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2753_vm 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2754 2024-09-19T04:15:30.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2755 2024-09-19T04:15:30.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2756 2024-09-19T04:15:30.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2757 2024-09-19T04:15:30.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2830 2024-09-19T04:15:30.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2831 2024-09-19T04:15:30.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2832 2024-09-19T04:15:30.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2833 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2834 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2835 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2836 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2837 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 04:15 2838 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2838_fsm 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2838_vm 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2839 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840_fsm 2024-09-19T04:15:30.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2840_vm 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2841 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2995 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2996 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3079_fsm 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079_vm 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3080 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3081 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3085 2024-09-19T04:15:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3118 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3119 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 04:15 3164 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3256 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3257 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3258 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3350 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3351 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3379 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3380 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3381 2024-09-19T04:15:30.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3394 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3394_fsm 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3394_vm 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3395 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3429 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3430 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3431 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3433 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3439 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3440 2024-09-19T04:15:30.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3455 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 04:15 3456 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3456_fsm 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3456_vm 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3466 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3467 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3468 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3501 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3502 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3503 2024-09-19T04:15:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3534 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3541_fsm 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541_vm 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3542 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3574 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3575 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3576 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3596 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3597 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3598 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3599 2024-09-19T04:15:30.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3600_fsm 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600_vm 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3601_fsm 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601_vm 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3602_fsm 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602_vm 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603_fsm 2024-09-19T04:15:30.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3603_vm 2024-09-19T04:15:30.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3604 2024-09-19T04:15:30.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3605 2024-09-19T04:15:30.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3606 2024-09-19T04:15:30.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3607 2024-09-19T04:15:30.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3608 2024-09-19T04:15:30.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 3609 2024-09-19T04:15:30.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3712 2024-09-19T04:15:30.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764 2024-09-19T04:15:30.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3764_fsm 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764_vm 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3766 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3767 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3997 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4143 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4144 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4145 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4146 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4147 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4148 2024-09-19T04:15:30.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4149 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4150 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4151 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4152 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4153 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4154 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4155 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4156 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4157 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4158 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4159 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4160 2024-09-19T04:15:30.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4161 2024-09-19T04:15:30.933 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4162 2024-09-19T04:15:30.933 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4163 2024-09-19T04:15:30.933 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4164 2024-09-19T04:15:30.933 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4165 2024-09-19T04:15:30.933 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4166 2024-09-19T04:15:30.933 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4167 2024-09-19T04:15:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4168 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4169 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4170 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4171 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4172 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4173 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4174 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 5002 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 548 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 549 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6102 2024-09-19T04:15:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6104 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6106 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6110 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6111 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6112 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6113 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6117 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 826 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 827 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 828 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 04:15 pg_filenode.map 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 04:15 PG_VERSION 2024-09-19T04:15:30.940 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/global: 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:total 538 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 Sep 19 04:15 . 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1213 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1213_fsm 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1213_vm 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1214 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1214_fsm 2024-09-19T04:15:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1214_vm 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 1232 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 1233 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1260 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1260_fsm 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1260_vm 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1261 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1261_fsm 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1261_vm 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1262 2024-09-19T04:15:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1262_fsm 2024-09-19T04:15:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1262_vm 2024-09-19T04:15:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2396 2024-09-19T04:15:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2396_fsm 2024-09-19T04:15:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2396_vm 2024-09-19T04:15:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2397 2024-09-19T04:15:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2671 2024-09-19T04:15:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2672 2024-09-19T04:15:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2676 2024-09-19T04:15:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2677 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2694 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2695 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2697 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2698 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2846 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2847 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2964 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2965 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2966 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2967 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3592 2024-09-19T04:15:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3593 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4060 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4061 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4175 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4176 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4177 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4178 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4181 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4182 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4183 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4184 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4185 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4186 2024-09-19T04:15:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6000 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6001 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6002 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6100 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6114 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6115 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 pg_control 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 04:15 pg_filenode.map 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/log: 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:total 3 2024-09-19T04:15:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:15 . 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 mappings 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:15 replorigin_checkpoint 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 snapshots 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:15 .. 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-19T04:15:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:15 .. 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:total 2 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 . 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 members 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 offsets 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-19T04:15:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:total 9 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 . 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 .. 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 0000 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:total 9 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 . 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 .. 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 0000 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-19T04:15:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-19T04:15:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:total 9 2024-09-19T04:15:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 . 2024-09-19T04:15:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 0000 2024-09-19T04:15:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-19T04:15:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:total 16386 2024-09-19T04:15:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:15 . 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:15 000000010000000000000001 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 archive_status 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:15 .. 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-19T04:15:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:total 9 2024-09-19T04:15:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 . 2024-09-19T04:15:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 19 04:15 .. 2024-09-19T04:15:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 0000 2024-09-19T04:15:31.001 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-19T04:15:31.171 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-19T04:15:31.257 INFO:teuthology.orchestra.run.smithi094.stderr:dropping old tables... 2024-09-19T04:15:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-19T04:15:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-19T04:15:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-19T04:15:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-19T04:15:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:creating tables... 2024-09-19T04:15:31.301 INFO:teuthology.orchestra.run.smithi094.stderr:generating data (client-side)... 2024-09-19T04:15:31.776 INFO:teuthology.orchestra.run.smithi094.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.45 s, remaining 14.06 s) 2024-09-19T04:15:31.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:31 smithi112 ceph-mon[35051]: pgmap v18: 97 pgs: 97 active+clean; 44 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 3.1 MiB/s wr, 99 op/s 2024-09-19T04:15:31.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:31 smithi094 ceph-mon[30972]: pgmap v18: 97 pgs: 97 active+clean; 44 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 3.1 MiB/s wr, 99 op/s 2024-09-19T04:15:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:31 smithi104 ceph-mon[34145]: pgmap v18: 97 pgs: 97 active+clean; 44 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 3.1 MiB/s wr, 99 op/s 2024-09-19T04:15:32.882 INFO:teuthology.orchestra.run.smithi094.stderr:200000 of 3200000 tuples (6%) done (elapsed 1.56 s, remaining 23.39 s) 2024-09-19T04:15:33.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:33 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:15:33.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:33 smithi112 ceph-mon[35051]: pgmap v19: 97 pgs: 97 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 6.9 MiB/s wr, 190 op/s 2024-09-19T04:15:33.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:33 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:15:33.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:33 smithi094 ceph-mon[30972]: pgmap v19: 97 pgs: 97 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 6.9 MiB/s wr, 190 op/s 2024-09-19T04:15:33.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:33 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:15:33.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:33 smithi104 ceph-mon[34145]: pgmap v19: 97 pgs: 97 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 6.9 MiB/s wr, 190 op/s 2024-09-19T04:15:34.027 INFO:teuthology.orchestra.run.smithi094.stderr:300000 of 3200000 tuples (9%) done (elapsed 2.71 s, remaining 26.15 s) 2024-09-19T04:15:34.735 INFO:teuthology.orchestra.run.smithi094.stderr:400000 of 3200000 tuples (12%) done (elapsed 3.41 s, remaining 23.89 s) 2024-09-19T04:15:35.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:35 smithi112 ceph-mon[35051]: pgmap v20: 97 pgs: 97 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 4.8 MiB/s wr, 118 op/s 2024-09-19T04:15:35.792 INFO:teuthology.orchestra.run.smithi094.stderr:500000 of 3200000 tuples (15%) done (elapsed 4.47 s, remaining 24.14 s) 2024-09-19T04:15:35.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:35 smithi094 ceph-mon[30972]: pgmap v20: 97 pgs: 97 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 4.8 MiB/s wr, 118 op/s 2024-09-19T04:15:35.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:35 smithi104 ceph-mon[34145]: pgmap v20: 97 pgs: 97 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 4.8 MiB/s wr, 118 op/s 2024-09-19T04:15:36.592 INFO:teuthology.orchestra.run.smithi094.stderr:600000 of 3200000 tuples (18%) done (elapsed 5.27 s, remaining 22.84 s) 2024-09-19T04:15:37.662 INFO:teuthology.orchestra.run.smithi094.stderr:700000 of 3200000 tuples (21%) done (elapsed 6.34 s, remaining 22.64 s) 2024-09-19T04:15:37.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:37 smithi112 ceph-mon[35051]: pgmap v21: 97 pgs: 97 active+clean; 194 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 24 MiB/s wr, 138 op/s 2024-09-19T04:15:37.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:37 smithi094 ceph-mon[30972]: pgmap v21: 97 pgs: 97 active+clean; 194 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 24 MiB/s wr, 138 op/s 2024-09-19T04:15:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:37 smithi104 ceph-mon[34145]: pgmap v21: 97 pgs: 97 active+clean; 194 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 24 MiB/s wr, 138 op/s 2024-09-19T04:15:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:800000 of 3200000 tuples (25%) done (elapsed 7.42 s, remaining 22.25 s) 2024-09-19T04:15:39.537 INFO:teuthology.orchestra.run.smithi094.stderr:900000 of 3200000 tuples (28%) done (elapsed 8.21 s, remaining 20.99 s) 2024-09-19T04:15:39.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:39 smithi112 ceph-mon[35051]: pgmap v22: 97 pgs: 97 active+clean; 194 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 23 MiB/s wr, 110 op/s 2024-09-19T04:15:39.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:39 smithi094 ceph-mon[30972]: pgmap v22: 97 pgs: 97 active+clean; 194 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 23 MiB/s wr, 110 op/s 2024-09-19T04:15:39.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:39 smithi104 ceph-mon[34145]: pgmap v22: 97 pgs: 97 active+clean; 194 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 23 MiB/s wr, 110 op/s 2024-09-19T04:15:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:1000000 of 3200000 tuples (31%) done (elapsed 9.28 s, remaining 20.42 s) 2024-09-19T04:15:41.670 INFO:teuthology.orchestra.run.smithi094.stderr:1100000 of 3200000 tuples (34%) done (elapsed 10.35 s, remaining 19.75 s) 2024-09-19T04:15:41.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:41 smithi112 ceph-mon[35051]: pgmap v23: 97 pgs: 97 active+clean; 194 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 23 MiB/s wr, 110 op/s 2024-09-19T04:15:41.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:41 smithi094 ceph-mon[30972]: pgmap v23: 97 pgs: 97 active+clean; 194 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 23 MiB/s wr, 110 op/s 2024-09-19T04:15:41.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:41 smithi104 ceph-mon[34145]: pgmap v23: 97 pgs: 97 active+clean; 194 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 23 MiB/s wr, 110 op/s 2024-09-19T04:15:42.442 INFO:teuthology.orchestra.run.smithi094.stderr:1200000 of 3200000 tuples (37%) done (elapsed 11.12 s, remaining 18.53 s) 2024-09-19T04:15:43.455 INFO:teuthology.orchestra.run.smithi094.stderr:1300000 of 3200000 tuples (40%) done (elapsed 12.13 s, remaining 17.73 s) 2024-09-19T04:15:43.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:43 smithi112 ceph-mon[35051]: pgmap v24: 97 pgs: 97 active+clean; 310 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 43 MiB/s wr, 127 op/s 2024-09-19T04:15:43.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:43 smithi094 ceph-mon[30972]: pgmap v24: 97 pgs: 97 active+clean; 310 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 43 MiB/s wr, 127 op/s 2024-09-19T04:15:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:43 smithi104 ceph-mon[34145]: pgmap v24: 97 pgs: 97 active+clean; 310 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 43 MiB/s wr, 127 op/s 2024-09-19T04:15:43.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:15:44.241 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:44.564 INFO:teuthology.orchestra.run.smithi094.stderr:1400000 of 3200000 tuples (43%) done (elapsed 13.24 s, remaining 17.03 s) 2024-09-19T04:15:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.785+0000 7f1fe7e71640 1 -- 172.21.15.94:0/1909202499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe0106a70 msgr2=0x7f1fe0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.785+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/1909202499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe0106a70 0x7f1fe0106e50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009930 tx=0x7f1fdc02a420 comp rx=0 tx=0).stop 2024-09-19T04:15:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.785+0000 7f1fe7e71640 1 -- 172.21.15.94:0/1909202499 shutdown_connections 2024-09-19T04:15:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.785+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/1909202499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe0101410 0x7f1fe010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.785+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/1909202499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fe0100a70 0x7f1fe0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.785+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/1909202499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe0106a70 0x7f1fe0106e50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.785+0000 7f1fe7e71640 1 -- 172.21.15.94:0/1909202499 >> 172.21.15.94:0/1909202499 conn(0x7f1fe00fc7d0 msgr2=0x7f1fe00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.785+0000 7f1fe7e71640 1 -- 172.21.15.94:0/1909202499 shutdown_connections 2024-09-19T04:15:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.785+0000 7f1fe7e71640 1 -- 172.21.15.94:0/1909202499 wait complete. 2024-09-19T04:15:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe7e71640 1 Processor -- start 2024-09-19T04:15:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe7e71640 1 -- start start 2024-09-19T04:15:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe7e71640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fe0100a70 0x7f1fe0102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe7e71640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe0101410 0x7f1fe0102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe7e71640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe0106a70 0x7f1fe01030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe7e71640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0110d20 con 0x7f1fe0101410 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe7e71640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0110ba0 con 0x7f1fe0106a70 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe7e71640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0110ea0 con 0x7f1fe0100a70 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe540c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe0101410 0x7f1fe0102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe5c0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fe0100a70 0x7f1fe0102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe540c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe0101410 0x7f1fe0102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47632/0 (socket says 172.21.15.94:47632) 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.786+0000 7f1fe540c640 1 -- 172.21.15.94:0/125365319 learned_addr learned my addr 172.21.15.94:0/125365319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fe640e640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe0106a70 0x7f1fe01030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fe5c0d640 1 -- 172.21.15.94:0/125365319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe0106a70 msgr2=0x7f1fe01030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fe5c0d640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe0106a70 0x7f1fe01030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fe5c0d640 1 -- 172.21.15.94:0/125365319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe0101410 msgr2=0x7f1fe0102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fe5c0d640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe0101410 0x7f1fe0102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fe5c0d640 1 -- 172.21.15.94:0/125365319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe01a6840 con 0x7f1fe0100a70 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fe5c0d640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fe0100a70 0x7f1fe0102640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009230 tx=0x7f1fdc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fd6ffd640 1 -- 172.21.15.94:0/125365319 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc03f070 con 0x7f1fe0100a70 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fe7e71640 1 -- 172.21.15.94:0/125365319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe01a6ad0 con 0x7f1fe0100a70 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fd6ffd640 1 -- 172.21.15.94:0/125365319 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fdc031cf0 con 0x7f1fe0100a70 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fd6ffd640 1 -- 172.21.15.94:0/125365319 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc035070 con 0x7f1fe0100a70 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.787+0000 7f1fe7e71640 1 -- 172.21.15.94:0/125365319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe01a6fb0 con 0x7f1fe0100a70 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.790+0000 7f1fd6ffd640 1 -- 172.21.15.94:0/125365319 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f1fdc003c10 con 0x7f1fe0100a70 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.790+0000 7f1fea24a740 1 Processor -- start 2024-09-19T04:15:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.790+0000 7f1fea24a740 1 -- start start 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b04880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fd6ffd640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1fcc0a3440 0x7f1fcc0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 0x55d708b0f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b0f680 0x55d708b11a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fea24a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d708a6a2e0 con 0x55d708b044a0 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fea24a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d708a11ce0 con 0x55d708b0f680 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fea24a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d708a662a0 con 0x55d708b04dc0 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe640e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b04880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe5c0d640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1fcc0a3440 0x7f1fcc0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe540c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 0x55d708b0f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe640e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b04880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47646/0 (socket says 172.21.15.94:47646) 2024-09-19T04:15:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe640e640 1 -- 172.21.15.94:0/2327783938 learned_addr learned my addr 172.21.15.94:0/2327783938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe640e640 1 --2- 172.21.15.94:0/2327783938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b0f680 0x55d708b11a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fd6ffd640 1 -- 172.21.15.94:0/125365319 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f1fdc0f0610 con 0x7f1fe0100a70 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe540c640 1 -- 172.21.15.94:0/2327783938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b0f680 msgr2=0x55d708b11a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe540c640 1 --2- 172.21.15.94:0/2327783938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b0f680 0x55d708b11a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe540c640 1 -- 172.21.15.94:0/2327783938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 msgr2=0x55d708b04880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe540c640 1 --2- 172.21.15.94:0/2327783938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b04880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.791+0000 7f1fe540c640 1 -- 172.21.15.94:0/2327783938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7089f6440 con 0x55d708b04dc0 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.792+0000 7f1fe640e640 1 --2- 172.21.15.94:0/2327783938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b04880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.792+0000 7f1fe5c0d640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1fcc0a3440 0x7f1fcc0a5900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0edc90 tx=0x7f1fdc0e74e0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.792+0000 7f1fe540c640 1 --2- 172.21.15.94:0/2327783938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 0x55d708b0f140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00058c0 tx=0x7f1fd000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98633badd43bb227 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.792+0000 7f1fcbfff640 1 -- 172.21.15.94:0/2327783938 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0004220 con 0x55d708b04dc0 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.792+0000 7f1fcbfff640 1 -- 172.21.15.94:0/2327783938 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd00043c0 con 0x55d708b04dc0 2024-09-19T04:15:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.792+0000 7f1fcbfff640 1 -- 172.21.15.94:0/2327783938 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00187c0 con 0x55d708b04dc0 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 -- 172.21.15.94:0/2327783938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 msgr2=0x55d708b0f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 --2- 172.21.15.94:0/2327783938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 0x55d708b0f140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00058c0 tx=0x7f1fd000f790 comp rx=0 tx=0).stop 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 -- 172.21.15.94:0/2327783938 shutdown_connections 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 --2- 172.21.15.94:0/2327783938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b0f680 0x55d708b11a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 --2- 172.21.15.94:0/2327783938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 0x55d708b0f140 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 --2- 172.21.15.94:0/2327783938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b04880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 -- 172.21.15.94:0/2327783938 >> 172.21.15.94:0/2327783938 conn(0x55d708b00a30 msgr2=0x55d708b00e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 -- 172.21.15.94:0/2327783938 shutdown_connections 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 -- 172.21.15.94:0/2327783938 wait complete. 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.793+0000 7f1fea24a740 1 -- start start 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fea24a740 1 Processor -- start 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b0ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 0x55d708b0c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 0x55d708b0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fea24a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d708b12250 con 0x55d708b044a0 2024-09-19T04:15:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fea24a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d708a25050 con 0x55d708b04dc0 2024-09-19T04:15:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fea24a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d708b123d0 con 0x55d708b0f680 2024-09-19T04:15:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fea24a740 10 client.0 _ll_register_callbacks cb 0x55d708a65160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fe640e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b0ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fe640e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b0ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47656/0 (socket says 172.21.15.94:47656) 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fe640e640 1 -- 172.21.15.94:0/298142502 learned_addr learned my addr 172.21.15.94:0/298142502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.794+0000 7f1fe640e640 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 0x55d708b0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fe540c640 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 0x55d708b0c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fe640e640 1 -- 172.21.15.94:0/298142502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 msgr2=0x55d708b0c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fe640e640 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 0x55d708b0c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fe640e640 1 -- 172.21.15.94:0/298142502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 msgr2=0x55d708b0c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fe640e640 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 0x55d708b0c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fe640e640 1 -- 172.21.15.94:0/298142502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7087db3c0 con 0x55d708b044a0 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fe640e640 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b0ef70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80117f0 tx=0x7f1fd8007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6aa118c942764cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fc97fa640 1 -- 172.21.15.94:0/298142502 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011e30 con 0x55d708b044a0 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fc97fa640 1 -- 172.21.15.94:0/298142502 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f1fd80110d0 con 0x55d708b044a0 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fea24a740 1 -- 172.21.15.94:0/298142502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd00043c0 con 0x55d708b044a0 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fc97fa640 1 -- 172.21.15.94:0/298142502 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1fd8026050 con 0x55d708b044a0 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fea24a740 10 client.15369 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.795+0000 7f1fea24a740 1 -- 172.21.15.94:0/298142502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d708b26420 con 0x55d708b044a0 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.796+0000 7f1fc97fa640 1 -- 172.21.15.94:0/298142502 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd00043c0 con 0x55d708b044a0 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.796+0000 7f1fc97fa640 1 -- 172.21.15.94:0/298142502 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1fd800f890 con 0x55d708b044a0 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.796+0000 7f1fea24a740 10 client.15369 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.796+0000 7f1fea24a740 10 client.15369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.796+0000 7f1fea24a740 20 client.15369 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.796+0000 7f1fea24a740 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d708b278e0 0x55d708b29d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.796+0000 7f1fea24a740 4 client.15369 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.796+0000 7f1fea24a740 1 -- 172.21.15.94:0/298142502 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d708a77850 con 0x55d708b278e0 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.796+0000 7f1fe540c640 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d708b278e0 0x55d708b29d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.797+0000 7f1fc97fa640 10 client.15369 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.797+0000 7f1fe540c640 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d708b278e0 0x55d708b29d30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74073e213ee6e6b server_cookie=3c25381b6e790b71 in_seq=0 out_seq=0 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fc97fa640 1 -- 172.21.15.94:0/298142502 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d708a77850 con 0x55d708b278e0 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fc97fa640 10 client.15369 handle_command_reply: tid=0 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 client.15369 shutdown 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 -- 172.21.15.94:0/298142502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 msgr2=0x55d708b0ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b0ef70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80117f0 tx=0x7f1fd8007520 comp rx=0 tx=0).stop 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 -- 172.21.15.94:0/298142502 shutdown_connections 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d708b278e0 0x55d708b29d30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fc97fa640 0 client.15369 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 0x55d708b0c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 0x55d708b0c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 --2- 172.21.15.94:0/298142502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b0ef70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 -- 172.21.15.94:0/298142502 >> 172.21.15.94:0/298142502 conn(0x55d708b00a30 msgr2=0x55d708affb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.798+0000 7f1fea24a740 1 -- 172.21.15.94:0/298142502 shutdown_connections 2024-09-19T04:15:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.799+0000 7f1fea24a740 1 -- 172.21.15.94:0/298142502 wait complete. 2024-09-19T04:15:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.799+0000 7f1fea24a740 20 client.15369 trim_cache size 0 max 16384 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fea24a740 1 Processor -- start 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fea24a740 1 -- start start 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b1b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 0x55d708b2f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 0x55d708b2fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fea24a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01addd0 con 0x55d708b044a0 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fea24a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01ab6a0 con 0x55d708b04dc0 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fea24a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01a9530 con 0x55d708b0f680 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fe640e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b1b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.819+0000 7f1fe640e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b1b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47670/0 (socket says 172.21.15.94:47670) 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fe640e640 1 -- 172.21.15.94:0/3710186287 learned_addr learned my addr 172.21.15.94:0/3710186287 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:44.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fe540c640 1 --2- 172.21.15.94:0/3710186287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 0x55d708b2f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fe640e640 1 --2- 172.21.15.94:0/3710186287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 0x55d708b2fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fe640e640 1 -- 172.21.15.94:0/3710186287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 msgr2=0x55d708b2fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fe640e640 1 --2- 172.21.15.94:0/3710186287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 0x55d708b2fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fe640e640 1 -- 172.21.15.94:0/3710186287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 msgr2=0x55d708b2f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fe640e640 1 --2- 172.21.15.94:0/3710186287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 0x55d708b2f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fe640e640 1 -- 172.21.15.94:0/3710186287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d708b0a770 con 0x55d708b044a0 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fe640e640 1 --2- 172.21.15.94:0/3710186287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b1b4b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8021560 tx=0x7f1fd8007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d5365576757ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fcb7fe640 1 -- 172.21.15.94:0/3710186287 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80112d0 con 0x55d708b044a0 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fcb7fe640 1 -- 172.21.15.94:0/3710186287 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd800cdf0 con 0x55d708b044a0 2024-09-19T04:15:44.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.820+0000 7f1fcb7fe640 1 -- 172.21.15.94:0/3710186287 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd800f7d0 con 0x55d708b044a0 2024-09-19T04:15:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.821+0000 7f1fea24a740 1 -- 172.21.15.94:0/3710186287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 msgr2=0x55d708b1b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.821+0000 7f1fea24a740 1 --2- 172.21.15.94:0/3710186287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b1b4b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8021560 tx=0x7f1fd8007c30 comp rx=0 tx=0).stop 2024-09-19T04:15:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.821+0000 7f1fea24a740 1 -- 172.21.15.94:0/3710186287 shutdown_connections 2024-09-19T04:15:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.821+0000 7f1fea24a740 1 --2- 172.21.15.94:0/3710186287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b0f680 0x55d708b2fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.821+0000 7f1fea24a740 1 --2- 172.21.15.94:0/3710186287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b04dc0 0x55d708b2f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.821+0000 7f1fea24a740 1 --2- 172.21.15.94:0/3710186287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b044a0 0x55d708b1b4b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.821+0000 7f1fea24a740 1 -- 172.21.15.94:0/3710186287 >> 172.21.15.94:0/3710186287 conn(0x55d708b00a30 msgr2=0x55d708affb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:44.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 -- 172.21.15.94:0/3710186287 shutdown_connections 2024-09-19T04:15:44.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 -- 172.21.15.94:0/3710186287 wait complete. 2024-09-19T04:15:44.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 -- start start 2024-09-19T04:15:44.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 Processor -- start 2024-09-19T04:15:44.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b044a0 0x55d708b17970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 0x55d708b17eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b0f680 0x55d708b14de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01a9b80 con 0x55d708b0f680 2024-09-19T04:15:44.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01aafd0 con 0x55d708b044a0 2024-09-19T04:15:44.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.822+0000 7f1fea24a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01aa230 con 0x55d708b04dc0 2024-09-19T04:15:44.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fea24a740 10 client.0 _ll_register_callbacks cb 0x7f1fd8007e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:44.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b0f680 0x55d708b14de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b0f680 0x55d708b14de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47684/0 (socket says 172.21.15.94:47684) 2024-09-19T04:15:44.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 -- 172.21.15.94:0/983867898 learned_addr learned my addr 172.21.15.94:0/983867898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:44.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe540c640 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 0x55d708b17eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b044a0 0x55d708b17970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 -- 172.21.15.94:0/983867898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 msgr2=0x55d708b17eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 0x55d708b17eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 -- 172.21.15.94:0/983867898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b0f680 msgr2=0x55d708b14de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b0f680 0x55d708b14de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 -- 172.21.15.94:0/983867898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d708b15610 con 0x55d708b044a0 2024-09-19T04:15:44.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fe640e640 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b044a0 0x55d708b17970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800f4b0 tx=0x7f1fd8007c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f29f5d352ec6db53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.823+0000 7f1fc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:15:44.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fc97fa640 1 -- 172.21.15.94:0/983867898 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8016070 con 0x55d708b044a0 2024-09-19T04:15:44.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fc97fa640 1 -- 172.21.15.94:0/983867898 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f1fd8011300 con 0x55d708b044a0 2024-09-19T04:15:44.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fea24a740 1 -- 172.21.15.94:0/983867898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d708a71d60 con 0x55d708b044a0 2024-09-19T04:15:44.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fc97fa640 1 -- 172.21.15.94:0/983867898 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1fd8003970 con 0x55d708b044a0 2024-09-19T04:15:44.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fea24a740 10 client.24980 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:44.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fea24a740 1 -- 172.21.15.94:0/983867898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d708b921f0 con 0x55d708b044a0 2024-09-19T04:15:44.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fc97fa640 1 -- 172.21.15.94:0/983867898 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d708a71d60 con 0x55d708b044a0 2024-09-19T04:15:44.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fc97fa640 1 -- 172.21.15.94:0/983867898 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1fd8029070 con 0x55d708b044a0 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fea24a740 10 client.24980 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fea24a740 10 client.24980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fea24a740 20 client.24980 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fea24a740 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d708b278e0 0x55d708b94710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fea24a740 4 client.24980 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.824+0000 7f1fea24a740 1 -- 172.21.15.94:0/983867898 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7089ee210 con 0x55d708b278e0 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.825+0000 7f1fe540c640 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d708b278e0 0x55d708b94710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.825+0000 7f1fc97fa640 10 client.24980 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.825+0000 7f1fe540c640 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d708b278e0 0x55d708b94710 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b472b3d0dcb501 server_cookie=37116eaad6ac3a37 in_seq=0 out_seq=0 2024-09-19T04:15:44.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.825+0000 7f1fc97fa640 1 -- 172.21.15.94:0/983867898 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d7089ee210 con 0x55d708b278e0 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.825+0000 7f1fc97fa640 10 client.24980 handle_command_reply: tid=0 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.825+0000 7f1fea24a740 1 client.24980 shutdown 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 -- 172.21.15.94:0/983867898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b044a0 msgr2=0x55d708b17970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b044a0 0x55d708b17970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800f4b0 tx=0x7f1fd8007c30 comp rx=0 tx=0).stop 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 -- 172.21.15.94:0/983867898 shutdown_connections 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d708b278e0 0x55d708b94710 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d708b0f680 0x55d708b14de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fc97fa640 0 client.24980 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d708b04dc0 0x55d708b17eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 --2- 172.21.15.94:0/983867898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d708b044a0 0x55d708b17970 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 -- 172.21.15.94:0/983867898 >> 172.21.15.94:0/983867898 conn(0x55d708b00a30 msgr2=0x55d708b0ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:44.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 -- 172.21.15.94:0/983867898 shutdown_connections 2024-09-19T04:15:44.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 1 -- 172.21.15.94:0/983867898 wait complete. 2024-09-19T04:15:44.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.826+0000 7f1fea24a740 20 client.24980 trim_cache size 0 max 16384 2024-09-19T04:15:44.836 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:15:44.836 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:15:44.836 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:15:44.836 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 -- 172.21.15.94:0/125365319 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1fcc0a3440 msgr2=0x7f1fcc0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1fcc0a3440 0x7f1fcc0a5900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0edc90 tx=0x7f1fdc0e74e0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 -- 172.21.15.94:0/125365319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fe0100a70 msgr2=0x7f1fe0102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fe0100a70 0x7f1fe0102640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009230 tx=0x7f1fdc004270 comp rx=0 tx=0).stop 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 -- 172.21.15.94:0/125365319 shutdown_connections 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1fcc0a3440 0x7f1fcc0a5900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe0106a70 0x7f1fe01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe0101410 0x7f1fe0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 --2- 172.21.15.94:0/125365319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fe0100a70 0x7f1fe0102640 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 -- 172.21.15.94:0/125365319 >> 172.21.15.94:0/125365319 conn(0x7f1fe00fc7d0 msgr2=0x7f1fe0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:44.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 -- 172.21.15.94:0/125365319 shutdown_connections 2024-09-19T04:15:44.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:44.827+0000 7f1fe7e71640 1 -- 172.21.15.94:0/125365319 wait complete. 2024-09-19T04:15:45.358 INFO:teuthology.orchestra.run.smithi094.stderr:1500000 of 3200000 tuples (46%) done (elapsed 14.04 s, remaining 15.91 s) 2024-09-19T04:15:45.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:45 smithi112 ceph-mon[35051]: pgmap v25: 97 pgs: 97 active+clean; 310 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 39 MiB/s wr, 36 op/s 2024-09-19T04:15:45.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:45 smithi094 ceph-mon[30972]: pgmap v25: 97 pgs: 97 active+clean; 310 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 39 MiB/s wr, 36 op/s 2024-09-19T04:15:45.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:45 smithi104 ceph-mon[34145]: pgmap v25: 97 pgs: 97 active+clean; 310 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 39 MiB/s wr, 36 op/s 2024-09-19T04:15:46.500 INFO:teuthology.orchestra.run.smithi094.stderr:1600000 of 3200000 tuples (50%) done (elapsed 15.18 s, remaining 15.18 s) 2024-09-19T04:15:47.124 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:15:47.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:15:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:1700000 of 3200000 tuples (53%) done (elapsed 15.97 s, remaining 14.09 s) 2024-09-19T04:15:47.424 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:47.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:47 smithi112 ceph-mon[35051]: pgmap v26: 97 pgs: 97 active+clean; 427 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 59 MiB/s wr, 51 op/s 2024-09-19T04:15:47.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:47 smithi094 ceph-mon[30972]: pgmap v26: 97 pgs: 97 active+clean; 427 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 59 MiB/s wr, 51 op/s 2024-09-19T04:15:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:47 smithi104 ceph-mon[34145]: pgmap v26: 97 pgs: 97 active+clean; 427 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 59 MiB/s wr, 51 op/s 2024-09-19T04:15:48.529 INFO:teuthology.orchestra.run.smithi094.stderr:1800000 of 3200000 tuples (56%) done (elapsed 17.21 s, remaining 13.38 s) 2024-09-19T04:15:49.692 INFO:teuthology.orchestra.run.smithi094.stderr:1900000 of 3200000 tuples (59%) done (elapsed 18.37 s, remaining 12.57 s) 2024-09-19T04:15:49.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:49 smithi112 ceph-mon[35051]: pgmap v27: 97 pgs: 97 active+clean; 427 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 40 MiB/s wr, 31 op/s 2024-09-19T04:15:49.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:49 smithi094 ceph-mon[30972]: pgmap v27: 97 pgs: 97 active+clean; 427 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 40 MiB/s wr, 31 op/s 2024-09-19T04:15:49.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:49 smithi104 ceph-mon[34145]: pgmap v27: 97 pgs: 97 active+clean; 427 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 40 MiB/s wr, 31 op/s 2024-09-19T04:15:50.490 INFO:teuthology.orchestra.run.smithi094.stderr:2000000 of 3200000 tuples (62%) done (elapsed 19.17 s, remaining 11.50 s) 2024-09-19T04:15:50.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.607+0000 7fb543790640 1 -- 172.21.15.94:0/4155866480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 msgr2=0x7fb53c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:50.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.607+0000 7fb543790640 1 --2- 172.21.15.94:0/4155866480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 0x7fb53c1029b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009900 tx=0x7fb52c02a480 comp rx=0 tx=0).stop 2024-09-19T04:15:50.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.607+0000 7fb543790640 1 -- 172.21.15.94:0/4155866480 shutdown_connections 2024-09-19T04:15:50.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.607+0000 7fb543790640 1 --2- 172.21.15.94:0/4155866480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c102ef0 0x7fb53c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.607+0000 7fb543790640 1 --2- 172.21.15.94:0/4155866480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 0x7fb53c1029b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.607+0000 7fb543790640 1 --2- 172.21.15.94:0/4155866480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c100e20 0x7fb53c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.607+0000 7fb543790640 1 -- 172.21.15.94:0/4155866480 >> 172.21.15.94:0/4155866480 conn(0x7fb53c0fc7d0 msgr2=0x7fb53c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.607+0000 7fb543790640 1 -- 172.21.15.94:0/4155866480 shutdown_connections 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.607+0000 7fb543790640 1 -- 172.21.15.94:0/4155866480 wait complete. 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb543790640 1 Processor -- start 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb543790640 1 -- start start 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb543790640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c100e20 0x7fb53c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb543790640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 0x7fb53c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb543790640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c102ef0 0x7fb53c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb543790640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c110e30 con 0x7fb53c102570 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb543790640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c110cb0 con 0x7fb53c102ef0 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb543790640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c110fb0 con 0x7fb53c100e20 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb540d2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 0x7fb53c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:50.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb540d2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 0x7fb53c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47710/0 (socket says 172.21.15.94:47710) 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb540d2b640 1 -- 172.21.15.94:0/2606809904 learned_addr learned my addr 172.21.15.94:0/2606809904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb541d2d640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c102ef0 0x7fb53c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb54152c640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c100e20 0x7fb53c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb540d2b640 1 -- 172.21.15.94:0/2606809904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c100e20 msgr2=0x7fb53c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb540d2b640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c100e20 0x7fb53c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb540d2b640 1 -- 172.21.15.94:0/2606809904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c102ef0 msgr2=0x7fb53c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb540d2b640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c102ef0 0x7fb53c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.608+0000 7fb540d2b640 1 -- 172.21.15.94:0/2606809904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c1a6900 con 0x7fb53c102570 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.609+0000 7fb540d2b640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 0x7fb53c0fd1c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb52c006010 tx=0x7fb52c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.609+0000 7fb5327fc640 1 -- 172.21.15.94:0/2606809904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c03f070 con 0x7fb53c102570 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.609+0000 7fb5327fc640 1 -- 172.21.15.94:0/2606809904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52c003aa0 con 0x7fb53c102570 2024-09-19T04:15:50.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.609+0000 7fb5327fc640 1 -- 172.21.15.94:0/2606809904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c035030 con 0x7fb53c102570 2024-09-19T04:15:50.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.609+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53c1a6b90 con 0x7fb53c102570 2024-09-19T04:15:50.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.609+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53c1a7070 con 0x7fb53c102570 2024-09-19T04:15:50.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.610+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53c101200 con 0x7fb53c102570 2024-09-19T04:15:50.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.611+0000 7fb5327fc640 1 -- 172.21.15.94:0/2606809904 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7fb52c042070 con 0x7fb53c102570 2024-09-19T04:15:50.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.612+0000 7fb5327fc640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5280a3440 0x7fb5280a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:50.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.612+0000 7fb54152c640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5280a3440 0x7fb5280a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:50.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.612+0000 7fb5327fc640 1 -- 172.21.15.94:0/2606809904 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb52c0f0720 con 0x7fb53c102570 2024-09-19T04:15:50.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.612+0000 7fb54152c640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5280a3440 0x7fb5280a5900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5380058c0 tx=0x7fb53800a250 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:50.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.617+0000 7fb5327fc640 1 -- 172.21.15.94:0/2606809904 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb52c0bc930 con 0x7fb53c102570 2024-09-19T04:15:50.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.907+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb53c10a0a0 con 0x7fb53c102570 2024-09-19T04:15:50.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.908+0000 7fb5327fc640 1 -- 172.21.15.94:0/2606809904 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11918 (secure 0 0 0) 0x7fb52c0c0c30 con 0x7fb53c102570 2024-09-19T04:15:50.909 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:15:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"btime":"2024-09-19T04:15:32:461979+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:15:31.810015+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24466,24467,24454]},"id":1}]} 2024-09-19T04:15:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5280a3440 msgr2=0x7fb5280a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5280a3440 0x7fb5280a5900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5380058c0 tx=0x7fb53800a250 comp rx=0 tx=0).stop 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 msgr2=0x7fb53c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 0x7fb53c0fd1c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb52c006010 tx=0x7fb52c0042f0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 shutdown_connections 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5280a3440 0x7fb5280a5900 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c102ef0 0x7fb53c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c102570 0x7fb53c0fd1c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 --2- 172.21.15.94:0/2606809904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c100e20 0x7fb53c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 >> 172.21.15.94:0/2606809904 conn(0x7fb53c0fc7d0 msgr2=0x7fb53c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 shutdown_connections 2024-09-19T04:15:50.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:50.910+0000 7fb543790640 1 -- 172.21.15.94:0/2606809904 wait complete. 2024-09-19T04:15:51.237 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:50 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2606809904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:15:51.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:50 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2606809904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:15:51.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:50 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2606809904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:15:51.535 INFO:teuthology.orchestra.run.smithi094.stderr:2100000 of 3200000 tuples (65%) done (elapsed 20.21 s, remaining 10.59 s) 2024-09-19T04:15:51.591 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:15:51.873 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:52.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:51 smithi112 ceph-mon[35051]: pgmap v28: 97 pgs: 97 active+clean; 427 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 40 MiB/s wr, 31 op/s 2024-09-19T04:15:52.292 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:51 smithi094 ceph-mon[30972]: pgmap v28: 97 pgs: 97 active+clean; 427 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 40 MiB/s wr, 31 op/s 2024-09-19T04:15:52.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:51 smithi104 ceph-mon[34145]: pgmap v28: 97 pgs: 97 active+clean; 427 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 40 MiB/s wr, 31 op/s 2024-09-19T04:15:52.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 -- 172.21.15.94:0/1644689756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c102570 msgr2=0x7fa86c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:52.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1644689756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c102570 0x7fa86c1029b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa860009900 tx=0x7fa86002a480 comp rx=0 tx=0).stop 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 -- 172.21.15.94:0/1644689756 shutdown_connections 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1644689756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa86c102ef0 0x7fa86c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1644689756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c102570 0x7fa86c1029b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1644689756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa86c100e20 0x7fa86c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 -- 172.21.15.94:0/1644689756 >> 172.21.15.94:0/1644689756 conn(0x7fa86c0fc7d0 msgr2=0x7fa86c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 -- 172.21.15.94:0/1644689756 shutdown_connections 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 -- 172.21.15.94:0/1644689756 wait complete. 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.384+0000 7fa8721a0640 1 Processor -- start 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa8721a0640 1 -- start start 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa8721a0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c100e20 0x7fa86c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa8721a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa86c102570 0x7fa86c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa8721a0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa86c102ef0 0x7fa86c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa8721a0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c110e30 con 0x7fa86c100e20 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa8721a0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c110cb0 con 0x7fa86c102ef0 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa8721a0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c110fb0 con 0x7fa86c102570 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c100e20 0x7fa86c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c100e20 0x7fa86c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48160/0 (socket says 172.21.15.94:48160) 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa86b7fe640 1 -- 172.21.15.94:0/1415558409 learned_addr learned my addr 172.21.15.94:0/1415558409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa86affd640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa86c102570 0x7fa86c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.385+0000 7fa86bfff640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa86c102ef0 0x7fa86c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa86b7fe640 1 -- 172.21.15.94:0/1415558409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa86c102570 msgr2=0x7fa86c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa86b7fe640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa86c102570 0x7fa86c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa86b7fe640 1 -- 172.21.15.94:0/1415558409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa86c102ef0 msgr2=0x7fa86c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa86b7fe640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa86c102ef0 0x7fa86c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa86b7fe640 1 -- 172.21.15.94:0/1415558409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c19d3e0 con 0x7fa86c100e20 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa86affd640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa86c102570 0x7fa86c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa86b7fe640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c100e20 0x7fa86c198430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00e9f0 tx=0x7fa85c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa868ff9640 1 -- 172.21.15.94:0/1415558409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c00cd80 con 0x7fa86c100e20 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa8721a0640 1 -- 172.21.15.94:0/1415558409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c19d6d0 con 0x7fa86c100e20 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa868ff9640 1 -- 172.21.15.94:0/1415558409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa85c0054d0 con 0x7fa86c100e20 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa868ff9640 1 -- 172.21.15.94:0/1415558409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c010490 con 0x7fa86c100e20 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.386+0000 7fa8721a0640 1 -- 172.21.15.94:0/1415558409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c1a4fb0 con 0x7fa86c100e20 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.388+0000 7fa868ff9640 1 -- 172.21.15.94:0/1415558409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7fa85c005020 con 0x7fa86c100e20 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa874579740 1 Processor -- start 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa874579740 1 -- start start 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa868ff9640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8540a3440 0x7fa8540a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa874579740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8beb64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa874579740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8beb6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa874579740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa874579740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55af8be182f0 con 0x55af8beb69e0 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa874579740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55af8bdbfcf0 con 0x55af8bec1780 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa874579740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55af8be142b0 con 0x55af8beb7f50 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa86affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8beb6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa86affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8beb6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48168/0 (socket says 172.21.15.94:48168) 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa86affd640 1 -- 172.21.15.94:0/4292823366 learned_addr learned my addr 172.21.15.94:0/4292823366 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.389+0000 7fa86b7fe640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8540a3440 0x7fa8540a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa868ff9640 1 -- 172.21.15.94:0/1415558409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fa85c0d0330 con 0x7fa86c100e20 2024-09-19T04:15:52.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86bfff640 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86bfff640 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8beb64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86affd640 1 -- 172.21.15.94:0/4292823366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 msgr2=0x55af8beb64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86affd640 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8beb64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86affd640 1 -- 172.21.15.94:0/4292823366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 msgr2=0x55af8bec3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86affd640 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86affd640 1 -- 172.21.15.94:0/4292823366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af8bda43a0 con 0x55af8beb69e0 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86bfff640 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8beb64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86b7fe640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8540a3440 0x7fa8540a5900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0cda10 tx=0x7fa85c01bb30 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa86affd640 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8beb6e20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa860009900 tx=0x7fa860032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d4597faf2fbc9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa851ffb640 1 -- 172.21.15.94:0/4292823366 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860035040 con 0x55af8beb69e0 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa851ffb640 1 -- 172.21.15.94:0/4292823366 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa860031730 con 0x55af8beb69e0 2024-09-19T04:15:52.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.390+0000 7fa851ffb640 1 -- 172.21.15.94:0/4292823366 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860003df0 con 0x55af8beb69e0 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.391+0000 7fa874579740 1 -- 172.21.15.94:0/4292823366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 msgr2=0x55af8beb6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.391+0000 7fa874579740 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8beb6e20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa860009900 tx=0x7fa860032f90 comp rx=0 tx=0).stop 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.391+0000 7fa874579740 1 -- 172.21.15.94:0/4292823366 shutdown_connections 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.391+0000 7fa874579740 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.391+0000 7fa874579740 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8beb6e20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.391+0000 7fa874579740 1 --2- 172.21.15.94:0/4292823366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8beb64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.391+0000 7fa874579740 1 -- 172.21.15.94:0/4292823366 >> 172.21.15.94:0/4292823366 conn(0x55af8be7b310 msgr2=0x55af8beb3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.391+0000 7fa874579740 1 -- 172.21.15.94:0/4292823366 shutdown_connections 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.391+0000 7fa874579740 1 -- 172.21.15.94:0/4292823366 wait complete. 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa874579740 1 -- start start 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa874579740 1 Processor -- start 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa874579740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8beb69e0 0x55af8bec1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa874579740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bebe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa874579740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8bec1780 0x55af8bebe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa874579740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55af8bec4420 con 0x55af8beb7f50 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa874579740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55af8bdd3060 con 0x55af8beb69e0 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa874579740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55af8bec45a0 con 0x55af8bec1780 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa874579740 10 client.0 _ll_register_callbacks cb 0x55af8bb268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa86affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bebe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa86affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bebe3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48172/0 (socket says 172.21.15.94:48172) 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa86affd640 1 -- 172.21.15.94:0/3310968379 learned_addr learned my addr 172.21.15.94:0/3310968379 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa86bfff640 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8beb69e0 0x55af8bec1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.392+0000 7fa86bfff640 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8bec1780 0x55af8bebe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa86affd640 1 -- 172.21.15.94:0/3310968379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8bec1780 msgr2=0x55af8bebe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa86affd640 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8bec1780 0x55af8bebe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa86affd640 1 -- 172.21.15.94:0/3310968379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8beb69e0 msgr2=0x55af8bec1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa86affd640 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8beb69e0 0x55af8bec1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa86affd640 1 -- 172.21.15.94:0/3310968379 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8bb893c0 con 0x55af8beb7f50 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa86bfff640 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8beb69e0 0x55af8bec1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa86affd640 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bebe3e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa860006010 tx=0x7fa860031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b6d7a03534e4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa84affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa84affd640 1 -- 172.21.15.94:0/3310968379 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86004e070 con 0x55af8beb7f50 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa84affd640 1 -- 172.21.15.94:0/3310968379 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fa860005260 con 0x55af8beb7f50 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa874579740 1 -- 172.21.15.94:0/3310968379 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa860031730 con 0x55af8beb7f50 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa84affd640 1 -- 172.21.15.94:0/3310968379 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa86005a050 con 0x55af8beb7f50 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa874579740 10 client.15393 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.393+0000 7fa874579740 1 -- 172.21.15.94:0/3310968379 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af8becfe70 con 0x55af8beb7f50 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa84affd640 1 -- 172.21.15.94:0/3310968379 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa860031730 con 0x55af8beb7f50 2024-09-19T04:15:52.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa84affd640 1 -- 172.21.15.94:0/3310968379 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa860003b50 con 0x55af8beb7f50 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa874579740 10 client.15393 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa874579740 10 client.15393 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa874579740 20 client.15393 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa874579740 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55af8bed11c0 0x55af8bed35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa874579740 4 client.15393 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa874579740 1 -- 172.21.15.94:0/3310968379 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af8be25860 con 0x55af8bed11c0 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa86bfff640 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55af8bed11c0 0x55af8bed35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa86bfff640 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55af8bed11c0 0x55af8bed35b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=202e240a69c82d32 server_cookie=50ebc8488ba6580f in_seq=0 out_seq=0 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.394+0000 7fa84affd640 10 client.15393 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa84affd640 1 -- 172.21.15.94:0/3310968379 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af8be25860 con 0x55af8bed11c0 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa84affd640 10 client.15393 handle_command_reply: tid=0 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 client.15393 shutdown 2024-09-19T04:15:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 -- 172.21.15.94:0/3310968379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 msgr2=0x55af8bebe3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bebe3e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa860006010 tx=0x7fa860031960 comp rx=0 tx=0).stop 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 -- 172.21.15.94:0/3310968379 shutdown_connections 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55af8bed11c0 0x55af8bed35b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8bec1780 0x55af8bebe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bebe3e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 --2- 172.21.15.94:0/3310968379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8beb69e0 0x55af8bec1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 -- 172.21.15.94:0/3310968379 >> 172.21.15.94:0/3310968379 conn(0x55af8be7b310 msgr2=0x55af8beb8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa84affd640 0 client.15393 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 -- 172.21.15.94:0/3310968379 shutdown_connections 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.395+0000 7fa874579740 1 -- 172.21.15.94:0/3310968379 wait complete. 2024-09-19T04:15:52.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.396+0000 7fa874579740 20 client.15393 trim_cache size 0 max 16384 2024-09-19T04:15:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.412+0000 7fa874579740 1 Processor -- start 2024-09-19T04:15:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.412+0000 7fa874579740 1 -- start start 2024-09-19T04:15:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.412+0000 7fa874579740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8becd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa874579740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8bec2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa874579740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa874579740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1addf0 con 0x55af8beb69e0 2024-09-19T04:15:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa874579740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1ab6c0 con 0x55af8bec1780 2024-09-19T04:15:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa874579740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1a9510 con 0x55af8beb7f50 2024-09-19T04:15:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8becd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8becd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48174/0 (socket says 172.21.15.94:48174) 2024-09-19T04:15:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 -- 172.21.15.94:0/983079483 learned_addr learned my addr 172.21.15.94:0/983079483 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86affd640 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8bec2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 -- 172.21.15.94:0/983079483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 msgr2=0x55af8bec2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8bec2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 -- 172.21.15.94:0/983079483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 msgr2=0x55af8bec30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 -- 172.21.15.94:0/983079483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af8bec3fb0 con 0x55af8beb69e0 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86affd640 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8bec2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa86bfff640 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8becd6c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f80 tx=0x7fa864004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89124fe9fe5f939 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.413+0000 7fa8517fa640 1 -- 172.21.15.94:0/983079483 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864014310 con 0x55af8beb69e0 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa8517fa640 1 -- 172.21.15.94:0/983079483 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa864028030 con 0x55af8beb69e0 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa8517fa640 1 -- 172.21.15.94:0/983079483 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864005950 con 0x55af8beb69e0 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa874579740 1 -- 172.21.15.94:0/983079483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 msgr2=0x55af8becd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa874579740 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8becd6c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f80 tx=0x7fa864004a40 comp rx=0 tx=0).stop 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa874579740 1 -- 172.21.15.94:0/983079483 shutdown_connections 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa874579740 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa874579740 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb7f50 0x55af8bec2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa874579740 1 --2- 172.21.15.94:0/983079483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb69e0 0x55af8becd6c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa874579740 1 -- 172.21.15.94:0/983079483 >> 172.21.15.94:0/983079483 conn(0x55af8be7b310 msgr2=0x55af8bebccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa874579740 1 -- 172.21.15.94:0/983079483 shutdown_connections 2024-09-19T04:15:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.414+0000 7fa874579740 1 -- 172.21.15.94:0/983079483 wait complete. 2024-09-19T04:15:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa874579740 1 -- start start 2024-09-19T04:15:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa874579740 1 Processor -- start 2024-09-19T04:15:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa874579740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb69e0 0x55af8bec9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa874579740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bec24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa874579740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa874579740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1a9b80 con 0x55af8beb7f50 2024-09-19T04:15:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa874579740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1aaff0 con 0x55af8bec1780 2024-09-19T04:15:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa874579740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1aa250 con 0x55af8beb69e0 2024-09-19T04:15:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa874579740 10 client.0 _ll_register_callbacks cb 0x7fa860040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa86affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bec24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.415+0000 7fa86affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bec24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48190/0 (socket says 172.21.15.94:48190) 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86affd640 1 -- 172.21.15.94:0/4165019773 learned_addr learned my addr 172.21.15.94:0/4165019773 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86bfff640 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb69e0 0x55af8bec9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86bfff640 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86affd640 1 -- 172.21.15.94:0/4165019773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb69e0 msgr2=0x55af8bec9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86affd640 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb69e0 0x55af8bec9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86affd640 1 -- 172.21.15.94:0/4165019773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 msgr2=0x55af8bec2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86affd640 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86affd640 1 -- 172.21.15.94:0/4165019773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8bec3230 con 0x55af8beb7f50 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86bfff640 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb69e0 0x55af8bec9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa86affd640 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bec24f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa860001eb0 tx=0x7fa860004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c0e6d5b3e2a64a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:52.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa84affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa84affd640 1 -- 172.21.15.94:0/4165019773 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86004e040 con 0x55af8beb7f50 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa84affd640 1 -- 172.21.15.94:0/4165019773 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fa860005260 con 0x55af8beb7f50 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa874579740 1 -- 172.21.15.94:0/4165019773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa864028030 con 0x55af8beb7f50 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa84affd640 1 -- 172.21.15.94:0/4165019773 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa860035040 con 0x55af8beb7f50 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa874579740 10 client.15399 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.416+0000 7fa874579740 1 -- 172.21.15.94:0/4165019773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af8bf3bb20 con 0x55af8beb7f50 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.417+0000 7fa84affd640 1 -- 172.21.15.94:0/4165019773 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa864028030 con 0x55af8beb7f50 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.417+0000 7fa84affd640 1 -- 172.21.15.94:0/4165019773 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa860005510 con 0x55af8beb7f50 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.417+0000 7fa874579740 10 client.15399 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.417+0000 7fa874579740 10 client.15399 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.417+0000 7fa874579740 20 client.15399 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.417+0000 7fa874579740 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55af8bf3c1d0 0x55af8bf3e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.417+0000 7fa874579740 4 client.15399 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.417+0000 7fa874579740 1 -- 172.21.15.94:0/4165019773 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af8be1fd70 con 0x55af8bf3c1d0 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.417+0000 7fa86bfff640 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55af8bf3c1d0 0x55af8bf3e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa84affd640 10 client.15399 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa86bfff640 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55af8bf3c1d0 0x55af8bf3e5f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=554a9aead4d86704 server_cookie=fc4d62bc07e89347 in_seq=0 out_seq=0 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa84affd640 1 -- 172.21.15.94:0/4165019773 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af8be1fd70 con 0x55af8bf3c1d0 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa84affd640 10 client.15399 handle_command_reply: tid=0 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 client.15399 shutdown 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 -- 172.21.15.94:0/4165019773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 msgr2=0x55af8bec24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bec24f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa860001eb0 tx=0x7fa860004370 comp rx=0 tx=0).stop 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 -- 172.21.15.94:0/4165019773 shutdown_connections 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55af8bf3c1d0 0x55af8bf3e5f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa84affd640 0 client.15399 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:15:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af8bec1780 0x55af8bec2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af8beb7f50 0x55af8bec24f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 --2- 172.21.15.94:0/4165019773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af8beb69e0 0x55af8bec9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 -- 172.21.15.94:0/4165019773 >> 172.21.15.94:0/4165019773 conn(0x55af8be7b310 msgr2=0x55af8bebfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 -- 172.21.15.94:0/4165019773 shutdown_connections 2024-09-19T04:15:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.418+0000 7fa874579740 1 -- 172.21.15.94:0/4165019773 wait complete. 2024-09-19T04:15:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa874579740 20 client.15399 trim_cache size 0 max 16384 2024-09-19T04:15:52.425 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:15:52.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 -- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8540a3440 msgr2=0x7fa8540a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:52.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8540a3440 0x7fa8540a5900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0cda10 tx=0x7fa85c01bb30 comp rx=0 tx=0).stop 2024-09-19T04:15:52.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 -- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c100e20 msgr2=0x7fa86c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:52.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c100e20 0x7fa86c198430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00e9f0 tx=0x7fa85c00edf0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 -- 172.21.15.94:0/1415558409 shutdown_connections 2024-09-19T04:15:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8540a3440 0x7fa8540a5900 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa86c102ef0 0x7fa86c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa86c102570 0x7fa86c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 --2- 172.21.15.94:0/1415558409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86c100e20 0x7fa86c198430 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 -- 172.21.15.94:0/1415558409 >> 172.21.15.94:0/1415558409 conn(0x7fa86c0fc7d0 msgr2=0x7fa86c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 -- 172.21.15.94:0/1415558409 shutdown_connections 2024-09-19T04:15:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:52.419+0000 7fa8721a0640 1 -- 172.21.15.94:0/1415558409 wait complete. 2024-09-19T04:15:52.647 INFO:teuthology.orchestra.run.smithi094.stderr:2200000 of 3200000 tuples (68%) done (elapsed 21.32 s, remaining 9.69 s) 2024-09-19T04:15:53.136 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:15:53.415 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:53.438 INFO:teuthology.orchestra.run.smithi094.stderr:2300000 of 3200000 tuples (71%) done (elapsed 22.12 s, remaining 8.65 s) 2024-09-19T04:15:53.732 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:53 smithi094 ceph-mon[30972]: pgmap v29: 97 pgs: 97 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 MiB/s wr, 47 op/s 2024-09-19T04:15:53.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:53 smithi112 ceph-mon[35051]: pgmap v29: 97 pgs: 97 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 MiB/s wr, 47 op/s 2024-09-19T04:15:53.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:53 smithi104 ceph-mon[34145]: pgmap v29: 97 pgs: 97 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 MiB/s wr, 47 op/s 2024-09-19T04:15:53.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.922+0000 7ff1069fe640 1 -- 172.21.15.94:0/2596856117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff100100de0 msgr2=0x7ff100104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.922+0000 7ff1069fe640 1 --2- 172.21.15.94:0/2596856117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff100100de0 0x7ff100104e30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009900 tx=0x7ff0f402a410 comp rx=0 tx=0).stop 2024-09-19T04:15:53.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.922+0000 7ff1069fe640 1 -- 172.21.15.94:0/2596856117 shutdown_connections 2024-09-19T04:15:53.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.922+0000 7ff1069fe640 1 --2- 172.21.15.94:0/2596856117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff100105560 0x7ff10010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.922+0000 7ff1069fe640 1 --2- 172.21.15.94:0/2596856117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff100100de0 0x7ff100104e30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.922+0000 7ff1069fe640 1 --2- 172.21.15.94:0/2596856117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff100100430 0x7ff100100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.922+0000 7ff1069fe640 1 -- 172.21.15.94:0/2596856117 >> 172.21.15.94:0/2596856117 conn(0x7ff100077230 msgr2=0x7ff1000fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:53.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.922+0000 7ff1069fe640 1 -- 172.21.15.94:0/2596856117 shutdown_connections 2024-09-19T04:15:53.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.922+0000 7ff1069fe640 1 -- 172.21.15.94:0/2596856117 wait complete. 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff1069fe640 1 Processor -- start 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff1069fe640 1 -- start start 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff1069fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff100100430 0x7ff1001991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff1069fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff100100de0 0x7ff1001996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff1069fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff100105560 0x7ff1001a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff1069fe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff100110460 con 0x7ff100105560 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff1069fe640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001102e0 con 0x7ff100100430 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff1069fe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001105e0 con 0x7ff100100de0 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff104f9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff100105560 0x7ff1001a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.923+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff100100de0 0x7ff1001996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff0fffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff100100430 0x7ff1001991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff100100de0 0x7ff1001996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54580/0 (socket says 172.21.15.94:54580) 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/460376411 learned_addr learned my addr 172.21.15.94:0/460376411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff104f9b640 1 -- 172.21.15.94:0/460376411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff100100de0 msgr2=0x7ff1001996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff104f9b640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff100100de0 0x7ff1001996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff104f9b640 1 -- 172.21.15.94:0/460376411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff100100430 msgr2=0x7ff1001991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff104f9b640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff100100430 0x7ff1001991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff104f9b640 1 -- 172.21.15.94:0/460376411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1001a4760 con 0x7ff100105560 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff100100de0 0x7ff1001996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff104f9b640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff100105560 0x7ff1001a4220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800cab0 tx=0x7ff0f800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff0fd7fa640 1 -- 172.21.15.94:0/460376411 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800ae20 con 0x7ff100105560 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff0fd7fa640 1 -- 172.21.15.94:0/460376411 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0f800f900 con 0x7ff100105560 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff1069fe640 1 -- 172.21.15.94:0/460376411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff100196f90 con 0x7ff100105560 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff0fd7fa640 1 -- 172.21.15.94:0/460376411 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8018a00 con 0x7ff100105560 2024-09-19T04:15:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.924+0000 7ff1069fe640 1 -- 172.21.15.94:0/460376411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff100197480 con 0x7ff100105560 2024-09-19T04:15:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.927+0000 7ff0fd7fa640 1 -- 172.21.15.94:0/460376411 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7ff0f800fab0 con 0x7ff100105560 2024-09-19T04:15:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.927+0000 7ff108dd7740 1 Processor -- start 2024-09-19T04:15:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.927+0000 7ff108dd7740 1 -- start start 2024-09-19T04:15:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaa9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 0x5571ebaa9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebab4370 0x5571ebab6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff108dd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5571eba0f2f0 con 0x5571ebaaab40 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff108dd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5571eb9b6cf0 con 0x5571ebab4370 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff108dd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5571eba0b2b0 con 0x5571ebaa95d0 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0fd7fa640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff0e80a3440 0x7ff0e80a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaa9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaa9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48208/0 (socket says 172.21.15.94:48208) 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3368733148 learned_addr learned my addr 172.21.15.94:0/3368733148 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0fffff640 1 --2- 172.21.15.94:0/3368733148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 0x5571ebaa9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff104f9b640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff0e80a3440 0x7ff0e80a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0fd7fa640 1 -- 172.21.15.94:0/460376411 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff0f80d0130 con 0x7ff100105560 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3368733148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 msgr2=0x5571ebaa9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/3368733148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 0x5571ebaa9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3368733148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebab4370 msgr2=0x5571ebab6760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/3368733148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebab4370 0x5571ebab6760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3368733148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571eb99b3a0 con 0x5571ebaaab40 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff104f9b640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff0e80a3440 0x7ff0e80a5900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80cd810 tx=0x7ff0f801baf0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.928+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/3368733148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaa9090 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4005ab0 tx=0x7ff0f40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7502940fa4847b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.929+0000 7ff0e67fc640 1 -- 172.21.15.94:0/3368733148 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4040070 con 0x5571ebaaab40 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.929+0000 7ff0e67fc640 1 -- 172.21.15.94:0/3368733148 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0f40047a0 con 0x5571ebaaab40 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.929+0000 7ff0e67fc640 1 -- 172.21.15.94:0/3368733148 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4035040 con 0x5571ebaaab40 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.929+0000 7ff108dd7740 1 -- 172.21.15.94:0/3368733148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 msgr2=0x5571ebaa9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.929+0000 7ff108dd7740 1 --2- 172.21.15.94:0/3368733148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaa9090 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4005ab0 tx=0x7ff0f40035f0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 -- 172.21.15.94:0/3368733148 shutdown_connections 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 --2- 172.21.15.94:0/3368733148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebab4370 0x5571ebab6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 --2- 172.21.15.94:0/3368733148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 0x5571ebaa9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 --2- 172.21.15.94:0/3368733148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaa9090 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 -- 172.21.15.94:0/3368733148 >> 172.21.15.94:0/3368733148 conn(0x5571eba72310 msgr2=0x5571ebaa5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 -- 172.21.15.94:0/3368733148 shutdown_connections 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 -- 172.21.15.94:0/3368733148 wait complete. 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 -- start start 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 Processor -- start 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebab3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.930+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 0x5571ebab1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 0x5571ebab1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff108dd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5571ebab7010 con 0x5571ebaa95d0 2024-09-19T04:15:53.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff108dd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5571eb9ca060 con 0x5571ebaaab40 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff108dd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5571ebab7190 con 0x5571ebab4370 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff108dd7740 10 client.0 _ll_register_callbacks cb 0x5571eb71d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebab3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebab3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48214/0 (socket says 172.21.15.94:48214) 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2100927454 learned_addr learned my addr 172.21.15.94:0/2100927454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0fffff640 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 0x5571ebab1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 0x5571ebab1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2100927454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 msgr2=0x5571ebab1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 0x5571ebab1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2100927454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 msgr2=0x5571ebab1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 0x5571ebab1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2100927454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571eb7803c0 con 0x5571ebaa95d0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebab3d60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff0f402ac00 tx=0x7ff0f402ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fd264ea7c7e561 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.931+0000 7ff0df7fe640 1 -- 172.21.15.94:0/2100927454 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4052060 con 0x5571ebaa95d0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff0df7fe640 1 -- 172.21.15.94:0/2100927454 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff0f4003850 con 0x5571ebaa95d0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff108dd7740 1 -- 172.21.15.94:0/2100927454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f40047a0 con 0x5571ebaa95d0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff0df7fe640 1 -- 172.21.15.94:0/2100927454 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0f4048070 con 0x5571ebaa95d0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff108dd7740 10 client.15411 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff108dd7740 1 -- 172.21.15.94:0/2100927454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571ebac2ae0 con 0x5571ebaa95d0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff0df7fe640 1 -- 172.21.15.94:0/2100927454 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0f40047a0 con 0x5571ebaa95d0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff0df7fe640 1 -- 172.21.15.94:0/2100927454 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff0f4031b00 con 0x5571ebaa95d0 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff108dd7740 10 client.15411 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff108dd7740 10 client.15411 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:15:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff108dd7740 20 client.15411 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5571ebac3e30 0x5571ebac6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff108dd7740 4 client.15411 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.932+0000 7ff108dd7740 1 -- 172.21.15.94:0/2100927454 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571eba1c860 con 0x5571ebac3e30 2024-09-19T04:15:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.933+0000 7ff0fffff640 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5571ebac3e30 0x5571ebac6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.933+0000 7ff0fffff640 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5571ebac3e30 0x5571ebac6220 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f566dcd312bf9bd9 server_cookie=34aad3fb23db70e0 in_seq=0 out_seq=0 2024-09-19T04:15:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.933+0000 7ff0df7fe640 10 client.15411 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:15:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff0df7fe640 1 -- 172.21.15.94:0/2100927454 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571eba1c860 con 0x5571ebac3e30 2024-09-19T04:15:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff0df7fe640 10 client.15411 handle_command_reply: tid=0 2024-09-19T04:15:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 client.15411 shutdown 2024-09-19T04:15:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 -- 172.21.15.94:0/2100927454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 msgr2=0x5571ebab3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebab3d60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff0f402ac00 tx=0x7ff0f402ab90 comp rx=0 tx=0).stop 2024-09-19T04:15:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 -- 172.21.15.94:0/2100927454 shutdown_connections 2024-09-19T04:15:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5571ebac3e30 0x5571ebac6220 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 0x5571ebab1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 0x5571ebab1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2100927454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebab3d60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 -- 172.21.15.94:0/2100927454 >> 172.21.15.94:0/2100927454 conn(0x5571eba72310 msgr2=0x5571ebaab990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff0df7fe640 0 client.15411 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:15:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 -- 172.21.15.94:0/2100927454 shutdown_connections 2024-09-19T04:15:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 1 -- 172.21.15.94:0/2100927454 wait complete. 2024-09-19T04:15:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.934+0000 7ff108dd7740 20 client.15411 trim_cache size 0 max 16384 2024-09-19T04:15:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff108dd7740 1 Processor -- start 2024-09-19T04:15:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff108dd7740 1 -- start start 2024-09-19T04:15:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 0x5571ebaa4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebab4370 0x5571ebaae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff108dd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001add90 con 0x5571ebaaab40 2024-09-19T04:15:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff108dd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001ab660 con 0x5571ebab4370 2024-09-19T04:15:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff108dd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001a94b0 con 0x5571ebaa95d0 2024-09-19T04:15:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff0fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff0fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48222/0 (socket says 172.21.15.94:48222) 2024-09-19T04:15:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.950+0000 7ff0fffff640 1 -- 172.21.15.94:0/2693865590 learned_addr learned my addr 172.21.15.94:0/2693865590 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebab4370 0x5571ebaae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 0x5571ebaa4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0fffff640 1 -- 172.21.15.94:0/2693865590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 msgr2=0x5571ebaa4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0fffff640 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 0x5571ebaa4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0fffff640 1 -- 172.21.15.94:0/2693865590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebab4370 msgr2=0x5571ebaae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0fffff640 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebab4370 0x5571ebaae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0fffff640 1 -- 172.21.15.94:0/2693865590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ebaaf560 con 0x5571ebaaab40 2024-09-19T04:15:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 0x5571ebaa4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0fffff640 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaae290 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5571ebab4340 tx=0x7ff0f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13b8ab19332a4fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0e5ffb640 1 -- 172.21.15.94:0/2693865590 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00234c0 con 0x5571ebaaab40 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0e5ffb640 1 -- 172.21.15.94:0/2693865590 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0f0028070 con 0x5571ebaaab40 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.951+0000 7ff0e5ffb640 1 -- 172.21.15.94:0/2693865590 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f001bbf0 con 0x5571ebaaab40 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 -- 172.21.15.94:0/2693865590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 msgr2=0x5571ebaae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaae290 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5571ebab4340 tx=0x7ff0f0004730 comp rx=0 tx=0).stop 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 -- 172.21.15.94:0/2693865590 shutdown_connections 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebab4370 0x5571ebaae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaaab40 0x5571ebaae290 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 --2- 172.21.15.94:0/2693865590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebaa95d0 0x5571ebaa4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 -- 172.21.15.94:0/2693865590 >> 172.21.15.94:0/2693865590 conn(0x5571eba72310 msgr2=0x5571ebaa4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:53.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 -- 172.21.15.94:0/2693865590 shutdown_connections 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 -- 172.21.15.94:0/2693865590 wait complete. 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.952+0000 7ff108dd7740 1 -- start start 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff108dd7740 1 Processor -- start 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebac6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 0x5571ebac70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff108dd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 0x5571ebac7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff108dd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001a9b20 con 0x5571ebaa95d0 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff108dd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001aaf90 con 0x5571ebaaab40 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff108dd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001aa1f0 con 0x5571ebab4370 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff108dd7740 10 client.0 _ll_register_callbacks cb 0x7ff0f000fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebac6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebac6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48228/0 (socket says 172.21.15.94:48228) 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/1177747363 learned_addr learned my addr 172.21.15.94:0/1177747363 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff0fffff640 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 0x5571ebac70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.953+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 0x5571ebac7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/1177747363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 msgr2=0x5571ebac7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 0x5571ebac7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/1177747363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 msgr2=0x5571ebac70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 0x5571ebac70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/1177747363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ebb2d770 con 0x5571ebaa95d0 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0ff7fe640 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebac6ba0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff0f403c670 tx=0x7ff0f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc84ae8783ad2bf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0df7fe640 1 -- 172.21.15.94:0/1177747363 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4063060 con 0x5571ebaa95d0 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0df7fe640 1 -- 172.21.15.94:0/1177747363 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff0f403cab0 con 0x5571ebaa95d0 2024-09-19T04:15:53.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff108dd7740 1 -- 172.21.15.94:0/1177747363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f0028070 con 0x5571ebaa95d0 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0df7fe640 1 -- 172.21.15.94:0/1177747363 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0f4052050 con 0x5571ebaa95d0 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff108dd7740 10 client.15417 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff108dd7740 1 -- 172.21.15.94:0/1177747363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571ebb2e9b0 con 0x5571ebaa95d0 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.954+0000 7ff0df7fe640 1 -- 172.21.15.94:0/1177747363 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0f403ccb0 con 0x5571ebaa95d0 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff0df7fe640 1 -- 172.21.15.94:0/1177747363 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff0f40035f0 con 0x5571ebaa95d0 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff108dd7740 10 client.15417 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff108dd7740 10 client.15417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff108dd7740 20 client.15417 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff108dd7740 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5571ebb2f070 0x5571ebb31460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff108dd7740 4 client.15417 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff108dd7740 1 -- 172.21.15.94:0/1177747363 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff0f0028070 con 0x5571ebb2f070 2024-09-19T04:15:53.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff0fffff640 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5571ebb2f070 0x5571ebb31460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff0fffff640 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5571ebb2f070 0x5571ebb31460 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=603f4b1135b710e2 server_cookie=1c2ab2f27dea3024 in_seq=0 out_seq=0 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.955+0000 7ff0df7fe640 10 client.15417 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff0df7fe640 1 -- 172.21.15.94:0/1177747363 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff0f0028070 con 0x5571ebb2f070 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff0df7fe640 10 client.15417 handle_command_reply: tid=0 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 client.15417 shutdown 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 -- 172.21.15.94:0/1177747363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 msgr2=0x5571ebac6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebac6ba0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff0f403c670 tx=0x7ff0f4004ab0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 -- 172.21.15.94:0/1177747363 shutdown_connections 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5571ebb2f070 0x5571ebb31460 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff0df7fe640 0 client.15417 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571ebab4370 0x5571ebac7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571ebaaab40 0x5571ebac70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 --2- 172.21.15.94:0/1177747363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5571ebaa95d0 0x5571ebac6ba0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 -- 172.21.15.94:0/1177747363 >> 172.21.15.94:0/1177747363 conn(0x5571eba72310 msgr2=0x5571ebaa4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:53.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 -- 172.21.15.94:0/1177747363 shutdown_connections 2024-09-19T04:15:53.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 1 -- 172.21.15.94:0/1177747363 wait complete. 2024-09-19T04:15:53.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.956+0000 7ff108dd7740 20 client.15417 trim_cache size 0 max 16384 2024-09-19T04:15:53.963 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 -- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff0e80a3440 msgr2=0x7ff0e80a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff0e80a3440 0x7ff0e80a5900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80cd810 tx=0x7ff0f801baf0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 -- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff100105560 msgr2=0x7ff1001a4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff100105560 0x7ff1001a4220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800cab0 tx=0x7ff0f800cf70 comp rx=0 tx=0).stop 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 -- 172.21.15.94:0/460376411 shutdown_connections 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff0e80a3440 0x7ff0e80a5900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff100105560 0x7ff1001a4220 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff100100de0 0x7ff1001996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 --2- 172.21.15.94:0/460376411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff100100430 0x7ff1001991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 -- 172.21.15.94:0/460376411 >> 172.21.15.94:0/460376411 conn(0x7ff100077230 msgr2=0x7ff1000fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:53.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 -- 172.21.15.94:0/460376411 shutdown_connections 2024-09-19T04:15:53.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:53.957+0000 7ff1069fe640 1 -- 172.21.15.94:0/460376411 wait complete. 2024-09-19T04:15:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2400000 of 3200000 tuples (75%) done (elapsed 23.21 s, remaining 7.74 s) 2024-09-19T04:15:54.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:15:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.549+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2985452777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac102ef0 msgr2=0x7f1bac10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.549+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2985452777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac102ef0 0x7f1bac10da40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009ca0 tx=0x7f1ba402a950 comp rx=0 tx=0).stop 2024-09-19T04:15:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.549+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2985452777 shutdown_connections 2024-09-19T04:15:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.549+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2985452777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac102ef0 0x7f1bac10da40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.549+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2985452777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac102570 0x7f1bac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.549+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2985452777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bac100e20 0x7f1bac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.549+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2985452777 >> 172.21.15.94:0/2985452777 conn(0x7f1bac0fc7d0 msgr2=0x7f1bac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.549+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2985452777 shutdown_connections 2024-09-19T04:15:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.549+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2985452777 wait complete. 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb36ad640 1 Processor -- start 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb36ad640 1 -- start start 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb36ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac100e20 0x7f1bac0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb36ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bac102570 0x7f1bac0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb36ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac102ef0 0x7f1bac0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb36ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac110e30 con 0x7f1bac100e20 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb36ad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac110cb0 con 0x7f1bac102570 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb36ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac110fb0 con 0x7f1bac102ef0 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb1449640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac100e20 0x7f1bac0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb1449640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac100e20 0x7f1bac0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48242/0 (socket says 172.21.15.94:48242) 2024-09-19T04:15:55.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb1449640 1 -- 172.21.15.94:0/2424641871 learned_addr learned my addr 172.21.15.94:0/2424641871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac102ef0 0x7f1bac0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb0c48640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bac102570 0x7f1bac0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb1449640 1 -- 172.21.15.94:0/2424641871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac102ef0 msgr2=0x7f1bac0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb1449640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac102ef0 0x7f1bac0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb1449640 1 -- 172.21.15.94:0/2424641871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bac102570 msgr2=0x7f1bac0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb1449640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bac102570 0x7f1bac0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb1449640 1 -- 172.21.15.94:0/2424641871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac1a6840 con 0x7f1bac100e20 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.550+0000 7f1bb0c48640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bac102570 0x7f1bac0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.551+0000 7f1bb1449640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac100e20 0x7f1bac0fcc80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800e9f0 tx=0x7f1ba800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.551+0000 7f1ba27fc640 1 -- 172.21.15.94:0/2424641871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800cd80 con 0x7f1bac100e20 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.551+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2424641871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bac1a6b30 con 0x7f1bac100e20 2024-09-19T04:15:55.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.551+0000 7f1ba27fc640 1 -- 172.21.15.94:0/2424641871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba80108e0 con 0x7f1bac100e20 2024-09-19T04:15:55.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.551+0000 7f1ba27fc640 1 -- 172.21.15.94:0/2424641871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8018920 con 0x7f1bac100e20 2024-09-19T04:15:55.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.551+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2424641871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bac1a70c0 con 0x7f1bac100e20 2024-09-19T04:15:55.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.554+0000 7f1ba27fc640 1 -- 172.21.15.94:0/2424641871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f1ba8018ac0 con 0x7f1bac100e20 2024-09-19T04:15:55.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.554+0000 7f1bb5a86740 1 Processor -- start 2024-09-19T04:15:55.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb5a86740 1 -- start start 2024-09-19T04:15:55.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace692090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace692a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace69f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555ace5f82f0 con 0x555ace6925d0 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555ace59fcf0 con 0x555ace693b40 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ace5f42b0 con 0x555ace69d370 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1ba27fc640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b980a34c0 0x7f1b980a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1449640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace692a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1449640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace692a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48244/0 (socket says 172.21.15.94:48244) 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1449640 1 -- 172.21.15.94:0/1766730018 learned_addr learned my addr 172.21.15.94:0/1766730018 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b980a34c0 0x7f1b980a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace692090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1ba27fc640 1 -- 172.21.15.94:0/2424641871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f1ba80d01f0 con 0x7f1bac100e20 2024-09-19T04:15:55.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb0c48640 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace69f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1449640 1 -- 172.21.15.94:0/1766730018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 msgr2=0x555ace69f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:55.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1449640 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace69f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1449640 1 -- 172.21.15.94:0/1766730018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 msgr2=0x555ace692090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:55.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1449640 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace692090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb1449640 1 -- 172.21.15.94:0/1766730018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ace5843a0 con 0x555ace6925d0 2024-09-19T04:15:55.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.555+0000 7f1bb0c48640 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace69f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:55.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.556+0000 7f1bb1449640 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace692a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80cd950 tx=0x7f1ba8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b414519673a25ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.556+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b980a34c0 0x7f1b980a5980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4005e30 tx=0x7f1ba4009800 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.556+0000 7f1b977fe640 1 -- 172.21.15.94:0/1766730018 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba809ca50 con 0x555ace6925d0 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.556+0000 7f1b977fe640 1 -- 172.21.15.94:0/1766730018 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba80a0700 con 0x555ace6925d0 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.556+0000 7f1b977fe640 1 -- 172.21.15.94:0/1766730018 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80cdb20 con 0x555ace6925d0 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1766730018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 msgr2=0x555ace692a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace692a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80cd950 tx=0x7f1ba8017000 comp rx=0 tx=0).stop 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1766730018 shutdown_connections 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace69f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace692a10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/1766730018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace692090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1766730018 >> 172.21.15.94:0/1766730018 conn(0x555ace65b310 msgr2=0x555ace68ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1766730018 shutdown_connections 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1766730018 wait complete. 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 -- start start 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.557+0000 7f1bb5a86740 1 Processor -- start 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace69cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace699ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace69d370 0x555ace69a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555ace6a0010 con 0x555ace69d370 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555ace5b3060 con 0x555ace693b40 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ace6a0190 con 0x555ace6925d0 2024-09-19T04:15:55.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb5a86740 10 client.0 _ll_register_callbacks cb 0x555ace3068c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:55.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1449640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace69cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1c4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace69d370 0x555ace69a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1449640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace69cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54658/0 (socket says 172.21.15.94:54658) 2024-09-19T04:15:55.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1449640 1 -- 172.21.15.94:0/2037736325 learned_addr learned my addr 172.21.15.94:0/2037736325 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:55.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb0c48640 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace699ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1c4a640 1 -- 172.21.15.94:0/2037736325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 msgr2=0x555ace69cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:55.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace69cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1c4a640 1 -- 172.21.15.94:0/2037736325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 msgr2=0x555ace699ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace699ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1c4a640 1 -- 172.21.15.94:0/2037736325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ace3693c0 con 0x555ace69d370 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1449640 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace69cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace69d370 0x555ace69a5e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1ba402ae70 tx=0x7f1ba402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29321e468dfb816f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1b94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.558+0000 7f1b94ff9640 1 -- 172.21.15.94:0/2037736325 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4034040 con 0x555ace69d370 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1b94ff9640 1 -- 172.21.15.94:0/2037736325 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f1ba4004250 con 0x555ace69d370 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1bb5a86740 1 -- 172.21.15.94:0/2037736325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ba80a0700 con 0x555ace69d370 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1b94ff9640 1 -- 172.21.15.94:0/2037736325 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ba4004490 con 0x555ace69d370 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1bb5a86740 10 client.15429 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1bb5a86740 1 -- 172.21.15.94:0/2037736325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555ace6aba40 con 0x555ace69d370 2024-09-19T04:15:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1b94ff9640 1 -- 172.21.15.94:0/2037736325 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba80a0700 con 0x555ace69d370 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1b94ff9640 1 -- 172.21.15.94:0/2037736325 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1ba4048040 con 0x555ace69d370 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1bb5a86740 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1bb5a86740 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1bb5a86740 20 client.15429 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ace6acfc0 0x555ace6af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1bb5a86740 4 client.15429 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.559+0000 7f1bb5a86740 1 -- 172.21.15.94:0/2037736325 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ace605860 con 0x555ace6acfc0 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.560+0000 7f1bb1449640 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ace6acfc0 0x555ace6af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.560+0000 7f1b94ff9640 10 client.15429 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.560+0000 7f1bb1449640 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ace6acfc0 0x555ace6af420 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346a684c9a2204ad server_cookie=6ab278132b0a242e in_seq=0 out_seq=0 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1b94ff9640 1 -- 172.21.15.94:0/2037736325 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ace605860 con 0x555ace6acfc0 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1b94ff9640 10 client.15429 handle_command_reply: tid=0 2024-09-19T04:15:55.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1bb5a86740 1 client.15429 shutdown 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1bb5a86740 1 -- 172.21.15.94:0/2037736325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace69d370 msgr2=0x555ace69a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace69d370 0x555ace69a5e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1ba402ae70 tx=0x7f1ba402a950 comp rx=0 tx=0).stop 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1bb5a86740 1 -- 172.21.15.94:0/2037736325 shutdown_connections 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ace6acfc0 0x555ace6af420 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace69d370 0x555ace69a5e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace699ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1b94ff9640 0 client.15429 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/2037736325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace69cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.561+0000 7f1bb5a86740 1 -- 172.21.15.94:0/2037736325 >> 172.21.15.94:0/2037736325 conn(0x555ace65b310 msgr2=0x555ace694990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.562+0000 7f1bb5a86740 1 -- 172.21.15.94:0/2037736325 shutdown_connections 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.562+0000 7f1bb5a86740 1 -- 172.21.15.94:0/2037736325 wait complete. 2024-09-19T04:15:55.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.562+0000 7f1bb5a86740 20 client.15429 trim_cache size 0 max 16384 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.575+0000 7f1bb5a86740 1 Processor -- start 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.575+0000 7f1bb5a86740 1 -- start start 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace6a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace68e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace68ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac1b2220 con 0x555ace6925d0 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac1afaf0 con 0x555ace693b40 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac1ad940 con 0x555ace69d370 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1449640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace6a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1449640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace6a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48248/0 (socket says 172.21.15.94:48248) 2024-09-19T04:15:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1449640 1 -- 172.21.15.94:0/1908978312 learned_addr learned my addr 172.21.15.94:0/1908978312 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace68ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb0c48640 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace68e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1449640 1 -- 172.21.15.94:0/1908978312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 msgr2=0x555ace68ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1449640 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace68ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1449640 1 -- 172.21.15.94:0/1908978312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 msgr2=0x555ace68e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1449640 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace68e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1449640 1 -- 172.21.15.94:0/1908978312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ace68f180 con 0x555ace6925d0 2024-09-19T04:15:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace68ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.576+0000 7f1bb1449640 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace6a92b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80987b0 tx=0x7f1ba809c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473e90ee9c41401e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1b96ffd640 1 -- 172.21.15.94:0/1908978312 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80a4fc0 con 0x555ace6925d0 2024-09-19T04:15:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1b96ffd640 1 -- 172.21.15.94:0/1908978312 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba809a650 con 0x555ace6925d0 2024-09-19T04:15:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1b96ffd640 1 -- 172.21.15.94:0/1908978312 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80980a0 con 0x555ace6925d0 2024-09-19T04:15:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1908978312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 msgr2=0x555ace6a92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace6a92b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80987b0 tx=0x7f1ba809c490 comp rx=0 tx=0).stop 2024-09-19T04:15:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1908978312 shutdown_connections 2024-09-19T04:15:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace69d370 0x555ace68ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace693b40 0x555ace68e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/1908978312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace6925d0 0x555ace6a92b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.577+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1908978312 >> 172.21.15.94:0/1908978312 conn(0x555ace65b310 msgr2=0x555ace699fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1908978312 shutdown_connections 2024-09-19T04:15:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 -- 172.21.15.94:0/1908978312 wait complete. 2024-09-19T04:15:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 -- start start 2024-09-19T04:15:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 Processor -- start 2024-09-19T04:15:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace6a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace693b40 0x555ace6a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace69d370 0x555ace6a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac1adfb0 con 0x555ace693b40 2024-09-19T04:15:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac1af420 con 0x555ace69d370 2024-09-19T04:15:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac1ae680 con 0x555ace6925d0 2024-09-19T04:15:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.578+0000 7f1bb5a86740 10 client.0 _ll_register_callbacks cb 0x7f1b9c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb0c48640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace693b40 0x555ace6a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb0c48640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace693b40 0x555ace6a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48264/0 (socket says 172.21.15.94:48264) 2024-09-19T04:15:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb0c48640 1 -- 172.21.15.94:0/505248577 learned_addr learned my addr 172.21.15.94:0/505248577 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb1449640 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace6a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb1c4a640 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace69d370 0x555ace6a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb0c48640 1 -- 172.21.15.94:0/505248577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 msgr2=0x555ace6a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb0c48640 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace6a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb0c48640 1 -- 172.21.15.94:0/505248577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace69d370 msgr2=0x555ace6a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb0c48640 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace69d370 0x555ace6a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb0c48640 1 -- 172.21.15.94:0/505248577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ace6a1560 con 0x555ace693b40 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb1449640 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace6a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb0c48640 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace693b40 0x555ace6a5d20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x555ace693070 tx=0x7f1b9c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f40117bf33001 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1b94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1b94ff9640 1 -- 172.21.15.94:0/505248577 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c010cf0 con 0x555ace693b40 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1b94ff9640 1 -- 172.21.15.94:0/505248577 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f1b9c01c3f0 con 0x555ace693b40 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.579+0000 7f1bb5a86740 1 -- 172.21.15.94:0/505248577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ba809a650 con 0x555ace693b40 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1b94ff9640 1 -- 172.21.15.94:0/505248577 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b9c010e90 con 0x555ace693b40 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1bb5a86740 10 client.15435 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1bb5a86740 1 -- 172.21.15.94:0/505248577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555ace7179d0 con 0x555ace693b40 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1b94ff9640 1 -- 172.21.15.94:0/505248577 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba809a650 con 0x555ace693b40 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1b94ff9640 1 -- 172.21.15.94:0/505248577 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1b9c025680 con 0x555ace693b40 2024-09-19T04:15:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1bb5a86740 10 client.15435 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1bb5a86740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1bb5a86740 20 client.15435 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ace718260 0x555ace71a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1bb5a86740 4 client.15435 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.580+0000 7f1bb5a86740 1 -- 172.21.15.94:0/505248577 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ace5ffd70 con 0x555ace718260 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb1449640 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ace718260 0x555ace71a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb1449640 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ace718260 0x555ace71a680 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3de6c676fcf98f server_cookie=d1f3b17fe44936d9 in_seq=0 out_seq=0 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1b94ff9640 10 client.15435 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1b94ff9640 1 -- 172.21.15.94:0/505248577 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ace5ffd70 con 0x555ace718260 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1b94ff9640 10 client.15435 handle_command_reply: tid=0 2024-09-19T04:15:55.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb5a86740 1 client.15435 shutdown 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb5a86740 1 -- 172.21.15.94:0/505248577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace693b40 msgr2=0x555ace6a5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace693b40 0x555ace6a5d20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x555ace693070 tx=0x7f1b9c011c20 comp rx=0 tx=0).stop 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb5a86740 1 -- 172.21.15.94:0/505248577 shutdown_connections 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ace718260 0x555ace71a680 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ace69d370 0x555ace6a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ace693b40 0x555ace6a5d20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb5a86740 1 --2- 172.21.15.94:0/505248577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ace6925d0 0x555ace6a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1bb5a86740 1 -- 172.21.15.94:0/505248577 >> 172.21.15.94:0/505248577 conn(0x555ace65b310 msgr2=0x555ace699fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.581+0000 7f1b94ff9640 0 client.15435 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb5a86740 1 -- 172.21.15.94:0/505248577 shutdown_connections 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb5a86740 1 -- 172.21.15.94:0/505248577 wait complete. 2024-09-19T04:15:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb5a86740 20 client.15435 trim_cache size 0 max 16384 2024-09-19T04:15:55.598 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:15:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b980a34c0 msgr2=0x7f1b980a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b980a34c0 0x7f1b980a5980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4005e30 tx=0x7f1ba4009800 comp rx=0 tx=0).stop 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac100e20 msgr2=0x7f1bac0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac100e20 0x7f1bac0fcc80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800e9f0 tx=0x7f1ba800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2424641871 shutdown_connections 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1b980a34c0 0x7f1b980a5980 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac102ef0 0x7f1bac0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bac102570 0x7f1bac0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 --2- 172.21.15.94:0/2424641871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bac100e20 0x7f1bac0fcc80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.582+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2424641871 >> 172.21.15.94:0/2424641871 conn(0x7f1bac0fc7d0 msgr2=0x7f1bac06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.583+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2424641871 shutdown_connections 2024-09-19T04:15:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:55.583+0000 7f1bb36ad640 1 -- 172.21.15.94:0/2424641871 wait complete. 2024-09-19T04:15:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2500000 of 3200000 tuples (78%) done (elapsed 24.34 s, remaining 6.81 s) 2024-09-19T04:15:55.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:55 smithi112 ceph-mon[35051]: pgmap v30: 97 pgs: 97 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 30 op/s 2024-09-19T04:15:55.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:55 smithi094 ceph-mon[30972]: pgmap v30: 97 pgs: 97 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 30 op/s 2024-09-19T04:15:55.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:55 smithi104 ceph-mon[34145]: pgmap v30: 97 pgs: 97 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 30 op/s 2024-09-19T04:15:56.384 INFO:teuthology.orchestra.run.smithi094.stderr:2600000 of 3200000 tuples (81%) done (elapsed 25.06 s, remaining 5.78 s) 2024-09-19T04:15:57.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:15:57.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:15:57.448 INFO:teuthology.orchestra.run.smithi094.stderr:2700000 of 3200000 tuples (84%) done (elapsed 26.13 s, remaining 4.84 s) 2024-09-19T04:15:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:57 smithi094 ceph-mon[30972]: pgmap v31: 97 pgs: 97 active+clean; 663 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 58 MiB/s wr, 46 op/s 2024-09-19T04:15:57.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:57 smithi112 ceph-mon[35051]: pgmap v31: 97 pgs: 97 active+clean; 663 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 58 MiB/s wr, 46 op/s 2024-09-19T04:15:57.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:57 smithi104 ceph-mon[34145]: pgmap v31: 97 pgs: 97 active+clean; 663 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 58 MiB/s wr, 46 op/s 2024-09-19T04:15:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.100+0000 7fe75649f640 1 -- 172.21.15.94:0/1456855309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 msgr2=0x7fe7501067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.100+0000 7fe75649f640 1 --2- 172.21.15.94:0/1456855309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 0x7fe7501067f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe744009900 tx=0x7fe74402a410 comp rx=0 tx=0).stop 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.100+0000 7fe75649f640 1 -- 172.21.15.94:0/1456855309 shutdown_connections 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.100+0000 7fe75649f640 1 --2- 172.21.15.94:0/1456855309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe750106f20 0x7fe75010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.100+0000 7fe75649f640 1 --2- 172.21.15.94:0/1456855309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 0x7fe7501067f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.100+0000 7fe75649f640 1 --2- 172.21.15.94:0/1456855309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe750101df0 0x7fe7501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.100+0000 7fe75649f640 1 -- 172.21.15.94:0/1456855309 >> 172.21.15.94:0/1456855309 conn(0x7fe7500fd8f0 msgr2=0x7fe7500ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.100+0000 7fe75649f640 1 -- 172.21.15.94:0/1456855309 shutdown_connections 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.100+0000 7fe75649f640 1 -- 172.21.15.94:0/1456855309 wait complete. 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe75649f640 1 Processor -- start 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe75649f640 1 -- start start 2024-09-19T04:15:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe75649f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe750101df0 0x7fe7501983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe75649f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 0x7fe750198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe75649f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe750106f20 0x7fe75019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe75649f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe750111e30 con 0x7fe7501027a0 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe75649f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe750111cb0 con 0x7fe750101df0 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe75649f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe750111fb0 con 0x7fe750106f20 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe74f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 0x7fe750198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe74f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 0x7fe750198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48282/0 (socket says 172.21.15.94:48282) 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe74f7fe640 1 -- 172.21.15.94:0/3467880629 learned_addr learned my addr 172.21.15.94:0/3467880629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe74ffff640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe750101df0 0x7fe7501983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.101+0000 7fe754a3c640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe750106f20 0x7fe75019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe74f7fe640 1 -- 172.21.15.94:0/3467880629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe750106f20 msgr2=0x7fe75019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe750106f20 0x7fe75019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe74f7fe640 1 -- 172.21.15.94:0/3467880629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe750101df0 msgr2=0x7fe7501983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe750101df0 0x7fe7501983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe74f7fe640 1 -- 172.21.15.94:0/3467880629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75019d3a0 con 0x7fe7501027a0 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 0x7fe750198930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe744005940 tx=0x7fe7440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe74d7fa640 1 -- 172.21.15.94:0/3467880629 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74403f030 con 0x7fe7501027a0 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe75649f640 1 -- 172.21.15.94:0/3467880629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75019d630 con 0x7fe7501027a0 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe74d7fa640 1 -- 172.21.15.94:0/3467880629 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe744003cb0 con 0x7fe7501027a0 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe74d7fa640 1 -- 172.21.15.94:0/3467880629 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744035070 con 0x7fe7501027a0 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.102+0000 7fe75649f640 1 -- 172.21.15.94:0/3467880629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7501a4f70 con 0x7fe7501027a0 2024-09-19T04:15:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe74d7fa640 1 -- 172.21.15.94:0/3467880629 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7fe744042030 con 0x7fe7501027a0 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe758878740 1 Processor -- start 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe758878740 1 -- start start 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe758878740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a4915f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe758878740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4920800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe758878740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4923130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe758878740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5567a487b280 con 0x5567a4916480 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe758878740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5567a4822b10 con 0x5567a4915b60 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe758878740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5567a4877240 con 0x5567a4920d40 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.105+0000 7fe74d7fa640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7380a34c0 0x7fe7380a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4920800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe754a3c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a4915f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4920800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48294/0 (socket says 172.21.15.94:48294) 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74ffff640 1 -- 172.21.15.94:0/1223525359 learned_addr learned my addr 172.21.15.94:0/1223525359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/1223525359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4923130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe754a3c640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7380a34c0 0x7fe7380a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74ffff640 1 -- 172.21.15.94:0/1223525359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 msgr2=0x5567a4923130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74ffff640 1 --2- 172.21.15.94:0/1223525359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4923130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74ffff640 1 -- 172.21.15.94:0/1223525359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 msgr2=0x5567a4915f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74ffff640 1 --2- 172.21.15.94:0/1223525359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a4915f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74ffff640 1 -- 172.21.15.94:0/1223525359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a482f1f0 con 0x5567a4916480 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/1223525359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4923130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74d7fa640 1 -- 172.21.15.94:0/3467880629 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe7440f05b0 con 0x7fe7501027a0 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe754a3c640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7380a34c0 0x7fe7380a5980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe748009fd0 tx=0x7fe748009600 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe74ffff640 1 --2- 172.21.15.94:0/1223525359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4920800 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe740009d20 tx=0x7fe740011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecae8115b68bcdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe7367fc640 1 -- 172.21.15.94:0/1223525359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74001e070 con 0x5567a4916480 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe7367fc640 1 -- 172.21.15.94:0/1223525359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe740019ae0 con 0x5567a4916480 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.106+0000 7fe7367fc640 1 -- 172.21.15.94:0/1223525359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74000eac0 con 0x5567a4916480 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.107+0000 7fe758878740 1 -- 172.21.15.94:0/1223525359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 msgr2=0x5567a4920800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.107+0000 7fe758878740 1 --2- 172.21.15.94:0/1223525359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4920800 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe740009d20 tx=0x7fe740011810 comp rx=0 tx=0).stop 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.107+0000 7fe758878740 1 -- 172.21.15.94:0/1223525359 shutdown_connections 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.107+0000 7fe758878740 1 --2- 172.21.15.94:0/1223525359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4923130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.107+0000 7fe758878740 1 --2- 172.21.15.94:0/1223525359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4920800 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.107+0000 7fe758878740 1 --2- 172.21.15.94:0/1223525359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a4915f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.107+0000 7fe758878740 1 -- 172.21.15.94:0/1223525359 >> 172.21.15.94:0/1223525359 conn(0x5567a49120f0 msgr2=0x5567a49124d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.107+0000 7fe758878740 1 -- 172.21.15.94:0/1223525359 shutdown_connections 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.107+0000 7fe758878740 1 -- 172.21.15.94:0/1223525359 wait complete. 2024-09-19T04:15:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe758878740 1 -- start start 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe758878740 1 Processor -- start 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe758878740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4915b60 0x5567a491dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe758878740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe758878740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4920d40 0x5567a491e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe758878740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5567a4923910 con 0x5567a4915b60 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe758878740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5567a4835f10 con 0x5567a4920d40 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe758878740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5567a4923a90 con 0x5567a4916480 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe758878740 10 client.0 _ll_register_callbacks cb 0x5567a47fea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4915b60 0x5567a491dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4915b60 0x5567a491dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48308/0 (socket says 172.21.15.94:48308) 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe74ffff640 1 -- 172.21.15.94:0/3365873032 learned_addr learned my addr 172.21.15.94:0/3365873032 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.108+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe754a3c640 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4920d40 0x5567a491e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe74ffff640 1 -- 172.21.15.94:0/3365873032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 msgr2=0x5567a491e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe74ffff640 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe74ffff640 1 -- 172.21.15.94:0/3365873032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4920d40 msgr2=0x5567a491e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe74ffff640 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4920d40 0x5567a491e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe74ffff640 1 -- 172.21.15.94:0/3365873032 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a45ec6a0 con 0x5567a4915b60 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe74ffff640 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4915b60 0x5567a491dc00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe74000e760 tx=0x7fe740004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecb777044efdbbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe72f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3365873032 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74001e050 con 0x5567a4915b60 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3365873032 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe740019630 con 0x5567a4915b60 2024-09-19T04:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe758878740 1 -- 172.21.15.94:0/3365873032 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe740019ae0 con 0x5567a4915b60 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3365873032 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe740032050 con 0x5567a4915b60 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe758878740 10 client.15447 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe758878740 1 -- 172.21.15.94:0/3365873032 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567a492f4f0 con 0x5567a4915b60 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.109+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3365873032 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe740019ae0 con 0x5567a4915b60 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3365873032 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe74002d030 con 0x5567a4915b60 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe758878740 10 client.15447 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe758878740 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe758878740 20 client.15447 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe758878740 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5567a49309b0 0x5567a4932da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe758878740 4 client.15447 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe758878740 1 -- 172.21.15.94:0/3365873032 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567a48887f0 con 0x5567a49309b0 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5567a49309b0 0x5567a4932da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5567a49309b0 0x5567a4932da0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24363e01396917bb server_cookie=d78de0c6690bfa79 in_seq=0 out_seq=0 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.110+0000 7fe72f7fe640 10 client.15447 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3365873032 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567a48887f0 con 0x5567a49309b0 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe72f7fe640 10 client.15447 handle_command_reply: tid=0 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 client.15447 shutdown 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 -- 172.21.15.94:0/3365873032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4915b60 msgr2=0x5567a491dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4915b60 0x5567a491dc00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe74000e760 tx=0x7fe740004270 comp rx=0 tx=0).stop 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 -- 172.21.15.94:0/3365873032 shutdown_connections 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5567a49309b0 0x5567a4932da0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4920d40 0x5567a491e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe72f7fe640 0 client.15447 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 --2- 172.21.15.94:0/3365873032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4915b60 0x5567a491dc00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 -- 172.21.15.94:0/3365873032 >> 172.21.15.94:0/3365873032 conn(0x5567a49120f0 msgr2=0x5567a4917e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 -- 172.21.15.94:0/3365873032 shutdown_connections 2024-09-19T04:15:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.111+0000 7fe758878740 1 -- 172.21.15.94:0/3365873032 wait complete. 2024-09-19T04:15:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.112+0000 7fe758878740 20 client.15447 trim_cache size 0 max 16384 2024-09-19T04:15:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe758878740 1 Processor -- start 2024-09-19T04:15:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe758878740 1 -- start start 2024-09-19T04:15:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe758878740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a4918590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe758878740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe758878740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4920d40 0x5567a4938580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe758878740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501ade90 con 0x5567a4920d40 2024-09-19T04:15:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe758878740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501ab760 con 0x5567a4915b60 2024-09-19T04:15:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe758878740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501a95b0 con 0x5567a4916480 2024-09-19T04:15:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a4918590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a4918590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:47958/0 (socket says 172.21.15.94:47958) 2024-09-19T04:15:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54712/0 (socket says 172.21.15.94:54712) 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74ffff640 1 -- 172.21.15.94:0/2889557368 learned_addr learned my addr 172.21.15.94:0/2889557368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe754a3c640 1 --2- 172.21.15.94:0/2889557368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4920d40 0x5567a4938580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74f7fe640 1 -- 172.21.15.94:0/2889557368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 msgr2=0x5567a4918590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/2889557368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a4918590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74f7fe640 1 -- 172.21.15.94:0/2889557368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4920d40 msgr2=0x5567a4938580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/2889557368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4920d40 0x5567a4938580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.134+0000 7fe74f7fe640 1 -- 172.21.15.94:0/2889557368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a4920200 con 0x5567a4916480 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.135+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/2889557368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491d160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe7440edbd0 tx=0x7fe7440b8390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af27f38b132f4809 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.135+0000 7fe735ffb640 1 -- 172.21.15.94:0/2889557368 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744049040 con 0x5567a4916480 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.135+0000 7fe735ffb640 1 -- 172.21.15.94:0/2889557368 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe74403b460 con 0x5567a4916480 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.135+0000 7fe735ffb640 1 -- 172.21.15.94:0/2889557368 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7440baaf0 con 0x5567a4916480 2024-09-19T04:15:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 -- 172.21.15.94:0/2889557368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 msgr2=0x5567a491d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 --2- 172.21.15.94:0/2889557368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491d160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe7440edbd0 tx=0x7fe7440b8390 comp rx=0 tx=0).stop 2024-09-19T04:15:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 -- 172.21.15.94:0/2889557368 shutdown_connections 2024-09-19T04:15:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 --2- 172.21.15.94:0/2889557368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4920d40 0x5567a4938580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 --2- 172.21.15.94:0/2889557368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4916480 0x5567a491d160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 --2- 172.21.15.94:0/2889557368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a4918590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 -- 172.21.15.94:0/2889557368 >> 172.21.15.94:0/2889557368 conn(0x5567a49120f0 msgr2=0x5567a4911270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 -- 172.21.15.94:0/2889557368 shutdown_connections 2024-09-19T04:15:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 -- 172.21.15.94:0/2889557368 wait complete. 2024-09-19T04:15:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 -- start start 2024-09-19T04:15:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.136+0000 7fe758878740 1 Processor -- start 2024-09-19T04:15:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe758878740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a49290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe758878740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4924420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe758878740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4924960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe758878740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501a9c20 con 0x5567a4916480 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe758878740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501ab090 con 0x5567a4915b60 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe758878740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501aa2f0 con 0x5567a4920d40 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe758878740 10 client.0 _ll_register_callbacks cb 0x7fe74001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe74f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4924420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe74f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4924420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48328/0 (socket says 172.21.15.94:48328) 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe74f7fe640 1 -- 172.21.15.94:0/3730208420 learned_addr learned my addr 172.21.15.94:0/3730208420 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe754a3c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4924960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe74ffff640 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a49290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe74f7fe640 1 -- 172.21.15.94:0/3730208420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 msgr2=0x5567a4924960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4924960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe74f7fe640 1 -- 172.21.15.94:0/3730208420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 msgr2=0x5567a49290f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a49290f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe74f7fe640 1 -- 172.21.15.94:0/3730208420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a4924ea0 con 0x5567a4916480 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.137+0000 7fe754a3c640 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4924960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe74f7fe640 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4924420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe7440b8bc0 tx=0x7fe74402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5106f1aa79ad747f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe72f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3730208420 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7440d9020 con 0x5567a4916480 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3730208420 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe744003800 con 0x5567a4916480 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe758878740 1 -- 172.21.15.94:0/3730208420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567a4882d00 con 0x5567a4916480 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3730208420 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe744049050 con 0x5567a4916480 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe758878740 10 client.15453 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe758878740 1 -- 172.21.15.94:0/3730208420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567a499b260 con 0x5567a4916480 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3730208420 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5567a4882d00 con 0x5567a4916480 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.138+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3730208420 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7440dc040 con 0x5567a4916480 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.139+0000 7fe758878740 10 client.15453 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.139+0000 7fe758878740 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.139+0000 7fe758878740 20 client.15453 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.139+0000 7fe758878740 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5567a499ba20 0x5567a499de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.139+0000 7fe758878740 4 client.15453 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.139+0000 7fe758878740 1 -- 172.21.15.94:0/3730208420 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5567a47ff110 con 0x5567a499ba20 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.139+0000 7fe74ffff640 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5567a499ba20 0x5567a499de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.139+0000 7fe72f7fe640 10 client.15453 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.140+0000 7fe74ffff640 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5567a499ba20 0x5567a499de40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1048eedc091a34 server_cookie=6ea47f01a88c34ab in_seq=0 out_seq=0 2024-09-19T04:15:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.140+0000 7fe72f7fe640 1 -- 172.21.15.94:0/3730208420 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5567a47ff110 con 0x5567a499ba20 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.140+0000 7fe72f7fe640 10 client.15453 handle_command_reply: tid=0 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.140+0000 7fe758878740 1 client.15453 shutdown 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 -- 172.21.15.94:0/3730208420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 msgr2=0x5567a4924420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4924420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe7440b8bc0 tx=0x7fe74402a410 comp rx=0 tx=0).stop 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 -- 172.21.15.94:0/3730208420 shutdown_connections 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5567a499ba20 0x5567a499de40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a4920d40 0x5567a4924960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567a4916480 0x5567a4924420 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 --2- 172.21.15.94:0/3730208420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567a4915b60 0x5567a49290f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 -- 172.21.15.94:0/3730208420 >> 172.21.15.94:0/3730208420 conn(0x5567a49120f0 msgr2=0x5567a491e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe72f7fe640 0 client.15453 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 -- 172.21.15.94:0/3730208420 shutdown_connections 2024-09-19T04:15:58.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 1 -- 172.21.15.94:0/3730208420 wait complete. 2024-09-19T04:15:58.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.141+0000 7fe758878740 20 client.15453 trim_cache size 0 max 16384 2024-09-19T04:15:58.148 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:15:58.148 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:15:58.148 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "ef800d3c-dbb4-41d8-9f15-e05a8238ede7", 2024-09-19T04:15:58.148 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:15:58.148 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:15:58.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 -- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7380a34c0 msgr2=0x7fe7380a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7380a34c0 0x7fe7380a5980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe748009fd0 tx=0x7fe748009600 comp rx=0 tx=0).stop 2024-09-19T04:15:58.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 -- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 msgr2=0x7fe750198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 0x7fe750198930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe744005940 tx=0x7fe7440042f0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 -- 172.21.15.94:0/3467880629 shutdown_connections 2024-09-19T04:15:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7380a34c0 0x7fe7380a5980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe750106f20 0x7fe75019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7501027a0 0x7fe750198930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 --2- 172.21.15.94:0/3467880629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe750101df0 0x7fe7501983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 -- 172.21.15.94:0/3467880629 >> 172.21.15.94:0/3467880629 conn(0x7fe7500fd8f0 msgr2=0x7fe7500fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 -- 172.21.15.94:0/3467880629 shutdown_connections 2024-09-19T04:15:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:58.142+0000 7fe75649f640 1 -- 172.21.15.94:0/3467880629 wait complete. 2024-09-19T04:15:58.244 INFO:teuthology.orchestra.run.smithi094.stderr:2800000 of 3200000 tuples (87%) done (elapsed 26.92 s, remaining 3.85 s) 2024-09-19T04:15:58.860 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:15:59.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:58 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:15:59.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:58 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:59.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:58 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:15:59.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:58 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:15:59.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:58 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:59.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:58 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:58 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:15:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:58 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:58 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:15:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:58 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:15:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:58 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:58 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:59.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:58 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:15:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:58 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:58 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:15:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:58 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:15:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:58 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:58 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:15:59.358 INFO:teuthology.orchestra.run.smithi094.stderr:2900000 of 3200000 tuples (90%) done (elapsed 28.04 s, remaining 2.90 s) 2024-09-19T04:15:59.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.643+0000 7f6c8f36a640 1 -- 172.21.15.94:0/666572324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88106f20 msgr2=0x7f6c8810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:59.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.643+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/666572324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88106f20 0x7f6c8810eaa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009ca0 tx=0x7f6c8002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.643+0000 7f6c8f36a640 1 -- 172.21.15.94:0/666572324 shutdown_connections 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.643+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/666572324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88106f20 0x7f6c8810eaa0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.643+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/666572324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c881027a0 0x7f6c881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.643+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/666572324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c88101df0 0x7f6c881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.643+0000 7f6c8f36a640 1 -- 172.21.15.94:0/666572324 >> 172.21.15.94:0/666572324 conn(0x7f6c880fd8f0 msgr2=0x7f6c880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.643+0000 7f6c8f36a640 1 -- 172.21.15.94:0/666572324 shutdown_connections 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.644+0000 7f6c8f36a640 1 -- 172.21.15.94:0/666572324 wait complete. 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.644+0000 7f6c8f36a640 1 Processor -- start 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.644+0000 7f6c8f36a640 1 -- start start 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.644+0000 7f6c8f36a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88101df0 0x7f6c88198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.644+0000 7f6c8f36a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c881027a0 0x7f6c88198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.644+0000 7f6c8f36a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c88106f20 0x7f6c8819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8f36a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88111e30 con 0x7f6c88101df0 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8f36a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88111cb0 con 0x7f6c88106f20 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8f36a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88111fb0 con 0x7f6c881027a0 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d106640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88101df0 0x7f6c88198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8c905640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c881027a0 0x7f6c88198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d106640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88101df0 0x7f6c88198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48352/0 (socket says 172.21.15.94:48352) 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d106640 1 -- 172.21.15.94:0/2284573792 learned_addr learned my addr 172.21.15.94:0/2284573792 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d907640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c88106f20 0x7f6c8819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d106640 1 -- 172.21.15.94:0/2284573792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c881027a0 msgr2=0x7f6c88198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d106640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c881027a0 0x7f6c88198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d106640 1 -- 172.21.15.94:0/2284573792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c88106f20 msgr2=0x7f6c8819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d106640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c88106f20 0x7f6c8819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d106640 1 -- 172.21.15.94:0/2284573792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8819d2e0 con 0x7f6c88101df0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d907640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c88106f20 0x7f6c8819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8d106640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88101df0 0x7f6c88198330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c890 tx=0x7f6c8400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c7e7fc640 1 -- 172.21.15.94:0/2284573792 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84004220 con 0x7f6c88101df0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c7e7fc640 1 -- 172.21.15.94:0/2284573792 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c840043c0 con 0x7f6c88101df0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.645+0000 7f6c8f36a640 1 -- 172.21.15.94:0/2284573792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8819d5d0 con 0x7f6c88101df0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.646+0000 7f6c7e7fc640 1 -- 172.21.15.94:0/2284573792 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84004a00 con 0x7f6c88101df0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.646+0000 7f6c8f36a640 1 -- 172.21.15.94:0/2284573792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c881a4eb0 con 0x7f6c88101df0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.648+0000 7f6c7e7fc640 1 -- 172.21.15.94:0/2284573792 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138531+0+0 (secure 0 0 0) 0x7f6c84004ba0 con 0x7f6c88101df0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.649+0000 7f6c7e7fc640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6c740a34c0 0x7f6c740a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.649+0000 7f6c91743740 1 Processor -- start 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.649+0000 7f6c8c905640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6c740a34c0 0x7f6c740a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.649+0000 7f6c7e7fc640 1 -- 172.21.15.94:0/2284573792 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f6c840cfb70 con 0x7f6c88101df0 2024-09-19T04:15:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.649+0000 7f6c91743740 1 -- start start 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8c905640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6c740a34c0 0x7f6c740a5980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c880ffe70 tx=0x7f6c7800b410 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c91743740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b871a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c91743740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 0x5561b8725140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c91743740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b8725680 0x5561b8727a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c91743740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5561b86802e0 con 0x5561b871a4a0 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c91743740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5561b8627ce0 con 0x5561b871adc0 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c91743740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5561b867c2a0 con 0x5561b8725680 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d106640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b871a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d106640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b871a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48368/0 (socket says 172.21.15.94:48368) 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d106640 1 -- 172.21.15.94:0/931027691 learned_addr learned my addr 172.21.15.94:0/931027691 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8c905640 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 0x5561b8725140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d907640 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b8725680 0x5561b8727a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d106640 1 -- 172.21.15.94:0/931027691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b8725680 msgr2=0x5561b8727a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d106640 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b8725680 0x5561b8727a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d106640 1 -- 172.21.15.94:0/931027691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 msgr2=0x5561b8725140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d106640 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 0x5561b8725140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d106640 1 -- 172.21.15.94:0/931027691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561b860c440 con 0x5561b871a4a0 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.650+0000 7f6c8d907640 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b8725680 0x5561b8727a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.651+0000 7f6c8d106640 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b871a880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6c840058c0 tx=0x7f6c8409b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fa9290896f08e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.651+0000 7f6c737fe640 1 -- 172.21.15.94:0/931027691 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8401d040 con 0x5561b871a4a0 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.651+0000 7f6c737fe640 1 -- 172.21.15.94:0/931027691 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c840a0020 con 0x5561b871a4a0 2024-09-19T04:15:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.651+0000 7f6c737fe640 1 -- 172.21.15.94:0/931027691 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c840cd440 con 0x5561b871a4a0 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 -- 172.21.15.94:0/931027691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 msgr2=0x5561b871a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b871a880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6c840058c0 tx=0x7f6c8409b690 comp rx=0 tx=0).stop 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 -- 172.21.15.94:0/931027691 shutdown_connections 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b8725680 0x5561b8727a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 0x5561b8725140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 --2- 172.21.15.94:0/931027691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b871a880 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 -- 172.21.15.94:0/931027691 >> 172.21.15.94:0/931027691 conn(0x5561b8716a30 msgr2=0x5561b8716e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 -- 172.21.15.94:0/931027691 shutdown_connections 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 -- 172.21.15.94:0/931027691 wait complete. 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 -- start start 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.652+0000 7f6c91743740 1 Processor -- start 2024-09-19T04:15:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c91743740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871a4a0 0x5561b8725010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c91743740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 0x5561b87222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c91743740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b8722840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c91743740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5561b8728250 con 0x5561b8725680 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c91743740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5561b863b050 con 0x5561b871adc0 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c91743740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5561b87283d0 con 0x5561b871a4a0 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c91743740 10 client.0 _ll_register_callbacks cb 0x5561b867b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d106640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871a4a0 0x5561b8725010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d907640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b8722840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d907640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b8722840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48376/0 (socket says 172.21.15.94:48376) 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d106640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871a4a0 0x5561b8725010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54762/0 (socket says 172.21.15.94:54762) 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d907640 1 -- 172.21.15.94:0/1304990708 learned_addr learned my addr 172.21.15.94:0/1304990708 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8c905640 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 0x5561b87222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d106640 1 -- 172.21.15.94:0/1304990708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 msgr2=0x5561b87222c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d106640 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 0x5561b87222c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d106640 1 -- 172.21.15.94:0/1304990708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 msgr2=0x5561b8722840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d106640 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b8722840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d106640 1 -- 172.21.15.94:0/1304990708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561b83f13c0 con 0x5561b871a4a0 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c8d106640 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871a4a0 0x5561b8725010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c840058c0 tx=0x7f6c8400a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69950f054bc79afd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.653+0000 7f6c70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c70ff9640 1 -- 172.21.15.94:0/1304990708 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8401d070 con 0x5561b871a4a0 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c70ff9640 1 -- 172.21.15.94:0/1304990708 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f6c84098350 con 0x5561b871a4a0 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c91743740 1 -- 172.21.15.94:0/1304990708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c840a0020 con 0x5561b871a4a0 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c70ff9640 1 -- 172.21.15.94:0/1304990708 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c840a48e0 con 0x5561b871a4a0 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c91743740 10 client.25027 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c91743740 1 -- 172.21.15.94:0/1304990708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561b8728250 con 0x5561b871a4a0 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c70ff9640 1 -- 172.21.15.94:0/1304990708 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c840a0020 con 0x5561b871a4a0 2024-09-19T04:15:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c70ff9640 1 -- 172.21.15.94:0/1304990708 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c840ba3a0 con 0x5561b871a4a0 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c91743740 10 client.25027 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c91743740 10 client.25027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c91743740 20 client.25027 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c91743740 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5561b8734e10 0x5561b8737200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c91743740 4 client.25027 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.654+0000 7f6c91743740 1 -- 172.21.15.94:0/1304990708 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561b868d850 con 0x5561b8734e10 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.655+0000 7f6c8c905640 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5561b8734e10 0x5561b8737200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.655+0000 7f6c70ff9640 10 client.25027 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.655+0000 7f6c8c905640 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5561b8734e10 0x5561b8737200 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df7d38a7ae33ce4c server_cookie=ee3c7e36521625c3 in_seq=0 out_seq=0 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c70ff9640 1 -- 172.21.15.94:0/1304990708 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561b868d850 con 0x5561b8734e10 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c70ff9640 10 client.25027 handle_command_reply: tid=0 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 client.25027 shutdown 2024-09-19T04:15:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 -- 172.21.15.94:0/1304990708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871a4a0 msgr2=0x5561b8725010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871a4a0 0x5561b8725010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c840058c0 tx=0x7f6c8400a3f0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 -- 172.21.15.94:0/1304990708 shutdown_connections 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5561b8734e10 0x5561b8737200 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b8722840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871adc0 0x5561b87222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 --2- 172.21.15.94:0/1304990708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871a4a0 0x5561b8725010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 -- 172.21.15.94:0/1304990708 >> 172.21.15.94:0/1304990708 conn(0x5561b8716a30 msgr2=0x5561b871c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c70ff9640 0 client.25027 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 -- 172.21.15.94:0/1304990708 shutdown_connections 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 1 -- 172.21.15.94:0/1304990708 wait complete. 2024-09-19T04:15:59.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.656+0000 7f6c91743740 20 client.25027 trim_cache size 0 max 16384 2024-09-19T04:15:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.680+0000 7f6c91743740 1 Processor -- start 2024-09-19T04:15:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c91743740 1 -- start start 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c91743740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b873c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c91743740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 0x5561b873cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c91743740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b8725680 0x5561b871f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c91743740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881b2370 con 0x5561b871a4a0 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c91743740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881afc40 con 0x5561b8725680 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c91743740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881ada90 con 0x5561b871adc0 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8d106640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b873c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8d106640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b873c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48378/0 (socket says 172.21.15.94:48378) 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8d106640 1 -- 172.21.15.94:0/47327170 learned_addr learned my addr 172.21.15.94:0/47327170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8c905640 1 --2- 172.21.15.94:0/47327170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 0x5561b873cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8d907640 1 --2- 172.21.15.94:0/47327170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b8725680 0x5561b871f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8d106640 1 -- 172.21.15.94:0/47327170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 msgr2=0x5561b873cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8d106640 1 --2- 172.21.15.94:0/47327170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 0x5561b873cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8d106640 1 -- 172.21.15.94:0/47327170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b8725680 msgr2=0x5561b871f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8d106640 1 --2- 172.21.15.94:0/47327170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b8725680 0x5561b871f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.681+0000 7f6c8d106640 1 -- 172.21.15.94:0/47327170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561b8717ae0 con 0x5561b871a4a0 2024-09-19T04:15:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.682+0000 7f6c8d106640 1 --2- 172.21.15.94:0/47327170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b873c9f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6c840a43b0 tx=0x7f6c84000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8b6397c84244fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.682+0000 7f6c72ffd640 1 -- 172.21.15.94:0/47327170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8401d070 con 0x5561b871a4a0 2024-09-19T04:15:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.682+0000 7f6c72ffd640 1 -- 172.21.15.94:0/47327170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c840a0020 con 0x5561b871a4a0 2024-09-19T04:15:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.682+0000 7f6c72ffd640 1 -- 172.21.15.94:0/47327170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8409b0c0 con 0x5561b871a4a0 2024-09-19T04:15:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.682+0000 7f6c91743740 1 -- 172.21.15.94:0/47327170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 msgr2=0x5561b873c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.682+0000 7f6c91743740 1 --2- 172.21.15.94:0/47327170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b873c9f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6c840a43b0 tx=0x7f6c84000bc0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.683+0000 7f6c91743740 1 -- 172.21.15.94:0/47327170 shutdown_connections 2024-09-19T04:15:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.683+0000 7f6c91743740 1 --2- 172.21.15.94:0/47327170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b8725680 0x5561b871f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.683+0000 7f6c91743740 1 --2- 172.21.15.94:0/47327170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 0x5561b873cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.683+0000 7f6c91743740 1 --2- 172.21.15.94:0/47327170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b871a4a0 0x5561b873c9f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.683+0000 7f6c91743740 1 -- 172.21.15.94:0/47327170 >> 172.21.15.94:0/47327170 conn(0x5561b8716a30 msgr2=0x5561b8715b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.683+0000 7f6c91743740 1 -- 172.21.15.94:0/47327170 shutdown_connections 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.683+0000 7f6c91743740 1 -- 172.21.15.94:0/47327170 wait complete. 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.683+0000 7f6c91743740 1 -- start start 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.683+0000 7f6c91743740 1 Processor -- start 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c91743740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871a4a0 0x5561b872b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c91743740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 0x5561b872b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c91743740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b872be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c91743740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881ae100 con 0x5561b8725680 2024-09-19T04:15:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c91743740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881af570 con 0x5561b871a4a0 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c91743740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881ae7d0 con 0x5561b871adc0 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c91743740 10 client.0 _ll_register_callbacks cb 0x7f6c7800b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d907640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b872be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d907640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b872be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48384/0 (socket says 172.21.15.94:48384) 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d907640 1 -- 172.21.15.94:0/3679841922 learned_addr learned my addr 172.21.15.94:0/3679841922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d106640 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871a4a0 0x5561b872b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8c905640 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 0x5561b872b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d907640 1 -- 172.21.15.94:0/3679841922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 msgr2=0x5561b872b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d907640 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 0x5561b872b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d907640 1 -- 172.21.15.94:0/3679841922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871a4a0 msgr2=0x5561b872b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d907640 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871a4a0 0x5561b872b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d907640 1 -- 172.21.15.94:0/3679841922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561b879ea00 con 0x5561b8725680 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d106640 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871a4a0 0x5561b872b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c8d907640 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b872be40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6c80031fd0 tx=0x7f6c80031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b531e0b2e9059c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.684+0000 7f6c70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.685+0000 7f6c70ff9640 1 -- 172.21.15.94:0/3679841922 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80003d80 con 0x5561b8725680 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.685+0000 7f6c70ff9640 1 -- 172.21.15.94:0/3679841922 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f6c80004040 con 0x5561b8725680 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.685+0000 7f6c91743740 1 -- 172.21.15.94:0/3679841922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c840a0020 con 0x5561b8725680 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.685+0000 7f6c70ff9640 1 -- 172.21.15.94:0/3679841922 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c80054050 con 0x5561b8725680 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.685+0000 7f6c91743740 10 client.15471 fetch_fsmap learned FSMap version 27 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.685+0000 7f6c91743740 1 -- 172.21.15.94:0/3679841922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561b879fcd0 con 0x5561b8725680 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.685+0000 7f6c70ff9640 1 -- 172.21.15.94:0/3679841922 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c840a0020 con 0x5561b8725680 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.686+0000 7f6c70ff9640 1 -- 172.21.15.94:0/3679841922 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c8004f030 con 0x5561b8725680 2024-09-19T04:15:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.686+0000 7f6c91743740 10 client.15471 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.686+0000 7f6c91743740 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.686+0000 7f6c91743740 20 client.15471 populate_metadata read hostname 'smithi094' 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.686+0000 7f6c91743740 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5561b87a04d0 0x5561b87a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.686+0000 7f6c91743740 4 client.15471 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.686+0000 7f6c91743740 1 -- 172.21.15.94:0/3679841922 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561b8687d60 con 0x5561b87a04d0 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.686+0000 7f6c8d106640 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5561b87a04d0 0x5561b87a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c70ff9640 10 client.15471 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c8d106640 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5561b87a04d0 0x5561b87a28f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24322b5e22deea7 server_cookie=86fe8bb7c7159f8e in_seq=0 out_seq=0 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c70ff9640 1 -- 172.21.15.94:0/3679841922 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5561b8687d60 con 0x5561b87a04d0 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c70ff9640 10 client.15471 handle_command_reply: tid=0 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 client.15471 shutdown 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 -- 172.21.15.94:0/3679841922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 msgr2=0x5561b872be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b872be40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6c80031fd0 tx=0x7f6c80031680 comp rx=0 tx=0).stop 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 -- 172.21.15.94:0/3679841922 shutdown_connections 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5561b87a04d0 0x5561b87a28f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561b8725680 0x5561b872be40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561b871adc0 0x5561b872b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 --2- 172.21.15.94:0/3679841922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b871a4a0 0x5561b872b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c70ff9640 0 client.15471 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 -- 172.21.15.94:0/3679841922 >> 172.21.15.94:0/3679841922 conn(0x5561b8716a30 msgr2=0x5561b8723b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 -- 172.21.15.94:0/3679841922 shutdown_connections 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 1 -- 172.21.15.94:0/3679841922 wait complete. 2024-09-19T04:15:59.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.687+0000 7f6c91743740 20 client.15471 trim_cache size 0 max 16384 2024-09-19T04:15:59.695 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:15:59.695 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:15:59.695 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:15:59.695 INFO:teuthology.orchestra.run.smithi094.stdout: "ef800d3c-dbb4-41d8-9f15-e05a8238ede7": { 2024-09-19T04:15:59.695 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:15:59.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "ef800d3c-dbb4-41d8-9f15-e05a8238ede7", 2024-09-19T04:15:59.695 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:15:59.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:15:59.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:15:59.696 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:15:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 -- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6c740a34c0 msgr2=0x7f6c740a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6c740a34c0 0x7f6c740a5980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c880ffe70 tx=0x7f6c7800b410 comp rx=0 tx=0).stop 2024-09-19T04:15:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 -- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88101df0 msgr2=0x7f6c88198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88101df0 0x7f6c88198330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c890 tx=0x7f6c8400cd50 comp rx=0 tx=0).stop 2024-09-19T04:15:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 -- 172.21.15.94:0/2284573792 shutdown_connections 2024-09-19T04:15:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6c740a34c0 0x7f6c740a5980 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c88106f20 0x7f6c8819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c881027a0 0x7f6c88198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 --2- 172.21.15.94:0/2284573792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c88101df0 0x7f6c88198330 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 -- 172.21.15.94:0/2284573792 >> 172.21.15.94:0/2284573792 conn(0x7f6c880fd8f0 msgr2=0x7f6c880fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 -- 172.21.15.94:0/2284573792 shutdown_connections 2024-09-19T04:15:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:15:59.688+0000 7f6c8f36a640 1 -- 172.21.15.94:0/2284573792 wait complete. 2024-09-19T04:16:00.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:59 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:16:00.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:59 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:00.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:59 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:16:00.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:59 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:00.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:15:59 smithi094 ceph-mon[30972]: pgmap v32: 97 pgs: 97 active+clean; 663 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 31 op/s 2024-09-19T04:16:00.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:59 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:16:00.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:59 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:00.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:59 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:16:00.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:59 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:00.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:15:59 smithi104 ceph-mon[34145]: pgmap v32: 97 pgs: 97 active+clean; 663 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 31 op/s 2024-09-19T04:16:00.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:59 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:16:00.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:59 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:00.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:59 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:16:00.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:59 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:00.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:15:59 smithi112 ceph-mon[35051]: pgmap v32: 97 pgs: 97 active+clean; 663 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 31 op/s 2024-09-19T04:16:00.409 INFO:teuthology.orchestra.run.smithi094.stderr:3000000 of 3200000 tuples (93%) done (elapsed 29.09 s, remaining 1.94 s) 2024-09-19T04:16:00.506 INFO:tasks.cephfs.filesystem:scrub status for tag:ef800d3c-dbb4-41d8-9f15-e05a8238ede7 - {'path': '/', 'tag': 'ef800d3c-dbb4-41d8-9f15-e05a8238ede7', 'options': 'recursive,force'} 2024-09-19T04:16:00.840 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:16:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:16:00.512+0000 7f9f5b6aa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:16:00.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:16:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:16:00.512+0000 7f9f5b6aa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:16:00.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:16:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:16:00.512+0000 7f9f5b6aa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:16:00.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:16:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:16:00.512+0000 7f9f5b6aa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:16:00.841 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:16:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:16:00.512+0000 7f9f5b6aa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:16:01.209 INFO:teuthology.orchestra.run.smithi094.stderr:3100000 of 3200000 tuples (96%) done (elapsed 29.89 s, remaining 0.96 s) 2024-09-19T04:16:01.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:01 smithi112 ceph-mon[35051]: pgmap v33: 97 pgs: 97 active+clean; 663 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 31 op/s 2024-09-19T04:16:01.788 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:01 smithi112 ceph-mon[35051]: mgrmap e24: x(active, since 60s), standbys: y, z 2024-09-19T04:16:01.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:01 smithi094 ceph-mon[30972]: pgmap v33: 97 pgs: 97 active+clean; 663 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 31 op/s 2024-09-19T04:16:01.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:01 smithi094 ceph-mon[30972]: mgrmap e24: x(active, since 60s), standbys: y, z 2024-09-19T04:16:01.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:01 smithi104 ceph-mon[34145]: pgmap v33: 97 pgs: 97 active+clean; 663 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 31 op/s 2024-09-19T04:16:01.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:01 smithi104 ceph-mon[34145]: mgrmap e24: x(active, since 60s), standbys: y, z 2024-09-19T04:16:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:3200000 of 3200000 tuples (100%) done (elapsed 31.00 s, remaining 0.00 s) 2024-09-19T04:16:02.413 INFO:teuthology.orchestra.run.smithi094.stderr:vacuuming... 2024-09-19T04:16:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:03 smithi112 ceph-mon[35051]: pgmap v34: 97 pgs: 97 active+clean; 806 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 58 MiB/s wr, 62 op/s 2024-09-19T04:16:03.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:03 smithi094 ceph-mon[30972]: pgmap v34: 97 pgs: 97 active+clean; 806 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 58 MiB/s wr, 62 op/s 2024-09-19T04:16:03.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:03 smithi104 ceph-mon[34145]: pgmap v34: 97 pgs: 97 active+clean; 806 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 58 MiB/s wr, 62 op/s 2024-09-19T04:16:05.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:05 smithi112 ceph-mon[35051]: pgmap v35: 97 pgs: 97 active+clean; 806 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 MiB/s wr, 46 op/s 2024-09-19T04:16:05.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:05 smithi094 ceph-mon[30972]: pgmap v35: 97 pgs: 97 active+clean; 806 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 MiB/s wr, 46 op/s 2024-09-19T04:16:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:05 smithi104 ceph-mon[34145]: pgmap v35: 97 pgs: 97 active+clean; 806 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 MiB/s wr, 46 op/s 2024-09-19T04:16:07.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:06 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:16:07.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:06 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:16:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:06 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:16:08.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:07 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:16:08.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:07 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:16:08.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:07 smithi112 ceph-mon[35051]: pgmap v36: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 64 MiB/s wr, 65 op/s 2024-09-19T04:16:08.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:07 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:16:08.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:07 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:16:08.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:07 smithi094 ceph-mon[30972]: pgmap v36: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 64 MiB/s wr, 65 op/s 2024-09-19T04:16:08.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:07 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:16:08.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:07 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:16:08.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:07 smithi104 ceph-mon[34145]: pgmap v36: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 64 MiB/s wr, 65 op/s 2024-09-19T04:16:09.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:08 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:16:09.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:08 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:16:09.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:08 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:16:09.449 INFO:teuthology.orchestra.run.smithi094.stderr:creating primary keys... 2024-09-19T04:16:10.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:09 smithi112 ceph-mon[35051]: pgmap v37: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 44 MiB/s wr, 48 op/s 2024-09-19T04:16:10.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:09 smithi094 ceph-mon[30972]: pgmap v37: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 44 MiB/s wr, 48 op/s 2024-09-19T04:16:10.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:09 smithi104 ceph-mon[34145]: pgmap v37: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 44 MiB/s wr, 48 op/s 2024-09-19T04:16:11.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:11 smithi112 ceph-mon[35051]: pgmap v38: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 44 MiB/s wr, 48 op/s 2024-09-19T04:16:11.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:11 smithi094 ceph-mon[30972]: pgmap v38: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 44 MiB/s wr, 48 op/s 2024-09-19T04:16:11.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:11 smithi104 ceph-mon[34145]: pgmap v38: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 44 MiB/s wr, 48 op/s 2024-09-19T04:16:12.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:12.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:12.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:12.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:16:12.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:12.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:12.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:12.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:16:12.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:12.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:12.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:12.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:16:13.755 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:13 smithi104 ceph-mon[34145]: pgmap v39: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 MiB/s wr, 71 op/s 2024-09-19T04:16:13.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:13 smithi112 ceph-mon[35051]: pgmap v39: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 MiB/s wr, 71 op/s 2024-09-19T04:16:13.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:13 smithi094 ceph-mon[30972]: pgmap v39: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 MiB/s wr, 71 op/s 2024-09-19T04:16:14.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:14 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:16:14.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:14 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:16:14.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:14 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:16:15.417 INFO:teuthology.orchestra.run.smithi094.stderr:done in 44.16 s (drop tables 0.00 s, create tables 0.04 s, client-side generate 31.11 s, vacuum 7.04 s, primary keys 5.97 s). 2024-09-19T04:16:15.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-19T04:16:15.601 INFO:teuthology.orchestra.run.smithi094.stderr:starting vacuum...end. 2024-09-19T04:16:15.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:15 smithi112 ceph-mon[35051]: pgmap v40: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 47 MiB/s wr, 40 op/s 2024-09-19T04:16:15.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:15 smithi094 ceph-mon[30972]: pgmap v40: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 47 MiB/s wr, 40 op/s 2024-09-19T04:16:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:15 smithi104 ceph-mon[34145]: pgmap v40: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 47 MiB/s wr, 40 op/s 2024-09-19T04:16:17.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:17 smithi112 ceph-mon[35051]: pgmap v41: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 62 MiB/s wr, 90 op/s 2024-09-19T04:16:17.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:17 smithi094 ceph-mon[30972]: pgmap v41: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 62 MiB/s wr, 90 op/s 2024-09-19T04:16:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:17 smithi104 ceph-mon[34145]: pgmap v41: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 62 MiB/s wr, 90 op/s 2024-09-19T04:16:19.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:19 smithi112 ceph-mon[35051]: pgmap v42: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 38 MiB/s wr, 71 op/s 2024-09-19T04:16:19.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:19 smithi094 ceph-mon[30972]: pgmap v42: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 38 MiB/s wr, 71 op/s 2024-09-19T04:16:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:19 smithi104 ceph-mon[34145]: pgmap v42: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 38 MiB/s wr, 71 op/s 2024-09-19T04:16:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 5.0 s, 2102.6 tps, lat 44.873 ms stddev 52.407 2024-09-19T04:16:21.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:21 smithi112 ceph-mon[35051]: pgmap v43: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 38 MiB/s wr, 71 op/s 2024-09-19T04:16:21.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:21 smithi094 ceph-mon[30972]: pgmap v43: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 38 MiB/s wr, 71 op/s 2024-09-19T04:16:21.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:21 smithi104 ceph-mon[34145]: pgmap v43: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 38 MiB/s wr, 71 op/s 2024-09-19T04:16:23.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:23 smithi112 ceph-mon[35051]: pgmap v44: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 43 MiB/s wr, 475 op/s 2024-09-19T04:16:23.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:23 smithi094 ceph-mon[30972]: pgmap v44: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 43 MiB/s wr, 475 op/s 2024-09-19T04:16:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:23 smithi104 ceph-mon[34145]: pgmap v44: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 43 MiB/s wr, 475 op/s 2024-09-19T04:16:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 10.0 s, 1538.4 tps, lat 64.816 ms stddev 18.309 2024-09-19T04:16:25.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:25 smithi112 ceph-mon[35051]: pgmap v45: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 452 op/s 2024-09-19T04:16:25.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:25 smithi094 ceph-mon[30972]: pgmap v45: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 452 op/s 2024-09-19T04:16:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:25 smithi104 ceph-mon[34145]: pgmap v45: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 452 op/s 2024-09-19T04:16:27.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:27 smithi112 ceph-mon[35051]: pgmap v46: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 27 MiB/s wr, 1.32k op/s 2024-09-19T04:16:27.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:27 smithi094 ceph-mon[30972]: pgmap v46: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 27 MiB/s wr, 1.32k op/s 2024-09-19T04:16:27.853 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:27 smithi104 ceph-mon[34145]: pgmap v46: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 27 MiB/s wr, 1.32k op/s 2024-09-19T04:16:29.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:29 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:16:29.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:29 smithi112 ceph-mon[35051]: pgmap v47: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-09-19T04:16:29.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:29 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:16:29.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:29 smithi094 ceph-mon[30972]: pgmap v47: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-09-19T04:16:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:29 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:16:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:29 smithi104 ceph-mon[34145]: pgmap v47: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-09-19T04:16:30.507 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:16:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 15.0 s, 1667.4 tps, lat 59.854 ms stddev 14.931 2024-09-19T04:16:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:16:31.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.497+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/3299209178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d64102570 msgr2=0x7f5d641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.497+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/3299209178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d64102570 0x7f5d641029b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000dcc0 tx=0x7f5d5002e420 comp rx=0 tx=0).stop 2024-09-19T04:16:31.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.497+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/3299209178 shutdown_connections 2024-09-19T04:16:31.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.497+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/3299209178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d64102ef0 0x7f5d6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.497+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/3299209178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d64102570 0x7f5d641029b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.497+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/3299209178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d64100e20 0x7f5d64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.497+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/3299209178 >> 172.21.15.94:0/3299209178 conn(0x7f5d640fc7d0 msgr2=0x7f5d640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.497+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/3299209178 shutdown_connections 2024-09-19T04:16:31.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.497+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/3299209178 wait complete. 2024-09-19T04:16:31.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.498+0000 7f5d6aa7b640 1 Processor -- start 2024-09-19T04:16:31.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.498+0000 7f5d6aa7b640 1 -- start start 2024-09-19T04:16:31.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.498+0000 7f5d6aa7b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d64100e20 0x7f5d64073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.498+0000 7f5d6aa7b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d64102570 0x7f5d64073d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.498+0000 7f5d6aa7b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d64102ef0 0x7f5d6406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.498+0000 7f5d6aa7b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410fd30 con 0x7f5d64100e20 2024-09-19T04:16:31.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.498+0000 7f5d6aa7b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410fbb0 con 0x7f5d64102570 2024-09-19T04:16:31.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.498+0000 7f5d6aa7b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410feb0 con 0x7f5d64102ef0 2024-09-19T04:16:31.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.498+0000 7f5d68817640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d64100e20 0x7f5d64073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d68817640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d64100e20 0x7f5d64073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50944/0 (socket says 172.21.15.94:50944) 2024-09-19T04:16:31.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d68817640 1 -- 172.21.15.94:0/1904083892 learned_addr learned my addr 172.21.15.94:0/1904083892 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:31.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d63fff640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d64102570 0x7f5d64073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d63fff640 1 -- 172.21.15.94:0/1904083892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d64102ef0 msgr2=0x7f5d6406c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d69018640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d64102ef0 0x7f5d6406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d63fff640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d64102ef0 0x7f5d6406c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d63fff640 1 -- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d64100e20 msgr2=0x7f5d64073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d63fff640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d64100e20 0x7f5d64073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d63fff640 1 -- 172.21.15.94:0/1904083892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6406d1e0 con 0x7f5d64102570 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d69018640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d64102ef0 0x7f5d6406c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d68817640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d64100e20 0x7f5d64073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.499+0000 7f5d63fff640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d64102570 0x7f5d64073d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d5002e940 tx=0x7f5d50003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.500+0000 7f5d61ffb640 1 -- 172.21.15.94:0/1904083892 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50047070 con 0x7f5d64102570 2024-09-19T04:16:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.500+0000 7f5d61ffb640 1 -- 172.21.15.94:0/1904083892 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d50003e00 con 0x7f5d64102570 2024-09-19T04:16:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.500+0000 7f5d61ffb640 1 -- 172.21.15.94:0/1904083892 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50035a90 con 0x7f5d64102570 2024-09-19T04:16:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.500+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/1904083892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6406d470 con 0x7f5d64102570 2024-09-19T04:16:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.500+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/1904083892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d64071500 con 0x7f5d64102570 2024-09-19T04:16:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.504+0000 7f5d6ce54740 1 Processor -- start 2024-09-19T04:16:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.504+0000 7f5d6ce54740 1 -- start start 2024-09-19T04:16:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d61ffb640 1 -- 172.21.15.94:0/1904083892 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138588+0+0 (secure 0 0 0) 0x7f5d50035c30 con 0x7f5d64102570 2024-09-19T04:16:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638601294a0 0x563860129880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x563860134140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860134680 0x563860136a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d6ce54740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56386008f2e0 con 0x563860129dc0 2024-09-19T04:16:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d6ce54740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563860036ce0 con 0x563860134680 2024-09-19T04:16:31.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d6ce54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56386008b2a0 con 0x5638601294a0 2024-09-19T04:16:31.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d68817640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x563860134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d68817640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x563860134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50946/0 (socket says 172.21.15.94:50946) 2024-09-19T04:16:31.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d68817640 1 -- 172.21.15.94:0/4245608849 learned_addr learned my addr 172.21.15.94:0/4245608849 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:31.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d63fff640 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860134680 0x563860136a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.505+0000 7f5d69018640 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638601294a0 0x563860129880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.506+0000 7f5d68817640 1 -- 172.21.15.94:0/4245608849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638601294a0 msgr2=0x563860129880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.506+0000 7f5d68817640 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638601294a0 0x563860129880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.506+0000 7f5d68817640 1 -- 172.21.15.94:0/4245608849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860134680 msgr2=0x563860136a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.506+0000 7f5d68817640 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860134680 0x563860136a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.506+0000 7f5d68817640 1 -- 172.21.15.94:0/4245608849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386001b440 con 0x563860129dc0 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.506+0000 7f5d69018640 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638601294a0 0x563860129880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.507+0000 7f5d68817640 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x563860134140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800f8d0 tx=0x7f5d5800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6300e05d91126153 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.507+0000 7f5d61ffb640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5d540a35d0 0x7f5d540a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.507+0000 7f5d4effd640 1 -- 172.21.15.94:0/4245608849 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58011c20 con 0x563860129dc0 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.507+0000 7f5d61ffb640 1 -- 172.21.15.94:0/1904083892 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f5d500f86e0 con 0x7f5d64102570 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.507+0000 7f5d69018640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5d540a35d0 0x7f5d540a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.507+0000 7f5d4effd640 1 -- 172.21.15.94:0/4245608849 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d58011dc0 con 0x563860129dc0 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.508+0000 7f5d4effd640 1 -- 172.21.15.94:0/4245608849 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58018790 con 0x563860129dc0 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.508+0000 7f5d69018640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5d540a35d0 0x7f5d540a5a90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009df0 tx=0x7f5d5c005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.508+0000 7f5d6ce54740 1 -- 172.21.15.94:0/4245608849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 msgr2=0x563860134140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.508+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x563860134140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800f8d0 tx=0x7f5d5800fd90 comp rx=0 tx=0).stop 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.508+0000 7f5d6ce54740 1 -- 172.21.15.94:0/4245608849 shutdown_connections 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.508+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860134680 0x563860136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.508+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x563860134140 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.508+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/4245608849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638601294a0 0x563860129880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.508+0000 7f5d6ce54740 1 -- 172.21.15.94:0/4245608849 >> 172.21.15.94:0/4245608849 conn(0x563860125a30 msgr2=0x563860125e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.509+0000 7f5d6ce54740 1 -- 172.21.15.94:0/4245608849 shutdown_connections 2024-09-19T04:16:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.509+0000 7f5d6ce54740 1 -- 172.21.15.94:0/4245608849 wait complete. 2024-09-19T04:16:31.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.509+0000 7f5d6ce54740 1 -- start start 2024-09-19T04:16:31.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.509+0000 7f5d6ce54740 1 Processor -- start 2024-09-19T04:16:31.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638601294a0 0x563860133f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860129dc0 0x5638601311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860134680 0x563860131730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d6ce54740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563860137250 con 0x5638601294a0 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d6ce54740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56386004a050 con 0x563860129dc0 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d6ce54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5638601373d0 con 0x563860134680 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d6ce54740 10 client.0 _ll_register_callbacks cb 0x56386008a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d69018640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860134680 0x563860131730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d69018640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860134680 0x563860131730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:43630/0 (socket says 172.21.15.94:43630) 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d69018640 1 -- 172.21.15.94:0/419758552 learned_addr learned my addr 172.21.15.94:0/419758552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.510+0000 7f5d63fff640 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860129dc0 0x5638601311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d68817640 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638601294a0 0x563860133f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d69018640 1 -- 172.21.15.94:0/419758552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860129dc0 msgr2=0x5638601311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d69018640 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860129dc0 0x5638601311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d69018640 1 -- 172.21.15.94:0/419758552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638601294a0 msgr2=0x563860133f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d69018640 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638601294a0 0x563860133f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d69018640 1 -- 172.21.15.94:0/419758552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56385fe003c0 con 0x563860134680 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d68817640 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638601294a0 0x563860133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d69018640 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860134680 0x563860131730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c01e660 tx=0x7f5d5c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d68596845ec6ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d43fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d43fff640 1 -- 172.21.15.94:0/419758552 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c02b060 con 0x563860134680 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.511+0000 7f5d43fff640 1 -- 172.21.15.94:0/419758552 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f5d5c0117b0 con 0x563860134680 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.512+0000 7f5d6ce54740 1 -- 172.21.15.94:0/419758552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d58011dc0 con 0x563860134680 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.512+0000 7f5d43fff640 1 -- 172.21.15.94:0/419758552 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d5c0119b0 con 0x563860134680 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.512+0000 7f5d6ce54740 10 client.25045 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.512+0000 7f5d6ce54740 1 -- 172.21.15.94:0/419758552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563860142c50 con 0x563860134680 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.512+0000 7f5d43fff640 1 -- 172.21.15.94:0/419758552 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d58011dc0 con 0x563860134680 2024-09-19T04:16:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.513+0000 7f5d43fff640 1 -- 172.21.15.94:0/419758552 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f5d5c01a040 con 0x563860134680 2024-09-19T04:16:31.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.513+0000 7f5d6ce54740 10 client.25045 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:31.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.513+0000 7f5d6ce54740 10 client.25045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:16:31.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.513+0000 7f5d6ce54740 20 client.25045 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:31.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.513+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563860144060 0x563860146520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.513+0000 7f5d6ce54740 4 client.25045 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:31.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.513+0000 7f5d6ce54740 1 -- 172.21.15.94:0/419758552 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56386009c850 con 0x563860144060 2024-09-19T04:16:31.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.513+0000 7f5d68817640 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563860144060 0x563860146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.514+0000 7f5d68817640 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563860144060 0x563860146520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ed6a98ec2dd1bd server_cookie=7574183f5bf94194 in_seq=0 out_seq=0 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.514+0000 7f5d43fff640 10 client.25045 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d43fff640 1 -- 172.21.15.94:0/419758552 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56386009c850 con 0x563860144060 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d43fff640 10 client.25045 handle_command_reply: tid=0 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 client.25045 shutdown 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 -- 172.21.15.94:0/419758552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860134680 msgr2=0x563860131730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860134680 0x563860131730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c01e660 tx=0x7f5d5c0046c0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 -- 172.21.15.94:0/419758552 shutdown_connections 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563860144060 0x563860146520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d43fff640 0 client.25045 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:31.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860134680 0x563860131730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563860129dc0 0x5638601311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/419758552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638601294a0 0x563860133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 -- 172.21.15.94:0/419758552 >> 172.21.15.94:0/419758552 conn(0x563860125a30 msgr2=0x56386012b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 -- 172.21.15.94:0/419758552 shutdown_connections 2024-09-19T04:16:31.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.515+0000 7f5d6ce54740 1 -- 172.21.15.94:0/419758552 wait complete. 2024-09-19T04:16:31.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.516+0000 7f5d6ce54740 20 client.25045 trim_cache size 0 max 16384 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.544+0000 7f5d6ce54740 1 Processor -- start 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.544+0000 7f5d6ce54740 1 -- start start 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.544+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860125a30 0x563860133160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.544+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x56386014bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x56386014c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d68817640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860125a30 0x563860133160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d68817640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860125a30 0x563860133160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:43644/0 (socket says 172.21.15.94:43644) 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d68817640 1 -- 172.21.15.94:0/895221442 learned_addr learned my addr 172.21.15.94:0/895221442 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d6ce54740 1 -- 172.21.15.94:0/895221442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641b6620 con 0x563860129dc0 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d6ce54740 1 -- 172.21.15.94:0/895221442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641b3e60 con 0x5638601294a0 2024-09-19T04:16:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d6ce54740 1 -- 172.21.15.94:0/895221442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641b1cb0 con 0x563860125a30 2024-09-19T04:16:31.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d63fff640 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x56386014bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d69018640 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x56386014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d68817640 1 -- 172.21.15.94:0/895221442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 msgr2=0x56386014bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d68817640 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x56386014bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d68817640 1 -- 172.21.15.94:0/895221442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 msgr2=0x56386014c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d68817640 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x56386014c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.545+0000 7f5d68817640 1 -- 172.21.15.94:0/895221442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563860125860 con 0x563860125a30 2024-09-19T04:16:31.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.546+0000 7f5d63fff640 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x56386014bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:31.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.546+0000 7f5d68817640 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860125a30 0x563860133160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009aa0 tx=0x7f5d58019f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bf609117bc2f1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.546+0000 7f5d4e7fc640 1 -- 172.21.15.94:0/895221442 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58003b70 con 0x563860125a30 2024-09-19T04:16:31.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.547+0000 7f5d4e7fc640 1 -- 172.21.15.94:0/895221442 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d5802a070 con 0x563860125a30 2024-09-19T04:16:31.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.547+0000 7f5d4e7fc640 1 -- 172.21.15.94:0/895221442 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58019420 con 0x563860125a30 2024-09-19T04:16:31.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.548+0000 7f5d6ce54740 1 -- 172.21.15.94:0/895221442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860125a30 msgr2=0x563860133160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.548+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860125a30 0x563860133160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009aa0 tx=0x7f5d58019f90 comp rx=0 tx=0).stop 2024-09-19T04:16:31.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.548+0000 7f5d6ce54740 1 -- 172.21.15.94:0/895221442 shutdown_connections 2024-09-19T04:16:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.548+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860129dc0 0x56386014c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.548+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x56386014bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.548+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/895221442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860125a30 0x563860133160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.548+0000 7f5d6ce54740 1 -- 172.21.15.94:0/895221442 >> 172.21.15.94:0/895221442 conn(0x563860131890 msgr2=0x563860132a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.548+0000 7f5d6ce54740 1 -- 172.21.15.94:0/895221442 shutdown_connections 2024-09-19T04:16:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.549+0000 7f5d6ce54740 1 -- 172.21.15.94:0/895221442 wait complete. 2024-09-19T04:16:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.549+0000 7f5d6ce54740 1 -- start start 2024-09-19T04:16:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.549+0000 7f5d6ce54740 1 Processor -- start 2024-09-19T04:16:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860125a30 0x56386013cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x563860139f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d6ce54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860129dc0 0x56386013a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d6ce54740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641b2320 con 0x563860125a30 2024-09-19T04:16:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d6ce54740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641b3790 con 0x5638601294a0 2024-09-19T04:16:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d63fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x563860139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d63fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x563860139f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56914/0 (socket says 172.21.15.94:56914) 2024-09-19T04:16:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d63fff640 1 -- 172.21.15.94:0/1207442081 learned_addr learned my addr 172.21.15.94:0/1207442081 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d68817640 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860125a30 0x56386013cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d6ce54740 1 -- 172.21.15.94:0/1207442081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641b29f0 con 0x563860129dc0 2024-09-19T04:16:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d6ce54740 10 client.0 _ll_register_callbacks cb 0x7f5d5c013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d69018640 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860129dc0 0x56386013a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d63fff640 1 -- 172.21.15.94:0/1207442081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860129dc0 msgr2=0x56386013a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d63fff640 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860129dc0 0x56386013a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d63fff640 1 -- 172.21.15.94:0/1207442081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860125a30 msgr2=0x56386013cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d63fff640 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860125a30 0x56386013cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.550+0000 7f5d63fff640 1 -- 172.21.15.94:0/1207442081 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386013aa30 con 0x5638601294a0 2024-09-19T04:16:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.551+0000 7f5d63fff640 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x563860139f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d50010090 tx=0x7f5d50005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454375168a6ba9a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.551+0000 7f5d43fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.551+0000 7f5d69018640 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860129dc0 0x56386013a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.551+0000 7f5d43fff640 1 -- 172.21.15.94:0/1207442081 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50050040 con 0x5638601294a0 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.551+0000 7f5d43fff640 1 -- 172.21.15.94:0/1207442081 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f5d500f6120 con 0x5638601294a0 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.551+0000 7f5d6ce54740 1 -- 172.21.15.94:0/1207442081 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5802a070 con 0x5638601294a0 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d43fff640 1 -- 172.21.15.94:0/1207442081 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d500c4a70 con 0x5638601294a0 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d6ce54740 10 client.25091 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d6ce54740 1 -- 172.21.15.94:0/1207442081 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638601aec20 con 0x5638601294a0 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d43fff640 1 -- 172.21.15.94:0/1207442081 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d500c53a0 con 0x5638601294a0 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d43fff640 1 -- 172.21.15.94:0/1207442081 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f5d500c09f0 con 0x5638601294a0 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d6ce54740 10 client.25091 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d6ce54740 10 client.25091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:16:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d6ce54740 20 client.25091 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563860134680 0x5638601b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d6ce54740 4 client.25091 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:16:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.552+0000 7f5d6ce54740 1 -- 172.21.15.94:0/1207442081 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5d5802a070 con 0x563860134680 2024-09-19T04:16:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.553+0000 7f5d68817640 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563860134680 0x5638601b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.553+0000 7f5d43fff640 10 client.25091 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.553+0000 7f5d68817640 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563860134680 0x5638601b13d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07bbb5894a5d76c server_cookie=7fa284b60ee58b6c in_seq=0 out_seq=0 2024-09-19T04:16:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d43fff640 1 -- 172.21.15.94:0/1207442081 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f5d5802a070 con 0x563860134680 2024-09-19T04:16:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d43fff640 10 client.25091 handle_command_reply: tid=0 2024-09-19T04:16:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d6ce54740 1 client.25091 shutdown 2024-09-19T04:16:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d6ce54740 1 -- 172.21.15.94:0/1207442081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 msgr2=0x563860139f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x563860139f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d50010090 tx=0x7f5d50005e10 comp rx=0 tx=0).stop 2024-09-19T04:16:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d6ce54740 1 -- 172.21.15.94:0/1207442081 shutdown_connections 2024-09-19T04:16:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563860134680 0x5638601b13d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860129dc0 0x56386013a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d43fff640 0 client.25091 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638601294a0 0x563860139f00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.554+0000 7f5d6ce54740 1 --2- 172.21.15.94:0/1207442081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563860125a30 0x56386013cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.555+0000 7f5d6ce54740 1 -- 172.21.15.94:0/1207442081 >> 172.21.15.94:0/1207442081 conn(0x563860131890 msgr2=0x56386012f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.555+0000 7f5d6ce54740 1 -- 172.21.15.94:0/1207442081 shutdown_connections 2024-09-19T04:16:31.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.555+0000 7f5d6ce54740 1 -- 172.21.15.94:0/1207442081 wait complete. 2024-09-19T04:16:31.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.555+0000 7f5d6ce54740 20 client.25091 trim_cache size 0 max 16384 2024-09-19T04:16:31.559 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:16:31.559 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:16:31.559 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:16:31.559 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:16:31.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.556+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5d540a35d0 msgr2=0x7f5d540a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.556+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5d540a35d0 0x7f5d540a5a90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009df0 tx=0x7f5d5c005f90 comp rx=0 tx=0).stop 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.556+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/1904083892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d64102570 msgr2=0x7f5d64073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.556+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d64102570 0x7f5d64073d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d5002e940 tx=0x7f5d50003c10 comp rx=0 tx=0).stop 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.556+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/1904083892 shutdown_connections 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.556+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5d540a35d0 0x7f5d540a5a90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.557+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d64102ef0 0x7f5d6406c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.557+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d64102570 0x7f5d64073d40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.557+0000 7f5d6aa7b640 1 --2- 172.21.15.94:0/1904083892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d64100e20 0x7f5d64073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.557+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/1904083892 >> 172.21.15.94:0/1904083892 conn(0x7f5d640fc7d0 msgr2=0x7f5d64103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.557+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/1904083892 shutdown_connections 2024-09-19T04:16:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:31.557+0000 7f5d6aa7b640 1 -- 172.21.15.94:0/1904083892 wait complete. 2024-09-19T04:16:31.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:31 smithi094 ceph-mon[30972]: pgmap v48: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-09-19T04:16:31.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:31 smithi112 ceph-mon[35051]: pgmap v48: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-09-19T04:16:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:31 smithi104 ceph-mon[34145]: pgmap v48: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-09-19T04:16:32.631 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:16:32.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:16:33.064 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:16:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.647+0000 7f997ef05640 1 -- 172.21.15.94:0/1009529091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978100e20 msgr2=0x7f9978101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.648+0000 7f997ef05640 1 --2- 172.21.15.94:0/1009529091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978100e20 0x7f9978101200 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f996c009900 tx=0x7f996c02a480 comp rx=0 tx=0).stop 2024-09-19T04:16:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.648+0000 7f997ef05640 1 -- 172.21.15.94:0/1009529091 shutdown_connections 2024-09-19T04:16:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.648+0000 7f997ef05640 1 --2- 172.21.15.94:0/1009529091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9978102ef0 0x7f997810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.648+0000 7f997ef05640 1 --2- 172.21.15.94:0/1009529091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9978102570 0x7f99781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.648+0000 7f997ef05640 1 --2- 172.21.15.94:0/1009529091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978100e20 0x7f9978101200 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.648+0000 7f997ef05640 1 -- 172.21.15.94:0/1009529091 >> 172.21.15.94:0/1009529091 conn(0x7f99780fc7d0 msgr2=0x7f99780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.648+0000 7f997ef05640 1 -- 172.21.15.94:0/1009529091 shutdown_connections 2024-09-19T04:16:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.649+0000 7f997ef05640 1 -- 172.21.15.94:0/1009529091 wait complete. 2024-09-19T04:16:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.650+0000 7f997ef05640 1 Processor -- start 2024-09-19T04:16:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.650+0000 7f997ef05640 1 -- start start 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.650+0000 7f997ef05640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9978100e20 0x7f99781a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.650+0000 7f997ef05640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978102570 0x7f99781a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.650+0000 7f997ef05640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9978102ef0 0x7f997819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.650+0000 7f997ef05640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f997810e8b0 con 0x7f9978102570 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.650+0000 7f997ef05640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f997810e730 con 0x7f9978100e20 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.650+0000 7f997ef05640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f997810ea30 con 0x7f9978102ef0 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997cca1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9978100e20 0x7f99781a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997d4a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9978102ef0 0x7f997819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997cca1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9978100e20 0x7f99781a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51380/0 (socket says 172.21.15.94:51380) 2024-09-19T04:16:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f9977fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978102570 0x7f99781a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997cca1640 1 -- 172.21.15.94:0/1152516015 learned_addr learned my addr 172.21.15.94:0/1152516015 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997cca1640 1 -- 172.21.15.94:0/1152516015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9978102ef0 msgr2=0x7f997819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997cca1640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9978102ef0 0x7f997819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997cca1640 1 -- 172.21.15.94:0/1152516015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978102570 msgr2=0x7f99781a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997cca1640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978102570 0x7f99781a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:33.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997cca1640 1 -- 172.21.15.94:0/1152516015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997819f630 con 0x7f9978100e20 2024-09-19T04:16:33.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997d4a2640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9978102ef0 0x7f997819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:33.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f997cca1640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9978100e20 0x7f99781a0170 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f996c02aa70 tx=0x7f996c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:33.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.651+0000 7f9975ffb640 1 -- 172.21.15.94:0/1152516015 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c03f070 con 0x7f9978100e20 2024-09-19T04:16:33.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.652+0000 7f9975ffb640 1 -- 172.21.15.94:0/1152516015 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f996c03bb20 con 0x7f9978100e20 2024-09-19T04:16:33.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.652+0000 7f9975ffb640 1 -- 172.21.15.94:0/1152516015 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c035030 con 0x7f9978100e20 2024-09-19T04:16:33.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.652+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997819f8c0 con 0x7f9978100e20 2024-09-19T04:16:33.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.652+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9978111ad0 con 0x7f9978100e20 2024-09-19T04:16:33.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.653+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99781021b0 con 0x7f9978100e20 2024-09-19T04:16:33.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.655+0000 7f9975ffb640 1 -- 172.21.15.94:0/1152516015 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f996c042070 con 0x7f9978100e20 2024-09-19T04:16:33.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.657+0000 7f9975ffb640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f99680a3430 0x7f99680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:33.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.657+0000 7f9977fff640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f99680a3430 0x7f99680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:33.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.657+0000 7f9975ffb640 1 -- 172.21.15.94:0/1152516015 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f996c0f0080 con 0x7f9978100e20 2024-09-19T04:16:33.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.658+0000 7f9977fff640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f99680a3430 0x7f99680a58f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99640095f0 tx=0x7f9964016040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:33.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:33.660+0000 7f9975ffb640 1 -- 172.21.15.94:0/1152516015 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f996c0bca50 con 0x7f9978100e20 2024-09-19T04:16:33.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:33 smithi112 ceph-mon[35051]: pgmap v49: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 19 MiB/s wr, 2.08k op/s 2024-09-19T04:16:33.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:33 smithi112 ceph-mon[35051]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-19T04:16:33.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:33 smithi094 ceph-mon[30972]: pgmap v49: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 19 MiB/s wr, 2.08k op/s 2024-09-19T04:16:33.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:33 smithi094 ceph-mon[30972]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-19T04:16:33.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:33 smithi104 ceph-mon[34145]: pgmap v49: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 19 MiB/s wr, 2.08k op/s 2024-09-19T04:16:33.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:33 smithi104 ceph-mon[34145]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-19T04:16:34.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.018+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f997810dcd0 con 0x7f9978100e20 2024-09-19T04:16:34.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.018+0000 7f9975ffb640 1 -- 172.21.15.94:0/1152516015 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11922 (secure 0 0 0) 0x7f996c0bc350 con 0x7f9978100e20 2024-09-19T04:16:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:16:34.020 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":30,"btime":"2024-09-19T04:16:28:467883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:16:27.778564+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:16:34.021 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 30 2024-09-19T04:16:34.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.022+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f99680a3430 msgr2=0x7f99680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:34.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.022+0000 7f997ef05640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f99680a3430 0x7f99680a58f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99640095f0 tx=0x7f9964016040 comp rx=0 tx=0).stop 2024-09-19T04:16:34.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.022+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9978100e20 msgr2=0x7f99781a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:34.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.022+0000 7f997ef05640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9978100e20 0x7f99781a0170 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f996c02aa70 tx=0x7f996c003c10 comp rx=0 tx=0).stop 2024-09-19T04:16:34.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.023+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 shutdown_connections 2024-09-19T04:16:34.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.023+0000 7f997ef05640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f99680a3430 0x7f99680a58f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:34.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.023+0000 7f997ef05640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9978102ef0 0x7f997819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:34.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.023+0000 7f997ef05640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978102570 0x7f99781a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:34.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.023+0000 7f997ef05640 1 --2- 172.21.15.94:0/1152516015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9978100e20 0x7f99781a0170 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:34.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.023+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 >> 172.21.15.94:0/1152516015 conn(0x7f99780fc7d0 msgr2=0x7f9978106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:34.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.023+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 shutdown_connections 2024-09-19T04:16:34.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:34.023+0000 7f997ef05640 1 -- 172.21.15.94:0/1152516015 wait complete. 2024-09-19T04:16:34.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:34 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1152516015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:16:34.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:34 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1152516015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:16:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:34 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1152516015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:16:34.862 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:16:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:16:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 20.0 s, 1185.8 tps, lat 81.374 ms stddev 28.381 2024-09-19T04:16:35.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:35 smithi094 ceph-mon[30972]: pgmap v50: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 1.67k op/s 2024-09-19T04:16:35.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:35 smithi112 ceph-mon[35051]: pgmap v50: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 1.67k op/s 2024-09-19T04:16:35.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.808+0000 7fb584679640 1 -- 172.21.15.94:0/4141902080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c10e4f0 msgr2=0x7fb57c1108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:35.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.808+0000 7fb584679640 1 --2- 172.21.15.94:0/4141902080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c10e4f0 0x7fb57c1108e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb574009ca0 tx=0x7fb57402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.808+0000 7fb584679640 1 -- 172.21.15.94:0/4141902080 shutdown_connections 2024-09-19T04:16:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.808+0000 7fb584679640 1 --2- 172.21.15.94:0/4141902080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c10e4f0 0x7fb57c1108e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.808+0000 7fb584679640 1 --2- 172.21.15.94:0/4141902080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb57c1012f0 0x7fb57c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.808+0000 7fb584679640 1 --2- 172.21.15.94:0/4141902080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c100940 0x7fb57c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.808+0000 7fb584679640 1 -- 172.21.15.94:0/4141902080 >> 172.21.15.94:0/4141902080 conn(0x7fb57c0fc7d0 msgr2=0x7fb57c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.808+0000 7fb584679640 1 -- 172.21.15.94:0/4141902080 shutdown_connections 2024-09-19T04:16:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.808+0000 7fb584679640 1 -- 172.21.15.94:0/4141902080 wait complete. 2024-09-19T04:16:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.809+0000 7fb584679640 1 Processor -- start 2024-09-19T04:16:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.809+0000 7fb584679640 1 -- start start 2024-09-19T04:16:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.809+0000 7fb584679640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c100940 0x7fb57c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.809+0000 7fb584679640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb57c1012f0 0x7fb57c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.809+0000 7fb584679640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c10e4f0 0x7fb57c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.809+0000 7fb584679640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1140f0 con 0x7fb57c100940 2024-09-19T04:16:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.809+0000 7fb584679640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c113f70 con 0x7fb57c10e4f0 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.809+0000 7fb584679640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c114270 con 0x7fb57c1012f0 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582415640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c100940 0x7fb57c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582415640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c100940 0x7fb57c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49934/0 (socket says 172.21.15.94:49934) 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582415640 1 -- 172.21.15.94:0/859231119 learned_addr learned my addr 172.21.15.94:0/859231119 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582c16640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c10e4f0 0x7fb57c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb581c14640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb57c1012f0 0x7fb57c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582415640 1 -- 172.21.15.94:0/859231119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb57c1012f0 msgr2=0x7fb57c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582415640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb57c1012f0 0x7fb57c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582415640 1 -- 172.21.15.94:0/859231119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c10e4f0 msgr2=0x7fb57c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:35.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582415640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c10e4f0 0x7fb57c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582415640 1 -- 172.21.15.94:0/859231119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c19f5b0 con 0x7fb57c100940 2024-09-19T04:16:35.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb582415640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c100940 0x7fb57c19a600 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb57800b4a0 tx=0x7fb57800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:35.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb5737fe640 1 -- 172.21.15.94:0/859231119 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578015020 con 0x7fb57c100940 2024-09-19T04:16:35.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb5737fe640 1 -- 172.21.15.94:0/859231119 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb578011ce0 con 0x7fb57c100940 2024-09-19T04:16:35.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb584679640 1 -- 172.21.15.94:0/859231119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c19f8a0 con 0x7fb57c100940 2024-09-19T04:16:35.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.810+0000 7fb5737fe640 1 -- 172.21.15.94:0/859231119 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578010dc0 con 0x7fb57c100940 2024-09-19T04:16:35.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.811+0000 7fb584679640 1 -- 172.21.15.94:0/859231119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c1a7180 con 0x7fb57c100940 2024-09-19T04:16:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.813+0000 7fb586a52740 1 Processor -- start 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.813+0000 7fb5737fe640 1 -- 172.21.15.94:0/859231119 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fb578018020 con 0x7fb57c100940 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb586a52740 1 -- start start 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb586a52740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca340090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb586a52740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca340a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb586a52740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 0x5572ca34d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb586a52740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5572ca2a62f0 con 0x5572ca34b370 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb586a52740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5572ca24dcf0 con 0x5572ca3405d0 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb586a52740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572ca2a22b0 con 0x5572ca341b40 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb582c16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca340090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb582c16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca340090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:38918/0 (socket says 172.21.15.94:38918) 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb582c16640 1 -- 172.21.15.94:0/2668413397 learned_addr learned my addr 172.21.15.94:0/2668413397 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb5737fe640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5680a3430 0x7fb5680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb582415640 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca340a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.814+0000 7fb581c14640 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 0x5572ca34d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb582c16640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5680a3430 0x7fb5680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb5737fe640 1 -- 172.21.15.94:0/859231119 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb5780cf4f0 con 0x7fb57c100940 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb582415640 1 -- 172.21.15.94:0/2668413397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 msgr2=0x5572ca340090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb582415640 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca340090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb582415640 1 -- 172.21.15.94:0/2668413397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 msgr2=0x5572ca34d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb582415640 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 0x5572ca34d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb582415640 1 -- 172.21.15.94:0/2668413397 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ca2323a0 con 0x5572ca3405d0 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb582c16640 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca340090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb582415640 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca340a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5780ccce0 tx=0x7fb578011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d8b723605f3592 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb570ff9640 1 -- 172.21.15.94:0/2668413397 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57809bf70 con 0x5572ca3405d0 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.815+0000 7fb570ff9640 1 -- 172.21.15.94:0/2668413397 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb57809b870 con 0x5572ca3405d0 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb570ff9640 1 -- 172.21.15.94:0/2668413397 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578097ee0 con 0x5572ca3405d0 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb586a52740 1 -- 172.21.15.94:0/2668413397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 msgr2=0x5572ca340a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb582c16640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5680a3430 0x7fb5680a58f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb574005fe0 tx=0x7fb57402a8c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb586a52740 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca340a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5780ccce0 tx=0x7fb578011000 comp rx=0 tx=0).stop 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb586a52740 1 -- 172.21.15.94:0/2668413397 shutdown_connections 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb586a52740 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 0x5572ca34d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb586a52740 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca340a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb586a52740 1 --2- 172.21.15.94:0/2668413397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca340090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb586a52740 1 -- 172.21.15.94:0/2668413397 >> 172.21.15.94:0/2668413397 conn(0x5572ca309310 msgr2=0x5572ca33cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb586a52740 1 -- 172.21.15.94:0/2668413397 shutdown_connections 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.816+0000 7fb586a52740 1 -- 172.21.15.94:0/2668413397 wait complete. 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb586a52740 1 -- start start 2024-09-19T04:16:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb586a52740 1 Processor -- start 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb586a52740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca34ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb586a52740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca348050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb586a52740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca34b370 0x5572ca3487e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb586a52740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5572ca34e0b0 con 0x5572ca341b40 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb586a52740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5572ca261060 con 0x5572ca3405d0 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb586a52740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572ca34e230 con 0x5572ca34b370 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb581c14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca348050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb586a52740 10 client.0 _ll_register_callbacks cb 0x5572c9fb48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb581c14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca348050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49958/0 (socket says 172.21.15.94:49958) 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb581c14640 1 -- 172.21.15.94:0/2744633504 learned_addr learned my addr 172.21.15.94:0/2744633504 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb582415640 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca34ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb581c14640 1 -- 172.21.15.94:0/2744633504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca34b370 msgr2=0x5572ca3487e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb581c14640 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca34b370 0x5572ca3487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb581c14640 1 -- 172.21.15.94:0/2744633504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 msgr2=0x5572ca34ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.817+0000 7fb581c14640 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca34ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.818+0000 7fb581c14640 1 -- 172.21.15.94:0/2744633504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ca0173c0 con 0x5572ca341b40 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.818+0000 7fb581c14640 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca348050 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009da0 tx=0x7fb56c00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c33b41a5c1c6036 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.818+0000 7fb565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.818+0000 7fb565ffb640 1 -- 172.21.15.94:0/2744633504 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c024060 con 0x5572ca341b40 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.818+0000 7fb586a52740 1 -- 172.21.15.94:0/2744633504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb57809b870 con 0x5572ca341b40 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.818+0000 7fb565ffb640 1 -- 172.21.15.94:0/2744633504 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb56c00bda0 con 0x5572ca341b40 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.818+0000 7fb565ffb640 1 -- 172.21.15.94:0/2744633504 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb56c028420 con 0x5572ca341b40 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.818+0000 7fb586a52740 10 client.15510 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.818+0000 7fb586a52740 1 -- 172.21.15.94:0/2744633504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572ca359bf0 con 0x5572ca341b40 2024-09-19T04:16:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.819+0000 7fb565ffb640 1 -- 172.21.15.94:0/2744633504 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb57809b870 con 0x5572ca341b40 2024-09-19T04:16:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.819+0000 7fb565ffb640 1 -- 172.21.15.94:0/2744633504 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb56c01e540 con 0x5572ca341b40 2024-09-19T04:16:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.819+0000 7fb586a52740 10 client.15510 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.819+0000 7fb586a52740 10 client.15510 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.819+0000 7fb586a52740 20 client.15510 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.819+0000 7fb586a52740 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5572ca35b500 0x5572ca35d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.819+0000 7fb586a52740 4 client.15510 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.819+0000 7fb586a52740 1 -- 172.21.15.94:0/2744633504 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572ca2b3860 con 0x5572ca35b500 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.819+0000 7fb582415640 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5572ca35b500 0x5572ca35d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb565ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb582415640 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5572ca35b500 0x5572ca35d9c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c9a00a032d6bf4c server_cookie=42e94d6cba103218 in_seq=0 out_seq=0 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb565ffb640 1 -- 172.21.15.94:0/2744633504 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572ca2b3860 con 0x5572ca35b500 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb565ffb640 10 client.15510 handle_command_reply: tid=0 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 client.15510 shutdown 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 -- 172.21.15.94:0/2744633504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 msgr2=0x5572ca348050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca348050 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009da0 tx=0x7fb56c00b510 comp rx=0 tx=0).stop 2024-09-19T04:16:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 -- 172.21.15.94:0/2744633504 shutdown_connections 2024-09-19T04:16:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5572ca35b500 0x5572ca35d9c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca34b370 0x5572ca3487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca348050 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 --2- 172.21.15.94:0/2744633504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca34ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 -- 172.21.15.94:0/2744633504 >> 172.21.15.94:0/2744633504 conn(0x5572ca309310 msgr2=0x5572ca3429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb565ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:16:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.820+0000 7fb586a52740 1 -- 172.21.15.94:0/2744633504 shutdown_connections 2024-09-19T04:16:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.821+0000 7fb586a52740 1 -- 172.21.15.94:0/2744633504 wait complete. 2024-09-19T04:16:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.821+0000 7fb586a52740 20 client.15510 trim_cache size 0 max 16384 2024-09-19T04:16:35.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb586a52740 1 Processor -- start 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb586a52740 1 -- start start 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb586a52740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca33ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb586a52740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca347910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb586a52740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca34b370 0x5572ca3632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb586a52740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1ae180 con 0x5572ca341b40 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb586a52740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1ab9c0 con 0x5572ca3405d0 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb586a52740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1a9830 con 0x5572ca34b370 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb581c14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca347910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb581c14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca347910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49968/0 (socket says 172.21.15.94:49968) 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb581c14640 1 -- 172.21.15.94:0/3834427560 learned_addr learned my addr 172.21.15.94:0/3834427560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb582c16640 1 --2- 172.21.15.94:0/3834427560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca34b370 0x5572ca3632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.835+0000 7fb582415640 1 --2- 172.21.15.94:0/3834427560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca33ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.836+0000 7fb581c14640 1 -- 172.21.15.94:0/3834427560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca34b370 msgr2=0x5572ca3632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.836+0000 7fb581c14640 1 --2- 172.21.15.94:0/3834427560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca34b370 0x5572ca3632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.836+0000 7fb581c14640 1 -- 172.21.15.94:0/3834427560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 msgr2=0x5572ca33ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.836+0000 7fb581c14640 1 --2- 172.21.15.94:0/3834427560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca33ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.836+0000 7fb581c14640 1 -- 172.21.15.94:0/3834427560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ca347000 con 0x5572ca341b40 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.836+0000 7fb581c14640 1 --2- 172.21.15.94:0/3834427560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca347910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0097c0 tx=0x7fb56c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d797a92ded6cdeee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.836+0000 7fb567fff640 1 -- 172.21.15.94:0/3834427560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c036070 con 0x5572ca341b40 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.836+0000 7fb567fff640 1 -- 172.21.15.94:0/3834427560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb56c01ed00 con 0x5572ca341b40 2024-09-19T04:16:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.836+0000 7fb567fff640 1 -- 172.21.15.94:0/3834427560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c028a60 con 0x5572ca341b40 2024-09-19T04:16:35.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 -- 172.21.15.94:0/3834427560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 msgr2=0x5572ca347910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:35.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 --2- 172.21.15.94:0/3834427560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca347910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0097c0 tx=0x7fb56c016040 comp rx=0 tx=0).stop 2024-09-19T04:16:35.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 -- 172.21.15.94:0/3834427560 shutdown_connections 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 --2- 172.21.15.94:0/3834427560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca34b370 0x5572ca3632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 --2- 172.21.15.94:0/3834427560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca341b40 0x5572ca347910 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 --2- 172.21.15.94:0/3834427560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca33ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 -- 172.21.15.94:0/3834427560 >> 172.21.15.94:0/3834427560 conn(0x5572ca309310 msgr2=0x5572ca33c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 -- 172.21.15.94:0/3834427560 shutdown_connections 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 -- 172.21.15.94:0/3834427560 wait complete. 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.837+0000 7fb586a52740 1 -- start start 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb586a52740 1 Processor -- start 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb586a52740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca34f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb586a52740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca34f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb586a52740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 0x5572ca34fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb586a52740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1a9e80 con 0x5572ca34b370 2024-09-19T04:16:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb586a52740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1ab2f0 con 0x5572ca3405d0 2024-09-19T04:16:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb586a52740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1aa550 con 0x5572ca341b40 2024-09-19T04:16:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb586a52740 10 client.0 _ll_register_callbacks cb 0x7fb56c014b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb581c14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca34f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb582c16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 0x5572ca34fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb582c16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 0x5572ca34fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49972/0 (socket says 172.21.15.94:49972) 2024-09-19T04:16:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb581c14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca34f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:38948/0 (socket says 172.21.15.94:38948) 2024-09-19T04:16:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb582c16640 1 -- 172.21.15.94:0/2998111503 learned_addr learned my addr 172.21.15.94:0/2998111503 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb582415640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca34f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb582415640 1 -- 172.21.15.94:0/2998111503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 msgr2=0x5572ca34f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb582415640 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca34f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb582415640 1 -- 172.21.15.94:0/2998111503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 msgr2=0x5572ca34fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb582415640 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 0x5572ca34fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.838+0000 7fb582415640 1 -- 172.21.15.94:0/2998111503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ca348b50 con 0x5572ca3405d0 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb582415640 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca34f150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb5780ccd10 tx=0x7fb578097b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41d0a0ea48a78bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb565ffb640 1 -- 172.21.15.94:0/2998111503 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57809b870 con 0x5572ca3405d0 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb586a52740 1 -- 172.21.15.94:0/2998111503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb56c01ed00 con 0x5572ca3405d0 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb565ffb640 1 -- 172.21.15.94:0/2998111503 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb578097ee0 con 0x5572ca3405d0 2024-09-19T04:16:35.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb565ffb640 1 -- 172.21.15.94:0/2998111503 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb57809bf70 con 0x5572ca3405d0 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb586a52740 10 client.25112 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb586a52740 1 -- 172.21.15.94:0/2998111503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572ca3c62f0 con 0x5572ca3405d0 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb565ffb640 1 -- 172.21.15.94:0/2998111503 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb56c01ed00 con 0x5572ca3405d0 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb565ffb640 1 -- 172.21.15.94:0/2998111503 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb57809cba0 con 0x5572ca3405d0 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb586a52740 10 client.25112 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb586a52740 10 client.25112 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb586a52740 20 client.25112 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb586a52740 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5572ca3c6520 0x5572ca3c8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb586a52740 4 client.25112 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:16:35.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.839+0000 7fb586a52740 1 -- 172.21.15.94:0/2998111503 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572ca2add70 con 0x5572ca3c6520 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb581c14640 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5572ca3c6520 0x5572ca3c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb565ffb640 10 client.25112 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb581c14640 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5572ca3c6520 0x5572ca3c8910 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef85675e807b11e server_cookie=34555ea3d01c0063 in_seq=0 out_seq=0 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb565ffb640 1 -- 172.21.15.94:0/2998111503 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572ca2add70 con 0x5572ca3c6520 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb565ffb640 10 client.25112 handle_command_reply: tid=0 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 client.25112 shutdown 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 -- 172.21.15.94:0/2998111503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 msgr2=0x5572ca34f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca34f150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb5780ccd10 tx=0x7fb578097b10 comp rx=0 tx=0).stop 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 -- 172.21.15.94:0/2998111503 shutdown_connections 2024-09-19T04:16:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5572ca3c6520 0x5572ca3c8910 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb565ffb640 0 client.25112 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:16:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572ca34b370 0x5572ca34fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ca341b40 0x5572ca34f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 --2- 172.21.15.94:0/2998111503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572ca3405d0 0x5572ca34f150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 -- 172.21.15.94:0/2998111503 >> 172.21.15.94:0/2998111503 conn(0x5572ca309310 msgr2=0x5572ca3498b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.840+0000 7fb586a52740 1 -- 172.21.15.94:0/2998111503 shutdown_connections 2024-09-19T04:16:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb586a52740 1 -- 172.21.15.94:0/2998111503 wait complete. 2024-09-19T04:16:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb586a52740 20 client.25112 trim_cache size 0 max 16384 2024-09-19T04:16:35.850 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:16:35.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 -- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5680a3430 msgr2=0x7fb5680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:35.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5680a3430 0x7fb5680a58f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb574005fe0 tx=0x7fb57402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 -- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c100940 msgr2=0x7fb57c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:35.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c100940 0x7fb57c19a600 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb57800b4a0 tx=0x7fb57800b8a0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 -- 172.21.15.94:0/859231119 shutdown_connections 2024-09-19T04:16:35.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5680a3430 0x7fb5680a58f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c10e4f0 0x7fb57c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb57c1012f0 0x7fb57c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 --2- 172.21.15.94:0/859231119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb57c100940 0x7fb57c19a600 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:35.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.841+0000 7fb584679640 1 -- 172.21.15.94:0/859231119 >> 172.21.15.94:0/859231119 conn(0x7fb57c0fc7d0 msgr2=0x7fb57c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:35.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.842+0000 7fb584679640 1 -- 172.21.15.94:0/859231119 shutdown_connections 2024-09-19T04:16:35.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:35.842+0000 7fb584679640 1 -- 172.21.15.94:0/859231119 wait complete. 2024-09-19T04:16:35.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:35 smithi104 ceph-mon[34145]: pgmap v50: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 1.67k op/s 2024-09-19T04:16:36.681 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:16:37.084 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:16:37.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.643+0000 7fb2b7a36640 1 -- 172.21.15.94:0/129986025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 msgr2=0x7fb2b01000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:37.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.643+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/129986025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 0x7fb2b01000b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009900 tx=0x7fb2ac02a480 comp rx=0 tx=0).stop 2024-09-19T04:16:37.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.643+0000 7fb2b7a36640 1 -- 172.21.15.94:0/129986025 shutdown_connections 2024-09-19T04:16:37.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.643+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/129986025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2b0100c10 0x7fb2b0110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.643+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/129986025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b0102ea0 0x7fb2b01005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.643+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/129986025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 0x7fb2b01000b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.643+0000 7fb2b7a36640 1 -- 172.21.15.94:0/129986025 >> 172.21.15.94:0/129986025 conn(0x7fb2b0077220 msgr2=0x7fb2b00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:37.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.643+0000 7fb2b7a36640 1 -- 172.21.15.94:0/129986025 shutdown_connections 2024-09-19T04:16:37.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.643+0000 7fb2b7a36640 1 -- 172.21.15.94:0/129986025 wait complete. 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.644+0000 7fb2b7a36640 1 Processor -- start 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.644+0000 7fb2b7a36640 1 -- start start 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.644+0000 7fb2b7a36640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2b0100c10 0x7fb2b010f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.644+0000 7fb2b7a36640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 0x7fb2b010f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b57d2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2b0100c10 0x7fb2b010f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b57d2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2b0100c10 0x7fb2b010f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51432/0 (socket says 172.21.15.94:51432) 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b4fd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 0x7fb2b010f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b7a36640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b0102ea0 0x7fb2b01a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b7a36640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b01045e0 con 0x7fb2b0102ac0 2024-09-19T04:16:37.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b7a36640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0104460 con 0x7fb2b0100c10 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b4fd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 0x7fb2b010f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49992/0 (socket says 172.21.15.94:49992) 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b7a36640 1 -- 172.21.15.94:0/1074877307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0104760 con 0x7fb2b0102ea0 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b57d2640 1 -- 172.21.15.94:0/1074877307 learned_addr learned my addr 172.21.15.94:0/1074877307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b0102ea0 0x7fb2b01a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b57d2640 1 -- 172.21.15.94:0/1074877307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b0102ea0 msgr2=0x7fb2b01a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b57d2640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b0102ea0 0x7fb2b01a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b57d2640 1 -- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 msgr2=0x7fb2b010f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b57d2640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 0x7fb2b010f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b57d2640 1 -- 172.21.15.94:0/1074877307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b01a4af0 con 0x7fb2b0100c10 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b0102ea0 0x7fb2b01a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.645+0000 7fb2b4fd1640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 0x7fb2b010f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.646+0000 7fb2b57d2640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2b0100c10 0x7fb2b010f380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac005ab0 tx=0x7fb2ac003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.646+0000 7fb2a67fc640 1 -- 172.21.15.94:0/1074877307 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac03f070 con 0x7fb2b0100c10 2024-09-19T04:16:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.646+0000 7fb2a67fc640 1 -- 172.21.15.94:0/1074877307 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2ac031b60 con 0x7fb2b0100c10 2024-09-19T04:16:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.646+0000 7fb2a67fc640 1 -- 172.21.15.94:0/1074877307 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac035030 con 0x7fb2b0100c10 2024-09-19T04:16:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.646+0000 7fb2b7a36640 1 -- 172.21.15.94:0/1074877307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b01a4d80 con 0x7fb2b0100c10 2024-09-19T04:16:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.646+0000 7fb2b7a36640 1 -- 172.21.15.94:0/1074877307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b01a5210 con 0x7fb2b0100c10 2024-09-19T04:16:37.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.649+0000 7fb2a67fc640 1 -- 172.21.15.94:0/1074877307 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fb2ac042070 con 0x7fb2b0100c10 2024-09-19T04:16:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.651+0000 7fb2b9e0f740 1 Processor -- start 2024-09-19T04:16:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.651+0000 7fb2b9e0f740 1 -- start start 2024-09-19T04:16:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.651+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e96eb40 0x56289e96d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.651+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e96da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.651+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e978370 0x56289e97a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.651+0000 7fb2b5fd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e96eb40 0x56289e96d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.652+0000 7fb2b5fd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e96eb40 0x56289e96d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51448/0 (socket says 172.21.15.94:51448) 2024-09-19T04:16:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.652+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56289e8d32f0 con 0x56289e96d5d0 2024-09-19T04:16:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.652+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56289e87acf0 con 0x56289e96eb40 2024-09-19T04:16:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.652+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56289e8cf2b0 con 0x56289e978370 2024-09-19T04:16:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.652+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/568183532 learned_addr learned my addr 172.21.15.94:0/568183532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.652+0000 7fb2b57d2640 1 --2- 172.21.15.94:0/568183532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e96da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.652+0000 7fb2a67fc640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb29c0a3500 0x7fb29c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.652+0000 7fb2b4fd1640 1 --2- 172.21.15.94:0/568183532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e978370 0x56289e97a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.652+0000 7fb2a67fc640 1 -- 172.21.15.94:0/1074877307 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb2ac0f03a0 con 0x7fb2b0100c10 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.653+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/568183532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e978370 msgr2=0x56289e97a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.653+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/568183532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e978370 0x56289e97a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.653+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/568183532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 msgr2=0x56289e96da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.653+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/568183532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e96da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.653+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/568183532 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56289e85f3a0 con 0x56289e96eb40 2024-09-19T04:16:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.653+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb29c0a3500 0x7fb29c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.653+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/568183532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e96eb40 0x56289e96d090 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80105d0 tx=0x7fb2a8010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=334fea18b9e8420 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.654+0000 7fb29b7fe640 1 -- 172.21.15.94:0/568183532 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8019070 con 0x56289e96eb40 2024-09-19T04:16:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.654+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb29c0a3500 0x7fb29c0a59c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8021000 tx=0x7fb2a801f000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:37.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.654+0000 7fb29b7fe640 1 -- 172.21.15.94:0/568183532 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2a8022070 con 0x56289e96eb40 2024-09-19T04:16:37.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.654+0000 7fb29b7fe640 1 -- 172.21.15.94:0/568183532 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8011bc0 con 0x56289e96eb40 2024-09-19T04:16:37.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.655+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/568183532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e96eb40 msgr2=0x56289e96d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:37.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.655+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/568183532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e96eb40 0x56289e96d090 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80105d0 tx=0x7fb2a8010a90 comp rx=0 tx=0).stop 2024-09-19T04:16:37.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.655+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/568183532 shutdown_connections 2024-09-19T04:16:37.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.655+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/568183532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e978370 0x56289e97a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.655+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/568183532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e96da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.655+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/568183532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e96eb40 0x56289e96d090 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.655+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/568183532 >> 172.21.15.94:0/568183532 conn(0x56289e936310 msgr2=0x56289e969c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:37.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.655+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/568183532 shutdown_connections 2024-09-19T04:16:37.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.655+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/568183532 wait complete. 2024-09-19T04:16:37.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.656+0000 7fb2b9e0f740 1 -- start start 2024-09-19T04:16:37.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.656+0000 7fb2b9e0f740 1 Processor -- start 2024-09-19T04:16:37.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.657+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e977dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.657+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 0x56289e975080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.657+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e975810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.657+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56289e97b070 con 0x56289e96eb40 2024-09-19T04:16:37.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.657+0000 7fb2b57d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e977dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.657+0000 7fb2b4fd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 0x56289e975080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b57d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e977dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:38976/0 (socket says 172.21.15.94:38976) 2024-09-19T04:16:37.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b57d2640 1 -- 172.21.15.94:0/2591408794 learned_addr learned my addr 172.21.15.94:0/2591408794 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:37.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.657+0000 7fb2b5fd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e975810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56289e88e060 con 0x56289e978370 2024-09-19T04:16:37.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56289e97b1f0 con 0x56289e96d5d0 2024-09-19T04:16:37.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b9e0f740 10 client.0 _ll_register_callbacks cb 0x56289e5e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/2591408794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 msgr2=0x56289e977dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e977dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/2591408794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 msgr2=0x56289e975080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 0x56289e975080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/2591408794 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56289e6443c0 con 0x56289e978370 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.658+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e975810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80105d0 tx=0x7fb2a8013270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12bca639f3980fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.659+0000 7fb298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.659+0000 7fb298ff9640 1 -- 172.21.15.94:0/2591408794 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8019040 con 0x56289e978370 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.659+0000 7fb298ff9640 1 -- 172.21.15.94:0/2591408794 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb2a8003970 con 0x56289e978370 2024-09-19T04:16:37.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.659+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a8022070 con 0x56289e978370 2024-09-19T04:16:37.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.659+0000 7fb298ff9640 1 -- 172.21.15.94:0/2591408794 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2a8009d10 con 0x56289e978370 2024-09-19T04:16:37.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.659+0000 7fb2b9e0f740 10 client.25124 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:37.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.659+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56289e986bc0 con 0x56289e978370 2024-09-19T04:16:37.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.660+0000 7fb298ff9640 1 -- 172.21.15.94:0/2591408794 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2a8013de0 con 0x56289e978370 2024-09-19T04:16:37.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.660+0000 7fb298ff9640 1 -- 172.21.15.94:0/2591408794 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb2a802d070 con 0x56289e978370 2024-09-19T04:16:37.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.660+0000 7fb2b9e0f740 10 client.25124 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:37.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.660+0000 7fb2b9e0f740 10 client.25124 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:16:37.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.660+0000 7fb2b9e0f740 20 client.25124 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:37.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.660+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56289e987f10 0x56289e98a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.660+0000 7fb2b9e0f740 4 client.25124 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:37.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.660+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb2a8022070 con 0x56289e987f10 2024-09-19T04:16:37.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.660+0000 7fb2b57d2640 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56289e987f10 0x56289e98a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.661+0000 7fb298ff9640 10 client.25124 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:16:37.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.661+0000 7fb2b57d2640 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56289e987f10 0x56289e98a300 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c7688d7a24ad7c2 server_cookie=bb262ee6a30c90ef in_seq=0 out_seq=0 2024-09-19T04:16:37.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.661+0000 7fb298ff9640 1 -- 172.21.15.94:0/2591408794 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb2a8022070 con 0x56289e987f10 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.662+0000 7fb298ff9640 10 client.25124 handle_command_reply: tid=0 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.662+0000 7fb2b9e0f740 1 client.25124 shutdown 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.662+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 msgr2=0x56289e975810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.662+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e975810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80105d0 tx=0x7fb2a8013270 comp rx=0 tx=0).stop 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.662+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 shutdown_connections 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.662+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56289e987f10 0x56289e98a300 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.662+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e975810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.662+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 0x56289e975080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.663+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2591408794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e977dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.663+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 >> 172.21.15.94:0/2591408794 conn(0x56289e936310 msgr2=0x56289e96f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:37.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.663+0000 7fb298ff9640 0 client.25124 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:16:37.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.663+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 shutdown_connections 2024-09-19T04:16:37.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.663+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2591408794 wait complete. 2024-09-19T04:16:37.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.663+0000 7fb2b9e0f740 20 client.25124 trim_cache size 0 max 16384 2024-09-19T04:16:37.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.686+0000 7fb2b9e0f740 1 Processor -- start 2024-09-19T04:16:37.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.686+0000 7fb2b9e0f740 1 -- start start 2024-09-19T04:16:37.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e975050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 0x56289e973fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e970120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b01b6710 con 0x56289e96eb40 2024-09-19T04:16:37.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b57d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e975050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b4fd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 0x56289e973fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b01b3f50 con 0x56289e978370 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b01b1da0 con 0x56289e96d5d0 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b57d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e975050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:38978/0 (socket says 172.21.15.94:38978) 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b57d2640 1 -- 172.21.15.94:0/3569436990 learned_addr learned my addr 172.21.15.94:0/3569436990 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.687+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/3569436990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e970120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.688+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/3569436990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 msgr2=0x56289e975050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.688+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/3569436990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e975050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.688+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/3569436990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 msgr2=0x56289e973fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.688+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/3569436990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 0x56289e973fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.688+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/3569436990 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56289e9735d0 con 0x56289e978370 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.689+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/3569436990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e970120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8011770 tx=0x7fb2a8010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1868f3ee4329e922 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:37.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.689+0000 7fb29affd640 1 -- 172.21.15.94:0/3569436990 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8019070 con 0x56289e978370 2024-09-19T04:16:37.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.689+0000 7fb29affd640 1 -- 172.21.15.94:0/3569436990 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2a8022040 con 0x56289e978370 2024-09-19T04:16:37.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.689+0000 7fb29affd640 1 -- 172.21.15.94:0/3569436990 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8013dc0 con 0x56289e978370 2024-09-19T04:16:37.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.690+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/3569436990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 msgr2=0x56289e970120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:37.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.690+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/3569436990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e970120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8011770 tx=0x7fb2a8010a90 comp rx=0 tx=0).stop 2024-09-19T04:16:37.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.690+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/3569436990 shutdown_connections 2024-09-19T04:16:37.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.690+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/3569436990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e970120 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.690+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/3569436990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96eb40 0x56289e973fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.690+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/3569436990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96d5d0 0x56289e975050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.690+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/3569436990 >> 172.21.15.94:0/3569436990 conn(0x56289e936310 msgr2=0x56289e974940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:37.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.691+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/3569436990 shutdown_connections 2024-09-19T04:16:37.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.691+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/3569436990 wait complete. 2024-09-19T04:16:37.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.691+0000 7fb2b9e0f740 1 -- start start 2024-09-19T04:16:37.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.691+0000 7fb2b9e0f740 1 Processor -- start 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e97ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96eb40 0x56289e97bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b9e0f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e9f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b01b2410 con 0x56289e96d5d0 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b01b3880 con 0x56289e978370 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b9e0f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b01b2ae0 con 0x56289e96eb40 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b9e0f740 10 client.0 _ll_register_callbacks cb 0x7fb2a0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b57d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e97ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b5fd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e9f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b57d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e97ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50018/0 (socket says 172.21.15.94:50018) 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b5fd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e9f0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51468/0 (socket says 172.21.15.94:51468) 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b57d2640 1 -- 172.21.15.94:0/2899505931 learned_addr learned my addr 172.21.15.94:0/2899505931 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.692+0000 7fb2b4fd1640 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96eb40 0x56289e97bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/2899505931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96eb40 msgr2=0x56289e97bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96eb40 0x56289e97bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/2899505931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 msgr2=0x56289e97ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e97ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb2b5fd3640 1 -- 172.21.15.94:0/2899505931 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56289e9f0fd0 con 0x56289e978370 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb2b57d2640 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e97ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb2b5fd3640 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e9f0a90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009ed0 tx=0x7fb2a8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92aa5cb412b14a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:16:37.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb298ff9640 1 -- 172.21.15.94:0/2899505931 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8044060 con 0x56289e978370 2024-09-19T04:16:37.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb298ff9640 1 -- 172.21.15.94:0/2899505931 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb2a80047e0 con 0x56289e978370 2024-09-19T04:16:37.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.693+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2899505931 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a8022040 con 0x56289e978370 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.694+0000 7fb298ff9640 1 -- 172.21.15.94:0/2899505931 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2a8019040 con 0x56289e978370 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.694+0000 7fb2b9e0f740 10 client.25130 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.694+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2899505931 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56289e9f22a0 con 0x56289e978370 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.694+0000 7fb298ff9640 1 -- 172.21.15.94:0/2899505931 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2a8022040 con 0x56289e978370 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.694+0000 7fb298ff9640 1 -- 172.21.15.94:0/2899505931 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb2a802d040 con 0x56289e978370 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.694+0000 7fb2b9e0f740 10 client.25130 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.694+0000 7fb2b9e0f740 10 client.25130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.695+0000 7fb2b9e0f740 20 client.25130 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.695+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56289e9f2930 0x56289e9f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.695+0000 7fb2b9e0f740 4 client.25130 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.695+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2899505931 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56289e8e0860 con 0x56289e9f2930 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.695+0000 7fb2b57d2640 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56289e9f2930 0x56289e9f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.695+0000 7fb2b57d2640 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56289e9f2930 0x56289e9f4d50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b332b136bd7c877 server_cookie=4249d9b1f3c00c66 in_seq=0 out_seq=0 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.695+0000 7fb298ff9640 10 client.25130 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb298ff9640 1 -- 172.21.15.94:0/2899505931 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56289e8e0860 con 0x56289e9f2930 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb298ff9640 10 client.25130 handle_command_reply: tid=0 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb2b9e0f740 1 client.25130 shutdown 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2899505931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 msgr2=0x56289e9f0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e9f0a90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009ed0 tx=0x7fb2a8005fd0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2899505931 shutdown_connections 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56289e9f2930 0x56289e9f4d50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56289e978370 0x56289e9f0a90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289e96eb40 0x56289e97bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb2b9e0f740 1 --2- 172.21.15.94:0/2899505931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56289e96d5d0 0x56289e97ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb298ff9640 0 client.25130 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:16:37.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.696+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2899505931 >> 172.21.15.94:0/2899505931 conn(0x56289e936310 msgr2=0x56289e977100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:37.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.697+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2899505931 shutdown_connections 2024-09-19T04:16:37.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.697+0000 7fb2b9e0f740 1 -- 172.21.15.94:0/2899505931 wait complete. 2024-09-19T04:16:37.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.697+0000 7fb2b9e0f740 20 client.25130 trim_cache size 0 max 16384 2024-09-19T04:16:37.701 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:16:37.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 -- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb29c0a3500 msgr2=0x7fb29c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:37.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb29c0a3500 0x7fb29c0a59c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8021000 tx=0x7fb2a801f000 comp rx=0 tx=0).stop 2024-09-19T04:16:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 -- 172.21.15.94:0/1074877307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2b0100c10 msgr2=0x7fb2b010f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2b0100c10 0x7fb2b010f380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac005ab0 tx=0x7fb2ac003c10 comp rx=0 tx=0).stop 2024-09-19T04:16:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 -- 172.21.15.94:0/1074877307 shutdown_connections 2024-09-19T04:16:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb29c0a3500 0x7fb29c0a59c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b0102ea0 0x7fb2b01a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2b0102ac0 0x7fb2b010f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 --2- 172.21.15.94:0/1074877307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2b0100c10 0x7fb2b010f380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 -- 172.21.15.94:0/1074877307 >> 172.21.15.94:0/1074877307 conn(0x7fb2b0077220 msgr2=0x7fb2b00fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 -- 172.21.15.94:0/1074877307 shutdown_connections 2024-09-19T04:16:37.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:37.698+0000 7fb2b7a36640 1 -- 172.21.15.94:0/1074877307 wait complete. 2024-09-19T04:16:37.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:37 smithi112 ceph-mon[35051]: pgmap v51: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 21 MiB/s wr, 2.32k op/s 2024-09-19T04:16:37.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:37 smithi094 ceph-mon[30972]: pgmap v51: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 21 MiB/s wr, 2.32k op/s 2024-09-19T04:16:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:37 smithi104 ceph-mon[34145]: pgmap v51: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 21 MiB/s wr, 2.32k op/s 2024-09-19T04:16:39.335 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:16:39.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:39 smithi094 ceph-mon[30972]: pgmap v52: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-19T04:16:39.736 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:16:39.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:39 smithi112 ceph-mon[35051]: pgmap v52: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-19T04:16:39.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:39 smithi104 ceph-mon[34145]: pgmap v52: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-19T04:16:40.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.307+0000 7faf73fff640 1 -- 172.21.15.94:0/4073578341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78035020 con 0x7faf7c102d30 2024-09-19T04:16:40.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.308+0000 7faf839c9640 1 -- 172.21.15.94:0/4073578341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf7c102d30 msgr2=0x7faf7c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.308+0000 7faf839c9640 1 --2- 172.21.15.94:0/4073578341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf7c102d30 0x7faf7c103110 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faf78009930 tx=0x7faf7802a420 comp rx=0 tx=0).stop 2024-09-19T04:16:40.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.308+0000 7faf839c9640 1 -- 172.21.15.94:0/4073578341 shutdown_connections 2024-09-19T04:16:40.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.308+0000 7faf839c9640 1 --2- 172.21.15.94:0/4073578341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c1082a0 0x7faf7c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.308+0000 7faf839c9640 1 --2- 172.21.15.94:0/4073578341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf7c1036e0 0x7faf7c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.308+0000 7faf839c9640 1 --2- 172.21.15.94:0/4073578341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf7c102d30 0x7faf7c103110 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.308+0000 7faf839c9640 1 -- 172.21.15.94:0/4073578341 >> 172.21.15.94:0/4073578341 conn(0x7faf7c0fc7d0 msgr2=0x7faf7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.309+0000 7faf839c9640 1 -- 172.21.15.94:0/4073578341 shutdown_connections 2024-09-19T04:16:40.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.309+0000 7faf839c9640 1 -- 172.21.15.94:0/4073578341 wait complete. 2024-09-19T04:16:40.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.309+0000 7faf839c9640 1 Processor -- start 2024-09-19T04:16:40.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.310+0000 7faf839c9640 1 -- start start 2024-09-19T04:16:40.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.310+0000 7faf839c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c102d30 0x7faf7c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.310+0000 7faf839c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf7c1036e0 0x7faf7c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.310+0000 7faf839c9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf7c1082a0 0x7faf7c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.310+0000 7faf839c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c111f40 con 0x7faf7c1036e0 2024-09-19T04:16:40.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.310+0000 7faf839c9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c111dc0 con 0x7faf7c1082a0 2024-09-19T04:16:40.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf839c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c1120c0 con 0x7faf7c102d30 2024-09-19T04:16:40.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81765640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c102d30 0x7faf7c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf80f64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf7c1036e0 0x7faf7c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81765640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c102d30 0x7faf7c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:39004/0 (socket says 172.21.15.94:39004) 2024-09-19T04:16:40.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81765640 1 -- 172.21.15.94:0/550137671 learned_addr learned my addr 172.21.15.94:0/550137671 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:40.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81f66640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf7c1082a0 0x7faf7c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81765640 1 -- 172.21.15.94:0/550137671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf7c1082a0 msgr2=0x7faf7c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81765640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf7c1082a0 0x7faf7c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81765640 1 -- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf7c1036e0 msgr2=0x7faf7c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81765640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf7c1036e0 0x7faf7c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81765640 1 -- 172.21.15.94:0/550137671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c19d450 con 0x7faf7c102d30 2024-09-19T04:16:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf80f64640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf7c1036e0 0x7faf7c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.311+0000 7faf81765640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c102d30 0x7faf7c1984a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faf78005ab0 tx=0x7faf78032a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.312+0000 7faf727fc640 1 -- 172.21.15.94:0/550137671 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7803f030 con 0x7faf7c102d30 2024-09-19T04:16:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.312+0000 7faf727fc640 1 -- 172.21.15.94:0/550137671 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf78003aa0 con 0x7faf7c102d30 2024-09-19T04:16:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.312+0000 7faf727fc640 1 -- 172.21.15.94:0/550137671 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78035070 con 0x7faf7c102d30 2024-09-19T04:16:40.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.312+0000 7faf839c9640 1 -- 172.21.15.94:0/550137671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c19d6e0 con 0x7faf7c102d30 2024-09-19T04:16:40.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.312+0000 7faf839c9640 1 -- 172.21.15.94:0/550137671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c19db70 con 0x7faf7c102d30 2024-09-19T04:16:40.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.316+0000 7faf85da2740 1 Processor -- start 2024-09-19T04:16:40.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.316+0000 7faf85da2740 1 -- start start 2024-09-19T04:16:40.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf85da2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462535ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf85da2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359120 0x564625359560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf85da2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x56462536ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf85da2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646252bf2f0 con 0x56462535a840 2024-09-19T04:16:40.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf85da2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564625266cf0 con 0x564625359b80 2024-09-19T04:16:40.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf85da2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646252bb2b0 con 0x564625359120 2024-09-19T04:16:40.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf80f64640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x56462536ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf81f66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462535ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf80f64640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x56462536ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51498/0 (socket says 172.21.15.94:51498) 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf80f64640 1 -- 172.21.15.94:0/1993365782 learned_addr learned my addr 172.21.15.94:0/1993365782 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf81f66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462535ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50038/0 (socket says 172.21.15.94:50038) 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf80f64640 1 -- 172.21.15.94:0/1993365782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359120 msgr2=0x564625359560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf80f64640 1 --2- 172.21.15.94:0/1993365782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359120 0x564625359560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf80f64640 1 -- 172.21.15.94:0/1993365782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 msgr2=0x56462535ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf80f64640 1 --2- 172.21.15.94:0/1993365782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462535ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.317+0000 7faf80f64640 1 -- 172.21.15.94:0/1993365782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462524b3a0 con 0x564625359b80 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.318+0000 7faf80f64640 1 --2- 172.21.15.94:0/1993365782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x56462536ef60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00bc60 tx=0x7faf6c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d4e7082bc79f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.318+0000 7faf677fe640 1 -- 172.21.15.94:0/1993365782 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c004030 con 0x564625359b80 2024-09-19T04:16:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.318+0000 7faf727fc640 1 -- 172.21.15.94:0/550137671 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7faf78042030 con 0x7faf7c102d30 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.319+0000 7faf677fe640 1 -- 172.21.15.94:0/1993365782 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf6c0054d0 con 0x564625359b80 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.319+0000 7faf677fe640 1 -- 172.21.15.94:0/1993365782 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c004bb0 con 0x564625359b80 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.319+0000 7faf727fc640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7faf680a34b0 0x7faf680a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf727fc640 1 -- 172.21.15.94:0/550137671 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7faf780efde0 con 0x7faf7c102d30 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf85da2740 1 -- 172.21.15.94:0/1993365782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 msgr2=0x56462536ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf85da2740 1 --2- 172.21.15.94:0/1993365782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x56462536ef60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00bc60 tx=0x7faf6c00f730 comp rx=0 tx=0).stop 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf81f66640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7faf680a34b0 0x7faf680a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf85da2740 1 -- 172.21.15.94:0/1993365782 shutdown_connections 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf85da2740 1 --2- 172.21.15.94:0/1993365782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x56462536ef60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf85da2740 1 --2- 172.21.15.94:0/1993365782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359120 0x564625359560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf85da2740 1 --2- 172.21.15.94:0/1993365782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462535ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf85da2740 1 -- 172.21.15.94:0/1993365782 >> 172.21.15.94:0/1993365782 conn(0x564625322310 msgr2=0x564625355c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf85da2740 1 -- 172.21.15.94:0/1993365782 shutdown_connections 2024-09-19T04:16:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.320+0000 7faf85da2740 1 -- 172.21.15.94:0/1993365782 wait complete. 2024-09-19T04:16:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.321+0000 7faf85da2740 1 -- start start 2024-09-19T04:16:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.321+0000 7faf81f66640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7faf680a34b0 0x7faf680a5970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf74005fd0 tx=0x7faf74009500 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.321+0000 7faf85da2740 1 Processor -- start 2024-09-19T04:16:40.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf85da2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x56462536eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf85da2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x56462536c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf85da2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462536c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf85da2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56462536f930 con 0x56462535a840 2024-09-19T04:16:40.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf85da2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56462527a060 con 0x564625359120 2024-09-19T04:16:40.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf85da2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56462536fab0 con 0x564625359b80 2024-09-19T04:16:40.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf81765640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x56462536eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf81765640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x56462536eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51508/0 (socket says 172.21.15.94:51508) 2024-09-19T04:16:40.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf81765640 1 -- 172.21.15.94:0/793166711 learned_addr learned my addr 172.21.15.94:0/793166711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:40.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf80f64640 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x56462536c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.322+0000 7faf81f66640 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462536c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.323+0000 7faf85da2740 10 client.0 _ll_register_callbacks cb 0x564624fcd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:40.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.323+0000 7faf80f64640 1 -- 172.21.15.94:0/793166711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 msgr2=0x56462536eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.323+0000 7faf80f64640 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x56462536eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.323+0000 7faf80f64640 1 -- 172.21.15.94:0/793166711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 msgr2=0x56462536c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.323+0000 7faf80f64640 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462536c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.323+0000 7faf80f64640 1 -- 172.21.15.94:0/793166711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646250303c0 con 0x564625359b80 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.324+0000 7faf81f66640 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462536c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.324+0000 7faf80f64640 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x56462536c170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00bc30 tx=0x7faf6c00bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2847b3ab55ab1d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.324+0000 7faf64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.324+0000 7faf64ff9640 1 -- 172.21.15.94:0/793166711 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c02b060 con 0x564625359b80 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.324+0000 7faf85da2740 1 -- 172.21.15.94:0/793166711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf6c0054d0 con 0x564625359b80 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.324+0000 7faf64ff9640 1 -- 172.21.15.94:0/793166711 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7faf6c01b660 con 0x564625359b80 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.324+0000 7faf64ff9640 1 -- 172.21.15.94:0/793166711 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf6c01b860 con 0x564625359b80 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.324+0000 7faf85da2740 10 client.25096 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.324+0000 7faf85da2740 1 -- 172.21.15.94:0/793166711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56462537b450 con 0x564625359b80 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.325+0000 7faf64ff9640 1 -- 172.21.15.94:0/793166711 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf6c0054d0 con 0x564625359b80 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.325+0000 7faf64ff9640 1 -- 172.21.15.94:0/793166711 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7faf6c011cd0 con 0x564625359b80 2024-09-19T04:16:40.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.325+0000 7faf85da2740 10 client.25096 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.325+0000 7faf85da2740 10 client.25096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:16:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.325+0000 7faf85da2740 20 client.25096 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.325+0000 7faf85da2740 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56462537c7a0 0x56462537eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.325+0000 7faf85da2740 4 client.25096 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.325+0000 7faf85da2740 1 -- 172.21.15.94:0/793166711 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646252cc860 con 0x56462537c7a0 2024-09-19T04:16:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.326+0000 7faf81765640 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56462537c7a0 0x56462537eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.326+0000 7faf64ff9640 10 client.25096 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.326+0000 7faf81765640 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56462537c7a0 0x56462537eb90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541234089de4c0ca server_cookie=429ed70e89f7e0a0 in_seq=0 out_seq=0 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf64ff9640 1 -- 172.21.15.94:0/793166711 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646252cc860 con 0x56462537c7a0 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf64ff9640 10 client.25096 handle_command_reply: tid=0 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf85da2740 1 client.25096 shutdown 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf85da2740 1 -- 172.21.15.94:0/793166711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 msgr2=0x56462536c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf85da2740 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x56462536c170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00bc30 tx=0x7faf6c00bf80 comp rx=0 tx=0).stop 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf85da2740 1 -- 172.21.15.94:0/793166711 shutdown_connections 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf85da2740 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56462537c7a0 0x56462537eb90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf85da2740 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x56462536c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf85da2740 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x56462536c170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf85da2740 1 --2- 172.21.15.94:0/793166711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x56462536eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf85da2740 1 -- 172.21.15.94:0/793166711 >> 172.21.15.94:0/793166711 conn(0x564625322310 msgr2=0x56462535b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.327+0000 7faf64ff9640 0 client.25096 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:40.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.328+0000 7faf85da2740 1 -- 172.21.15.94:0/793166711 shutdown_connections 2024-09-19T04:16:40.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.328+0000 7faf85da2740 1 -- 172.21.15.94:0/793166711 wait complete. 2024-09-19T04:16:40.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.328+0000 7faf85da2740 20 client.25096 trim_cache size 0 max 16384 2024-09-19T04:16:40.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf85da2740 1 Processor -- start 2024-09-19T04:16:40.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf85da2740 1 -- start start 2024-09-19T04:16:40.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf85da2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564625359120 0x56462536c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf85da2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x564625378c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf85da2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56462535a840 0x56462536e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf85da2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c1adfa0 con 0x564625359120 2024-09-19T04:16:40.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf85da2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c1ab7e0 con 0x564625359b80 2024-09-19T04:16:40.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf85da2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c1a9630 con 0x56462535a840 2024-09-19T04:16:40.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf81765640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564625359120 0x56462536c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf81765640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564625359120 0x56462536c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50066/0 (socket says 172.21.15.94:50066) 2024-09-19T04:16:40.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf81f66640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56462535a840 0x56462536e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf81765640 1 -- 172.21.15.94:0/3668587474 learned_addr learned my addr 172.21.15.94:0/3668587474 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:40.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.351+0000 7faf80f64640 1 --2- 172.21.15.94:0/3668587474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x564625378c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.352+0000 7faf81f66640 1 -- 172.21.15.94:0/3668587474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 msgr2=0x564625378c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.352+0000 7faf81f66640 1 --2- 172.21.15.94:0/3668587474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x564625378c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.352+0000 7faf81f66640 1 -- 172.21.15.94:0/3668587474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564625359120 msgr2=0x56462536c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.352+0000 7faf81f66640 1 --2- 172.21.15.94:0/3668587474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564625359120 0x56462536c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.352+0000 7faf81f66640 1 -- 172.21.15.94:0/3668587474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564625355510 con 0x56462535a840 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.352+0000 7faf81f66640 1 --2- 172.21.15.94:0/3668587474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56462535a840 0x56462536e0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf74009e20 tx=0x7faf74019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff1f3b5dba6c1b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.352+0000 7faf66ffd640 1 -- 172.21.15.94:0/3668587474 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7402f070 con 0x56462535a840 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.352+0000 7faf66ffd640 1 -- 172.21.15.94:0/3668587474 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf7401db40 con 0x56462535a840 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.352+0000 7faf66ffd640 1 -- 172.21.15.94:0/3668587474 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7402aae0 con 0x56462535a840 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.353+0000 7faf85da2740 1 -- 172.21.15.94:0/3668587474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56462535a840 msgr2=0x56462536e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.353+0000 7faf85da2740 1 --2- 172.21.15.94:0/3668587474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56462535a840 0x56462536e0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf74009e20 tx=0x7faf74019000 comp rx=0 tx=0).stop 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.353+0000 7faf85da2740 1 -- 172.21.15.94:0/3668587474 shutdown_connections 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.353+0000 7faf85da2740 1 --2- 172.21.15.94:0/3668587474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56462535a840 0x56462536e0e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.353+0000 7faf85da2740 1 --2- 172.21.15.94:0/3668587474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359b80 0x564625378c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.353+0000 7faf85da2740 1 --2- 172.21.15.94:0/3668587474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564625359120 0x56462536c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.353+0000 7faf85da2740 1 -- 172.21.15.94:0/3668587474 >> 172.21.15.94:0/3668587474 conn(0x564625322310 msgr2=0x56462536d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.353+0000 7faf85da2740 1 -- 172.21.15.94:0/3668587474 shutdown_connections 2024-09-19T04:16:40.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.353+0000 7faf85da2740 1 -- 172.21.15.94:0/3668587474 wait complete. 2024-09-19T04:16:40.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.354+0000 7faf85da2740 1 -- start start 2024-09-19T04:16:40.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.354+0000 7faf85da2740 1 Processor -- start 2024-09-19T04:16:40.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.354+0000 7faf85da2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x564625383df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.354+0000 7faf85da2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x564625384330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.354+0000 7faf85da2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x564625384870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.354+0000 7faf85da2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c1a9ca0 con 0x56462535a840 2024-09-19T04:16:40.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.354+0000 7faf85da2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c1ab110 con 0x564625359120 2024-09-19T04:16:40.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.354+0000 7faf85da2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c1aa370 con 0x564625359b80 2024-09-19T04:16:40.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.354+0000 7faf85da2740 10 client.0 _ll_register_callbacks cb 0x7faf6c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:40.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf80f64640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x564625384330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81f66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x564625384870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81765640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x564625383df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81f66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x564625384870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50070/0 (socket says 172.21.15.94:50070) 2024-09-19T04:16:40.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81765640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x564625383df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51530/0 (socket says 172.21.15.94:51530) 2024-09-19T04:16:40.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81f66640 1 -- 172.21.15.94:0/2866163381 learned_addr learned my addr 172.21.15.94:0/2866163381 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:40.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81765640 1 -- 172.21.15.94:0/2866163381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 msgr2=0x564625384330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81765640 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x564625384330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81765640 1 -- 172.21.15.94:0/2866163381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 msgr2=0x564625384870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81765640 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x564625384870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81765640 1 -- 172.21.15.94:0/2866163381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646253733d0 con 0x564625359120 2024-09-19T04:16:40.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf80f64640 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x564625384330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:40.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf81765640 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x564625383df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf780b8a20 tx=0x7faf78005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d423320854ca3a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:16:40.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.355+0000 7faf64ff9640 1 -- 172.21.15.94:0/2866163381 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78049040 con 0x564625359120 2024-09-19T04:16:40.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf64ff9640 1 -- 172.21.15.94:0/2866163381 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7faf7803b460 con 0x564625359120 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf85da2740 1 -- 172.21.15.94:0/2866163381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7401db40 con 0x564625359120 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf64ff9640 1 -- 172.21.15.94:0/2866163381 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf780c0400 con 0x564625359120 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf85da2740 10 client.25148 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf85da2740 1 -- 172.21.15.94:0/2866163381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646253f8520 con 0x564625359120 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf64ff9640 1 -- 172.21.15.94:0/2866163381 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf7401db40 con 0x564625359120 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf64ff9640 1 -- 172.21.15.94:0/2866163381 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7faf780d7030 con 0x564625359120 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf85da2740 10 client.25148 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf85da2740 10 client.25148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf85da2740 20 client.25148 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf85da2740 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5646253f8c80 0x5646253fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf85da2740 4 client.25148 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:16:40.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.356+0000 7faf85da2740 1 -- 172.21.15.94:0/2866163381 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646252c6d70 con 0x5646253f8c80 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.357+0000 7faf80f64640 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5646253f8c80 0x5646253fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.357+0000 7faf64ff9640 10 client.25148 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.357+0000 7faf80f64640 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5646253f8c80 0x5646253fb0a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64614db74ee6cb1b server_cookie=50459d5453a310dd in_seq=0 out_seq=0 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf64ff9640 1 -- 172.21.15.94:0/2866163381 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646252c6d70 con 0x5646253f8c80 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf64ff9640 10 client.25148 handle_command_reply: tid=0 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf85da2740 1 client.25148 shutdown 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf85da2740 1 -- 172.21.15.94:0/2866163381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 msgr2=0x564625383df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf85da2740 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x564625383df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf780b8a20 tx=0x7faf78005b70 comp rx=0 tx=0).stop 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf85da2740 1 -- 172.21.15.94:0/2866163381 shutdown_connections 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf85da2740 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5646253f8c80 0x5646253fb0a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf85da2740 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56462535a840 0x564625384870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf64ff9640 0 client.25148 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf85da2740 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564625359b80 0x564625384330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf85da2740 1 --2- 172.21.15.94:0/2866163381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564625359120 0x564625383df0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.358+0000 7faf85da2740 1 -- 172.21.15.94:0/2866163381 >> 172.21.15.94:0/2866163381 conn(0x564625322310 msgr2=0x564625354d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.359+0000 7faf85da2740 1 -- 172.21.15.94:0/2866163381 shutdown_connections 2024-09-19T04:16:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.359+0000 7faf85da2740 1 -- 172.21.15.94:0/2866163381 wait complete. 2024-09-19T04:16:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.359+0000 7faf85da2740 20 client.25148 trim_cache size 0 max 16384 2024-09-19T04:16:40.366 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:16:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 -- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7faf680a34b0 msgr2=0x7faf680a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7faf680a34b0 0x7faf680a5970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf74005fd0 tx=0x7faf74009500 comp rx=0 tx=0).stop 2024-09-19T04:16:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 -- 172.21.15.94:0/550137671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c102d30 msgr2=0x7faf7c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c102d30 0x7faf7c1984a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faf78005ab0 tx=0x7faf78032a00 comp rx=0 tx=0).stop 2024-09-19T04:16:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 -- 172.21.15.94:0/550137671 shutdown_connections 2024-09-19T04:16:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7faf680a34b0 0x7faf680a5970 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf7c1082a0 0x7faf7c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf7c1036e0 0x7faf7c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 --2- 172.21.15.94:0/550137671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c102d30 0x7faf7c1984a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.360+0000 7faf839c9640 1 -- 172.21.15.94:0/550137671 >> 172.21.15.94:0/550137671 conn(0x7faf7c0fc7d0 msgr2=0x7faf7c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.361+0000 7faf839c9640 1 -- 172.21.15.94:0/550137671 shutdown_connections 2024-09-19T04:16:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:40.361+0000 7faf839c9640 1 -- 172.21.15.94:0/550137671 wait complete. 2024-09-19T04:16:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 25.0 s, 1466.2 tps, lat 70.733 ms stddev 64.377 2024-09-19T04:16:41.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:41 smithi112 ceph-mon[35051]: pgmap v53: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-19T04:16:41.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:41 smithi094 ceph-mon[30972]: pgmap v53: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-19T04:16:41.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:41 smithi104 ceph-mon[34145]: pgmap v53: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-19T04:16:42.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:16:42.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:16:42.520 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:16:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.104+0000 7ff1d2db2640 1 -- 172.21.15.94:0/1023244424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1049a0 msgr2=0x7ff1cc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.104+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/1023244424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1049a0 0x7ff1cc1089f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009860 tx=0x7ff1b802a340 comp rx=0 tx=0).stop 2024-09-19T04:16:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.104+0000 7ff1d2db2640 1 -- 172.21.15.94:0/1023244424 shutdown_connections 2024-09-19T04:16:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.104+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/1023244424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1cc109120 0x7ff1cc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.104+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/1023244424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1049a0 0x7ff1cc1089f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.104+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/1023244424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc103ff0 0x7ff1cc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.104+0000 7ff1d2db2640 1 -- 172.21.15.94:0/1023244424 >> 172.21.15.94:0/1023244424 conn(0x7ff1cc0ffab0 msgr2=0x7ff1cc101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.104+0000 7ff1d2db2640 1 -- 172.21.15.94:0/1023244424 shutdown_connections 2024-09-19T04:16:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.104+0000 7ff1d2db2640 1 -- 172.21.15.94:0/1023244424 wait complete. 2024-09-19T04:16:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.105+0000 7ff1d2db2640 1 Processor -- start 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d2db2640 1 -- start start 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d2db2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1cc103ff0 0x7ff1cc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d2db2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1049a0 0x7ff1cc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d2db2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc109120 0x7ff1cc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d2db2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc113fd0 con 0x7ff1cc103ff0 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1049a0 0x7ff1cc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d0b4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1cc103ff0 0x7ff1cc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d134f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc109120 0x7ff1cc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d0b4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1cc103ff0 0x7ff1cc19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54444/0 (socket says 172.21.15.94:54444) 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d0b4e640 1 -- 172.21.15.94:0/2668282124 learned_addr learned my addr 172.21.15.94:0/2668282124 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc113e50 con 0x7ff1cc109120 2024-09-19T04:16:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.106+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc114150 con 0x7ff1cc1049a0 2024-09-19T04:16:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1d134f640 1 -- 172.21.15.94:0/2668282124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1049a0 msgr2=0x7ff1cc19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1d134f640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1049a0 0x7ff1cc19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1d134f640 1 -- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1cc103ff0 msgr2=0x7ff1cc19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1d134f640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1cc103ff0 0x7ff1cc19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1d134f640 1 -- 172.21.15.94:0/2668282124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc19f540 con 0x7ff1cc109120 2024-09-19T04:16:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1049a0 0x7ff1cc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1d0b4e640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1cc103ff0 0x7ff1cc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1d134f640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc109120 0x7ff1cc19ee60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400ec50 tx=0x7ff1c400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1c9ffb640 1 -- 172.21.15.94:0/2668282124 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4003b70 con 0x7ff1cc109120 2024-09-19T04:16:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.107+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc19f830 con 0x7ff1cc109120 2024-09-19T04:16:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.108+0000 7ff1c9ffb640 1 -- 172.21.15.94:0/2668282124 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1c4003d10 con 0x7ff1cc109120 2024-09-19T04:16:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.108+0000 7ff1c9ffb640 1 -- 172.21.15.94:0/2668282124 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4010760 con 0x7ff1cc109120 2024-09-19T04:16:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.108+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc1a7110 con 0x7ff1cc109120 2024-09-19T04:16:43.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.111+0000 7ff1c9ffb640 1 -- 172.21.15.94:0/2668282124 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff1c4004020 con 0x7ff1cc109120 2024-09-19T04:16:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.112+0000 7ff1d518b740 1 Processor -- start 2024-09-19T04:16:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.112+0000 7ff1d518b740 1 -- start start 2024-09-19T04:16:43.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.112+0000 7ff1c9ffb640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1bc0a34b0 0x7ff1bc0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba81f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1bc0a34b0 0x7ff1bc0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1c9ffb640 1 -- 172.21.15.94:0/2668282124 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff1c40d02d0 con 0x7ff1cc109120 2024-09-19T04:16:43.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d518b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cffb9e7280 con 0x55cffba8cd40 2024-09-19T04:16:43.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d518b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cffb98eb10 con 0x55cffba82480 2024-09-19T04:16:43.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d518b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cffb9e3240 con 0x55cffba81b60 2024-09-19T04:16:43.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d0b4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba81f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d134f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d134f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54460/0 (socket says 172.21.15.94:54460) 2024-09-19T04:16:43.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d134f640 1 -- 172.21.15.94:0/2408622886 learned_addr learned my addr 172.21.15.94:0/2408622886 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:43.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1d0b4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba81f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:33038/0 (socket says 172.21.15.94:33038) 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1bc0a34b0 0x7ff1bc0a5970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1b802a930 tx=0x7ff1b8004040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1cbfff640 1 -- 172.21.15.94:0/2408622886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 msgr2=0x55cffba81f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/2408622886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba81f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1cbfff640 1 -- 172.21.15.94:0/2408622886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 msgr2=0x55cffba8f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/2408622886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.113+0000 7ff1cbfff640 1 -- 172.21.15.94:0/2408622886 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cffb99b1f0 con 0x55cffba82480 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.114+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/2408622886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8c800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8005940 tx=0x7ff1b8047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f42470d53bc27d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.114+0000 7ff1b6ffd640 1 -- 172.21.15.94:0/2408622886 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8037040 con 0x55cffba82480 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.115+0000 7ff1b6ffd640 1 -- 172.21.15.94:0/2408622886 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b8004340 con 0x55cffba82480 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.115+0000 7ff1b6ffd640 1 -- 172.21.15.94:0/2408622886 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80326a0 con 0x55cffba82480 2024-09-19T04:16:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.115+0000 7ff1d518b740 1 -- 172.21.15.94:0/2408622886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 msgr2=0x55cffba8c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.115+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2408622886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8c800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8005940 tx=0x7ff1b8047000 comp rx=0 tx=0).stop 2024-09-19T04:16:43.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.116+0000 7ff1d518b740 1 -- 172.21.15.94:0/2408622886 shutdown_connections 2024-09-19T04:16:43.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.116+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2408622886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.116+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2408622886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8c800 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.116+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2408622886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba81f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.116+0000 7ff1d518b740 1 -- 172.21.15.94:0/2408622886 >> 172.21.15.94:0/2408622886 conn(0x55cffba7e0f0 msgr2=0x55cffba7e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.116+0000 7ff1d518b740 1 -- 172.21.15.94:0/2408622886 shutdown_connections 2024-09-19T04:16:43.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.116+0000 7ff1d518b740 1 -- 172.21.15.94:0/2408622886 wait complete. 2024-09-19T04:16:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.117+0000 7ff1d518b740 1 -- start start 2024-09-19T04:16:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.117+0000 7ff1d518b740 1 Processor -- start 2024-09-19T04:16:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba89c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d134f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d0b4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba89c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d134f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54476/0 (socket says 172.21.15.94:54476) 2024-09-19T04:16:43.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d0b4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba89c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:33046/0 (socket says 172.21.15.94:33046) 2024-09-19T04:16:43.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50446/0 (socket says 172.21.15.94:50446) 2024-09-19T04:16:43.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d134f640 1 -- 172.21.15.94:0/3292357719 learned_addr learned my addr 172.21.15.94:0/3292357719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:43.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cffba8f910 con 0x55cffba8cd40 2024-09-19T04:16:43.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cffb9a1f10 con 0x55cffba82480 2024-09-19T04:16:43.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cffba8fa90 con 0x55cffba81b60 2024-09-19T04:16:43.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d518b740 10 client.0 _ll_register_callbacks cb 0x55cffb96aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:43.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d134f640 1 -- 172.21.15.94:0/3292357719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 msgr2=0x55cffba89c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d134f640 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba89c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d134f640 1 -- 172.21.15.94:0/3292357719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 msgr2=0x55cffba8a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d134f640 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.118+0000 7ff1d134f640 1 -- 172.21.15.94:0/3292357719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cffb7586a0 con 0x55cffba8cd40 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.119+0000 7ff1d134f640 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8a6f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cba0 tx=0x7ff1c4009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d3c6da2bfbc361 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.119+0000 7ff1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.119+0000 7ff1abfff640 1 -- 172.21.15.94:0/3292357719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40ba060 con 0x55cffba8cd40 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.119+0000 7ff1abfff640 1 -- 172.21.15.94:0/3292357719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff1c40b52b0 con 0x55cffba8cd40 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.119+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8004340 con 0x55cffba8cd40 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.119+0000 7ff1abfff640 1 -- 172.21.15.94:0/3292357719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff1c40d0040 con 0x55cffba8cd40 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.120+0000 7ff1d518b740 10 client.15564 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.120+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55cffba9b430 con 0x55cffba8cd40 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.120+0000 7ff1abfff640 1 -- 172.21.15.94:0/3292357719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff1c4004b40 con 0x55cffba8cd40 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.120+0000 7ff1d518b740 10 client.15564 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.120+0000 7ff1d518b740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.121+0000 7ff1d518b740 20 client.15564 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.121+0000 7ff1d518b740 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55cffba9c7e0 0x55cffba9ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.121+0000 7ff1d518b740 4 client.15564 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.121+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff1b8004340 con 0x55cffba9c7e0 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.121+0000 7ff1d0b4e640 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55cffba9c7e0 0x55cffba9ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.122+0000 7ff1d0b4e640 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55cffba9c7e0 0x55cffba9ec30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67102d0791b98ca1 server_cookie=7635623cfcb55e21 in_seq=0 out_seq=0 2024-09-19T04:16:43.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.122+0000 7ff1abfff640 10 client.15564 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1abfff640 1 -- 172.21.15.94:0/3292357719 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff1b8004340 con 0x55cffba9c7e0 2024-09-19T04:16:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1abfff640 10 client.15564 handle_command_reply: tid=0 2024-09-19T04:16:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1d518b740 1 client.15564 shutdown 2024-09-19T04:16:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 msgr2=0x55cffba8a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1d518b740 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8a6f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cba0 tx=0x7ff1c4009960 comp rx=0 tx=0).stop 2024-09-19T04:16:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 shutdown_connections 2024-09-19T04:16:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1d518b740 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55cffba9c7e0 0x55cffba9ec30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1abfff640 0 client.15564 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1d518b740 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba8cd40 0x55cffba8a6f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1d518b740 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba82480 0x55cffba8a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1d518b740 1 --2- 172.21.15.94:0/3292357719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba89c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.123+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 >> 172.21.15.94:0/3292357719 conn(0x55cffba7e0f0 msgr2=0x55cffba83dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.124+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 shutdown_connections 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.124+0000 7ff1d518b740 1 -- 172.21.15.94:0/3292357719 wait complete. 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.124+0000 7ff1d518b740 20 client.15564 trim_cache size 0 max 16384 2024-09-19T04:16:43.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d518b740 1 Processor -- start 2024-09-19T04:16:43.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d518b740 1 -- start start 2024-09-19T04:16:43.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba81b60 0x55cffba845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba82480 0x55cffba8a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbaa4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d518b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc072820 con 0x55cffba81b60 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d518b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc070060 con 0x55cffba8cd40 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d518b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc06deb0 con 0x55cffba82480 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d0b4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba81b60 0x55cffba845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d0b4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba81b60 0x55cffba845c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54492/0 (socket says 172.21.15.94:54492) 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d0b4e640 1 -- 172.21.15.94:0/1707013837 learned_addr learned my addr 172.21.15.94:0/1707013837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba82480 0x55cffba8a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.157+0000 7ff1d134f640 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbaa4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.158+0000 7ff1cbfff640 1 -- 172.21.15.94:0/1707013837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 msgr2=0x55cffbaa4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.158+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbaa4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.158+0000 7ff1cbfff640 1 -- 172.21.15.94:0/1707013837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba81b60 msgr2=0x55cffba845c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.158+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba81b60 0x55cffba845c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.158+0000 7ff1cbfff640 1 -- 172.21.15.94:0/1707013837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cffba8f7f0 con 0x55cffba82480 2024-09-19T04:16:43.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.158+0000 7ff1d0b4e640 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba81b60 0x55cffba845c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:43.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.158+0000 7ff1cbfff640 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba82480 0x55cffba8a540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8035040 tx=0x7ff1b8043040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8945a5ec52386fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.158+0000 7ff1b67fc640 1 -- 172.21.15.94:0/1707013837 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80035f0 con 0x55cffba82480 2024-09-19T04:16:43.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.159+0000 7ff1b67fc640 1 -- 172.21.15.94:0/1707013837 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b8059070 con 0x55cffba82480 2024-09-19T04:16:43.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.159+0000 7ff1b67fc640 1 -- 172.21.15.94:0/1707013837 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80326a0 con 0x55cffba82480 2024-09-19T04:16:43.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.159+0000 7ff1d518b740 1 -- 172.21.15.94:0/1707013837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba82480 msgr2=0x55cffba8a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.159+0000 7ff1d518b740 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba82480 0x55cffba8a540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8035040 tx=0x7ff1b8043040 comp rx=0 tx=0).stop 2024-09-19T04:16:43.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.160+0000 7ff1d518b740 1 -- 172.21.15.94:0/1707013837 shutdown_connections 2024-09-19T04:16:43.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.160+0000 7ff1d518b740 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbaa4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.160+0000 7ff1d518b740 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba82480 0x55cffba8a540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.160+0000 7ff1d518b740 1 --2- 172.21.15.94:0/1707013837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba81b60 0x55cffba845c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.160+0000 7ff1d518b740 1 -- 172.21.15.94:0/1707013837 >> 172.21.15.94:0/1707013837 conn(0x55cffba7e0f0 msgr2=0x55cffba8bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.160+0000 7ff1d518b740 1 -- 172.21.15.94:0/1707013837 shutdown_connections 2024-09-19T04:16:43.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.160+0000 7ff1d518b740 1 -- 172.21.15.94:0/1707013837 wait complete. 2024-09-19T04:16:43.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.160+0000 7ff1d518b740 1 -- start start 2024-09-19T04:16:43.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1d518b740 1 Processor -- start 2024-09-19T04:16:43.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba90db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba82480 0x55cffba912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1d518b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbb064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1d518b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc06e520 con 0x55cffba82480 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1d518b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc06f990 con 0x55cffba8cd40 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1d518b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc06ebf0 con 0x55cffba81b60 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1d518b740 10 client.0 _ll_register_callbacks cb 0x7ff1c40cd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1d134f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbb064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.161+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba82480 0x55cffba912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1d134f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbb064d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50470/0 (socket says 172.21.15.94:50470) 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1d134f640 1 -- 172.21.15.94:0/2117647355 learned_addr learned my addr 172.21.15.94:0/2117647355 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1d0b4e640 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba90db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1d134f640 1 -- 172.21.15.94:0/2117647355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 msgr2=0x55cffba90db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1d134f640 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba90db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1d134f640 1 -- 172.21.15.94:0/2117647355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba82480 msgr2=0x55cffba912f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1d134f640 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba82480 0x55cffba912f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1d134f640 1 -- 172.21.15.94:0/2117647355 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cffba91bd0 con 0x55cffba8cd40 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1d134f640 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbb064d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40094d0 tx=0x7ff1c400ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f7c76dc6ad44e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.162+0000 7ff1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.163+0000 7ff1abfff640 1 -- 172.21.15.94:0/2117647355 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40ba040 con 0x55cffba8cd40 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.163+0000 7ff1abfff640 1 -- 172.21.15.94:0/2117647355 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff1c409b990 con 0x55cffba8cd40 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.163+0000 7ff1d518b740 1 -- 172.21.15.94:0/2117647355 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8059070 con 0x55cffba8cd40 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.163+0000 7ff1abfff640 1 -- 172.21.15.94:0/2117647355 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff1c407d030 con 0x55cffba8cd40 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.163+0000 7ff1d518b740 10 client.25166 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:43.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.163+0000 7ff1d518b740 1 -- 172.21.15.94:0/2117647355 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cffbb07b60 con 0x55cffba8cd40 2024-09-19T04:16:43.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.163+0000 7ff1abfff640 1 -- 172.21.15.94:0/2117647355 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b8059070 con 0x55cffba8cd40 2024-09-19T04:16:43.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.164+0000 7ff1abfff640 1 -- 172.21.15.94:0/2117647355 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff1c409b400 con 0x55cffba8cd40 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.164+0000 7ff1d518b740 10 client.25166 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.164+0000 7ff1d518b740 10 client.25166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.164+0000 7ff1d518b740 20 client.25166 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.164+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55cffba9c7e0 0x55cffbb0a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.164+0000 7ff1d518b740 4 client.25166 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.164+0000 7ff1d518b740 1 -- 172.21.15.94:0/2117647355 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cffb9f47f0 con 0x55cffba9c7e0 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.164+0000 7ff1d0b4e640 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55cffba9c7e0 0x55cffbb0a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.165+0000 7ff1d0b4e640 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55cffba9c7e0 0x55cffbb0a0f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87cac290822e861a server_cookie=35f91b75130f897b in_seq=0 out_seq=0 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.165+0000 7ff1abfff640 10 client.25166 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.166+0000 7ff1abfff640 1 -- 172.21.15.94:0/2117647355 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cffb9f47f0 con 0x55cffba9c7e0 2024-09-19T04:16:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.166+0000 7ff1abfff640 10 client.25166 handle_command_reply: tid=0 2024-09-19T04:16:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.166+0000 7ff1d518b740 1 client.25166 shutdown 2024-09-19T04:16:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.166+0000 7ff1d518b740 1 -- 172.21.15.94:0/2117647355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 msgr2=0x55cffbb064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.166+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbb064d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40094d0 tx=0x7ff1c400ef50 comp rx=0 tx=0).stop 2024-09-19T04:16:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1d518b740 1 -- 172.21.15.94:0/2117647355 shutdown_connections 2024-09-19T04:16:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55cffba9c7e0 0x55cffbb0a0f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cffba8cd40 0x55cffbb064d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1abfff640 0 client.25166 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cffba82480 0x55cffba912f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1d518b740 1 --2- 172.21.15.94:0/2117647355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cffba81b60 0x55cffba90db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1d518b740 1 -- 172.21.15.94:0/2117647355 >> 172.21.15.94:0/2117647355 conn(0x55cffba7e0f0 msgr2=0x55cffba97b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1d518b740 1 -- 172.21.15.94:0/2117647355 shutdown_connections 2024-09-19T04:16:43.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1d518b740 1 -- 172.21.15.94:0/2117647355 wait complete. 2024-09-19T04:16:43.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.167+0000 7ff1d518b740 20 client.25166 trim_cache size 0 max 16384 2024-09-19T04:16:43.171 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:16:43.171 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:16:43.171 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "f7d480a3-2049-430d-9251-7143138b8ae7", 2024-09-19T04:16:43.171 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:16:43.171 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:16:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.168+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1bc0a34b0 msgr2=0x7ff1bc0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.168+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1bc0a34b0 0x7ff1bc0a5970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1b802a930 tx=0x7ff1b8004040 comp rx=0 tx=0).stop 2024-09-19T04:16:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.168+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc109120 msgr2=0x7ff1cc19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.168+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc109120 0x7ff1cc19ee60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400ec50 tx=0x7ff1c400c6a0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.169+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 shutdown_connections 2024-09-19T04:16:43.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.169+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1bc0a34b0 0x7ff1bc0a5970 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.169+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc109120 0x7ff1cc19ee60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.169+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1049a0 0x7ff1cc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.169+0000 7ff1d2db2640 1 --2- 172.21.15.94:0/2668282124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1cc103ff0 0x7ff1cc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.169+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 >> 172.21.15.94:0/2668282124 conn(0x7ff1cc0ffab0 msgr2=0x7ff1cc100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.169+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 shutdown_connections 2024-09-19T04:16:43.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:43.169+0000 7ff1d2db2640 1 -- 172.21.15.94:0/2668282124 wait complete. 2024-09-19T04:16:43.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:43 smithi112 ceph-mon[35051]: pgmap v54: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 2.23k op/s 2024-09-19T04:16:43.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:43 smithi094 ceph-mon[30972]: pgmap v54: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 2.23k op/s 2024-09-19T04:16:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:43 smithi104 ceph-mon[34145]: pgmap v54: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 2.23k op/s 2024-09-19T04:16:43.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:16:44.328 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:16:44.769 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:16:44.769 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.769 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:16:44.769 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:16:44.769 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.770 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:16:44.770 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:16:44.770 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.770 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.770 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:44 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:16:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:16:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:16:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:16:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:16:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:44 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:16:44.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:16:44.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:16:44.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:16:44.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:16:44.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:44 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:16:44.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.900+0000 7f06ab395640 1 -- 172.21.15.94:0/3347826216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a4106a80 msgr2=0x7f06a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.900+0000 7f06ab395640 1 --2- 172.21.15.94:0/3347826216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a4106a80 0x7f06a4106e60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009930 tx=0x7f06a002a420 comp rx=0 tx=0).stop 2024-09-19T04:16:44.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.901+0000 7f06ab395640 1 -- 172.21.15.94:0/3347826216 shutdown_connections 2024-09-19T04:16:44.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.901+0000 7f06ab395640 1 --2- 172.21.15.94:0/3347826216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06a4101420 0x7f06a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.901+0000 7f06ab395640 1 --2- 172.21.15.94:0/3347826216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06a4100a80 0x7f06a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.901+0000 7f06ab395640 1 --2- 172.21.15.94:0/3347826216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a4106a80 0x7f06a4106e60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.901+0000 7f06ab395640 1 -- 172.21.15.94:0/3347826216 >> 172.21.15.94:0/3347826216 conn(0x7f06a40fc7a0 msgr2=0x7f06a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:44.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.901+0000 7f06ab395640 1 -- 172.21.15.94:0/3347826216 shutdown_connections 2024-09-19T04:16:44.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.901+0000 7f06ab395640 1 -- 172.21.15.94:0/3347826216 wait complete. 2024-09-19T04:16:44.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.902+0000 7f06ab395640 1 Processor -- start 2024-09-19T04:16:44.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.902+0000 7f06ab395640 1 -- start start 2024-09-19T04:16:44.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06ab395640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06a4100a80 0x7f06a4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06ab395640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a4101420 0x7f06a41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06ab395640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06a4106a80 0x7f06a419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06a8930640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a4101420 0x7f06a41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06a8930640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a4101420 0x7f06a41986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54518/0 (socket says 172.21.15.94:54518) 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06a8930640 1 -- 172.21.15.94:0/2753074816 learned_addr learned my addr 172.21.15.94:0/2753074816 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06a9131640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06a4100a80 0x7f06a4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4110d40 con 0x7f06a4101420 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4110bc0 con 0x7f06a4100a80 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4110ec0 con 0x7f06a4106a80 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06a9932640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06a4106a80 0x7f06a419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06a9131640 1 -- 172.21.15.94:0/2753074816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06a4106a80 msgr2=0x7f06a419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06a9131640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06a4106a80 0x7f06a419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.903+0000 7f06a9131640 1 -- 172.21.15.94:0/2753074816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a4101420 msgr2=0x7f06a41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.904+0000 7f06a9131640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a4101420 0x7f06a41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.904+0000 7f06a9131640 1 -- 172.21.15.94:0/2753074816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a419d140 con 0x7f06a4100a80 2024-09-19T04:16:44.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.904+0000 7f06a9131640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06a4100a80 0x7f06a4198190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06a002aa10 tx=0x7f06a00305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:44.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.904+0000 7f069a7fc640 1 -- 172.21.15.94:0/2753074816 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a003f030 con 0x7f06a4100a80 2024-09-19T04:16:44.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.904+0000 7f069a7fc640 1 -- 172.21.15.94:0/2753074816 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06a0030e00 con 0x7f06a4100a80 2024-09-19T04:16:44.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.904+0000 7f069a7fc640 1 -- 172.21.15.94:0/2753074816 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0035070 con 0x7f06a4100a80 2024-09-19T04:16:44.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.904+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a419d3d0 con 0x7f06a4100a80 2024-09-19T04:16:44.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.905+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a419d7a0 con 0x7f06a4100a80 2024-09-19T04:16:44.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.908+0000 7f069a7fc640 1 -- 172.21.15.94:0/2753074816 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f06a0042030 con 0x7f06a4100a80 2024-09-19T04:16:44.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f06ad76e740 1 Processor -- start 2024-09-19T04:16:44.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f06ad76e740 1 -- start start 2024-09-19T04:16:44.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f069a7fc640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f06900a3480 0x7f06900a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d4dc0 0x55bc761df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc761e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f06ad76e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc7613a2e0 con 0x55bc761d4dc0 2024-09-19T04:16:44.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f06ad76e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc760e1ce0 con 0x55bc761df680 2024-09-19T04:16:44.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f06ad76e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc761362a0 con 0x55bc761d44a0 2024-09-19T04:16:44.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f06a9932640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f06900a3480 0x7f06900a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.909+0000 7f069a7fc640 1 -- 172.21.15.94:0/2753074816 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f06a00f0250 con 0x7f06a4100a80 2024-09-19T04:16:44.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a8930640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d4dc0 0x55bc761df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a9131640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a8930640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d4dc0 0x55bc761df140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54534/0 (socket says 172.21.15.94:54534) 2024-09-19T04:16:44.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a8930640 1 -- 172.21.15.94:0/181067463 learned_addr learned my addr 172.21.15.94:0/181067463 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:44.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a9932640 1 --2- 172.21.15.94:0/181067463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc761e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a8930640 1 -- 172.21.15.94:0/181067463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 msgr2=0x55bc761d4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:44.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a8930640 1 --2- 172.21.15.94:0/181067463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761d4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a8930640 1 -- 172.21.15.94:0/181067463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 msgr2=0x55bc761e1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:44.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a8930640 1 --2- 172.21.15.94:0/181067463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc761e1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a9932640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f06900a3480 0x7f06900a5940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f069c009fd0 tx=0x7f069c009d80 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:44.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.910+0000 7f06a8930640 1 -- 172.21.15.94:0/181067463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc760c6440 con 0x55bc761d4dc0 2024-09-19T04:16:44.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.911+0000 7f06a8930640 1 --2- 172.21.15.94:0/181067463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d4dc0 0x55bc761df140 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0694009df0 tx=0x7f0694011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3efd72a5eaf11f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:44.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.912+0000 7f068f7fe640 1 -- 172.21.15.94:0/181067463 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069401e070 con 0x55bc761d4dc0 2024-09-19T04:16:44.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.912+0000 7f068f7fe640 1 -- 172.21.15.94:0/181067463 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0694019ae0 con 0x55bc761d4dc0 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.912+0000 7f068f7fe640 1 -- 172.21.15.94:0/181067463 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069400fc40 con 0x55bc761d4dc0 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.913+0000 7f06ad76e740 1 -- 172.21.15.94:0/181067463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d4dc0 msgr2=0x55bc761df140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.913+0000 7f06ad76e740 1 --2- 172.21.15.94:0/181067463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d4dc0 0x55bc761df140 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0694009df0 tx=0x7f0694011810 comp rx=0 tx=0).stop 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.913+0000 7f06ad76e740 1 -- 172.21.15.94:0/181067463 shutdown_connections 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.913+0000 7f06ad76e740 1 --2- 172.21.15.94:0/181067463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc761e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.913+0000 7f06ad76e740 1 --2- 172.21.15.94:0/181067463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d4dc0 0x55bc761df140 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.913+0000 7f06ad76e740 1 --2- 172.21.15.94:0/181067463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761d4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.913+0000 7f06ad76e740 1 -- 172.21.15.94:0/181067463 >> 172.21.15.94:0/181067463 conn(0x55bc761d0a30 msgr2=0x55bc761d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.913+0000 7f06ad76e740 1 -- 172.21.15.94:0/181067463 shutdown_connections 2024-09-19T04:16:44.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.913+0000 7f06ad76e740 1 -- 172.21.15.94:0/181067463 wait complete. 2024-09-19T04:16:44.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.914+0000 7f06ad76e740 1 -- start start 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.914+0000 7f06ad76e740 1 Processor -- start 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.914+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.914+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 0x55bc761dc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.914+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc761dc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.914+0000 7f06a9131640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.914+0000 7f06a8930640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 0x55bc761dc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.914+0000 7f06a9131640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761def50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54550/0 (socket says 172.21.15.94:54550) 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06a9131640 1 -- 172.21.15.94:0/2497736166 learned_addr learned my addr 172.21.15.94:0/2497736166 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc761e22f0 con 0x55bc761d44a0 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc760f5050 con 0x55bc761df680 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc761e2470 con 0x55bc761d4dc0 2024-09-19T04:16:44.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06ad76e740 10 client.0 _ll_register_callbacks cb 0x55bc76135160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06a9932640 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc761dc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06a8930640 1 -- 172.21.15.94:0/2497736166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 msgr2=0x55bc761dc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06a8930640 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc761dc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06a8930640 1 -- 172.21.15.94:0/2497736166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 msgr2=0x55bc761def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06a8930640 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06a8930640 1 -- 172.21.15.94:0/2497736166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc75eab3c0 con 0x55bc761d4dc0 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06a9131640 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f06a8930640 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 0x55bc761dc200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f069401c040 tx=0x7f06940044c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb1ceca9837a8654 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f068cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:16:44.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.915+0000 7f068cff9640 1 -- 172.21.15.94:0/2497736166 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694027020 con 0x55bc761d4dc0 2024-09-19T04:16:44.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.916+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0694019ae0 con 0x55bc761d4dc0 2024-09-19T04:16:44.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.916+0000 7f068cff9640 1 -- 172.21.15.94:0/2497736166 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f0694019630 con 0x55bc761d4dc0 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.916+0000 7f068cff9640 1 -- 172.21.15.94:0/2497736166 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0694032050 con 0x55bc761d4dc0 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.916+0000 7f06ad76e740 10 client.25132 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.916+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc761edc70 con 0x55bc761d4dc0 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.916+0000 7f068cff9640 1 -- 172.21.15.94:0/2497736166 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0694019ae0 con 0x55bc761d4dc0 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.917+0000 7f068cff9640 1 -- 172.21.15.94:0/2497736166 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f069400f8d0 con 0x55bc761d4dc0 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.917+0000 7f06ad76e740 10 client.25132 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.917+0000 7f06ad76e740 10 client.25132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.917+0000 7f06ad76e740 20 client.25132 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.917+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc761ef080 0x55bc761f1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.917+0000 7f06ad76e740 4 client.25132 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.917+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc76147850 con 0x55bc761ef080 2024-09-19T04:16:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.918+0000 7f06a9131640 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc761ef080 0x55bc761f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.918+0000 7f068cff9640 10 client.25132 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.918+0000 7f06a9131640 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc761ef080 0x55bc761f1470 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d34e0d66856ebc5d server_cookie=a8bdd6a0ebb727d1 in_seq=0 out_seq=0 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.919+0000 7f068cff9640 1 -- 172.21.15.94:0/2497736166 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc76147850 con 0x55bc761ef080 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.919+0000 7f068cff9640 10 client.25132 handle_command_reply: tid=0 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.919+0000 7f06ad76e740 1 client.25132 shutdown 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.919+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 msgr2=0x55bc761dc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.919+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 0x55bc761dc200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f069401c040 tx=0x7f06940044c0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.919+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 shutdown_connections 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.919+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc761ef080 0x55bc761f1470 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.920+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc761dc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.920+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 0x55bc761dc200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.920+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2497736166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.920+0000 7f068cff9640 0 client.25132 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:44.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.920+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 >> 172.21.15.94:0/2497736166 conn(0x55bc761d0a30 msgr2=0x55bc761cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:44.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.920+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 shutdown_connections 2024-09-19T04:16:44.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.920+0000 7f06ad76e740 1 -- 172.21.15.94:0/2497736166 wait complete. 2024-09-19T04:16:44.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.920+0000 7f06ad76e740 20 client.25132 trim_cache size 0 max 16384 2024-09-19T04:16:44.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06ad76e740 1 Processor -- start 2024-09-19T04:16:44.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06ad76e740 1 -- start start 2024-09-19T04:16:44.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761d4dc0 0x55bc761f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761df680 0x55bc761f7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06ad76e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41b67b0 con 0x55bc761df680 2024-09-19T04:16:44.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06ad76e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41b3ff0 con 0x55bc761d4dc0 2024-09-19T04:16:44.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06ad76e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41b1e40 con 0x55bc761d44a0 2024-09-19T04:16:44.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06a9131640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06a9131640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761eb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:33092/0 (socket says 172.21.15.94:33092) 2024-09-19T04:16:44.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06a9131640 1 -- 172.21.15.94:0/2532698720 learned_addr learned my addr 172.21.15.94:0/2532698720 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:44.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06a8930640 1 --2- 172.21.15.94:0/2532698720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761d4dc0 0x55bc761f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06a9932640 1 --2- 172.21.15.94:0/2532698720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761df680 0x55bc761f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06a9131640 1 -- 172.21.15.94:0/2532698720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761d4dc0 msgr2=0x55bc761f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:44.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06a9131640 1 --2- 172.21.15.94:0/2532698720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761d4dc0 0x55bc761f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06a9131640 1 -- 172.21.15.94:0/2532698720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761df680 msgr2=0x55bc761f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:44.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.950+0000 7f06a9131640 1 --2- 172.21.15.94:0/2532698720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761df680 0x55bc761f7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.951+0000 7f06a9131640 1 -- 172.21.15.94:0/2532698720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc761db1b0 con 0x55bc761d44a0 2024-09-19T04:16:44.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.951+0000 7f06a9131640 1 --2- 172.21.15.94:0/2532698720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761eb5a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b89a0 tx=0x7f06a000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd316ab919e8e771 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:44.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.951+0000 7f068effd640 1 -- 172.21.15.94:0/2532698720 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a00d5070 con 0x55bc761d44a0 2024-09-19T04:16:44.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.951+0000 7f068effd640 1 -- 172.21.15.94:0/2532698720 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06a000b0b0 con 0x55bc761d44a0 2024-09-19T04:16:44.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.951+0000 7f068effd640 1 -- 172.21.15.94:0/2532698720 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a00a5070 con 0x55bc761d44a0 2024-09-19T04:16:44.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.952+0000 7f06ad76e740 1 -- 172.21.15.94:0/2532698720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 msgr2=0x55bc761eb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.952+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2532698720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761eb5a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b89a0 tx=0x7f06a000b040 comp rx=0 tx=0).stop 2024-09-19T04:16:44.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.952+0000 7f06ad76e740 1 -- 172.21.15.94:0/2532698720 shutdown_connections 2024-09-19T04:16:44.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.952+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2532698720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761df680 0x55bc761f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.952+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2532698720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761d4dc0 0x55bc761f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.952+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2532698720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d44a0 0x55bc761eb5a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.952+0000 7f06ad76e740 1 -- 172.21.15.94:0/2532698720 >> 172.21.15.94:0/2532698720 conn(0x55bc761d0a30 msgr2=0x55bc761d70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.953+0000 7f06ad76e740 1 -- 172.21.15.94:0/2532698720 shutdown_connections 2024-09-19T04:16:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.953+0000 7f06ad76e740 1 -- 172.21.15.94:0/2532698720 wait complete. 2024-09-19T04:16:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.953+0000 7f06ad76e740 1 -- start start 2024-09-19T04:16:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.954+0000 7f06ad76e740 1 Processor -- start 2024-09-19T04:16:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.954+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761e2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 0x55bc761e3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06ad76e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc76251bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06ad76e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41b24b0 con 0x55bc761d44a0 2024-09-19T04:16:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06ad76e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41b3920 con 0x55bc761df680 2024-09-19T04:16:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06ad76e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41b2b80 con 0x55bc761d4dc0 2024-09-19T04:16:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06ad76e740 10 client.0 _ll_register_callbacks cb 0x7f069401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06a9932640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc76251bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06a9932640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc76251bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50526/0 (socket says 172.21.15.94:50526) 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06a9932640 1 -- 172.21.15.94:0/2226703686 learned_addr learned my addr 172.21.15.94:0/2226703686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06a9131640 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761e2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06a8930640 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 0x55bc761e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.955+0000 7f06a9932640 1 -- 172.21.15.94:0/2226703686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 msgr2=0x55bc761e3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.956+0000 7f06a9932640 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 0x55bc761e3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.956+0000 7f06a9932640 1 -- 172.21.15.94:0/2226703686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 msgr2=0x55bc761e2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.956+0000 7f06a9932640 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761e2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.956+0000 7f06a9932640 1 -- 172.21.15.94:0/2226703686 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc762522c0 con 0x55bc761df680 2024-09-19T04:16:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.956+0000 7f06a9932640 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc76251bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f069c013380 tx=0x7f069c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483f7a685dfdeac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.956+0000 7f068cff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:16:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.956+0000 7f068cff9640 1 -- 172.21.15.94:0/2226703686 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c040060 con 0x55bc761df680 2024-09-19T04:16:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.956+0000 7f068cff9640 1 -- 172.21.15.94:0/2226703686 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f069c01c040 con 0x55bc761df680 2024-09-19T04:16:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.956+0000 7f06ad76e740 1 -- 172.21.15.94:0/2226703686 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06a000b0b0 con 0x55bc761df680 2024-09-19T04:16:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.957+0000 7f068cff9640 1 -- 172.21.15.94:0/2226703686 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f069c003b80 con 0x55bc761df680 2024-09-19T04:16:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.957+0000 7f06ad76e740 10 client.25184 fetch_fsmap learned FSMap version 30 2024-09-19T04:16:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.957+0000 7f06ad76e740 1 -- 172.21.15.94:0/2226703686 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc762595c0 con 0x55bc761df680 2024-09-19T04:16:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.957+0000 7f068cff9640 1 -- 172.21.15.94:0/2226703686 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06a000b0b0 con 0x55bc761df680 2024-09-19T04:16:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.958+0000 7f068cff9640 1 -- 172.21.15.94:0/2226703686 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f069c03a030 con 0x55bc761df680 2024-09-19T04:16:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.958+0000 7f06ad76e740 10 client.25184 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:16:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.958+0000 7f06ad76e740 10 client.25184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:16:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.958+0000 7f06ad76e740 20 client.25184 populate_metadata read hostname 'smithi094' 2024-09-19T04:16:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.958+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc76259d20 0x55bc7625c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.958+0000 7f06ad76e740 4 client.25184 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:16:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.958+0000 7f06ad76e740 1 -- 172.21.15.94:0/2226703686 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc76141d60 con 0x55bc76259d20 2024-09-19T04:16:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.959+0000 7f06a9131640 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc76259d20 0x55bc7625c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.959+0000 7f06a9131640 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc76259d20 0x55bc7625c140 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e2fcced8fd6934 server_cookie=e49552187c300130 in_seq=0 out_seq=0 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.959+0000 7f068cff9640 10 client.25184 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.959+0000 7f068cff9640 1 -- 172.21.15.94:0/2226703686 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bc76141d60 con 0x55bc76259d20 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.959+0000 7f068cff9640 10 client.25184 handle_command_reply: tid=0 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.959+0000 7f06ad76e740 1 client.25184 shutdown 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f06ad76e740 1 -- 172.21.15.94:0/2226703686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 msgr2=0x55bc76251bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc76251bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f069c013380 tx=0x7f069c004ab0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f06ad76e740 1 -- 172.21.15.94:0/2226703686 shutdown_connections 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc76259d20 0x55bc7625c140 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc761df680 0x55bc76251bd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc761d4dc0 0x55bc761e3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f06ad76e740 1 --2- 172.21.15.94:0/2226703686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc761d44a0 0x55bc761e2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f06ad76e740 1 -- 172.21.15.94:0/2226703686 >> 172.21.15.94:0/2226703686 conn(0x55bc761d0a30 msgr2=0x55bc761de280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f068cff9640 0 client.25184 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.960+0000 7f06ad76e740 1 -- 172.21.15.94:0/2226703686 shutdown_connections 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.961+0000 7f06ad76e740 1 -- 172.21.15.94:0/2226703686 wait complete. 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.961+0000 7f06ad76e740 20 client.25184 trim_cache size 0 max 16384 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stdout: "f7d480a3-2049-430d-9251-7143138b8ae7": { 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "f7d480a3-2049-430d-9251-7143138b8ae7", 2024-09-19T04:16:44.967 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:16:44.968 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:16:44.968 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:16:44.968 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:16:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.961+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f06900a3480 msgr2=0x7f06900a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.961+0000 7f06ab395640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f06900a3480 0x7f06900a5940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f069c009fd0 tx=0x7f069c009d80 comp rx=0 tx=0).stop 2024-09-19T04:16:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.961+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06a4100a80 msgr2=0x7f06a4198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.961+0000 7f06ab395640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06a4100a80 0x7f06a4198190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06a002aa10 tx=0x7f06a00305d0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.962+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 shutdown_connections 2024-09-19T04:16:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.962+0000 7f06ab395640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f06900a3480 0x7f06900a5940 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.962+0000 7f06ab395640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06a4106a80 0x7f06a419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.962+0000 7f06ab395640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a4101420 0x7f06a41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.962+0000 7f06ab395640 1 --2- 172.21.15.94:0/2753074816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06a4100a80 0x7f06a4198190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.962+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 >> 172.21.15.94:0/2753074816 conn(0x7f06a40fc7a0 msgr2=0x7f06a4104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:44.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.962+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 shutdown_connections 2024-09-19T04:16:44.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:16:44.962+0000 7f06ab395640 1 -- 172.21.15.94:0/2753074816 wait complete. 2024-09-19T04:16:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 30.0 s, 1466.6 tps, lat 66.757 ms stddev 19.560 2024-09-19T04:16:45.753 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:45 smithi094 ceph-mon[30972]: pgmap v55: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:16:45.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:45 smithi112 ceph-mon[35051]: pgmap v55: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:16:45.801 INFO:tasks.cephfs.filesystem:scrub status for tag:f7d480a3-2049-430d-9251-7143138b8ae7 - {'path': '/', 'tag': 'f7d480a3-2049-430d-9251-7143138b8ae7', 'options': 'recursive,force'} 2024-09-19T04:16:45.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:45 smithi104 ceph-mon[34145]: pgmap v55: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:16:47.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:47 smithi112 ceph-mon[35051]: pgmap v56: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 2.22k op/s 2024-09-19T04:16:47.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:47 smithi094 ceph-mon[30972]: pgmap v56: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 2.22k op/s 2024-09-19T04:16:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:47 smithi104 ceph-mon[34145]: pgmap v56: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 2.22k op/s 2024-09-19T04:16:49.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:49 smithi112 ceph-mon[35051]: pgmap v57: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.57k op/s 2024-09-19T04:16:49.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:49 smithi094 ceph-mon[30972]: pgmap v57: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.57k op/s 2024-09-19T04:16:49.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:49 smithi104 ceph-mon[34145]: pgmap v57: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.57k op/s 2024-09-19T04:16:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 35.0 s, 1516.2 tps, lat 67.378 ms stddev 331.831 2024-09-19T04:16:51.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:51 smithi112 ceph-mon[35051]: pgmap v58: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.57k op/s 2024-09-19T04:16:51.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:51 smithi094 ceph-mon[30972]: pgmap v58: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.57k op/s 2024-09-19T04:16:51.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:51 smithi104 ceph-mon[34145]: pgmap v58: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.57k op/s 2024-09-19T04:16:52.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:52 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:16:52.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:52 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:16:52.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:52 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:16:53.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:53 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:16:53.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:53 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:16:53.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:53 smithi112 ceph-mon[35051]: pgmap v59: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 2.45k op/s 2024-09-19T04:16:53.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:53 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:16:53.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:53 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:16:53.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:53 smithi094 ceph-mon[30972]: pgmap v59: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 2.45k op/s 2024-09-19T04:16:53.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:53 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:16:53.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:53 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:16:53.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:53 smithi104 ceph-mon[34145]: pgmap v59: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 2.45k op/s 2024-09-19T04:16:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 40.0 s, 1688.8 tps, lat 59.131 ms stddev 15.401 2024-09-19T04:16:55.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:55 smithi112 ceph-mon[35051]: pgmap v60: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-09-19T04:16:55.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:55 smithi094 ceph-mon[30972]: pgmap v60: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-09-19T04:16:55.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:55 smithi104 ceph-mon[34145]: pgmap v60: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-09-19T04:16:57.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:57 smithi112 ceph-mon[35051]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 2.43k op/s 2024-09-19T04:16:57.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:57 smithi094 ceph-mon[30972]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 2.43k op/s 2024-09-19T04:16:57.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:57 smithi104 ceph-mon[34145]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 2.43k op/s 2024-09-19T04:16:59.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:16:59 smithi112 ceph-mon[35051]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 1.64k op/s 2024-09-19T04:16:59.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:16:59 smithi094 ceph-mon[30972]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 1.64k op/s 2024-09-19T04:16:59.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:16:59 smithi104 ceph-mon[34145]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 1.64k op/s 2024-09-19T04:17:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 45.0 s, 1533.6 tps, lat 65.202 ms stddev 56.622 2024-09-19T04:17:01.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:01 smithi094 ceph-mon[30972]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 1.64k op/s 2024-09-19T04:17:01.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:01 smithi112 ceph-mon[35051]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 1.64k op/s 2024-09-19T04:17:01.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:01 smithi104 ceph-mon[34145]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 1.64k op/s 2024-09-19T04:17:03.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:03 smithi112 ceph-mon[35051]: pgmap v64: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 2.48k op/s 2024-09-19T04:17:03.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:03 smithi094 ceph-mon[30972]: pgmap v64: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 2.48k op/s 2024-09-19T04:17:03.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:03 smithi104 ceph-mon[34145]: pgmap v64: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 2.48k op/s 2024-09-19T04:17:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 50.0 s, 1723.6 tps, lat 58.001 ms stddev 16.771 2024-09-19T04:17:05.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:05 smithi112 ceph-mon[35051]: pgmap v65: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.60k op/s 2024-09-19T04:17:05.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:05 smithi094 ceph-mon[30972]: pgmap v65: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.60k op/s 2024-09-19T04:17:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:05 smithi104 ceph-mon[34145]: pgmap v65: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.60k op/s 2024-09-19T04:17:07.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:07 smithi112 ceph-mon[35051]: pgmap v66: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 22 MiB/s wr, 2.44k op/s 2024-09-19T04:17:07.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:07 smithi094 ceph-mon[30972]: pgmap v66: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 22 MiB/s wr, 2.44k op/s 2024-09-19T04:17:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:07 smithi104 ceph-mon[34145]: pgmap v66: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 22 MiB/s wr, 2.44k op/s 2024-09-19T04:17:09.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:09 smithi112 ceph-mon[35051]: pgmap v67: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-09-19T04:17:09.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:09 smithi094 ceph-mon[30972]: pgmap v67: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-09-19T04:17:09.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:09 smithi104 ceph-mon[34145]: pgmap v67: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-09-19T04:17:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 55.0 s, 1675.8 tps, lat 59.667 ms stddev 15.517 2024-09-19T04:17:11.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:11 smithi112 ceph-mon[35051]: pgmap v68: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-09-19T04:17:11.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:11 smithi094 ceph-mon[30972]: pgmap v68: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-09-19T04:17:11.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:11 smithi104 ceph-mon[34145]: pgmap v68: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-09-19T04:17:12.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:12 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:12.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:12 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:12.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:12 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 22 MiB/s wr, 2.48k op/s 2024-09-19T04:17:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:14.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:13 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 22 MiB/s wr, 2.48k op/s 2024-09-19T04:17:14.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:14.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:14.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:13 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 22 MiB/s wr, 2.48k op/s 2024-09-19T04:17:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:14.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:14.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:13 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:17:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 60.0 s, 1660.0 tps, lat 60.215 ms stddev 15.341 2024-09-19T04:17:15.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:15 smithi112 ceph-mon[35051]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:17:15.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:17:15.839 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:15 smithi094 ceph-mon[30972]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:17:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:15 smithi104 ceph-mon[34145]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:17:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:16.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.784+0000 7f6129d94640 1 -- 172.21.15.94:0/709019378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6124106ac0 msgr2=0x7f6124106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.784+0000 7f6129d94640 1 --2- 172.21.15.94:0/709019378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6124106ac0 0x7f6124106ea0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6118009860 tx=0x7f611802a340 comp rx=0 tx=0).stop 2024-09-19T04:17:16.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.785+0000 7f6129d94640 1 -- 172.21.15.94:0/709019378 shutdown_connections 2024-09-19T04:17:16.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.785+0000 7f6129d94640 1 --2- 172.21.15.94:0/709019378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6124100970 0x7f6124105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.785+0000 7f6129d94640 1 --2- 172.21.15.94:0/709019378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61240ffed0 0x7f6124100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.785+0000 7f6129d94640 1 --2- 172.21.15.94:0/709019378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6124106ac0 0x7f6124106ea0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.785+0000 7f6129d94640 1 -- 172.21.15.94:0/709019378 >> 172.21.15.94:0/709019378 conn(0x7f6124077010 msgr2=0x7f61240fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.785+0000 7f6129d94640 1 -- 172.21.15.94:0/709019378 shutdown_connections 2024-09-19T04:17:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.785+0000 7f6129d94640 1 -- 172.21.15.94:0/709019378 wait complete. 2024-09-19T04:17:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6129d94640 1 Processor -- start 2024-09-19T04:17:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6129d94640 1 -- start start 2024-09-19T04:17:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6129d94640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61240ffed0 0x7f61241a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6129d94640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6124100970 0x7f61241a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6129d94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6124106ac0 0x7f612419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6129d94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6124118320 con 0x7f6124106ac0 2024-09-19T04:17:16.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6129d94640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f61241181a0 con 0x7f6124100970 2024-09-19T04:17:16.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6129d94640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61241184a0 con 0x7f61240ffed0 2024-09-19T04:17:16.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6128d92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61240ffed0 0x7f61241a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6129593640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6124106ac0 0x7f612419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6128d92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61240ffed0 0x7f61241a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58550/0 (socket says 172.21.15.94:58550) 2024-09-19T04:17:16.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.786+0000 7f6128d92640 1 -- 172.21.15.94:0/96639795 learned_addr learned my addr 172.21.15.94:0/96639795 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:16.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.787+0000 7f6123fff640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6124100970 0x7f61241a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.787+0000 7f6129593640 1 -- 172.21.15.94:0/96639795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61240ffed0 msgr2=0x7f61241a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.787+0000 7f6129593640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61240ffed0 0x7f61241a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.787+0000 7f6129593640 1 -- 172.21.15.94:0/96639795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6124100970 msgr2=0x7f61241a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.787+0000 7f6129593640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6124100970 0x7f61241a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.787+0000 7f6129593640 1 -- 172.21.15.94:0/96639795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612419eec0 con 0x7f6124106ac0 2024-09-19T04:17:16.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.787+0000 7f6123fff640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6124100970 0x7f61241a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:16.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.787+0000 7f6128d92640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61240ffed0 0x7f61241a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:16.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.788+0000 7f6129593640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6124106ac0 0x7f612419e650 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f611c00ecb0 tx=0x7f611c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.788+0000 7f6121ffb640 1 -- 172.21.15.94:0/96639795 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c004b20 con 0x7f6124106ac0 2024-09-19T04:17:16.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.788+0000 7f6129d94640 1 -- 172.21.15.94:0/96639795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612419f1b0 con 0x7f6124106ac0 2024-09-19T04:17:16.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.788+0000 7f6121ffb640 1 -- 172.21.15.94:0/96639795 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f611c004cc0 con 0x7f6124106ac0 2024-09-19T04:17:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.789+0000 7f6129d94640 1 -- 172.21.15.94:0/96639795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241ab500 con 0x7f6124106ac0 2024-09-19T04:17:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.789+0000 7f6121ffb640 1 -- 172.21.15.94:0/96639795 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c010690 con 0x7f6124106ac0 2024-09-19T04:17:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.792+0000 7f6121ffb640 1 -- 172.21.15.94:0/96639795 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f611c005020 con 0x7f6124106ac0 2024-09-19T04:17:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.793+0000 7f612d3cf740 1 Processor -- start 2024-09-19T04:17:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.793+0000 7f612d3cf740 1 -- start start 2024-09-19T04:17:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.794+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde7a670 0x560ecde7aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.794+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde85310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.794+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 0x560ecde87c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.794+0000 7f612d3cf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ecdde02e0 con 0x560ecde85850 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.794+0000 7f612d3cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560ecdd87ce0 con 0x560ecde7a670 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.794+0000 7f612d3cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560ecdddc2a0 con 0x560ecde7af90 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.794+0000 7f6121ffb640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f61140a3310 0x7f61140a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde7a670 0x560ecde7aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde7a670 0x560ecde7aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:35000/0 (socket says 172.21.15.94:35000) 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6129593640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f61140a3310 0x7f61140a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 -- 172.21.15.94:0/3081553692 learned_addr learned my addr 172.21.15.94:0/3081553692 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 --2- 172.21.15.94:0/3081553692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 0x560ecde87c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6128d92640 1 --2- 172.21.15.94:0/3081553692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde85310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6121ffb640 1 -- 172.21.15.94:0/96639795 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f611c0d0400 con 0x7f6124106ac0 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 -- 172.21.15.94:0/3081553692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 msgr2=0x560ecde85310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 --2- 172.21.15.94:0/3081553692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde85310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 -- 172.21.15.94:0/3081553692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 msgr2=0x560ecde87c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 --2- 172.21.15.94:0/3081553692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 0x560ecde87c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 -- 172.21.15.94:0/3081553692 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ecdd6c440 con 0x560ecde7a670 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6123fff640 1 --2- 172.21.15.94:0/3081553692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde7a670 0x560ecde7aa50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6110014930 tx=0x7f6110009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1ff06a8544fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.795+0000 7f6129593640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f61140a3310 0x7f61140a57d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f611c098b30 tx=0x7f611c0c74a0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.796+0000 7f610effd640 1 -- 172.21.15.94:0/3081553692 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100142d0 con 0x560ecde7a670 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.796+0000 7f610effd640 1 -- 172.21.15.94:0/3081553692 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f611000f760 con 0x560ecde7a670 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.796+0000 7f610effd640 1 -- 172.21.15.94:0/3081553692 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100044e0 con 0x560ecde7a670 2024-09-19T04:17:16.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.797+0000 7f612d3cf740 1 -- 172.21.15.94:0/3081553692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde7a670 msgr2=0x560ecde7aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.797+0000 7f612d3cf740 1 --2- 172.21.15.94:0/3081553692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde7a670 0x560ecde7aa50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6110014930 tx=0x7f6110009660 comp rx=0 tx=0).stop 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.797+0000 7f612d3cf740 1 -- 172.21.15.94:0/3081553692 shutdown_connections 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.797+0000 7f612d3cf740 1 --2- 172.21.15.94:0/3081553692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 0x560ecde87c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.797+0000 7f612d3cf740 1 --2- 172.21.15.94:0/3081553692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde85310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.797+0000 7f612d3cf740 1 --2- 172.21.15.94:0/3081553692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde7a670 0x560ecde7aa50 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6110014930 tx=0x7f6110009660 comp rx=0 tx=0).stop 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.797+0000 7f612d3cf740 1 -- 172.21.15.94:0/3081553692 >> 172.21.15.94:0/3081553692 conn(0x560ecde76c00 msgr2=0x560ecde76fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.797+0000 7f612d3cf740 1 -- 172.21.15.94:0/3081553692 shutdown_connections 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.798+0000 7f612d3cf740 1 -- 172.21.15.94:0/3081553692 wait complete. 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.798+0000 7f612d3cf740 1 -- start start 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.798+0000 7f612d3cf740 1 Processor -- start 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 0x560ecde82690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde85850 0x560ecde82bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 0x560ecdea28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f612d3cf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ecde993d0 con 0x560ecde7af90 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f612d3cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560ecdd9b050 con 0x560ecde83110 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f612d3cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560ecde99550 con 0x560ecde85850 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f612d3cf740 10 client.0 _ll_register_callbacks cb 0x560ecdddb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f6128d92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde85850 0x560ecde82bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f6128d92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde85850 0x560ecde82bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58558/0 (socket says 172.21.15.94:58558) 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f6128d92640 1 -- 172.21.15.94:0/1656401929 learned_addr learned my addr 172.21.15.94:0/1656401929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.799+0000 7f6123fff640 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 0x560ecdea28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.800+0000 7f6128d92640 1 -- 172.21.15.94:0/1656401929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 msgr2=0x560ecdea28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.800+0000 7f6128d92640 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 0x560ecdea28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.800+0000 7f6128d92640 1 -- 172.21.15.94:0/1656401929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 msgr2=0x560ecde82690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.800+0000 7f6128d92640 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 0x560ecde82690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.800+0000 7f6128d92640 1 -- 172.21.15.94:0/1656401929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ecdb513c0 con 0x560ecde85850 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.800+0000 7f6123fff640 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 0x560ecdea28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.800+0000 7f6128d92640 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde85850 0x560ecde82bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6118009980 tx=0x7f6118004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4af4d4fef3c56232 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.800+0000 7f6103fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.801+0000 7f6103fff640 1 -- 172.21.15.94:0/1656401929 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118037040 con 0x560ecde85850 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.801+0000 7f612d3cf740 1 -- 172.21.15.94:0/1656401929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611000f760 con 0x560ecde85850 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.801+0000 7f6103fff640 1 -- 172.21.15.94:0/1656401929 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f6118030650 con 0x560ecde85850 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.801+0000 7f6103fff640 1 -- 172.21.15.94:0/1656401929 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6118030850 con 0x560ecde85850 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.801+0000 7f612d3cf740 10 client.25150 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.801+0000 7f612d3cf740 1 -- 172.21.15.94:0/1656401929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ecdea5450 con 0x560ecde85850 2024-09-19T04:17:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.801+0000 7f6103fff640 1 -- 172.21.15.94:0/1656401929 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f611000f760 con 0x560ecde85850 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.801+0000 7f6103fff640 1 -- 172.21.15.94:0/1656401929 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f61180038f0 con 0x560ecde85850 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.802+0000 7f612d3cf740 10 client.25150 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.802+0000 7f612d3cf740 10 client.25150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.802+0000 7f612d3cf740 20 client.25150 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.802+0000 7f612d3cf740 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x560ecdea6cb0 0x560ecdea9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.802+0000 7f612d3cf740 4 client.25150 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.802+0000 7f612d3cf740 1 -- 172.21.15.94:0/1656401929 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ecdded850 con 0x560ecdea6cb0 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.802+0000 7f6123fff640 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x560ecdea6cb0 0x560ecdea9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.803+0000 7f6123fff640 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x560ecdea6cb0 0x560ecdea9170 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4078c75770ec6ce0 server_cookie=f5e8c325b128a95d in_seq=0 out_seq=0 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.803+0000 7f6103fff640 10 client.25150 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f6103fff640 1 -- 172.21.15.94:0/1656401929 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ecdded850 con 0x560ecdea6cb0 2024-09-19T04:17:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f6103fff640 10 client.25150 handle_command_reply: tid=0 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f612d3cf740 1 client.25150 shutdown 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f612d3cf740 1 -- 172.21.15.94:0/1656401929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde85850 msgr2=0x560ecde82bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f612d3cf740 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde85850 0x560ecde82bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6118009980 tx=0x7f6118004250 comp rx=0 tx=0).stop 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f612d3cf740 1 -- 172.21.15.94:0/1656401929 shutdown_connections 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f612d3cf740 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x560ecdea6cb0 0x560ecdea9170 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f612d3cf740 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 0x560ecdea28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f612d3cf740 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde85850 0x560ecde82bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f612d3cf740 1 --2- 172.21.15.94:0/1656401929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 0x560ecde82690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f612d3cf740 1 -- 172.21.15.94:0/1656401929 >> 172.21.15.94:0/1656401929 conn(0x560ecde76c00 msgr2=0x560ecde75d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.804+0000 7f6103fff640 0 client.25150 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.805+0000 7f612d3cf740 1 -- 172.21.15.94:0/1656401929 shutdown_connections 2024-09-19T04:17:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.805+0000 7f612d3cf740 1 -- 172.21.15.94:0/1656401929 wait complete. 2024-09-19T04:17:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.805+0000 7f612d3cf740 20 client.25150 trim_cache size 0 max 16384 2024-09-19T04:17:16.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.837+0000 7f612d3cf740 1 Processor -- start 2024-09-19T04:17:16.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.838+0000 7f612d3cf740 1 -- start start 2024-09-19T04:17:16.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.838+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 0x560ecdea2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.838+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde83110 0x560ecdea2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.838+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde85850 0x560ecdeaea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.838+0000 7f612d3cf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ad9d0 con 0x560ecde7af90 2024-09-19T04:17:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.838+0000 7f612d3cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6124110b40 con 0x560ecde85850 2024-09-19T04:17:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.838+0000 7f612d3cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61241010f0 con 0x560ecde83110 2024-09-19T04:17:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6123fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 0x560ecdea2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6123fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 0x560ecdea2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53878/0 (socket says 172.21.15.94:53878) 2024-09-19T04:17:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6123fff640 1 -- 172.21.15.94:0/3678398165 learned_addr learned my addr 172.21.15.94:0/3678398165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6123fff640 1 --2- 172.21.15.94:0/3678398165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde85850 0x560ecdeaea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6128d92640 1 --2- 172.21.15.94:0/3678398165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde83110 0x560ecdea2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6123fff640 1 -- 172.21.15.94:0/3678398165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde83110 msgr2=0x560ecdea2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6123fff640 1 --2- 172.21.15.94:0/3678398165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde83110 0x560ecdea2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6123fff640 1 -- 172.21.15.94:0/3678398165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 msgr2=0x560ecdea2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6123fff640 1 --2- 172.21.15.94:0/3678398165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 0x560ecdea2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.839+0000 7f6123fff640 1 -- 172.21.15.94:0/3678398165 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ecde7b810 con 0x560ecde85850 2024-09-19T04:17:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.840+0000 7f6123fff640 1 --2- 172.21.15.94:0/3678398165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde85850 0x560ecdeaea80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6110009e40 tx=0x7f6110023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=977c264ed17fe6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.840+0000 7f610e7fc640 1 -- 172.21.15.94:0/3678398165 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100050a0 con 0x560ecde85850 2024-09-19T04:17:16.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.840+0000 7f610e7fc640 1 -- 172.21.15.94:0/3678398165 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f611000f730 con 0x560ecde85850 2024-09-19T04:17:16.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.840+0000 7f610e7fc640 1 -- 172.21.15.94:0/3678398165 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110003870 con 0x560ecde85850 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.841+0000 7f612d3cf740 1 -- 172.21.15.94:0/3678398165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde85850 msgr2=0x560ecdeaea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.841+0000 7f612d3cf740 1 --2- 172.21.15.94:0/3678398165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde85850 0x560ecdeaea80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6110009e40 tx=0x7f6110023040 comp rx=0 tx=0).stop 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.841+0000 7f612d3cf740 1 -- 172.21.15.94:0/3678398165 shutdown_connections 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.841+0000 7f612d3cf740 1 --2- 172.21.15.94:0/3678398165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde85850 0x560ecdeaea80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.841+0000 7f612d3cf740 1 --2- 172.21.15.94:0/3678398165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde83110 0x560ecdea2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.841+0000 7f612d3cf740 1 --2- 172.21.15.94:0/3678398165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde7af90 0x560ecdea2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.841+0000 7f612d3cf740 1 -- 172.21.15.94:0/3678398165 >> 172.21.15.94:0/3678398165 conn(0x560ecde76c00 msgr2=0x560ecde84710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.842+0000 7f612d3cf740 1 -- 172.21.15.94:0/3678398165 shutdown_connections 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.842+0000 7f612d3cf740 1 -- 172.21.15.94:0/3678398165 wait complete. 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.842+0000 7f612d3cf740 1 -- start start 2024-09-19T04:17:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f612d3cf740 1 Processor -- start 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde7ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 0x560ecde9e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f612d3cf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 0x560ecde9ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f612d3cf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f612419fb80 con 0x560ecde85850 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f612d3cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6124110470 con 0x560ecde83110 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f612d3cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f612410f6d0 con 0x560ecde7af90 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f612d3cf740 10 client.0 _ll_register_callbacks cb 0x7f611c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f6123fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde7ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f6123fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde7ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58576/0 (socket says 172.21.15.94:58576) 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f6123fff640 1 -- 172.21.15.94:0/627847272 learned_addr learned my addr 172.21.15.94:0/627847272 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f6123fff640 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 0x560ecde9ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.843+0000 7f6128d92640 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 0x560ecde9e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f6123fff640 1 -- 172.21.15.94:0/627847272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 msgr2=0x560ecde9e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f6123fff640 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 0x560ecde9e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f6123fff640 1 -- 172.21.15.94:0/627847272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 msgr2=0x560ecde9ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f6123fff640 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 0x560ecde9ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f6123fff640 1 -- 172.21.15.94:0/627847272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ecde9f2b0 con 0x560ecde7af90 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f6123fff640 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde7ff10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6110005f70 tx=0x7f6110009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=257038013f195cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f6103fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f6103fff640 1 -- 172.21.15.94:0/627847272 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100052a0 con 0x560ecde7af90 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f6103fff640 1 -- 172.21.15.94:0/627847272 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f6110005440 con 0x560ecde7af90 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.844+0000 7f612d3cf740 1 -- 172.21.15.94:0/627847272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611000f730 con 0x560ecde7af90 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f6103fff640 1 -- 172.21.15.94:0/627847272 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f611000f8e0 con 0x560ecde7af90 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f612d3cf740 10 client.25156 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f612d3cf740 1 -- 172.21.15.94:0/627847272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ecdf10a50 con 0x560ecde7af90 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f6103fff640 1 -- 172.21.15.94:0/627847272 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f611000f730 con 0x560ecde7af90 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f6103fff640 1 -- 172.21.15.94:0/627847272 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6110037040 con 0x560ecde7af90 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f612d3cf740 10 client.25156 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f612d3cf740 10 client.25156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:17:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f612d3cf740 20 client.25156 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f612d3cf740 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x560ecdf10dd0 0x560ecdf131f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f612d3cf740 4 client.25156 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.845+0000 7f612d3cf740 1 -- 172.21.15.94:0/627847272 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ecdde7d60 con 0x560ecdf10dd0 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.846+0000 7f6128d92640 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x560ecdf10dd0 0x560ecdf131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.846+0000 7f6128d92640 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x560ecdf10dd0 0x560ecdf131f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0cd5d6f92427b4 server_cookie=300ddfecfd99b619 in_seq=0 out_seq=0 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.846+0000 7f6103fff640 10 client.25156 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.846+0000 7f6103fff640 1 -- 172.21.15.94:0/627847272 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560ecdde7d60 con 0x560ecdf10dd0 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.846+0000 7f6103fff640 10 client.25156 handle_command_reply: tid=0 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f612d3cf740 1 client.25156 shutdown 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f612d3cf740 1 -- 172.21.15.94:0/627847272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 msgr2=0x560ecde7ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f612d3cf740 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde7ff10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6110005f70 tx=0x7f6110009660 comp rx=0 tx=0).stop 2024-09-19T04:17:16.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f612d3cf740 1 -- 172.21.15.94:0/627847272 shutdown_connections 2024-09-19T04:17:16.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f612d3cf740 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x560ecdf10dd0 0x560ecdf131f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f612d3cf740 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ecde85850 0x560ecde9ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f612d3cf740 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecde83110 0x560ecde9e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f612d3cf740 1 --2- 172.21.15.94:0/627847272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ecde7af90 0x560ecde7ff10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f612d3cf740 1 -- 172.21.15.94:0/627847272 >> 172.21.15.94:0/627847272 conn(0x560ecde76c00 msgr2=0x560ecde75500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.847+0000 7f6103fff640 0 client.25156 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.848+0000 7f612d3cf740 1 -- 172.21.15.94:0/627847272 shutdown_connections 2024-09-19T04:17:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.848+0000 7f612d3cf740 1 -- 172.21.15.94:0/627847272 wait complete. 2024-09-19T04:17:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.848+0000 7f612d3cf740 20 client.25156 trim_cache size 0 max 16384 2024-09-19T04:17:16.853 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:17:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:17:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:17:16.853 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:17:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 -- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f61140a3310 msgr2=0x7f61140a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f61140a3310 0x7f61140a57d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f611c098b30 tx=0x7f611c0c74a0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 -- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6124106ac0 msgr2=0x7f612419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6124106ac0 0x7f612419e650 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f611c00ecb0 tx=0x7f611c00c6a0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 -- 172.21.15.94:0/96639795 shutdown_connections 2024-09-19T04:17:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f61140a3310 0x7f61140a57d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6124106ac0 0x7f612419e650 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6124100970 0x7f61241a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 --2- 172.21.15.94:0/96639795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61240ffed0 0x7f61241a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 -- 172.21.15.94:0/96639795 >> 172.21.15.94:0/96639795 conn(0x7f6124077010 msgr2=0x7f6124103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 -- 172.21.15.94:0/96639795 shutdown_connections 2024-09-19T04:17:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:16.849+0000 7f6129d94640 1 -- 172.21.15.94:0/96639795 wait complete. 2024-09-19T04:17:17.552 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:17 smithi094 ceph-mon[30972]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 2.37k op/s 2024-09-19T04:17:17.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:17 smithi112 ceph-mon[35051]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 2.37k op/s 2024-09-19T04:17:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:17 smithi104 ceph-mon[34145]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 2.37k op/s 2024-09-19T04:17:17.959 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:17:17.960 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:18.421 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.002+0000 7fcb1e01c640 1 -- 172.21.15.94:0/3555031071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb18109830 msgr2=0x7fcb18109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.002+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/3555031071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb18109830 0x7fcb18109c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009900 tx=0x7fcb0c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.002+0000 7fcb1e01c640 1 -- 172.21.15.94:0/3555031071 shutdown_connections 2024-09-19T04:17:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.002+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/3555031071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb1811b140 0x7fcb1811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.002+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/3555031071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb18109830 0x7fcb18109c90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.002+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/3555031071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb18109450 0x7fcb18110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.002+0000 7fcb1e01c640 1 -- 172.21.15.94:0/3555031071 >> 172.21.15.94:0/3555031071 conn(0x7fcb1806e140 msgr2=0x7fcb1806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.003+0000 7fcb1e01c640 1 -- 172.21.15.94:0/3555031071 shutdown_connections 2024-09-19T04:17:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.003+0000 7fcb1e01c640 1 -- 172.21.15.94:0/3555031071 wait complete. 2024-09-19T04:17:19.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.003+0000 7fcb1e01c640 1 Processor -- start 2024-09-19T04:17:19.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb1e01c640 1 -- start start 2024-09-19T04:17:19.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb1e01c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb18109450 0x7fcb18118720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb1e01c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb18109830 0x7fcb181117b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb1e01c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb1811b140 0x7fcb18111cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb1e01c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18120ca0 con 0x7fcb1811b140 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb1e01c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18120b20 con 0x7fcb18109450 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb1e01c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18120e20 con 0x7fcb18109830 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb177fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb18109450 0x7fcb18118720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb177fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb18109450 0x7fcb18118720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:35044/0 (socket says 172.21.15.94:35044) 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb177fe640 1 -- 172.21.15.94:0/1656711628 learned_addr learned my addr 172.21.15.94:0/1656711628 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb17fff640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb1811b140 0x7fcb18111cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.004+0000 7fcb16ffd640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb18109830 0x7fcb181117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.005+0000 7fcb17fff640 1 -- 172.21.15.94:0/1656711628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb18109830 msgr2=0x7fcb181117b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.005+0000 7fcb17fff640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb18109830 0x7fcb181117b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.005+0000 7fcb17fff640 1 -- 172.21.15.94:0/1656711628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb18109450 msgr2=0x7fcb18118720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.005+0000 7fcb17fff640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb18109450 0x7fcb18118720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.005+0000 7fcb17fff640 1 -- 172.21.15.94:0/1656711628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb181122e0 con 0x7fcb1811b140 2024-09-19T04:17:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.005+0000 7fcb17fff640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb1811b140 0x7fcb18111cf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000ec50 tx=0x7fcb1000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.006+0000 7fcb14ff9640 1 -- 172.21.15.94:0/1656711628 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10003b70 con 0x7fcb1811b140 2024-09-19T04:17:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.006+0000 7fcb14ff9640 1 -- 172.21.15.94:0/1656711628 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb1000ee60 con 0x7fcb1811b140 2024-09-19T04:17:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.006+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb181125d0 con 0x7fcb1811b140 2024-09-19T04:17:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.006+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb18116590 con 0x7fcb1811b140 2024-09-19T04:17:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.006+0000 7fcb14ff9640 1 -- 172.21.15.94:0/1656711628 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb100107a0 con 0x7fcb1811b140 2024-09-19T04:17:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.008+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb04005290 con 0x7fcb1811b140 2024-09-19T04:17:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.009+0000 7fcb14ff9640 1 -- 172.21.15.94:0/1656711628 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fcb10004b40 con 0x7fcb1811b140 2024-09-19T04:17:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.011+0000 7fcb14ff9640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcb000a3580 0x7fcb000a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.011+0000 7fcb14ff9640 1 -- 172.21.15.94:0/1656711628 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fcb100d0260 con 0x7fcb1811b140 2024-09-19T04:17:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.011+0000 7fcb177fe640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcb000a3580 0x7fcb000a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:19.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.012+0000 7fcb177fe640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcb000a3580 0x7fcb000a5a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb08005fd0 tx=0x7fcb0800b480 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.016+0000 7fcb14ff9640 1 -- 172.21.15.94:0/1656711628 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcb1009cc50 con 0x7fcb1811b140 2024-09-19T04:17:19.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.393+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb04005580 con 0x7fcb1811b140 2024-09-19T04:17:19.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.394+0000 7fcb14ff9640 1 -- 172.21.15.94:0/1656711628 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11922 (secure 0 0 0) 0x7fcb1009c550 con 0x7fcb1811b140 2024-09-19T04:17:19.397 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:17:19.397 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":30,"btime":"2024-09-19T04:16:28:467883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:16:27.778564+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:17:19.397 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 30 2024-09-19T04:17:19.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcb000a3580 msgr2=0x7fcb000a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcb000a3580 0x7fcb000a5a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb08005fd0 tx=0x7fcb0800b480 comp rx=0 tx=0).stop 2024-09-19T04:17:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb1811b140 msgr2=0x7fcb18111cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb1811b140 0x7fcb18111cf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000ec50 tx=0x7fcb1000c6a0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 shutdown_connections 2024-09-19T04:17:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcb000a3580 0x7fcb000a5a40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb1811b140 0x7fcb18111cf0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb18109830 0x7fcb181117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 --2- 172.21.15.94:0/1656711628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb18109450 0x7fcb18118720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.397+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 >> 172.21.15.94:0/1656711628 conn(0x7fcb1806e140 msgr2=0x7fcb1811c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.398+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 shutdown_connections 2024-09-19T04:17:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:19.398+0000 7fcb1e01c640 1 -- 172.21.15.94:0/1656711628 wait complete. 2024-09-19T04:17:19.739 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:19 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1656711628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:19.739 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:19 smithi094 ceph-mon[30972]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:17:19.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:19 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1656711628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:19.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:19 smithi112 ceph-mon[35051]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:17:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:19 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1656711628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:19 smithi104 ceph-mon[34145]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:17:20.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:17:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:20.642 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 65.0 s, 1435.8 tps, lat 69.396 ms stddev 24.469 2024-09-19T04:17:21.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.214+0000 7fecc831e640 1 -- 172.21.15.94:0/3489665205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc010d520 msgr2=0x7fecc01150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.214+0000 7fecc831e640 1 --2- 172.21.15.94:0/3489665205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc010d520 0x7fecc01150a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009be0 tx=0x7fecb802a6c0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.214+0000 7fecc831e640 1 -- 172.21.15.94:0/3489665205 shutdown_connections 2024-09-19T04:17:21.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.214+0000 7fecc831e640 1 --2- 172.21.15.94:0/3489665205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc010d520 0x7fecc01150a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.214+0000 7fecc831e640 1 --2- 172.21.15.94:0/3489665205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecc0108da0 0x7fecc010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.214+0000 7fecc831e640 1 --2- 172.21.15.94:0/3489665205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecc01083f0 0x7fecc01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.214+0000 7fecc831e640 1 -- 172.21.15.94:0/3489665205 >> 172.21.15.94:0/3489665205 conn(0x7fecc0077af0 msgr2=0x7fecc0079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:21.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.214+0000 7fecc831e640 1 -- 172.21.15.94:0/3489665205 shutdown_connections 2024-09-19T04:17:21.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.214+0000 7fecc831e640 1 -- 172.21.15.94:0/3489665205 wait complete. 2024-09-19T04:17:21.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc831e640 1 Processor -- start 2024-09-19T04:17:21.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc831e640 1 -- start start 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc831e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecc01083f0 0x7fecc0073a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc831e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecc0108da0 0x7fecc006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc831e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc010d520 0x7fecc006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc60ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecc01083f0 0x7fecc0073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc831e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0118400 con 0x7fecc01083f0 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc60ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecc01083f0 0x7fecc0073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53940/0 (socket says 172.21.15.94:53940) 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc60ba640 1 -- 172.21.15.94:0/2797054191 learned_addr learned my addr 172.21.15.94:0/2797054191 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc831e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0118280 con 0x7fecc010d520 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc831e640 1 -- 172.21.15.94:0/2797054191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0118580 con 0x7fecc0108da0 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.215+0000 7fecc58b9640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecc0108da0 0x7fecc006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.216+0000 7fecc60ba640 1 -- 172.21.15.94:0/2797054191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecc0108da0 msgr2=0x7fecc006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.216+0000 7fecc60ba640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecc0108da0 0x7fecc006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.216+0000 7fecc60ba640 1 -- 172.21.15.94:0/2797054191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc010d520 msgr2=0x7fecc006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.216+0000 7fecc60ba640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc010d520 0x7fecc006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.216+0000 7fecc60ba640 1 -- 172.21.15.94:0/2797054191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc006d5e0 con 0x7fecc01083f0 2024-09-19T04:17:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.216+0000 7fecc60ba640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecc01083f0 0x7fecc0073a20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00e9c0 tx=0x7fecbc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:21.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.217+0000 7fecb77fe640 1 -- 172.21.15.94:0/2797054191 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00cc60 con 0x7fecc01083f0 2024-09-19T04:17:21.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.217+0000 7fecb77fe640 1 -- 172.21.15.94:0/2797054191 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecbc00ce00 con 0x7fecc01083f0 2024-09-19T04:17:21.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.217+0000 7fecc831e640 1 -- 172.21.15.94:0/2797054191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc006d8d0 con 0x7fecc01083f0 2024-09-19T04:17:21.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.217+0000 7fecb77fe640 1 -- 172.21.15.94:0/2797054191 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc010430 con 0x7fecc01083f0 2024-09-19T04:17:21.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.217+0000 7fecc831e640 1 -- 172.21.15.94:0/2797054191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc0071910 con 0x7fecc01083f0 2024-09-19T04:17:21.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.220+0000 7fecb77fe640 1 -- 172.21.15.94:0/2797054191 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fecbc004f50 con 0x7fecc01083f0 2024-09-19T04:17:21.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.220+0000 7fecb77fe640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fecac0a3430 0x7fecac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.220+0000 7fecca6f7740 1 Processor -- start 2024-09-19T04:17:21.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecb77fe640 1 -- 172.21.15.94:0/2797054191 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fecbc0cff20 con 0x7fecc01083f0 2024-09-19T04:17:21.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecca6f7740 1 -- start start 2024-09-19T04:17:21.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecc58b9640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fecac0a3430 0x7fecac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 0x564a34769090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 0x564a34769a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a34774370 0x564a34776760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecca6f7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564a346cf2f0 con 0x564a34774370 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecca6f7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564a34676cf0 con 0x564a3476ab40 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecca6f7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564a346cb2b0 con 0x564a347695d0 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecc60ba640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 0x564a34769090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecc60ba640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 0x564a34769090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:35070/0 (socket says 172.21.15.94:35070) 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecc60ba640 1 -- 172.21.15.94:0/1289954281 learned_addr learned my addr 172.21.15.94:0/1289954281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.221+0000 7fecc68bb640 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a34774370 0x564a34776760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.222+0000 7fecc58b9640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fecac0a3430 0x7fecac0a58f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fecc006e270 tx=0x7fecb000b390 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.222+0000 7fecc58b9640 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 0x564a34769a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.222+0000 7fecc68bb640 1 -- 172.21.15.94:0/1289954281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 msgr2=0x564a34769a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.222+0000 7fecc68bb640 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 0x564a34769a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.222+0000 7fecc68bb640 1 -- 172.21.15.94:0/1289954281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 msgr2=0x564a34769090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:21.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.222+0000 7fecc68bb640 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 0x564a34769090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.222+0000 7fecc68bb640 1 -- 172.21.15.94:0/1289954281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a3465b3a0 con 0x564a34774370 2024-09-19T04:17:21.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.222+0000 7fecc58b9640 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 0x564a34769a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.222+0000 7fecc68bb640 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a34774370 0x564a34776760 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009b30 tx=0x7fecb8030450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9be566e9777aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.223+0000 7fecb4dea640 1 -- 172.21.15.94:0/1289954281 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8040020 con 0x564a34774370 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.223+0000 7fecb4dea640 1 -- 172.21.15.94:0/1289954281 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb8003e20 con 0x564a34774370 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.223+0000 7fecb4dea640 1 -- 172.21.15.94:0/1289954281 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8036040 con 0x564a34774370 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289954281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a34774370 msgr2=0x564a34776760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a34774370 0x564a34776760 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009b30 tx=0x7fecb8030450 comp rx=0 tx=0).stop 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289954281 shutdown_connections 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a34774370 0x564a34776760 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 0x564a34769a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289954281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 0x564a34769090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289954281 >> 172.21.15.94:0/1289954281 conn(0x564a34732310 msgr2=0x564a34765c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289954281 shutdown_connections 2024-09-19T04:17:21.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289954281 wait complete. 2024-09-19T04:17:21.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.224+0000 7fecca6f7740 1 -- start start 2024-09-19T04:17:21.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecca6f7740 1 Processor -- start 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34773da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 0x564a34771050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a34774370 0x564a347717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecca6f7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564a34777010 con 0x564a347695d0 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecca6f7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564a3468a060 con 0x564a3476ab40 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecca6f7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564a34777190 con 0x564a34774370 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecc60ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34773da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecc60ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34773da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53952/0 (socket says 172.21.15.94:53952) 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecc60ba640 1 -- 172.21.15.94:0/1289900627 learned_addr learned my addr 172.21.15.94:0/1289900627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecca6f7740 10 client.0 _ll_register_callbacks cb 0x564a343dd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.225+0000 7fecc58b9640 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 0x564a34771050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7fecc60ba640 1 -- 172.21.15.94:0/1289900627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a34774370 msgr2=0x564a347717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7fecc60ba640 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a34774370 0x564a347717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7fecc60ba640 1 -- 172.21.15.94:0/1289900627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 msgr2=0x564a34771050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7fecc60ba640 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 0x564a34771050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7fecc60ba640 1 -- 172.21.15.94:0/1289900627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a344403c0 con 0x564a347695d0 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7fecc60ba640 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34773da0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fecbc09bef0 tx=0x7fecbc09c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2676c869e718a47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7feca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7feca9ffb640 1 -- 172.21.15.94:0/1289900627 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0b4070 con 0x564a347695d0 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289900627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb8003e20 con 0x564a347695d0 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.226+0000 7feca9ffb640 1 -- 172.21.15.94:0/1289900627 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fecbc0a0170 con 0x564a347695d0 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7feca9ffb640 1 -- 172.21.15.94:0/1289900627 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fecbc0a0370 con 0x564a347695d0 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7fecca6f7740 10 client.15621 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289900627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a34782ab0 con 0x564a347695d0 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7feca9ffb640 1 -- 172.21.15.94:0/1289900627 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb8003e20 con 0x564a347695d0 2024-09-19T04:17:21.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7feca9ffb640 1 -- 172.21.15.94:0/1289900627 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fecbc005240 con 0x564a347695d0 2024-09-19T04:17:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7fecca6f7740 10 client.15621 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7fecca6f7740 10 client.15621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:17:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7fecca6f7740 20 client.15621 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x564a34783ec0 0x564a34786380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7fecca6f7740 4 client.15621 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.227+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289900627 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a346dc860 con 0x564a34783ec0 2024-09-19T04:17:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.228+0000 7fecc58b9640 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x564a34783ec0 0x564a34786380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.228+0000 7feca9ffb640 10 client.15621 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.228+0000 7fecc58b9640 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x564a34783ec0 0x564a34786380 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b77efb0d055472e server_cookie=6772358ed86d190b in_seq=0 out_seq=0 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7feca9ffb640 1 -- 172.21.15.94:0/1289900627 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a346dc860 con 0x564a34783ec0 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7feca9ffb640 10 client.15621 handle_command_reply: tid=0 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 client.15621 shutdown 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289900627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 msgr2=0x564a34773da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34773da0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fecbc09bef0 tx=0x7fecbc09c2b0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289900627 shutdown_connections 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x564a34783ec0 0x564a34786380 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a34774370 0x564a347717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a3476ab40 0x564a34771050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7feca9ffb640 0 client.15621 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 --2- 172.21.15.94:0/1289900627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34773da0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289900627 >> 172.21.15.94:0/1289900627 conn(0x564a34732310 msgr2=0x564a3476b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289900627 shutdown_connections 2024-09-19T04:17:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 1 -- 172.21.15.94:0/1289900627 wait complete. 2024-09-19T04:17:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.229+0000 7fecca6f7740 20 client.15621 trim_cache size 0 max 16384 2024-09-19T04:17:21.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.247+0000 7fecca6f7740 1 Processor -- start 2024-09-19T04:17:21.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecca6f7740 1 -- start start 2024-09-19T04:17:21.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34765e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a3476ab40 0x564a3476a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 0x564a34770910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecca6f7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01bf080 con 0x564a347695d0 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecca6f7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01bc8c0 con 0x564a34774370 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecca6f7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01ba710 con 0x564a3476ab40 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc60ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34765e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc60ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34765e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53960/0 (socket says 172.21.15.94:53960) 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc60ba640 1 -- 172.21.15.94:0/709267558 learned_addr learned my addr 172.21.15.94:0/709267558 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc58b9640 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a3476ab40 0x564a3476a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc68bb640 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 0x564a34770910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc58b9640 1 -- 172.21.15.94:0/709267558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 msgr2=0x564a34770910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:21.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc58b9640 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 0x564a34770910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc58b9640 1 -- 172.21.15.94:0/709267558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 msgr2=0x564a34765e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:21.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc58b9640 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34765e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc58b9640 1 -- 172.21.15.94:0/709267558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a34780790 con 0x564a3476ab40 2024-09-19T04:17:21.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.248+0000 7fecc60ba640 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34765e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:21.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.249+0000 7fecc58b9640 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a3476ab40 0x564a3476a300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x564a34772690 tx=0x7fecb0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=561889cb719c563c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:21.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.249+0000 7fecabfff640 1 -- 172.21.15.94:0/709267558 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb001e4e0 con 0x564a3476ab40 2024-09-19T04:17:21.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.249+0000 7fecabfff640 1 -- 172.21.15.94:0/709267558 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb001eb30 con 0x564a3476ab40 2024-09-19T04:17:21.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.249+0000 7fecabfff640 1 -- 172.21.15.94:0/709267558 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb00278f0 con 0x564a3476ab40 2024-09-19T04:17:21.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.250+0000 7fecca6f7740 1 -- 172.21.15.94:0/709267558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a3476ab40 msgr2=0x564a3476a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.250+0000 7fecca6f7740 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a3476ab40 0x564a3476a300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x564a34772690 tx=0x7fecb0013000 comp rx=0 tx=0).stop 2024-09-19T04:17:21.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.250+0000 7fecca6f7740 1 -- 172.21.15.94:0/709267558 shutdown_connections 2024-09-19T04:17:21.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.250+0000 7fecca6f7740 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 0x564a34770910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.250+0000 7fecca6f7740 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a3476ab40 0x564a3476a300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.250+0000 7fecca6f7740 1 --2- 172.21.15.94:0/709267558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a347695d0 0x564a34765e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.250+0000 7fecca6f7740 1 -- 172.21.15.94:0/709267558 >> 172.21.15.94:0/709267558 conn(0x564a34732310 msgr2=0x564a34765700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.250+0000 7fecca6f7740 1 -- 172.21.15.94:0/709267558 shutdown_connections 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.250+0000 7fecca6f7740 1 -- 172.21.15.94:0/709267558 wait complete. 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.251+0000 7fecca6f7740 1 -- start start 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.251+0000 7fecca6f7740 1 Processor -- start 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 0x564a3477ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a3476ab40 0x564a34777fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecca6f7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 0x564a347785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecca6f7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01bad80 con 0x564a3476ab40 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecca6f7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01bc1f0 con 0x564a34774370 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecca6f7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01bb450 con 0x564a347695d0 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc58b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a3476ab40 0x564a34777fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecca6f7740 10 client.0 _ll_register_callbacks cb 0x7fecbc09c910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:21.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc58b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a3476ab40 0x564a34777fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53966/0 (socket says 172.21.15.94:53966) 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc58b9640 1 -- 172.21.15.94:0/898203184 learned_addr learned my addr 172.21.15.94:0/898203184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc60ba640 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 0x564a3477ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc68bb640 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 0x564a347785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc58b9640 1 -- 172.21.15.94:0/898203184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 msgr2=0x564a3477ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc58b9640 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 0x564a3477ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc58b9640 1 -- 172.21.15.94:0/898203184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 msgr2=0x564a347785f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc58b9640 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 0x564a347785f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc58b9640 1 -- 172.21.15.94:0/898203184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a34778e20 con 0x564a3476ab40 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc68bb640 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 0x564a347785f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7feca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.252+0000 7fecc58b9640 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a3476ab40 0x564a34777fe0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x564a34772690 tx=0x7fecb0005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02fa1f84daac3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:21.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.253+0000 7feca9ffb640 1 -- 172.21.15.94:0/898203184 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb001ece0 con 0x564a3476ab40 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.253+0000 7feca9ffb640 1 -- 172.21.15.94:0/898203184 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fecb002d040 con 0x564a3476ab40 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.253+0000 7fecca6f7740 1 -- 172.21.15.94:0/898203184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb001eb30 con 0x564a3476ab40 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.253+0000 7feca9ffb640 1 -- 172.21.15.94:0/898203184 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fecb001e4e0 con 0x564a3476ab40 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.253+0000 7fecca6f7740 10 client.15627 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.253+0000 7fecca6f7740 1 -- 172.21.15.94:0/898203184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a347eeda0 con 0x564a3476ab40 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.253+0000 7feca9ffb640 1 -- 172.21.15.94:0/898203184 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb0005040 con 0x564a3476ab40 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.253+0000 7feca9ffb640 1 -- 172.21.15.94:0/898203184 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fecb0038030 con 0x564a3476ab40 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.254+0000 7fecca6f7740 10 client.15627 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.254+0000 7fecca6f7740 10 client.15627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.254+0000 7fecca6f7740 20 client.15627 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.254+0000 7fecca6f7740 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x564a34783ec0 0x564a347f1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.254+0000 7fecca6f7740 4 client.15627 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:17:21.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.254+0000 7fecca6f7740 1 -- 172.21.15.94:0/898203184 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fecb001eb30 con 0x564a34783ec0 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.254+0000 7fecc60ba640 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x564a34783ec0 0x564a347f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.254+0000 7fecc60ba640 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x564a34783ec0 0x564a347f1560 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7e62fe3530ec09f server_cookie=e49ded08e5f59a98 in_seq=0 out_seq=0 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.254+0000 7feca9ffb640 10 client.15627 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7feca9ffb640 1 -- 172.21.15.94:0/898203184 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fecb001eb30 con 0x564a34783ec0 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7feca9ffb640 10 client.15627 handle_command_reply: tid=0 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 client.15627 shutdown 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 -- 172.21.15.94:0/898203184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a3476ab40 msgr2=0x564a34777fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a3476ab40 0x564a34777fe0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x564a34772690 tx=0x7fecb0005d40 comp rx=0 tx=0).stop 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 -- 172.21.15.94:0/898203184 shutdown_connections 2024-09-19T04:17:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x564a34783ec0 0x564a347f1560 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a34774370 0x564a347785f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a3476ab40 0x564a34777fe0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 --2- 172.21.15.94:0/898203184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a347695d0 0x564a3477ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 -- 172.21.15.94:0/898203184 >> 172.21.15.94:0/898203184 conn(0x564a34732310 msgr2=0x564a3476b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7feca9ffb640 0 client.15627 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:17:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 -- 172.21.15.94:0/898203184 shutdown_connections 2024-09-19T04:17:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 1 -- 172.21.15.94:0/898203184 wait complete. 2024-09-19T04:17:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.255+0000 7fecca6f7740 20 client.15627 trim_cache size 0 max 16384 2024-09-19T04:17:21.261 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:17:21.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 -- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fecac0a3430 msgr2=0x7fecac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fecac0a3430 0x7fecac0a58f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fecc006e270 tx=0x7fecb000b390 comp rx=0 tx=0).stop 2024-09-19T04:17:21.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 -- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecc01083f0 msgr2=0x7fecc0073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecc01083f0 0x7fecc0073a20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00e9c0 tx=0x7fecbc00ee80 comp rx=0 tx=0).stop 2024-09-19T04:17:21.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 -- 172.21.15.94:0/2797054191 shutdown_connections 2024-09-19T04:17:21.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fecac0a3430 0x7fecac0a58f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc010d520 0x7fecc006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecc0108da0 0x7fecc006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 --2- 172.21.15.94:0/2797054191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecc01083f0 0x7fecc0073a20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.256+0000 7fecc831e640 1 -- 172.21.15.94:0/2797054191 >> 172.21.15.94:0/2797054191 conn(0x7fecc0077af0 msgr2=0x7fecc0078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:21.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.257+0000 7fecc831e640 1 -- 172.21.15.94:0/2797054191 shutdown_connections 2024-09-19T04:17:21.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:21.257+0000 7fecc831e640 1 -- 172.21.15.94:0/2797054191 wait complete. 2024-09-19T04:17:21.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:21 smithi112 ceph-mon[35051]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:17:21.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:21 smithi094 ceph-mon[30972]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:17:21.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:21 smithi104 ceph-mon[34145]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:17:22.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:17:23.019 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:23.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:23 smithi112 ceph-mon[35051]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 2.18k op/s 2024-09-19T04:17:23.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:23 smithi094 ceph-mon[30972]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 2.18k op/s 2024-09-19T04:17:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:23 smithi104 ceph-mon[34145]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 2.18k op/s 2024-09-19T04:17:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 70.0 s, 1276.0 tps, lat 78.643 ms stddev 67.860 2024-09-19T04:17:25.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:25 smithi112 ceph-mon[35051]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.37k op/s 2024-09-19T04:17:25.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:25 smithi094 ceph-mon[30972]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.37k op/s 2024-09-19T04:17:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:25 smithi104 ceph-mon[34145]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.37k op/s 2024-09-19T04:17:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.541+0000 7f9f04457640 1 -- 172.21.15.94:0/438511292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9efc1082a0 msgr2=0x7f9efc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.541+0000 7f9f04457640 1 --2- 172.21.15.94:0/438511292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9efc1082a0 0x7f9efc10ec40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009cd0 tx=0x7f9ef402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.542+0000 7f9f04457640 1 -- 172.21.15.94:0/438511292 shutdown_connections 2024-09-19T04:17:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.542+0000 7f9f04457640 1 --2- 172.21.15.94:0/438511292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9efc1082a0 0x7f9efc10ec40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.542+0000 7f9f04457640 1 --2- 172.21.15.94:0/438511292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc1036e0 0x7f9efc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.542+0000 7f9f04457640 1 --2- 172.21.15.94:0/438511292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc102d30 0x7f9efc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.542+0000 7f9f04457640 1 -- 172.21.15.94:0/438511292 >> 172.21.15.94:0/438511292 conn(0x7f9efc0fc7d0 msgr2=0x7f9efc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.542+0000 7f9f04457640 1 -- 172.21.15.94:0/438511292 shutdown_connections 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.542+0000 7f9f04457640 1 -- 172.21.15.94:0/438511292 wait complete. 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.543+0000 7f9f04457640 1 Processor -- start 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.544+0000 7f9f04457640 1 -- start start 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.544+0000 7f9f04457640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc102d30 0x7f9efc10a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.544+0000 7f9f04457640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc1036e0 0x7f9efc10aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f04457640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9efc1082a0 0x7f9efc1a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f021f3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc102d30 0x7f9efc10a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f021f3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc102d30 0x7f9efc10a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:46126/0 (socket says 172.21.15.94:46126) 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f021f3640 1 -- 172.21.15.94:0/643802050 learned_addr learned my addr 172.21.15.94:0/643802050 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc100d70 con 0x7f9efc1082a0 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc100bf0 con 0x7f9efc102d30 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc100ef0 con 0x7f9efc1036e0 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f019f2640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc1036e0 0x7f9efc10aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f029f4640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9efc1082a0 0x7f9efc1a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f021f3640 1 -- 172.21.15.94:0/643802050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc1036e0 msgr2=0x7f9efc10aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f021f3640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc1036e0 0x7f9efc10aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f021f3640 1 -- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9efc1082a0 msgr2=0x7f9efc1a33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f021f3640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9efc1082a0 0x7f9efc1a33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.545+0000 7f9f021f3640 1 -- 172.21.15.94:0/643802050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc1a3ba0 con 0x7f9efc102d30 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.546+0000 7f9f021f3640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc102d30 0x7f9efc10a980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800d950 tx=0x7f9ef800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.546+0000 7f9ef37fe640 1 -- 172.21.15.94:0/643802050 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8017070 con 0x7f9efc102d30 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.546+0000 7f9ef37fe640 1 -- 172.21.15.94:0/643802050 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ef8009c30 con 0x7f9efc102d30 2024-09-19T04:17:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.546+0000 7f9ef37fe640 1 -- 172.21.15.94:0/643802050 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800f620 con 0x7f9efc102d30 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.546+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9efc1a3e90 con 0x7f9efc102d30 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.546+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9efc1ab7a0 con 0x7f9efc102d30 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.549+0000 7f9ef37fe640 1 -- 172.21.15.94:0/643802050 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9ef80109a0 con 0x7f9efc102d30 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9f06830740 1 Processor -- start 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9ef37fe640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ee80a3430 0x7f9ee80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9ef37fe640 1 -- 172.21.15.94:0/643802050 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f9ef80cf840 con 0x7f9efc102d30 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9f029f4640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ee80a3430 0x7f9ee80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9f06830740 1 -- start start 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9f06830740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcf3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9f06830740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf35d0 0x563c0fcf3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9f06830740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fd00760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9f06830740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563c0fc592f0 con 0x563c0fcf35d0 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9f06830740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563c0fc00cf0 con 0x563c0fcfe370 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.550+0000 7f9f06830740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563c0fc552b0 con 0x563c0fcf4b40 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f019f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf35d0 0x563c0fcf3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f019f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf35d0 0x563c0fcf3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55244/0 (socket says 172.21.15.94:55244) 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f019f2640 1 -- 172.21.15.94:0/3824704331 learned_addr learned my addr 172.21.15.94:0/3824704331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f029f4640 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fd00760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f021f3640 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcf3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f029f4640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ee80a3430 0x7f9ee80a58f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4005e10 tx=0x7f9ef4005da0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f019f2640 1 -- 172.21.15.94:0/3824704331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 msgr2=0x563c0fcf3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f019f2640 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcf3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f019f2640 1 -- 172.21.15.94:0/3824704331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 msgr2=0x563c0fd00760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f019f2640 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fd00760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f019f2640 1 -- 172.21.15.94:0/3824704331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0fbe53a0 con 0x563c0fcf35d0 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f021f3640 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcf3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9f019f2640 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf35d0 0x563c0fcf3a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0045e0 tx=0x7f9eec004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a091ad14f5b61d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9ef0dea640 1 -- 172.21.15.94:0/3824704331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec003af0 con 0x563c0fcf35d0 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9ef0dea640 1 -- 172.21.15.94:0/3824704331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eec015070 con 0x563c0fcf35d0 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.551+0000 7f9ef0dea640 1 -- 172.21.15.94:0/3824704331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec004cf0 con 0x563c0fcf35d0 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 -- 172.21.15.94:0/3824704331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf35d0 msgr2=0x563c0fcf3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf35d0 0x563c0fcf3a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0045e0 tx=0x7f9eec004610 comp rx=0 tx=0).stop 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 -- 172.21.15.94:0/3824704331 shutdown_connections 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fd00760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf35d0 0x563c0fcf3a10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 --2- 172.21.15.94:0/3824704331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcf3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 -- 172.21.15.94:0/3824704331 >> 172.21.15.94:0/3824704331 conn(0x563c0fcbc310 msgr2=0x563c0fcefc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 -- 172.21.15.94:0/3824704331 shutdown_connections 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 -- 172.21.15.94:0/3824704331 wait complete. 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.552+0000 7f9f06830740 1 -- start start 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f06830740 1 Processor -- start 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f06830740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcf35d0 0x563c0fcfddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f06830740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcfb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f06830740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcfe370 0x563c0fcfb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f06830740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563c0fd01070 con 0x563c0fcfe370 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f06830740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563c0fc14060 con 0x563c0fcf35d0 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f06830740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563c0fd011f0 con 0x563c0fcf4b40 2024-09-19T04:17:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f06830740 10 client.0 _ll_register_callbacks cb 0x563c0f9678c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f019f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcfb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f019f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcfb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:42326/0 (socket says 172.21.15.94:42326) 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f019f2640 1 -- 172.21.15.94:0/2654953057 learned_addr learned my addr 172.21.15.94:0/2654953057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f029f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcfe370 0x563c0fcfb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f021f3640 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcf35d0 0x563c0fcfddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f019f2640 1 -- 172.21.15.94:0/2654953057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcf35d0 msgr2=0x563c0fcfddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f019f2640 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcf35d0 0x563c0fcfddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f019f2640 1 -- 172.21.15.94:0/2654953057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcfe370 msgr2=0x563c0fcfb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f019f2640 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcfe370 0x563c0fcfb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f019f2640 1 -- 172.21.15.94:0/2654953057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0f9ca3c0 con 0x563c0fcf4b40 2024-09-19T04:17:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f021f3640 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcf35d0 0x563c0fcfddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f029f4640 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcfe370 0x563c0fcfb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9f019f2640 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcfb070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9eec005880 tx=0x7f9eec003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cfaad2be7145491 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.553+0000 7f9ee5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/2654953057 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec028060 con 0x563c0fcf4b40 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9f06830740 1 -- 172.21.15.94:0/2654953057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eec015070 con 0x563c0fcf4b40 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/2654953057 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f9eec011850 con 0x563c0fcf4b40 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/2654953057 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9eec0046c0 con 0x563c0fcf4b40 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9f06830740 10 client.25186 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9f06830740 1 -- 172.21.15.94:0/2654953057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c0fd0cb80 con 0x563c0fcf4b40 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/2654953057 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eec015070 con 0x563c0fcf4b40 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/2654953057 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f9eec022070 con 0x563c0fcf4b40 2024-09-19T04:17:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9f06830740 10 client.25186 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9f06830740 10 client.25186 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9f06830740 20 client.25186 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9f06830740 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563c0fd0df30 0x563c0fd10320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9f06830740 4 client.25186 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.554+0000 7f9f06830740 1 -- 172.21.15.94:0/2654953057 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c0fc66860 con 0x563c0fd0df30 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.555+0000 7f9f021f3640 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563c0fd0df30 0x563c0fd10320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.555+0000 7f9f021f3640 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563c0fd0df30 0x563c0fd10320 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2ceb24c75007acf server_cookie=b0452581d6311f94 in_seq=0 out_seq=0 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.555+0000 7f9ee5ffb640 10 client.25186 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.555+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/2654953057 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c0fc66860 con 0x563c0fd0df30 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.555+0000 7f9ee5ffb640 10 client.25186 handle_command_reply: tid=0 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.555+0000 7f9f06830740 1 client.25186 shutdown 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 -- 172.21.15.94:0/2654953057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 msgr2=0x563c0fcfb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcfb070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9eec005880 tx=0x7f9eec003b20 comp rx=0 tx=0).stop 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 -- 172.21.15.94:0/2654953057 shutdown_connections 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563c0fd0df30 0x563c0fd10320 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcfe370 0x563c0fcfb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf4b40 0x563c0fcfb070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9ee5ffb640 0 client.25186 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 --2- 172.21.15.94:0/2654953057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcf35d0 0x563c0fcfddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 -- 172.21.15.94:0/2654953057 >> 172.21.15.94:0/2654953057 conn(0x563c0fcbc310 msgr2=0x563c0fcf5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 -- 172.21.15.94:0/2654953057 shutdown_connections 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 1 -- 172.21.15.94:0/2654953057 wait complete. 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.556+0000 7f9f06830740 20 client.25186 trim_cache size 0 max 16384 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.578+0000 7f9f06830740 1 Processor -- start 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.578+0000 7f9f06830740 1 -- start start 2024-09-19T04:17:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.578+0000 7f9f06830740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 0x563c0fcfcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.578+0000 7f9f06830740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.578+0000 7f9f06830740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fcf6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.578+0000 7f9f06830740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1b67d0 con 0x563c0fcf4b40 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.578+0000 7f9f06830740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1b4010 con 0x563c0fcfe370 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.578+0000 7f9f06830740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1b1e60 con 0x563c0fcf35d0 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.578+0000 7f9f019f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.579+0000 7f9f019f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55268/0 (socket says 172.21.15.94:55268) 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.579+0000 7f9f019f2640 1 -- 172.21.15.94:0/3775139171 learned_addr learned my addr 172.21.15.94:0/3775139171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.579+0000 7f9f029f4640 1 --2- 172.21.15.94:0/3775139171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fcf6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.579+0000 7f9f019f2640 1 -- 172.21.15.94:0/3775139171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 msgr2=0x563c0fcfcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.579+0000 7f9f019f2640 1 --2- 172.21.15.94:0/3775139171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 0x563c0fcfcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.437 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:17:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.579+0000 7f9f019f2640 1 -- 172.21.15.94:0/3775139171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 msgr2=0x563c0fcf6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.579+0000 7f9f019f2640 1 --2- 172.21.15.94:0/3775139171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fcf6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.579+0000 7f9f019f2640 1 -- 172.21.15.94:0/3775139171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0fcfbb70 con 0x563c0fcf4b40 2024-09-19T04:17:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.579+0000 7f9f019f2640 1 --2- 172.21.15.94:0/3775139171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf6120 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0199b0 tx=0x7f9eec004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b7b6ff55c217d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.580+0000 7f9ee7fff640 1 -- 172.21.15.94:0/3775139171 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec028040 con 0x563c0fcf4b40 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.580+0000 7f9ee7fff640 1 -- 172.21.15.94:0/3775139171 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eec015070 con 0x563c0fcf4b40 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.580+0000 7f9ee7fff640 1 -- 172.21.15.94:0/3775139171 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011b00 con 0x563c0fcf4b40 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.581+0000 7f9f06830740 1 -- 172.21.15.94:0/3775139171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 msgr2=0x563c0fcf6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.581+0000 7f9f06830740 1 --2- 172.21.15.94:0/3775139171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf6120 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0199b0 tx=0x7f9eec004080 comp rx=0 tx=0).stop 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.581+0000 7f9f06830740 1 -- 172.21.15.94:0/3775139171 shutdown_connections 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.581+0000 7f9f06830740 1 --2- 172.21.15.94:0/3775139171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fcf6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.581+0000 7f9f06830740 1 --2- 172.21.15.94:0/3775139171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf6120 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.581+0000 7f9f06830740 1 --2- 172.21.15.94:0/3775139171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 0x563c0fcfcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.581+0000 7f9f06830740 1 -- 172.21.15.94:0/3775139171 >> 172.21.15.94:0/3775139171 conn(0x563c0fcbc310 msgr2=0x563c0fcfc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.581+0000 7f9f06830740 1 -- 172.21.15.94:0/3775139171 shutdown_connections 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.581+0000 7f9f06830740 1 -- 172.21.15.94:0/3775139171 wait complete. 2024-09-19T04:17:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.582+0000 7f9f06830740 1 -- start start 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.582+0000 7f9f06830740 1 Processor -- start 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f06830740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 0x563c0fcf8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f06830740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f06830740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fcf8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f06830740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1b24d0 con 0x563c0fcf4b40 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f06830740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1b3940 con 0x563c0fcfe370 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f06830740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1b2ba0 con 0x563c0fcf35d0 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f06830740 10 client.0 _ll_register_callbacks cb 0x7f9ef4009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f021f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 0x563c0fcf8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f021f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 0x563c0fcf8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:42340/0 (socket says 172.21.15.94:42340) 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f021f3640 1 -- 172.21.15.94:0/3407905900 learned_addr learned my addr 172.21.15.94:0/3407905900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f029f4640 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fcf8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f019f2640 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f029f4640 1 -- 172.21.15.94:0/3407905900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 msgr2=0x563c0fcf8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f029f4640 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 0x563c0fcf8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f029f4640 1 -- 172.21.15.94:0/3407905900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 msgr2=0x563c0fcf8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f029f4640 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.583+0000 7f9f029f4640 1 -- 172.21.15.94:0/3407905900 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0fd779d0 con 0x563c0fcfe370 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.584+0000 7f9f029f4640 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fcf8b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009ca0 tx=0x7f9ef403e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc21c609aa297a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.584+0000 7f9ee5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.584+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/3407905900 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4057060 con 0x563c0fcfe370 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.584+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/3407905900 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f9ef40307a0 con 0x563c0fcfe370 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.584+0000 7f9f06830740 1 -- 172.21.15.94:0/3407905900 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eec015070 con 0x563c0fcfe370 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/3407905900 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ef40309a0 con 0x563c0fcfe370 2024-09-19T04:17:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9f06830740 10 client.25232 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9f06830740 1 -- 172.21.15.94:0/3407905900 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c0fd78c90 con 0x563c0fcfe370 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/3407905900 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eec015070 con 0x563c0fcfe370 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/3407905900 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f9ef4030070 con 0x563c0fcfe370 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9f06830740 10 client.25232 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9f06830740 10 client.25232 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9f06830740 20 client.25232 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9f06830740 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563c0fd793c0 0x563c0fd7b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9f06830740 4 client.25232 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.585+0000 7f9f06830740 1 -- 172.21.15.94:0/3407905900 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c0fc60d70 con 0x563c0fd793c0 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.586+0000 7f9f021f3640 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563c0fd793c0 0x563c0fd7b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.586+0000 7f9ee5ffb640 10 client.25232 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.586+0000 7f9f021f3640 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563c0fd793c0 0x563c0fd7b7e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1242abdb65375455 server_cookie=f7cda3975c2ba09d in_seq=0 out_seq=0 2024-09-19T04:17:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.586+0000 7f9ee5ffb640 1 -- 172.21.15.94:0/3407905900 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c0fc60d70 con 0x563c0fd793c0 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.586+0000 7f9ee5ffb640 10 client.25232 handle_command_reply: tid=0 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.586+0000 7f9f06830740 1 client.25232 shutdown 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 -- 172.21.15.94:0/3407905900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 msgr2=0x563c0fcf8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fcf8b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009ca0 tx=0x7f9ef403e040 comp rx=0 tx=0).stop 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 -- 172.21.15.94:0/3407905900 shutdown_connections 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563c0fd793c0 0x563c0fd7b7e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c0fcfe370 0x563c0fcf8b80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c0fcf4b40 0x563c0fcf8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 --2- 172.21.15.94:0/3407905900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c0fcf35d0 0x563c0fcf8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 -- 172.21.15.94:0/3407905900 >> 172.21.15.94:0/3407905900 conn(0x563c0fcbc310 msgr2=0x563c0fcfd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9ee5ffb640 0 client.25232 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 -- 172.21.15.94:0/3407905900 shutdown_connections 2024-09-19T04:17:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 1 -- 172.21.15.94:0/3407905900 wait complete. 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.587+0000 7f9f06830740 20 client.25232 trim_cache size 0 max 16384 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.588+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ee80a3430 msgr2=0x7f9ee80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.588+0000 7f9f04457640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ee80a3430 0x7f9ee80a58f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4005e10 tx=0x7f9ef4005da0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.588+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc102d30 msgr2=0x7f9efc10a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.588+0000 7f9f04457640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc102d30 0x7f9efc10a980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800d950 tx=0x7f9ef800de10 comp rx=0 tx=0).stop 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.588+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 shutdown_connections 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.588+0000 7f9f04457640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ee80a3430 0x7f9ee80a58f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.589+0000 7f9f04457640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9efc1082a0 0x7f9efc1a33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.589+0000 7f9f04457640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc1036e0 0x7f9efc10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.589+0000 7f9f04457640 1 --2- 172.21.15.94:0/643802050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc102d30 0x7f9efc10a980 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.589+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 >> 172.21.15.94:0/643802050 conn(0x7f9efc0fc7d0 msgr2=0x7f9efc0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.589+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 shutdown_connections 2024-09-19T04:17:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:26.589+0000 7f9f04457640 1 -- 172.21.15.94:0/643802050 wait complete. 2024-09-19T04:17:27.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:27 smithi112 ceph-mon[35051]: pgmap v76: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.99k op/s 2024-09-19T04:17:27.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:27 smithi094 ceph-mon[30972]: pgmap v76: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.99k op/s 2024-09-19T04:17:27.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:27 smithi104 ceph-mon[34145]: pgmap v76: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.99k op/s 2024-09-19T04:17:28.874 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:17:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:29.717 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:29 smithi094 ceph-mon[30972]: pgmap v77: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 1.26k op/s 2024-09-19T04:17:29.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:29 smithi112 ceph-mon[35051]: pgmap v77: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 1.26k op/s 2024-09-19T04:17:29.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.842+0000 7f060a621640 1 -- 172.21.15.94:0/3949155254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0604102780 msgr2=0x7f06041067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.842+0000 7f060a621640 1 --2- 172.21.15.94:0/3949155254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0604102780 0x7f06041067d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009900 tx=0x7f05f402a410 comp rx=0 tx=0).stop 2024-09-19T04:17:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.843+0000 7f060a621640 1 -- 172.21.15.94:0/3949155254 shutdown_connections 2024-09-19T04:17:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.843+0000 7f060a621640 1 --2- 172.21.15.94:0/3949155254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0604106f00 0x7f060410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.843+0000 7f060a621640 1 --2- 172.21.15.94:0/3949155254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0604102780 0x7f06041067d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.843+0000 7f060a621640 1 --2- 172.21.15.94:0/3949155254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0604101dd0 0x7f06041021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.843+0000 7f060a621640 1 -- 172.21.15.94:0/3949155254 >> 172.21.15.94:0/3949155254 conn(0x7f06040fd8b0 msgr2=0x7f06040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.843+0000 7f060a621640 1 -- 172.21.15.94:0/3949155254 shutdown_connections 2024-09-19T04:17:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.843+0000 7f060a621640 1 -- 172.21.15.94:0/3949155254 wait complete. 2024-09-19T04:17:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.844+0000 7f060a621640 1 Processor -- start 2024-09-19T04:17:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.844+0000 7f060a621640 1 -- start start 2024-09-19T04:17:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.844+0000 7f060a621640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0604101dd0 0x7f0604073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.844+0000 7f060a621640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0604102780 0x7f060406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.844+0000 7f060a621640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0604106f00 0x7f060406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.844+0000 7f0603fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0604101dd0 0x7f0604073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.844+0000 7f0603fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0604101dd0 0x7f0604073b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:46170/0 (socket says 172.21.15.94:46170) 2024-09-19T04:17:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f0603fff640 1 -- 172.21.15.94:0/1734395904 learned_addr learned my addr 172.21.15.94:0/1734395904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f06037fe640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0604102780 0x7f060406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0604111e10 con 0x7f0604106f00 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f0608bbe640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0604106f00 0x7f060406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0604111c90 con 0x7f0604101dd0 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0604111f90 con 0x7f0604102780 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f0603fff640 1 -- 172.21.15.94:0/1734395904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0604102780 msgr2=0x7f060406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f0603fff640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0604102780 0x7f060406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f0603fff640 1 -- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0604106f00 msgr2=0x7f060406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f0603fff640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0604106f00 0x7f060406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.845+0000 7f0603fff640 1 -- 172.21.15.94:0/1734395904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060406d900 con 0x7f0604101dd0 2024-09-19T04:17:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.846+0000 7f0603fff640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0604101dd0 0x7f0604073b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05f800e9f0 tx=0x7f05f800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.846+0000 7f06017fa640 1 -- 172.21.15.94:0/1734395904 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f800cd80 con 0x7f0604101dd0 2024-09-19T04:17:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.846+0000 7f06017fa640 1 -- 172.21.15.94:0/1734395904 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f80054d0 con 0x7f0604101dd0 2024-09-19T04:17:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.846+0000 7f06017fa640 1 -- 172.21.15.94:0/1734395904 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8010490 con 0x7f0604101dd0 2024-09-19T04:17:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.846+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06040716c0 con 0x7f0604101dd0 2024-09-19T04:17:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.846+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0604071b80 con 0x7f0604101dd0 2024-09-19T04:17:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.850+0000 7f06017fa640 1 -- 172.21.15.94:0/1734395904 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f05f80106b0 con 0x7f0604101dd0 2024-09-19T04:17:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.850+0000 7f060c9fa740 1 Processor -- start 2024-09-19T04:17:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f060c9fa740 1 -- start start 2024-09-19T04:17:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 0x55ac65666090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65666a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65673760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f060c9fa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ac655cc2f0 con 0x55ac65671370 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f060c9fa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ac65573cf0 con 0x55ac656665d0 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f060c9fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac655c82b0 con 0x55ac65667b40 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f06017fa640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f05ec0a7840 0x7f05ec0a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f06037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65673760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f0603fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65666a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f06037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65673760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55300/0 (socket says 172.21.15.94:55300) 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f06037fe640 1 -- 172.21.15.94:0/2980667100 learned_addr learned my addr 172.21.15.94:0/2980667100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.851+0000 7f0608bbe640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f05ec0a7840 0x7f05ec0a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f0608bbe640 1 --2- 172.21.15.94:0/2980667100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 0x55ac65666090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f0603fff640 1 -- 172.21.15.94:0/2980667100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 msgr2=0x55ac65666090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f0603fff640 1 --2- 172.21.15.94:0/2980667100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 0x55ac65666090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f0603fff640 1 -- 172.21.15.94:0/2980667100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 msgr2=0x55ac65673760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f0603fff640 1 --2- 172.21.15.94:0/2980667100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65673760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f0603fff640 1 -- 172.21.15.94:0/2980667100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac655583a0 con 0x55ac656665d0 2024-09-19T04:17:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f0603fff640 1 --2- 172.21.15.94:0/2980667100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65666a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f05f80ce6e0 tx=0x7f05f8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceb989823302aa1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:29.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:29 smithi104 ceph-mon[34145]: pgmap v77: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 1.26k op/s 2024-09-19T04:17:29.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f0608bbe640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f05ec0a7840 0x7f05ec0a9d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009fd0 tx=0x7f05fc009d80 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f06017fa640 1 -- 172.21.15.94:0/1734395904 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f05f80d0fe0 con 0x7f0604101dd0 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.852+0000 7f05ea7fc640 1 -- 172.21.15.94:0/2980667100 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f809d970 con 0x55ac656665d0 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.853+0000 7f05ea7fc640 1 -- 172.21.15.94:0/2980667100 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f809d270 con 0x55ac656665d0 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.853+0000 7f05ea7fc640 1 -- 172.21.15.94:0/2980667100 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f809b640 con 0x55ac656665d0 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.853+0000 7f060c9fa740 1 -- 172.21.15.94:0/2980667100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 msgr2=0x55ac65666a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.853+0000 7f060c9fa740 1 --2- 172.21.15.94:0/2980667100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65666a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f05f80ce6e0 tx=0x7f05f8014000 comp rx=0 tx=0).stop 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.854+0000 7f060c9fa740 1 -- 172.21.15.94:0/2980667100 shutdown_connections 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.854+0000 7f060c9fa740 1 --2- 172.21.15.94:0/2980667100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65673760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.854+0000 7f060c9fa740 1 --2- 172.21.15.94:0/2980667100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65666a10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.854+0000 7f060c9fa740 1 --2- 172.21.15.94:0/2980667100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 0x55ac65666090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.854+0000 7f060c9fa740 1 -- 172.21.15.94:0/2980667100 >> 172.21.15.94:0/2980667100 conn(0x55ac6562f310 msgr2=0x55ac65662c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.854+0000 7f060c9fa740 1 -- 172.21.15.94:0/2980667100 shutdown_connections 2024-09-19T04:17:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.854+0000 7f060c9fa740 1 -- 172.21.15.94:0/2980667100 wait complete. 2024-09-19T04:17:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.855+0000 7f060c9fa740 1 -- start start 2024-09-19T04:17:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.855+0000 7f060c9fa740 1 Processor -- start 2024-09-19T04:17:29.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65670d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac6566e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac6566e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f060c9fa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ac65674070 con 0x55ac65667b40 2024-09-19T04:17:29.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f060c9fa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ac65587060 con 0x55ac656665d0 2024-09-19T04:17:29.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f060c9fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac656741f0 con 0x55ac65671370 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f060c9fa740 10 client.0 _ll_register_callbacks cb 0x55ac652da8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f0608bbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac6566e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f0603fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65670d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f0608bbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac6566e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:42378/0 (socket says 172.21.15.94:42378) 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f0608bbe640 1 -- 172.21.15.94:0/4118880923 learned_addr learned my addr 172.21.15.94:0/4118880923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f06037fe640 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac6566e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f0608bbe640 1 -- 172.21.15.94:0/4118880923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 msgr2=0x55ac65670d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f0608bbe640 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65670d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.856+0000 7f0608bbe640 1 -- 172.21.15.94:0/4118880923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 msgr2=0x55ac6566e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.857+0000 7f0608bbe640 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac6566e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.857+0000 7f0608bbe640 1 -- 172.21.15.94:0/4118880923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac6533d3c0 con 0x55ac65671370 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.857+0000 7f06037fe640 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac6566e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.857+0000 7f0608bbe640 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac6566e640 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05fc012fd0 tx=0x7f05fc0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d13970b2df6545e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.857+0000 7f05e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:17:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.857+0000 7f05e37fe640 1 -- 172.21.15.94:0/4118880923 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc024070 con 0x55ac65671370 2024-09-19T04:17:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.857+0000 7f060c9fa740 1 -- 172.21.15.94:0/4118880923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05f809d270 con 0x55ac65671370 2024-09-19T04:17:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.857+0000 7f05e37fe640 1 -- 172.21.15.94:0/4118880923 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f05fc029460 con 0x55ac65671370 2024-09-19T04:17:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.858+0000 7f05e37fe640 1 -- 172.21.15.94:0/4118880923 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f05fc029660 con 0x55ac65671370 2024-09-19T04:17:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.858+0000 7f060c9fa740 10 client.25201 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.858+0000 7f060c9fa740 1 -- 172.21.15.94:0/4118880923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac6567fb90 con 0x55ac65671370 2024-09-19T04:17:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.858+0000 7f05e37fe640 1 -- 172.21.15.94:0/4118880923 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f809d270 con 0x55ac65671370 2024-09-19T04:17:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.858+0000 7f05e37fe640 1 -- 172.21.15.94:0/4118880923 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f05fc01f070 con 0x55ac65671370 2024-09-19T04:17:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.859+0000 7f060c9fa740 10 client.25201 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.859+0000 7f060c9fa740 10 client.25201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.859+0000 7f060c9fa740 20 client.25201 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.859+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ac656813b0 0x55ac65683830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.860+0000 7f060c9fa740 4 client.25201 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.860+0000 7f060c9fa740 1 -- 172.21.15.94:0/4118880923 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac655d9860 con 0x55ac656813b0 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.860+0000 7f0603fff640 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ac656813b0 0x55ac65683830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.860+0000 7f05e37fe640 10 client.25201 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.860+0000 7f0603fff640 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ac656813b0 0x55ac65683830 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef6112f85ec86d03 server_cookie=8046e202da26eb58 in_seq=0 out_seq=0 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.861+0000 7f05e37fe640 1 -- 172.21.15.94:0/4118880923 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac655d9860 con 0x55ac656813b0 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.861+0000 7f05e37fe640 10 client.25201 handle_command_reply: tid=0 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.861+0000 7f060c9fa740 1 client.25201 shutdown 2024-09-19T04:17:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 -- 172.21.15.94:0/4118880923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 msgr2=0x55ac6566e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac6566e640 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05fc012fd0 tx=0x7f05fc0096a0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 -- 172.21.15.94:0/4118880923 shutdown_connections 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ac656813b0 0x55ac65683830 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac6566e640 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac6566e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4118880923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65670d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 -- 172.21.15.94:0/4118880923 >> 172.21.15.94:0/4118880923 conn(0x55ac6562f310 msgr2=0x55ac656689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f05e37fe640 0 client.25201 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 -- 172.21.15.94:0/4118880923 shutdown_connections 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 1 -- 172.21.15.94:0/4118880923 wait complete. 2024-09-19T04:17:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.862+0000 7f060c9fa740 20 client.25201 trim_cache size 0 max 16384 2024-09-19T04:17:29.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.883+0000 7f060c9fa740 1 Processor -- start 2024-09-19T04:17:29.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.883+0000 7f060c9fa740 1 -- start start 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65661bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 0x55ac65669120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65669660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f060c9fa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06041b6710 con 0x55ac65671370 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f060c9fa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f06041b3f50 con 0x55ac656665d0 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f060c9fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06041b1da0 con 0x55ac65667b40 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f0603fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65661bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f06037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 0x55ac65669120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f0608bbe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65669660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f0603fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65661bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:46184/0 (socket says 172.21.15.94:46184) 2024-09-19T04:17:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.884+0000 7f0603fff640 1 -- 172.21.15.94:0/3472810904 learned_addr learned my addr 172.21.15.94:0/3472810904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:29.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.885+0000 7f06037fe640 1 -- 172.21.15.94:0/3472810904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 msgr2=0x55ac65661bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.885+0000 7f06037fe640 1 --2- 172.21.15.94:0/3472810904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65661bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.885+0000 7f06037fe640 1 -- 172.21.15.94:0/3472810904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 msgr2=0x55ac65669660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.885+0000 7f06037fe640 1 --2- 172.21.15.94:0/3472810904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65669660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.885+0000 7f06037fe640 1 -- 172.21.15.94:0/3472810904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac6566c580 con 0x55ac65667b40 2024-09-19T04:17:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.885+0000 7f0608bbe640 1 --2- 172.21.15.94:0/3472810904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65669660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.885+0000 7f06037fe640 1 --2- 172.21.15.94:0/3472810904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 0x55ac65669120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55ac65667070 tx=0x7f05f4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=629360de692d4787 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:29.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.885+0000 7f05e9ffb640 1 -- 172.21.15.94:0/3472810904 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4035040 con 0x55ac65667b40 2024-09-19T04:17:29.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.885+0000 7f05e9ffb640 1 -- 172.21.15.94:0/3472810904 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f40316a0 con 0x55ac65667b40 2024-09-19T04:17:29.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.886+0000 7f05e9ffb640 1 -- 172.21.15.94:0/3472810904 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4004750 con 0x55ac65667b40 2024-09-19T04:17:29.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.887+0000 7f060c9fa740 1 -- 172.21.15.94:0/3472810904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 msgr2=0x55ac65669120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:29.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.887+0000 7f060c9fa740 1 --2- 172.21.15.94:0/3472810904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 0x55ac65669120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55ac65667070 tx=0x7f05f4005fd0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.887+0000 7f060c9fa740 1 -- 172.21.15.94:0/3472810904 shutdown_connections 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.887+0000 7f060c9fa740 1 --2- 172.21.15.94:0/3472810904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65671370 0x55ac65669660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.887+0000 7f060c9fa740 1 --2- 172.21.15.94:0/3472810904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65667b40 0x55ac65669120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.887+0000 7f060c9fa740 1 --2- 172.21.15.94:0/3472810904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac65661bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.887+0000 7f060c9fa740 1 -- 172.21.15.94:0/3472810904 >> 172.21.15.94:0/3472810904 conn(0x55ac6562f310 msgr2=0x55ac656614e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.887+0000 7f060c9fa740 1 -- 172.21.15.94:0/3472810904 shutdown_connections 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.888+0000 7f060c9fa740 1 -- 172.21.15.94:0/3472810904 wait complete. 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.888+0000 7f060c9fa740 1 -- start start 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.888+0000 7f060c9fa740 1 Processor -- start 2024-09-19T04:17:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac656749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac65674f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f060c9fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac656ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f060c9fa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06041b2410 con 0x55ac65667b40 2024-09-19T04:17:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f060c9fa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f06041b3880 con 0x55ac656665d0 2024-09-19T04:17:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f060c9fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06041b2ae0 con 0x55ac65671370 2024-09-19T04:17:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f060c9fa740 10 client.0 _ll_register_callbacks cb 0x7f05fc0124f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f0608bbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac656ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f06037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac65674f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f06037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac65674f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55324/0 (socket says 172.21.15.94:55324) 2024-09-19T04:17:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f06037fe640 1 -- 172.21.15.94:0/4186861791 learned_addr learned my addr 172.21.15.94:0/4186861791 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.889+0000 7f0603fff640 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac656749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f0608bbe640 1 -- 172.21.15.94:0/4186861791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 msgr2=0x55ac656749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f0608bbe640 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac656749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f0608bbe640 1 -- 172.21.15.94:0/4186861791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 msgr2=0x55ac65674f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f0608bbe640 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac65674f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f0608bbe640 1 -- 172.21.15.94:0/4186861791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac656ea670 con 0x55ac65671370 2024-09-19T04:17:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f06037fe640 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac65674f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f0603fff640 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac656749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f0608bbe640 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac656ea100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f05fc010870 tx=0x7f05fc020ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c01ce58151ceafd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f05e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:17:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f05e37fe640 1 -- 172.21.15.94:0/4186861791 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc035020 con 0x55ac65671370 2024-09-19T04:17:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f05e37fe640 1 -- 172.21.15.94:0/4186861791 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f05fc020400 con 0x55ac65671370 2024-09-19T04:17:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f060c9fa740 1 -- 172.21.15.94:0/4186861791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05f40316a0 con 0x55ac65671370 2024-09-19T04:17:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f05e37fe640 1 -- 172.21.15.94:0/4186861791 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f05fc040050 con 0x55ac65671370 2024-09-19T04:17:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f060c9fa740 10 client.25207 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.890+0000 7f060c9fa740 1 -- 172.21.15.94:0/4186861791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac656eb970 con 0x55ac65671370 2024-09-19T04:17:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.891+0000 7f05e37fe640 1 -- 172.21.15.94:0/4186861791 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f40316a0 con 0x55ac65671370 2024-09-19T04:17:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.891+0000 7f05e37fe640 1 -- 172.21.15.94:0/4186861791 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f05fc029c50 con 0x55ac65671370 2024-09-19T04:17:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.891+0000 7f060c9fa740 10 client.25207 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.891+0000 7f060c9fa740 10 client.25207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.891+0000 7f060c9fa740 20 client.25207 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.891+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ac656ec110 0x55ac656ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.891+0000 7f060c9fa740 4 client.25207 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.891+0000 7f060c9fa740 1 -- 172.21.15.94:0/4186861791 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac655d3d70 con 0x55ac656ec110 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.891+0000 7f0603fff640 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ac656ec110 0x55ac656ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f0603fff640 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ac656ec110 0x55ac656ee530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8b8cf4df783a137 server_cookie=2f0e8c288bedf0c6 in_seq=0 out_seq=0 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f05e37fe640 10 client.25207 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f05e37fe640 1 -- 172.21.15.94:0/4186861791 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac655d3d70 con 0x55ac656ec110 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f05e37fe640 10 client.25207 handle_command_reply: tid=0 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 client.25207 shutdown 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 -- 172.21.15.94:0/4186861791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 msgr2=0x55ac656ea100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac656ea100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f05fc010870 tx=0x7f05fc020ce0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 -- 172.21.15.94:0/4186861791 shutdown_connections 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ac656ec110 0x55ac656ee530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac65671370 0x55ac656ea100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ac65667b40 0x55ac65674f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 --2- 172.21.15.94:0/4186861791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac656665d0 0x55ac656749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 -- 172.21.15.94:0/4186861791 >> 172.21.15.94:0/4186861791 conn(0x55ac6562f310 msgr2=0x55ac656700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f05e37fe640 0 client.25207 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 -- 172.21.15.94:0/4186861791 shutdown_connections 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.892+0000 7f060c9fa740 1 -- 172.21.15.94:0/4186861791 wait complete. 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060c9fa740 20 client.25207 trim_cache size 0 max 16384 2024-09-19T04:17:29.899 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:17:29.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f05ec0a7840 msgr2=0x7f05ec0a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:29.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f05ec0a7840 0x7f05ec0a9d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009fd0 tx=0x7f05fc009d80 comp rx=0 tx=0).stop 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0604101dd0 msgr2=0x7f0604073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0604101dd0 0x7f0604073b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05f800e9f0 tx=0x7f05f800edf0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 shutdown_connections 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f05ec0a7840 0x7f05ec0a9d00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0604106f00 0x7f060406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0604102780 0x7f060406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 --2- 172.21.15.94:0/1734395904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0604101dd0 0x7f0604073b20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.893+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 >> 172.21.15.94:0/1734395904 conn(0x7f06040fd8b0 msgr2=0x7f06040fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.894+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 shutdown_connections 2024-09-19T04:17:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:29.894+0000 7f060a621640 1 -- 172.21.15.94:0/1734395904 wait complete. 2024-09-19T04:17:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 75.0 s, 1417.0 tps, lat 70.723 ms stddev 28.029 2024-09-19T04:17:31.491 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:31 smithi094 ceph-mon[30972]: pgmap v78: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 1.26k op/s 2024-09-19T04:17:31.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:31 smithi112 ceph-mon[35051]: pgmap v78: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 1.26k op/s 2024-09-19T04:17:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:31 smithi104 ceph-mon[34145]: pgmap v78: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 1.26k op/s 2024-09-19T04:17:32.553 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:17:32.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:17:32.953 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:33.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.578+0000 7f3facdbe640 1 -- 172.21.15.94:0/601989458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fa81008d0 msgr2=0x7f3fa8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:33.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.578+0000 7f3facdbe640 1 --2- 172.21.15.94:0/601989458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fa81008d0 0x7f3fa8100cb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f98013150 tx=0x7f3f98032760 comp rx=0 tx=0).stop 2024-09-19T04:17:33.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.578+0000 7f3facdbe640 1 -- 172.21.15.94:0/601989458 shutdown_connections 2024-09-19T04:17:33.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.578+0000 7f3facdbe640 1 --2- 172.21.15.94:0/601989458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa810e480 0x7f3fa8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.578+0000 7f3facdbe640 1 --2- 172.21.15.94:0/601989458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa8101280 0x7f3fa810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.578+0000 7f3facdbe640 1 --2- 172.21.15.94:0/601989458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fa81008d0 0x7f3fa8100cb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.578+0000 7f3facdbe640 1 -- 172.21.15.94:0/601989458 >> 172.21.15.94:0/601989458 conn(0x7f3fa80fc7a0 msgr2=0x7f3fa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:33.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.578+0000 7f3facdbe640 1 -- 172.21.15.94:0/601989458 shutdown_connections 2024-09-19T04:17:33.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.578+0000 7f3facdbe640 1 -- 172.21.15.94:0/601989458 wait complete. 2024-09-19T04:17:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.579+0000 7f3facdbe640 1 Processor -- start 2024-09-19T04:17:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.579+0000 7f3facdbe640 1 -- start start 2024-09-19T04:17:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.579+0000 7f3facdbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa81008d0 0x7f3fa81966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.579+0000 7f3facdbe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa8101280 0x7f3fa8196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.579+0000 7f3facdbe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fa810e480 0x7f3fa819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3facdbe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8105310 con 0x7f3fa8101280 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3facdbe640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8105190 con 0x7f3fa810e480 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3facdbe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8105490 con 0x7f3fa81008d0 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa81008d0 0x7f3fa81966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa81008d0 0x7f3fa81966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58808/0 (socket says 172.21.15.94:58808) 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa659c640 1 -- 172.21.15.94:0/3058821160 learned_addr learned my addr 172.21.15.94:0/3058821160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa6d9d640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fa810e480 0x7f3fa819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa8101280 0x7f3fa8196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa659c640 1 -- 172.21.15.94:0/3058821160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fa810e480 msgr2=0x7f3fa819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa659c640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fa810e480 0x7f3fa819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa659c640 1 -- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa8101280 msgr2=0x7f3fa8196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa659c640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa8101280 0x7f3fa8196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa659c640 1 -- 172.21.15.94:0/3058821160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa819b150 con 0x7f3fa81008d0 2024-09-19T04:17:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.580+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa8101280 0x7f3fa8196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.581+0000 7f3fa659c640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa81008d0 0x7f3fa81966d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3f98000c00 tx=0x7f3f98011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.581+0000 7f3f977fe640 1 -- 172.21.15.94:0/3058821160 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98047070 con 0x7f3fa81008d0 2024-09-19T04:17:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.581+0000 7f3f977fe640 1 -- 172.21.15.94:0/3058821160 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f98004b50 con 0x7f3fa81008d0 2024-09-19T04:17:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.581+0000 7f3f977fe640 1 -- 172.21.15.94:0/3058821160 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f980038e0 con 0x7f3fa81008d0 2024-09-19T04:17:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.581+0000 7f3facdbe640 1 -- 172.21.15.94:0/3058821160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa819b3e0 con 0x7f3fa81008d0 2024-09-19T04:17:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.581+0000 7f3facdbe640 1 -- 172.21.15.94:0/3058821160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa81a2cd0 con 0x7f3fa81008d0 2024-09-19T04:17:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.584+0000 7f3f977fe640 1 -- 172.21.15.94:0/3058821160 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3f9804a070 con 0x7f3fa81008d0 2024-09-19T04:17:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.585+0000 7f3faf197740 1 Processor -- start 2024-09-19T04:17:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.585+0000 7f3faf197740 1 -- start start 2024-09-19T04:17:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.585+0000 7f3faf197740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc8b60 0x561e74fc8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3faf197740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3faf197740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3faf197740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561e74f2e280 con 0x561e74fd3d40 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3faf197740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561e74ed5b10 con 0x561e74fc9480 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3faf197740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e74f2a240 con 0x561e74fc8b60 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3f977fe640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3f900a3430 0x7f3f900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:41350/0 (socket says 172.21.15.94:41350) 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa659c640 1 -- 172.21.15.94:0/150017189 learned_addr learned my addr 172.21.15.94:0/150017189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3f977fe640 1 -- 172.21.15.94:0/3058821160 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3f980f82e0 con 0x7f3fa81008d0 2024-09-19T04:17:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa659c640 1 -- 172.21.15.94:0/150017189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc8b60 msgr2=0x561e74fc8f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:17:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/150017189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa659c640 1 --2- 172.21.15.94:0/150017189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc8b60 0x561e74fc8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa659c640 1 -- 172.21.15.94:0/150017189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 msgr2=0x561e74fd6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa659c640 1 --2- 172.21.15.94:0/150017189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa659c640 1 -- 172.21.15.94:0/150017189 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e74ee21f0 con 0x561e74fc9480 2024-09-19T04:17:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/150017189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa6d9d640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3f900a3430 0x7f3f900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.586+0000 7f3fa659c640 1 --2- 172.21.15.94:0/150017189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd3800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f980cccf0 tx=0x7f3f98032760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac707a1466fe857f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.587+0000 7f3f94ff9640 1 -- 172.21.15.94:0/150017189 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98051070 con 0x561e74fc9480 2024-09-19T04:17:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.587+0000 7f3f94ff9640 1 -- 172.21.15.94:0/150017189 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f980c4660 con 0x561e74fc9480 2024-09-19T04:17:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.587+0000 7f3f94ff9640 1 -- 172.21.15.94:0/150017189 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f980f7070 con 0x561e74fc9480 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.588+0000 7f3fa6d9d640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3f900a3430 0x7f3f900a58f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80699f0 tx=0x7f3fa0009500 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.588+0000 7f3faf197740 1 -- 172.21.15.94:0/150017189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 msgr2=0x561e74fd3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.588+0000 7f3faf197740 1 --2- 172.21.15.94:0/150017189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd3800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f980cccf0 tx=0x7f3f98032760 comp rx=0 tx=0).stop 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.588+0000 7f3faf197740 1 -- 172.21.15.94:0/150017189 shutdown_connections 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.588+0000 7f3faf197740 1 --2- 172.21.15.94:0/150017189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.588+0000 7f3faf197740 1 --2- 172.21.15.94:0/150017189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd3800 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.588+0000 7f3faf197740 1 --2- 172.21.15.94:0/150017189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc8b60 0x561e74fc8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.588+0000 7f3faf197740 1 -- 172.21.15.94:0/150017189 >> 172.21.15.94:0/150017189 conn(0x561e74fc50f0 msgr2=0x561e74fc54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.589+0000 7f3faf197740 1 -- 172.21.15.94:0/150017189 shutdown_connections 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.589+0000 7f3faf197740 1 -- 172.21.15.94:0/150017189 wait complete. 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.589+0000 7f3faf197740 1 -- start start 2024-09-19T04:17:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.589+0000 7f3faf197740 1 Processor -- start 2024-09-19T04:17:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3faf197740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc8b60 0x561e74fd0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3faf197740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc9480 0x561e74fd1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3faf197740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3faf197740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561e74fd66d0 con 0x561e74fd3d40 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3faf197740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561e74ee8f10 con 0x561e74fc8b60 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3faf197740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e74fd6850 con 0x561e74fc9480 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3faf197740 10 client.0 _ll_register_callbacks cb 0x561e74eb1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc8b60 0x561e74fd0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc8b60 0x561e74fd0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:41358/0 (socket says 172.21.15.94:41358) 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa659c640 1 -- 172.21.15.94:0/1187481424 learned_addr learned my addr 172.21.15.94:0/1187481424 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa6d9d640 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc9480 0x561e74fd1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa659c640 1 -- 172.21.15.94:0/1187481424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc9480 msgr2=0x561e74fd1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa659c640 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc9480 0x561e74fd1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa659c640 1 -- 172.21.15.94:0/1187481424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 msgr2=0x561e74fd1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa659c640 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.590+0000 7f3fa659c640 1 -- 172.21.15.94:0/1187481424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e74c9f6a0 con 0x561e74fc8b60 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3fa659c640 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc8b60 0x561e74fd0be0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f98005ab0 tx=0x7f3f98032ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3dea1836b03c034 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3f8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1187481424 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98051030 con 0x561e74fc8b60 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1187481424 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3f980c0040 con 0x561e74fc8b60 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3faf197740 1 -- 172.21.15.94:0/1187481424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f980c4660 con 0x561e74fc8b60 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1187481424 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3f980ab050 con 0x561e74fc8b60 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3faf197740 10 client.25262 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3faf197740 1 -- 172.21.15.94:0/1187481424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e74fd66d0 con 0x561e74fc8b60 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1187481424 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f980c4660 con 0x561e74fc8b60 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1187481424 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3f980dd240 con 0x561e74fc8b60 2024-09-19T04:17:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.591+0000 7f3faf197740 10 client.25262 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:33.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.592+0000 7f3faf197740 10 client.25262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:17:33.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.592+0000 7f3faf197740 20 client.25262 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:33.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.592+0000 7f3faf197740 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561e74fe35d0 0x561e74fe5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.592+0000 7f3faf197740 4 client.25262 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:33.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.592+0000 7f3faf197740 1 -- 172.21.15.94:0/1187481424 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e74f3b7f0 con 0x561e74fe35d0 2024-09-19T04:17:33.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.592+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561e74fe35d0 0x561e74fe5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.592+0000 7f3f8dffb640 10 client.25262 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:33.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.592+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561e74fe35d0 0x561e74fe5a50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3886835c2c1b0b85 server_cookie=8eb9a0a03edcac31 in_seq=0 out_seq=0 2024-09-19T04:17:33.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.593+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1187481424 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e74f3b7f0 con 0x561e74fe35d0 2024-09-19T04:17:33.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.593+0000 7f3f8dffb640 10 client.25262 handle_command_reply: tid=0 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.593+0000 7f3faf197740 1 client.25262 shutdown 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.593+0000 7f3faf197740 1 -- 172.21.15.94:0/1187481424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc8b60 msgr2=0x561e74fd0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.593+0000 7f3faf197740 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc8b60 0x561e74fd0be0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f98005ab0 tx=0x7f3f98032ee0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3faf197740 1 -- 172.21.15.94:0/1187481424 shutdown_connections 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3faf197740 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561e74fe35d0 0x561e74fe5a50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3faf197740 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3faf197740 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc9480 0x561e74fd1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3faf197740 1 --2- 172.21.15.94:0/1187481424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc8b60 0x561e74fd0be0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3faf197740 1 -- 172.21.15.94:0/1187481424 >> 172.21.15.94:0/1187481424 conn(0x561e74fc50f0 msgr2=0x561e74fcadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3f8dffb640 0 client.25262 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3faf197740 1 -- 172.21.15.94:0/1187481424 shutdown_connections 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3faf197740 1 -- 172.21.15.94:0/1187481424 wait complete. 2024-09-19T04:17:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.594+0000 7f3faf197740 20 client.25262 trim_cache size 0 max 16384 2024-09-19T04:17:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.625+0000 7f3faf197740 1 Processor -- start 2024-09-19T04:17:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.625+0000 7f3faf197740 1 -- start start 2024-09-19T04:17:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3faf197740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fc8b60 0x561e74fcb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3faf197740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3faf197740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fd3d40 0x561e74feb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3faf197740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81adfc0 con 0x561e74fc8b60 2024-09-19T04:17:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3faf197740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81ab800 con 0x561e74fc9480 2024-09-19T04:17:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3faf197740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81a9650 con 0x561e74fd3d40 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fd3d40 0x561e74feb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:41360/0 (socket says 172.21.15.94:41360) 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa5d9b640 1 -- 172.21.15.94:0/2970969149 learned_addr learned my addr 172.21.15.94:0/2970969149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa659c640 1 --2- 172.21.15.94:0/2970969149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fc8b60 0x561e74fcb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa5d9b640 1 -- 172.21.15.94:0/2970969149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fd3d40 msgr2=0x561e74feb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/2970969149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fd3d40 0x561e74feb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa5d9b640 1 -- 172.21.15.94:0/2970969149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fc8b60 msgr2=0x561e74fcb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/2970969149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fc8b60 0x561e74fcb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.626+0000 7f3fa5d9b640 1 -- 172.21.15.94:0/2970969149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e74fc4f20 con 0x561e74fc9480 2024-09-19T04:17:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.627+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/2970969149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd2a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3fa810dff0 tx=0x7f3f9c014e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6217a595f4930a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.627+0000 7f3f8ffff640 1 -- 172.21.15.94:0/2970969149 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c014190 con 0x561e74fc9480 2024-09-19T04:17:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.627+0000 7f3f8ffff640 1 -- 172.21.15.94:0/2970969149 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f9c014330 con 0x561e74fc9480 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.627+0000 7f3f8ffff640 1 -- 172.21.15.94:0/2970969149 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c012670 con 0x561e74fc9480 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.628+0000 7f3faf197740 1 -- 172.21.15.94:0/2970969149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 msgr2=0x561e74fd2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.628+0000 7f3faf197740 1 --2- 172.21.15.94:0/2970969149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd2a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3fa810dff0 tx=0x7f3f9c014e50 comp rx=0 tx=0).stop 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.628+0000 7f3faf197740 1 -- 172.21.15.94:0/2970969149 shutdown_connections 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.628+0000 7f3faf197740 1 --2- 172.21.15.94:0/2970969149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fd3d40 0x561e74feb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.628+0000 7f3faf197740 1 --2- 172.21.15.94:0/2970969149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd2a40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.628+0000 7f3faf197740 1 --2- 172.21.15.94:0/2970969149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fc8b60 0x561e74fcb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.628+0000 7f3faf197740 1 -- 172.21.15.94:0/2970969149 >> 172.21.15.94:0/2970969149 conn(0x561e74fc50f0 msgr2=0x561e74fd2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.629+0000 7f3faf197740 1 -- 172.21.15.94:0/2970969149 shutdown_connections 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.629+0000 7f3faf197740 1 -- 172.21.15.94:0/2970969149 wait complete. 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.629+0000 7f3faf197740 1 -- start start 2024-09-19T04:17:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.629+0000 7f3faf197740 1 Processor -- start 2024-09-19T04:17:33.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3faf197740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc8b60 0x561e74fd7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3faf197740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3faf197740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3faf197740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81a9cc0 con 0x561e74fd3d40 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3faf197740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81ab130 con 0x561e74fc9480 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3faf197740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81aa390 con 0x561e74fc8b60 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3faf197740 10 client.0 _ll_register_callbacks cb 0x7f3f98005580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60064/0 (socket says 172.21.15.94:60064) 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa6d9d640 1 -- 172.21.15.94:0/1562442214 learned_addr learned my addr 172.21.15.94:0/1562442214 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa659c640 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc8b60 0x561e74fd7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa5d9b640 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa6d9d640 1 -- 172.21.15.94:0/1562442214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc8b60 msgr2=0x561e74fd7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa6d9d640 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc8b60 0x561e74fd7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa6d9d640 1 -- 172.21.15.94:0/1562442214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 msgr2=0x561e74fd7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa6d9d640 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.630+0000 7f3fa6d9d640 1 -- 172.21.15.94:0/1562442214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7504d360 con 0x561e74fd3d40 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.631+0000 7f3fa6d9d640 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd81f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0004300 tx=0x7f3fa0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c483b3d0b31905d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.631+0000 7f3f8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.631+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1562442214 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa002b060 con 0x561e74fd3d40 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.631+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1562442214 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3fa001bd00 con 0x561e74fd3d40 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.631+0000 7f3faf197740 1 -- 172.21.15.94:0/1562442214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f9c014330 con 0x561e74fd3d40 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.631+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1562442214 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fa0026400 con 0x561e74fd3d40 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.631+0000 7f3faf197740 10 client.15678 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.631+0000 7f3faf197740 1 -- 172.21.15.94:0/1562442214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e7504e6c0 con 0x561e74fd3d40 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.632+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1562442214 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f9c014330 con 0x561e74fd3d40 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.632+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1562442214 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3fa0018040 con 0x561e74fd3d40 2024-09-19T04:17:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.632+0000 7f3faf197740 10 client.15678 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:33.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.632+0000 7f3faf197740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:17:33.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.632+0000 7f3faf197740 20 client.15678 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:33.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.632+0000 7f3faf197740 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561e7504ee80 0x561e750512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:33.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.632+0000 7f3faf197740 4 client.15678 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:17:33.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.632+0000 7f3faf197740 1 -- 172.21.15.94:0/1562442214 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561e74f35d00 con 0x561e7504ee80 2024-09-19T04:17:33.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.632+0000 7f3fa659c640 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561e7504ee80 0x561e750512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:33.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.633+0000 7f3f8dffb640 10 client.15678 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.633+0000 7f3fa659c640 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561e7504ee80 0x561e750512a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0aea10c4eebc62 server_cookie=62a2a0ffbb66f5c6 in_seq=0 out_seq=0 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3f8dffb640 1 -- 172.21.15.94:0/1562442214 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561e74f35d00 con 0x561e7504ee80 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3f8dffb640 10 client.15678 handle_command_reply: tid=0 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3faf197740 1 client.15678 shutdown 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3faf197740 1 -- 172.21.15.94:0/1562442214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 msgr2=0x561e74fd81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3faf197740 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd81f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0004300 tx=0x7f3fa0004ab0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3faf197740 1 -- 172.21.15.94:0/1562442214 shutdown_connections 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3faf197740 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561e7504ee80 0x561e750512a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3f8dffb640 0 client.15678 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3faf197740 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e74fd3d40 0x561e74fd81f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3faf197740 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e74fc9480 0x561e74fd7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3faf197740 1 --2- 172.21.15.94:0/1562442214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e74fc8b60 0x561e74fd7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.634+0000 7f3faf197740 1 -- 172.21.15.94:0/1562442214 >> 172.21.15.94:0/1562442214 conn(0x561e74fc50f0 msgr2=0x561e74fc4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:33.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.635+0000 7f3faf197740 1 -- 172.21.15.94:0/1562442214 shutdown_connections 2024-09-19T04:17:33.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.635+0000 7f3faf197740 1 -- 172.21.15.94:0/1562442214 wait complete. 2024-09-19T04:17:33.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.635+0000 7f3faf197740 20 client.15678 trim_cache size 0 max 16384 2024-09-19T04:17:33.639 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:17:33.639 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:17:33.639 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "934765be-5a26-4a82-be9a-a6502f3004fe", 2024-09-19T04:17:33.639 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:17:33.639 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:17:33.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 -- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3f900a3430 msgr2=0x7f3f900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:33.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3f900a3430 0x7f3f900a58f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80699f0 tx=0x7f3fa0009500 comp rx=0 tx=0).stop 2024-09-19T04:17:33.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 -- 172.21.15.94:0/3058821160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa81008d0 msgr2=0x7f3fa81966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:33.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa81008d0 0x7f3fa81966d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3f98000c00 tx=0x7f3f98011440 comp rx=0 tx=0).stop 2024-09-19T04:17:33.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 -- 172.21.15.94:0/3058821160 shutdown_connections 2024-09-19T04:17:33.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3f900a3430 0x7f3f900a58f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fa810e480 0x7f3fa819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa8101280 0x7f3fa8196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 --2- 172.21.15.94:0/3058821160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa81008d0 0x7f3fa81966d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:33.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 -- 172.21.15.94:0/3058821160 >> 172.21.15.94:0/3058821160 conn(0x7f3fa80fc7a0 msgr2=0x7f3fa80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:33.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 -- 172.21.15.94:0/3058821160 shutdown_connections 2024-09-19T04:17:33.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:33.636+0000 7f3facdbe640 1 -- 172.21.15.94:0/3058821160 wait complete. 2024-09-19T04:17:33.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:33 smithi112 ceph-mon[35051]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 2.05k op/s 2024-09-19T04:17:33.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:33 smithi094 ceph-mon[30972]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 2.05k op/s 2024-09-19T04:17:33.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:33 smithi104 ceph-mon[34145]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 2.05k op/s 2024-09-19T04:17:34.370 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:17:34.745 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:34 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:17:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:34 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:34 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:17:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:34 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:17:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:34 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:17:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:34 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:34 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:34 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:34 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:17:34.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:34 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:34 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:17:34.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:34 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:17:34.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:34 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:17:34.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:34 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:34 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:34 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:34 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:17:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:34 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:34 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:17:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:34 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:17:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:34 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:17:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:34 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:34 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:34 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:17:35.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.314+0000 7ff8045c3640 1 -- 172.21.15.94:0/3947535186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 msgr2=0x7ff7fc106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.314+0000 7ff8045c3640 1 --2- 172.21.15.94:0/3947535186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 0x7ff7fc106e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009930 tx=0x7ff7f802a420 comp rx=0 tx=0).stop 2024-09-19T04:17:35.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.314+0000 7ff8045c3640 1 -- 172.21.15.94:0/3947535186 shutdown_connections 2024-09-19T04:17:35.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.314+0000 7ff8045c3640 1 --2- 172.21.15.94:0/3947535186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7fc101430 0x7ff7fc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.314+0000 7ff8045c3640 1 --2- 172.21.15.94:0/3947535186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc100a90 0x7ff7fc100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.314+0000 7ff8045c3640 1 --2- 172.21.15.94:0/3947535186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 0x7ff7fc106e70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.314+0000 7ff8045c3640 1 -- 172.21.15.94:0/3947535186 >> 172.21.15.94:0/3947535186 conn(0x7ff7fc0fc7d0 msgr2=0x7ff7fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:35.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.314+0000 7ff8045c3640 1 -- 172.21.15.94:0/3947535186 shutdown_connections 2024-09-19T04:17:35.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.314+0000 7ff8045c3640 1 -- 172.21.15.94:0/3947535186 wait complete. 2024-09-19T04:17:35.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.315+0000 7ff8045c3640 1 Processor -- start 2024-09-19T04:17:35.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.315+0000 7ff8045c3640 1 -- start start 2024-09-19T04:17:35.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.315+0000 7ff8045c3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7fc100a90 0x7ff7fc1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.315+0000 7ff8045c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc101430 0x7ff7fc1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.315+0000 7ff8045c3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 0x7ff7fc19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.315+0000 7ff8045c3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc110d00 con 0x7ff7fc106a90 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.315+0000 7ff8045c3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc110b80 con 0x7ff7fc100a90 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.315+0000 7ff8045c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc110e80 con 0x7ff7fc101430 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff80235f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7fc100a90 0x7ff7fc1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff802b60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 0x7ff7fc19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff802b60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 0x7ff7fc19bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60070/0 (socket says 172.21.15.94:60070) 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff80235f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7fc100a90 0x7ff7fc1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:41394/0 (socket says 172.21.15.94:41394) 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff802b60640 1 -- 172.21.15.94:0/2252533913 learned_addr learned my addr 172.21.15.94:0/2252533913 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff801b5e640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc101430 0x7ff7fc1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff802b60640 1 -- 172.21.15.94:0/2252533913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc101430 msgr2=0x7ff7fc1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff802b60640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc101430 0x7ff7fc1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff802b60640 1 -- 172.21.15.94:0/2252533913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7fc100a90 msgr2=0x7ff7fc1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff802b60640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7fc100a90 0x7ff7fc1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff802b60640 1 -- 172.21.15.94:0/2252533913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc19c250 con 0x7ff7fc106a90 2024-09-19T04:17:35.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff802b60640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 0x7ff7fc19bb70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400eaa0 tx=0x7ff7f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:35.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.316+0000 7ff7f37fe640 1 -- 172.21.15.94:0/2252533913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f400cdb0 con 0x7ff7fc106a90 2024-09-19T04:17:35.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.317+0000 7ff8045c3640 1 -- 172.21.15.94:0/2252533913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7fc19c540 con 0x7ff7fc106a90 2024-09-19T04:17:35.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.317+0000 7ff8045c3640 1 -- 172.21.15.94:0/2252533913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7fc06a790 con 0x7ff7fc106a90 2024-09-19T04:17:35.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.317+0000 7ff7f37fe640 1 -- 172.21.15.94:0/2252533913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7f4005020 con 0x7ff7fc106a90 2024-09-19T04:17:35.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.317+0000 7ff7f37fe640 1 -- 172.21.15.94:0/2252533913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40116d0 con 0x7ff7fc106a90 2024-09-19T04:17:35.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.320+0000 7ff7f37fe640 1 -- 172.21.15.94:0/2252533913 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff7f40051d0 con 0x7ff7fc106a90 2024-09-19T04:17:35.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.321+0000 7ff80699c740 1 Processor -- start 2024-09-19T04:17:35.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.321+0000 7ff80699c740 1 -- start start 2024-09-19T04:17:35.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80699c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac2625880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80699c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2630140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80699c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2630680 0x557ac2632a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80699c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557ac258b2e0 con 0x557ac2630680 2024-09-19T04:17:35.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80699c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ac2532ce0 con 0x557ac2625dc0 2024-09-19T04:17:35.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80699c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557ac25872a0 con 0x557ac26254a0 2024-09-19T04:17:35.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80235f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2630140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff7f37fe640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7e80a3430 0x7ff7e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80235f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2630140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:41398/0 (socket says 172.21.15.94:41398) 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80235f640 1 -- 172.21.15.94:0/671931890 learned_addr learned my addr 172.21.15.94:0/671931890 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff801b5e640 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac2625880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff801b5e640 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2630680 0x557ac2632a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff7f37fe640 1 -- 172.21.15.94:0/2252533913 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff7f40cfd90 con 0x7ff7fc106a90 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80235f640 1 -- 172.21.15.94:0/671931890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 msgr2=0x557ac2625880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80235f640 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac2625880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80235f640 1 -- 172.21.15.94:0/671931890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2630680 msgr2=0x557ac2632a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80235f640 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2630680 0x557ac2632a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80235f640 1 -- 172.21.15.94:0/671931890 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ac2517440 con 0x557ac2625dc0 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff801b5e640 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac2625880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:35.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff802b60640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7e80a3430 0x7ff7e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.322+0000 7ff80235f640 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2630140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff7f802a940 tx=0x7ff7f8005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4d8bf3cf086e973 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:35.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.323+0000 7ff7f0ff9640 1 -- 172.21.15.94:0/671931890 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8035040 con 0x557ac2625dc0 2024-09-19T04:17:35.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.323+0000 7ff7f0ff9640 1 -- 172.21.15.94:0/671931890 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7f8004030 con 0x557ac2625dc0 2024-09-19T04:17:35.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.323+0000 7ff7f0ff9640 1 -- 172.21.15.94:0/671931890 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8005550 con 0x557ac2625dc0 2024-09-19T04:17:35.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff80699c740 1 -- 172.21.15.94:0/671931890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 msgr2=0x557ac2630140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff80699c740 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2630140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff7f802a940 tx=0x7ff7f8005be0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff802b60640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7e80a3430 0x7ff7e80a58f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff7f409c0a0 tx=0x7ff7f4009270 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:35.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff80699c740 1 -- 172.21.15.94:0/671931890 shutdown_connections 2024-09-19T04:17:35.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff80699c740 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2630680 0x557ac2632a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff80699c740 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2630140 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff80699c740 1 --2- 172.21.15.94:0/671931890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac2625880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff80699c740 1 -- 172.21.15.94:0/671931890 >> 172.21.15.94:0/671931890 conn(0x557ac2621a30 msgr2=0x557ac2621e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:35.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff80699c740 1 -- 172.21.15.94:0/671931890 shutdown_connections 2024-09-19T04:17:35.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.324+0000 7ff80699c740 1 -- 172.21.15.94:0/671931890 wait complete. 2024-09-19T04:17:35.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.325+0000 7ff80699c740 1 -- start start 2024-09-19T04:17:35.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.325+0000 7ff80699c740 1 Processor -- start 2024-09-19T04:17:35.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.325+0000 7ff80699c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac262ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.325+0000 7ff80699c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac262d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.325+0000 7ff80699c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 0x557ac262d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.325+0000 7ff80699c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557ac2633250 con 0x557ac26254a0 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.325+0000 7ff80699c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ac2546050 con 0x557ac2625dc0 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.325+0000 7ff80699c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557ac26333d0 con 0x557ac2630680 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff80699c740 10 client.0 _ll_register_callbacks cb 0x557ac2586160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff80235f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac262d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff801b5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac262ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff80235f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac262d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:41402/0 (socket says 172.21.15.94:41402) 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff80235f640 1 -- 172.21.15.94:0/2363523481 learned_addr learned my addr 172.21.15.94:0/2363523481 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff801b5e640 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 0x557ac262d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff801b5e640 1 -- 172.21.15.94:0/2363523481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 msgr2=0x557ac262d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff801b5e640 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 0x557ac262d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff801b5e640 1 -- 172.21.15.94:0/2363523481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 msgr2=0x557ac262d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff801b5e640 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac262d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.326+0000 7ff801b5e640 1 -- 172.21.15.94:0/2363523481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ac22fc3c0 con 0x557ac26254a0 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.327+0000 7ff801b5e640 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac262ff90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0097e0 tx=0x7ff7ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755ec1266a3c4944 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.327+0000 7ff7e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.327+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/2363523481 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00fd20 con 0x557ac26254a0 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.327+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/2363523481 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff7ec005020 con 0x557ac26254a0 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.327+0000 7ff80699c740 1 -- 172.21.15.94:0/2363523481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f8004030 con 0x557ac26254a0 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.327+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/2363523481 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7ec005220 con 0x557ac26254a0 2024-09-19T04:17:35.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.327+0000 7ff80699c740 10 client.15690 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:35.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.327+0000 7ff80699c740 1 -- 172.21.15.94:0/2363523481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557ac263ec30 con 0x557ac26254a0 2024-09-19T04:17:35.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/2363523481 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7ec004c50 con 0x557ac26254a0 2024-09-19T04:17:35.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/2363523481 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff7ec027390 con 0x557ac26254a0 2024-09-19T04:17:35.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff80699c740 10 client.15690 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:35.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff80699c740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:17:35.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff80699c740 20 client.15690 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:35.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff80699c740 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557ac2640590 0x557ac26429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff80699c740 4 client.15690 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:35.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff80699c740 1 -- 172.21.15.94:0/2363523481 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff7f8004030 con 0x557ac2640590 2024-09-19T04:17:35.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff80235f640 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557ac2640590 0x557ac26429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.328+0000 7ff7e5ffb640 10 client.15690 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.329+0000 7ff80235f640 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557ac2640590 0x557ac26429f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5a6f57cd2107e2 server_cookie=259ce636a9291922 in_seq=0 out_seq=0 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.329+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/2363523481 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff7f8004030 con 0x557ac2640590 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.329+0000 7ff7e5ffb640 10 client.15690 handle_command_reply: tid=0 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.329+0000 7ff80699c740 1 client.15690 shutdown 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.330+0000 7ff80699c740 1 -- 172.21.15.94:0/2363523481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 msgr2=0x557ac262ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.330+0000 7ff80699c740 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac262ff90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0097e0 tx=0x7ff7ec004730 comp rx=0 tx=0).stop 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.330+0000 7ff80699c740 1 -- 172.21.15.94:0/2363523481 shutdown_connections 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.330+0000 7ff80699c740 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557ac2640590 0x557ac26429f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.330+0000 7ff80699c740 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 0x557ac262d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.330+0000 7ff7e5ffb640 0 client.15690 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:35.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.330+0000 7ff80699c740 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac262d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.330+0000 7ff80699c740 1 --2- 172.21.15.94:0/2363523481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac262ff90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.330+0000 7ff80699c740 1 -- 172.21.15.94:0/2363523481 >> 172.21.15.94:0/2363523481 conn(0x557ac2621a30 msgr2=0x557ac2627710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:35.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.331+0000 7ff80699c740 1 -- 172.21.15.94:0/2363523481 shutdown_connections 2024-09-19T04:17:35.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.331+0000 7ff80699c740 1 -- 172.21.15.94:0/2363523481 wait complete. 2024-09-19T04:17:35.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.331+0000 7ff80699c740 20 client.15690 trim_cache size 0 max 16384 2024-09-19T04:17:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.357+0000 7ff80699c740 1 Processor -- start 2024-09-19T04:17:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.357+0000 7ff80699c740 1 -- start start 2024-09-19T04:17:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff80699c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac262eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff80699c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2625dc0 0x557ac2648300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff80699c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2630680 0x557ac2648840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff80699c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1ae080 con 0x557ac2625dc0 2024-09-19T04:17:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff80699c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1ab8c0 con 0x557ac2630680 2024-09-19T04:17:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff80699c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1a9710 con 0x557ac26254a0 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff80235f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2625dc0 0x557ac2648300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff80235f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2625dc0 0x557ac2648300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60086/0 (socket says 172.21.15.94:60086) 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff801b5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac262eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff80235f640 1 -- 172.21.15.94:0/1858262380 learned_addr learned my addr 172.21.15.94:0/1858262380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff801b5e640 1 --2- 172.21.15.94:0/1858262380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2630680 0x557ac2648840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff801b5e640 1 -- 172.21.15.94:0/1858262380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2630680 msgr2=0x557ac2648840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.358+0000 7ff801b5e640 1 --2- 172.21.15.94:0/1858262380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2630680 0x557ac2648840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.359+0000 7ff801b5e640 1 -- 172.21.15.94:0/1858262380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2625dc0 msgr2=0x557ac2648300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.359+0000 7ff801b5e640 1 --2- 172.21.15.94:0/1858262380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2625dc0 0x557ac2648300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.359+0000 7ff801b5e640 1 -- 172.21.15.94:0/1858262380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ac2625bf0 con 0x557ac26254a0 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.359+0000 7ff80235f640 1 --2- 172.21.15.94:0/1858262380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2625dc0 0x557ac2648300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.359+0000 7ff801b5e640 1 --2- 172.21.15.94:0/1858262380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac262eaa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec001eb0 tx=0x7ff7ec005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cce8bbb5535785a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.359+0000 7ff7e7fff640 1 -- 172.21.15.94:0/1858262380 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec010920 con 0x557ac26254a0 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.359+0000 7ff7e7fff640 1 -- 172.21.15.94:0/1858262380 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7ec027e20 con 0x557ac26254a0 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff7e7fff640 1 -- 172.21.15.94:0/1858262380 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec027980 con 0x557ac26254a0 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff80699c740 1 -- 172.21.15.94:0/1858262380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 msgr2=0x557ac262eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff80699c740 1 --2- 172.21.15.94:0/1858262380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac262eaa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec001eb0 tx=0x7ff7ec005f90 comp rx=0 tx=0).stop 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff80699c740 1 -- 172.21.15.94:0/1858262380 shutdown_connections 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff80699c740 1 --2- 172.21.15.94:0/1858262380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2630680 0x557ac2648840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff80699c740 1 --2- 172.21.15.94:0/1858262380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac2625dc0 0x557ac2648300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff80699c740 1 --2- 172.21.15.94:0/1858262380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac26254a0 0x557ac262eaa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff80699c740 1 -- 172.21.15.94:0/1858262380 >> 172.21.15.94:0/1858262380 conn(0x557ac2621a30 msgr2=0x557ac2620b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:35.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff80699c740 1 -- 172.21.15.94:0/1858262380 shutdown_connections 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.360+0000 7ff80699c740 1 -- 172.21.15.94:0/1858262380 wait complete. 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff80699c740 1 -- start start 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff80699c740 1 Processor -- start 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff80699c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac2638d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff80699c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2636060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff80699c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 0x557ac26365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff80699c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1a9d80 con 0x557ac26254a0 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff80699c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1ab1f0 con 0x557ac2625dc0 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff80699c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1aa450 con 0x557ac2630680 2024-09-19T04:17:35.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff80699c740 10 client.0 _ll_register_callbacks cb 0x7ff7ec009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:35.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff801b5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac2638d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.361+0000 7ff801b5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac2638d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60096/0 (socket says 172.21.15.94:60096) 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.362+0000 7ff801b5e640 1 -- 172.21.15.94:0/452367183 learned_addr learned my addr 172.21.15.94:0/452367183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.362+0000 7ff80235f640 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2636060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.362+0000 7ff801b5e640 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 0x557ac26365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.362+0000 7ff801b5e640 1 -- 172.21.15.94:0/452367183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 msgr2=0x557ac26365a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.362+0000 7ff801b5e640 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 0x557ac26365a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.362+0000 7ff801b5e640 1 -- 172.21.15.94:0/452367183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 msgr2=0x557ac2636060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.362+0000 7ff801b5e640 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2636060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.362+0000 7ff801b5e640 1 -- 172.21.15.94:0/452367183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ac2636d80 con 0x557ac26254a0 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.362+0000 7ff80235f640 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2636060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.363+0000 7ff801b5e640 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac2638d10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec028050 tx=0x7ff7ec0053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eec5ce6a3deda36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:35.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.363+0000 7ff7e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:17:35.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.363+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/452367183 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec027980 con 0x557ac26254a0 2024-09-19T04:17:35.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.363+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/452367183 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff7ec027b20 con 0x557ac26254a0 2024-09-19T04:17:35.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.363+0000 7ff80699c740 1 -- 172.21.15.94:0/452367183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7ec027e20 con 0x557ac26254a0 2024-09-19T04:17:35.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.363+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/452367183 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7ec00fd30 con 0x557ac26254a0 2024-09-19T04:17:35.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff80699c740 10 client.15696 fetch_fsmap learned FSMap version 30 2024-09-19T04:17:35.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff80699c740 1 -- 172.21.15.94:0/452367183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557ac26ab2a0 con 0x557ac26254a0 2024-09-19T04:17:35.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/452367183 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7ec027e20 con 0x557ac26254a0 2024-09-19T04:17:35.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/452367183 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff7ec034040 con 0x557ac26254a0 2024-09-19T04:17:35.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff80699c740 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:17:35.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff80699c740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:17:35.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff80699c740 20 client.15696 populate_metadata read hostname 'smithi094' 2024-09-19T04:17:35.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff80699c740 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557ac2640590 0x557ac26ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff80699c740 4 client.15696 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:17:35.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.364+0000 7ff80699c740 1 -- 172.21.15.94:0/452367183 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ac2598850 con 0x557ac2640590 2024-09-19T04:17:35.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.365+0000 7ff80235f640 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557ac2640590 0x557ac26ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.365+0000 7ff7e5ffb640 10 client.15696 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:35.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.365+0000 7ff80235f640 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557ac2640590 0x557ac26ad920 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe2abcc3572c4c1 server_cookie=8ac494ccb4fa027f in_seq=0 out_seq=0 2024-09-19T04:17:35.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff7e5ffb640 1 -- 172.21.15.94:0/452367183 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557ac2598850 con 0x557ac2640590 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff7e5ffb640 10 client.15696 handle_command_reply: tid=0 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 client.15696 shutdown 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 -- 172.21.15.94:0/452367183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 msgr2=0x557ac2638d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac2638d10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec028050 tx=0x7ff7ec0053c0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 -- 172.21.15.94:0/452367183 shutdown_connections 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557ac2640590 0x557ac26ad920 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ac2630680 0x557ac26365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ac2625dc0 0x557ac2636060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 --2- 172.21.15.94:0/452367183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ac26254a0 0x557ac2638d10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 -- 172.21.15.94:0/452367183 >> 172.21.15.94:0/452367183 conn(0x557ac2621a30 msgr2=0x557ac262ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff7e5ffb640 0 client.15696 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 -- 172.21.15.94:0/452367183 shutdown_connections 2024-09-19T04:17:35.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 1 -- 172.21.15.94:0/452367183 wait complete. 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "934765be-5a26-4a82-be9a-a6502f3004fe": { 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "934765be-5a26-4a82-be9a-a6502f3004fe", 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:17:35.373 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:17:35.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.366+0000 7ff80699c740 20 client.15696 trim_cache size 0 max 16384 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 -- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7e80a3430 msgr2=0x7ff7e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7e80a3430 0x7ff7e80a58f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff7f409c0a0 tx=0x7ff7f4009270 comp rx=0 tx=0).stop 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 -- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 msgr2=0x7ff7fc19bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 0x7ff7fc19bb70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400eaa0 tx=0x7ff7f400ef60 comp rx=0 tx=0).stop 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 -- 172.21.15.94:0/2252533913 shutdown_connections 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7e80a3430 0x7ff7e80a58f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7fc106a90 0x7ff7fc19bb70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc101430 0x7ff7fc1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 --2- 172.21.15.94:0/2252533913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7fc100a90 0x7ff7fc1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.367+0000 7ff8045c3640 1 -- 172.21.15.94:0/2252533913 >> 172.21.15.94:0/2252533913 conn(0x7ff7fc0fc7d0 msgr2=0x7ff7fc0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.368+0000 7ff8045c3640 1 -- 172.21.15.94:0/2252533913 shutdown_connections 2024-09-19T04:17:35.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:17:35.368+0000 7ff8045c3640 1 -- 172.21.15.94:0/2252533913 wait complete. 2024-09-19T04:17:35.501 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:35 smithi094 ceph-mon[30972]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-09-19T04:17:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 80.0 s, 1476.8 tps, lat 67.488 ms stddev 34.748 2024-09-19T04:17:35.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:35 smithi112 ceph-mon[35051]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-09-19T04:17:35.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:35 smithi104 ceph-mon[34145]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-09-19T04:17:36.220 INFO:tasks.cephfs.filesystem:scrub status for tag:934765be-5a26-4a82-be9a-a6502f3004fe - {'path': '/', 'tag': '934765be-5a26-4a82-be9a-a6502f3004fe', 'options': 'recursive,force'} 2024-09-19T04:17:37.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:37 smithi112 ceph-mon[35051]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 2.18k op/s 2024-09-19T04:17:37.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:37 smithi094 ceph-mon[30972]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 2.18k op/s 2024-09-19T04:17:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:37 smithi104 ceph-mon[34145]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 2.18k op/s 2024-09-19T04:17:39.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:39 smithi112 ceph-mon[35051]: pgmap v82: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.56k op/s 2024-09-19T04:17:39.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:39 smithi094 ceph-mon[30972]: pgmap v82: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.56k op/s 2024-09-19T04:17:39.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:39 smithi104 ceph-mon[34145]: pgmap v82: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.56k op/s 2024-09-19T04:17:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 85.0 s, 1706.8 tps, lat 58.654 ms stddev 15.050 2024-09-19T04:17:41.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:41 smithi112 ceph-mon[35051]: pgmap v83: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.56k op/s 2024-09-19T04:17:41.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:41 smithi094 ceph-mon[30972]: pgmap v83: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.56k op/s 2024-09-19T04:17:41.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:41 smithi104 ceph-mon[34145]: pgmap v83: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.56k op/s 2024-09-19T04:17:42.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:42 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:17:42.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:42 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:17:42.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:42 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:17:43.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:43 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:17:43.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:43 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:17:43.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:43 smithi112 ceph-mon[35051]: pgmap v84: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 20 MiB/s wr, 2.36k op/s 2024-09-19T04:17:43.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:43 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:17:43.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:43 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:17:43.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:43 smithi094 ceph-mon[30972]: pgmap v84: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 20 MiB/s wr, 2.36k op/s 2024-09-19T04:17:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:43 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:17:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:43 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:17:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:43 smithi104 ceph-mon[34145]: pgmap v84: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 20 MiB/s wr, 2.36k op/s 2024-09-19T04:17:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 90.0 s, 1592.6 tps, lat 62.760 ms stddev 15.801 2024-09-19T04:17:45.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:45 smithi112 ceph-mon[35051]: pgmap v85: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.57k op/s 2024-09-19T04:17:45.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:45 smithi094 ceph-mon[30972]: pgmap v85: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.57k op/s 2024-09-19T04:17:45.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:45 smithi104 ceph-mon[34145]: pgmap v85: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.57k op/s 2024-09-19T04:17:47.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:47 smithi112 ceph-mon[35051]: pgmap v86: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 21 MiB/s wr, 2.26k op/s 2024-09-19T04:17:47.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:47 smithi094 ceph-mon[30972]: pgmap v86: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 21 MiB/s wr, 2.26k op/s 2024-09-19T04:17:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:47 smithi104 ceph-mon[34145]: pgmap v86: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 21 MiB/s wr, 2.26k op/s 2024-09-19T04:17:49.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:49 smithi112 ceph-mon[35051]: pgmap v87: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 1.49k op/s 2024-09-19T04:17:49.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:49 smithi094 ceph-mon[30972]: pgmap v87: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 1.49k op/s 2024-09-19T04:17:49.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:49 smithi104 ceph-mon[34145]: pgmap v87: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 1.49k op/s 2024-09-19T04:17:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 95.0 s, 1405.0 tps, lat 71.078 ms stddev 60.892 2024-09-19T04:17:51.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:51 smithi112 ceph-mon[35051]: pgmap v88: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 1.49k op/s 2024-09-19T04:17:51.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:51 smithi094 ceph-mon[30972]: pgmap v88: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 1.49k op/s 2024-09-19T04:17:51.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:51 smithi104 ceph-mon[34145]: pgmap v88: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 1.49k op/s 2024-09-19T04:17:53.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:53 smithi112 ceph-mon[35051]: pgmap v89: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 20 MiB/s wr, 2.17k op/s 2024-09-19T04:17:53.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:53 smithi094 ceph-mon[30972]: pgmap v89: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 20 MiB/s wr, 2.17k op/s 2024-09-19T04:17:53.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:53 smithi104 ceph-mon[34145]: pgmap v89: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 20 MiB/s wr, 2.17k op/s 2024-09-19T04:17:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 100.0 s, 1516.2 tps, lat 66.088 ms stddev 16.627 2024-09-19T04:17:55.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:55 smithi112 ceph-mon[35051]: pgmap v90: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.37k op/s 2024-09-19T04:17:55.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:55 smithi094 ceph-mon[30972]: pgmap v90: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.37k op/s 2024-09-19T04:17:55.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:55 smithi104 ceph-mon[34145]: pgmap v90: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.37k op/s 2024-09-19T04:17:57.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:57 smithi112 ceph-mon[35051]: pgmap v91: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 2.09k op/s 2024-09-19T04:17:57.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:57 smithi094 ceph-mon[30972]: pgmap v91: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 2.09k op/s 2024-09-19T04:17:57.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:57 smithi104 ceph-mon[34145]: pgmap v91: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 2.09k op/s 2024-09-19T04:17:59.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:17:59 smithi112 ceph-mon[35051]: pgmap v92: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:17:59.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:17:59 smithi094 ceph-mon[30972]: pgmap v92: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:17:59.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:17:59 smithi104 ceph-mon[34145]: pgmap v92: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:18:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 105.0 s, 1610.8 tps, lat 62.040 ms stddev 17.271 2024-09-19T04:18:01.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:01 smithi112 ceph-mon[35051]: pgmap v93: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:18:01.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:01 smithi094 ceph-mon[30972]: pgmap v93: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:18:01.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:01 smithi104 ceph-mon[34145]: pgmap v93: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:18:03.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:03 smithi112 ceph-mon[35051]: pgmap v94: 97 pgs: 97 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 2.19k op/s 2024-09-19T04:18:03.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:03 smithi094 ceph-mon[30972]: pgmap v94: 97 pgs: 97 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 2.19k op/s 2024-09-19T04:18:03.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:03 smithi104 ceph-mon[34145]: pgmap v94: 97 pgs: 97 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 2.19k op/s 2024-09-19T04:18:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 110.0 s, 1702.0 tps, lat 58.684 ms stddev 17.307 2024-09-19T04:18:05.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:05 smithi112 ceph-mon[35051]: pgmap v95: 97 pgs: 97 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-09-19T04:18:05.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:05 smithi094 ceph-mon[30972]: pgmap v95: 97 pgs: 97 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-09-19T04:18:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:05 smithi104 ceph-mon[34145]: pgmap v95: 97 pgs: 97 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-09-19T04:18:06.221 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:18:06.635 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.217+0000 7f5a7247b640 1 -- 172.21.15.94:0/3286540209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6c102ef0 msgr2=0x7f5a6c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.217+0000 7f5a7247b640 1 --2- 172.21.15.94:0/3286540209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6c102ef0 0x7f5a6c10da40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009ca0 tx=0x7f5a6402a950 comp rx=0 tx=0).stop 2024-09-19T04:18:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.217+0000 7f5a7247b640 1 -- 172.21.15.94:0/3286540209 shutdown_connections 2024-09-19T04:18:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.217+0000 7f5a7247b640 1 --2- 172.21.15.94:0/3286540209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6c102ef0 0x7f5a6c10da40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.217+0000 7f5a7247b640 1 --2- 172.21.15.94:0/3286540209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c102570 0x7f5a6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.217+0000 7f5a7247b640 1 --2- 172.21.15.94:0/3286540209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c100e20 0x7f5a6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.217+0000 7f5a7247b640 1 -- 172.21.15.94:0/3286540209 >> 172.21.15.94:0/3286540209 conn(0x7f5a6c0fc7d0 msgr2=0x7f5a6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.218+0000 7f5a7247b640 1 -- 172.21.15.94:0/3286540209 shutdown_connections 2024-09-19T04:18:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.218+0000 7f5a7247b640 1 -- 172.21.15.94:0/3286540209 wait complete. 2024-09-19T04:18:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.218+0000 7f5a7247b640 1 Processor -- start 2024-09-19T04:18:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a7247b640 1 -- start start 2024-09-19T04:18:07.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a7247b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c100e20 0x7f5a6c1a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a7247b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6c102570 0x7f5a6c1a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a7247b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c102ef0 0x7f5a6c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a7247b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c110e80 con 0x7f5a6c102570 2024-09-19T04:18:07.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a63fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6c102570 0x7f5a6c1a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a7247b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c110d00 con 0x7f5a6c100e20 2024-09-19T04:18:07.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a7247b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c111000 con 0x7f5a6c102ef0 2024-09-19T04:18:07.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a63fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6c102570 0x7f5a6c1a3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58004/0 (socket says 172.21.15.94:58004) 2024-09-19T04:18:07.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a63fff640 1 -- 172.21.15.94:0/417249661 learned_addr learned my addr 172.21.15.94:0/417249661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:07.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c100e20 0x7f5a6c1a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.219+0000 7f5a70a18640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c102ef0 0x7f5a6c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.220+0000 7f5a6bfff640 1 -- 172.21.15.94:0/417249661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c102ef0 msgr2=0x7f5a6c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.220+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c102ef0 0x7f5a6c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.220+0000 7f5a6bfff640 1 -- 172.21.15.94:0/417249661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6c102570 msgr2=0x7f5a6c1a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.220+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6c102570 0x7f5a6c1a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.220+0000 7f5a6bfff640 1 -- 172.21.15.94:0/417249661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c19f3c0 con 0x7f5a6c100e20 2024-09-19T04:18:07.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.220+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c100e20 0x7f5a6c1a35c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e9f0 tx=0x7f5a5c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.220+0000 7f5a69ffb640 1 -- 172.21.15.94:0/417249661 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00ccc0 con 0x7f5a6c100e20 2024-09-19T04:18:07.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.221+0000 7f5a69ffb640 1 -- 172.21.15.94:0/417249661 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a5c0108e0 con 0x7f5a6c100e20 2024-09-19T04:18:07.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.221+0000 7f5a69ffb640 1 -- 172.21.15.94:0/417249661 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c018880 con 0x7f5a6c100e20 2024-09-19T04:18:07.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.221+0000 7f5a7247b640 1 -- 172.21.15.94:0/417249661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c19f6b0 con 0x7f5a6c100e20 2024-09-19T04:18:07.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.221+0000 7f5a7247b640 1 -- 172.21.15.94:0/417249661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c1ab4f0 con 0x7f5a6c100e20 2024-09-19T04:18:07.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.224+0000 7f5a69ffb640 1 -- 172.21.15.94:0/417249661 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5a5c010a90 con 0x7f5a6c100e20 2024-09-19T04:18:07.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.225+0000 7f5a69ffb640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a580a3430 0x7f5a580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.225+0000 7f5a74854740 1 Processor -- start 2024-09-19T04:18:07.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a69ffb640 1 -- 172.21.15.94:0/417249661 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f5a5c0cffe0 con 0x7f5a6c100e20 2024-09-19T04:18:07.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a74854740 1 -- start start 2024-09-19T04:18:07.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a74854740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330c4a0 0x559fa330c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a74854740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330cdc0 0x559fa3317140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330c4a0 0x559fa330c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a74854740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 0x559fa3319a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330c4a0 0x559fa330c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44732/0 (socket says 172.21.15.94:44732) 2024-09-19T04:18:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a6bfff640 1 -- 172.21.15.94:0/1947958565 learned_addr learned my addr 172.21.15.94:0/1947958565 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a74854740 1 -- 172.21.15.94:0/1947958565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559fa32722e0 con 0x559fa330cdc0 2024-09-19T04:18:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a74854740 1 -- 172.21.15.94:0/1947958565 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559fa3219ce0 con 0x559fa3317680 2024-09-19T04:18:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a74854740 1 -- 172.21.15.94:0/1947958565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559fa326e2a0 con 0x559fa330c4a0 2024-09-19T04:18:07.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a63fff640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a580a3430 0x7f5a580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.226+0000 7f5a70a18640 1 --2- 172.21.15.94:0/1947958565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 0x559fa3319a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.227+0000 7f5a6bfff640 1 -- 172.21.15.94:0/1947958565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 msgr2=0x559fa3319a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.227+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/1947958565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 0x559fa3319a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.227+0000 7f5a6bfff640 1 -- 172.21.15.94:0/1947958565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330cdc0 msgr2=0x559fa3317140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.227+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/1947958565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330cdc0 0x559fa3317140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.227+0000 7f5a6bfff640 1 -- 172.21.15.94:0/1947958565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa31fe440 con 0x559fa330c4a0 2024-09-19T04:18:07.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.227+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/1947958565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330c4a0 0x559fa330c880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c098d30 tx=0x7f5a5c09c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c221b6075de997e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.227+0000 7f5a627fc640 1 -- 172.21.15.94:0/1947958565 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0a4f20 con 0x559fa330c4a0 2024-09-19T04:18:07.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.227+0000 7f5a627fc640 1 -- 172.21.15.94:0/1947958565 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a5c0cd9a0 con 0x559fa330c4a0 2024-09-19T04:18:07.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.227+0000 7f5a627fc640 1 -- 172.21.15.94:0/1947958565 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c09ba10 con 0x559fa330c4a0 2024-09-19T04:18:07.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.228+0000 7f5a74854740 1 -- 172.21.15.94:0/1947958565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330c4a0 msgr2=0x559fa330c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.228+0000 7f5a74854740 1 --2- 172.21.15.94:0/1947958565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330c4a0 0x559fa330c880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c098d30 tx=0x7f5a5c09c360 comp rx=0 tx=0).stop 2024-09-19T04:18:07.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.228+0000 7f5a63fff640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a580a3430 0x7f5a580a58f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x559fa3309890 tx=0x7f5a54009990 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.228+0000 7f5a74854740 1 -- 172.21.15.94:0/1947958565 shutdown_connections 2024-09-19T04:18:07.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.228+0000 7f5a74854740 1 --2- 172.21.15.94:0/1947958565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 0x559fa3319a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.229+0000 7f5a74854740 1 --2- 172.21.15.94:0/1947958565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330cdc0 0x559fa3317140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.229+0000 7f5a74854740 1 --2- 172.21.15.94:0/1947958565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330c4a0 0x559fa330c880 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.229+0000 7f5a74854740 1 -- 172.21.15.94:0/1947958565 >> 172.21.15.94:0/1947958565 conn(0x559fa3308a30 msgr2=0x559fa3308e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.229+0000 7f5a74854740 1 -- 172.21.15.94:0/1947958565 shutdown_connections 2024-09-19T04:18:07.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.229+0000 7f5a74854740 1 -- 172.21.15.94:0/1947958565 wait complete. 2024-09-19T04:18:07.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.229+0000 7f5a74854740 1 -- start start 2024-09-19T04:18:07.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a74854740 1 Processor -- start 2024-09-19T04:18:07.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a74854740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa3316f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a74854740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa3314220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a74854740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 0x559fa3314760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a74854740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559fa331a2a0 con 0x559fa330c4a0 2024-09-19T04:18:07.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a74854740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559fa322d050 con 0x559fa3317680 2024-09-19T04:18:07.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a74854740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559fa331a420 con 0x559fa330cdc0 2024-09-19T04:18:07.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa3314220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa3314220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44746/0 (socket says 172.21.15.94:44746) 2024-09-19T04:18:07.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a63fff640 1 -- 172.21.15.94:0/3378735572 learned_addr learned my addr 172.21.15.94:0/3378735572 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:07.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.230+0000 7f5a74854740 10 client.0 _ll_register_callbacks cb 0x559fa326d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:07.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa3316f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a70a18640 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 0x559fa3314760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a6bfff640 1 -- 172.21.15.94:0/3378735572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 msgr2=0x559fa3314220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa3314220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a6bfff640 1 -- 172.21.15.94:0/3378735572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 msgr2=0x559fa3314760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 0x559fa3314760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a6bfff640 1 -- 172.21.15.94:0/3378735572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa2fe33c0 con 0x559fa330c4a0 2024-09-19T04:18:07.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a63fff640 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa3314220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:07.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa3316f70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c09c2f0 tx=0x7f5a5c0cdb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e41a194b4ba797 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.231+0000 7f5a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:18:07.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.232+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/3378735572 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0b52a0 con 0x559fa330c4a0 2024-09-19T04:18:07.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.232+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/3378735572 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f5a5c09a660 con 0x559fa330c4a0 2024-09-19T04:18:07.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.232+0000 7f5a74854740 1 -- 172.21.15.94:0/3378735572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a5c0cd9a0 con 0x559fa330c4a0 2024-09-19T04:18:07.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.232+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/3378735572 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a5c010430 con 0x559fa330c4a0 2024-09-19T04:18:07.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.232+0000 7f5a74854740 10 client.15705 fetch_fsmap learned FSMap version 30 2024-09-19T04:18:07.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.232+0000 7f5a74854740 1 -- 172.21.15.94:0/3378735572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fa3325c10 con 0x559fa330c4a0 2024-09-19T04:18:07.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.233+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/3378735572 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a5c0105d0 con 0x559fa330c4a0 2024-09-19T04:18:07.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.233+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/3378735572 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f5a5c09b220 con 0x559fa330c4a0 2024-09-19T04:18:07.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.233+0000 7f5a74854740 10 client.15705 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:18:07.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.233+0000 7f5a74854740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:07.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.233+0000 7f5a74854740 20 client.15705 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:07.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.233+0000 7f5a74854740 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559fa3326f60 0x559fa3329350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.233+0000 7f5a74854740 4 client.15705 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:07.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.233+0000 7f5a74854740 1 -- 172.21.15.94:0/3378735572 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5a5c0cd9a0 con 0x559fa3326f60 2024-09-19T04:18:07.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.233+0000 7f5a63fff640 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559fa3326f60 0x559fa3329350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.234+0000 7f5a4b7fe640 10 client.15705 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:07.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.234+0000 7f5a63fff640 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559fa3326f60 0x559fa3329350 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba3bcab06e35b7 server_cookie=be2874572b3d380a in_seq=0 out_seq=0 2024-09-19T04:18:07.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.234+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/3378735572 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5a5c0cd9a0 con 0x559fa3326f60 2024-09-19T04:18:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a4b7fe640 10 client.15705 handle_command_reply: tid=0 2024-09-19T04:18:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a74854740 1 client.15705 shutdown 2024-09-19T04:18:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a74854740 1 -- 172.21.15.94:0/3378735572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 msgr2=0x559fa3316f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a74854740 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa3316f70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c09c2f0 tx=0x7f5a5c0cdb50 comp rx=0 tx=0).stop 2024-09-19T04:18:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a74854740 1 -- 172.21.15.94:0/3378735572 shutdown_connections 2024-09-19T04:18:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a74854740 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559fa3326f60 0x559fa3329350 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a74854740 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa3317680 0x559fa3314760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a74854740 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa3314220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a74854740 1 --2- 172.21.15.94:0/3378735572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa3316f70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.235+0000 7f5a74854740 1 -- 172.21.15.94:0/3378735572 >> 172.21.15.94:0/3378735572 conn(0x559fa3308a30 msgr2=0x559fa330e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.236+0000 7f5a4b7fe640 0 client.15705 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:07.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.236+0000 7f5a74854740 1 -- 172.21.15.94:0/3378735572 shutdown_connections 2024-09-19T04:18:07.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.236+0000 7f5a74854740 1 -- 172.21.15.94:0/3378735572 wait complete. 2024-09-19T04:18:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.236+0000 7f5a74854740 20 client.15705 trim_cache size 0 max 16384 2024-09-19T04:18:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.268+0000 7f5a74854740 1 Processor -- start 2024-09-19T04:18:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.268+0000 7f5a74854740 1 -- start start 2024-09-19T04:18:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.268+0000 7f5a74854740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330c4a0 0x559fa3307a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.268+0000 7f5a74854740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa332bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.268+0000 7f5a74854740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa3317680 0x559fa332c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a74854740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1ae070 con 0x559fa3317680 2024-09-19T04:18:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a74854740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1ab8b0 con 0x559fa330c4a0 2024-09-19T04:18:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a74854740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c19cc00 con 0x559fa330cdc0 2024-09-19T04:18:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330c4a0 0x559fa3307a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330c4a0 0x559fa3307a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:43878/0 (socket says 172.21.15.94:43878) 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a6bfff640 1 -- 172.21.15.94:0/437319226 learned_addr learned my addr 172.21.15.94:0/437319226 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a63fff640 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa332bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a6bfff640 1 -- 172.21.15.94:0/437319226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 msgr2=0x559fa332bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa332bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a70a18640 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa3317680 0x559fa332c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a6bfff640 1 -- 172.21.15.94:0/437319226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa3317680 msgr2=0x559fa332c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa3317680 0x559fa332c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a6bfff640 1 -- 172.21.15.94:0/437319226 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa33084d0 con 0x559fa330c4a0 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a63fff640 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa332bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a70a18640 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa3317680 0x559fa332c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.269+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330c4a0 0x559fa3307a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0a50e0 tx=0x7f5a5c09bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9107301119aef838 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.270+0000 7f5a61ffb640 1 -- 172.21.15.94:0/437319226 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0cd9a0 con 0x559fa330c4a0 2024-09-19T04:18:07.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.270+0000 7f5a61ffb640 1 -- 172.21.15.94:0/437319226 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a5c0b52a0 con 0x559fa330c4a0 2024-09-19T04:18:07.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.270+0000 7f5a61ffb640 1 -- 172.21.15.94:0/437319226 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0a1990 con 0x559fa330c4a0 2024-09-19T04:18:07.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.271+0000 7f5a74854740 1 -- 172.21.15.94:0/437319226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330c4a0 msgr2=0x559fa3307a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.271+0000 7f5a74854740 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330c4a0 0x559fa3307a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0a50e0 tx=0x7f5a5c09bfd0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.271+0000 7f5a74854740 1 -- 172.21.15.94:0/437319226 shutdown_connections 2024-09-19T04:18:07.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.271+0000 7f5a74854740 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa3317680 0x559fa332c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.271+0000 7f5a74854740 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa330cdc0 0x559fa332bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.271+0000 7f5a74854740 1 --2- 172.21.15.94:0/437319226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330c4a0 0x559fa3307a80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.271+0000 7f5a74854740 1 -- 172.21.15.94:0/437319226 >> 172.21.15.94:0/437319226 conn(0x559fa3308a30 msgr2=0x559fa3307370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.271+0000 7f5a74854740 1 -- 172.21.15.94:0/437319226 shutdown_connections 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.271+0000 7f5a74854740 1 -- 172.21.15.94:0/437319226 wait complete. 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.272+0000 7f5a74854740 1 -- start start 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.272+0000 7f5a74854740 1 Processor -- start 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a74854740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa332af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a74854740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330cdc0 0x559fa332b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a74854740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa3317680 0x559fa332b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a74854740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c19d270 con 0x559fa330c4a0 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a74854740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c19e6e0 con 0x559fa330cdc0 2024-09-19T04:18:07.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a74854740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c19d940 con 0x559fa3317680 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a63fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330cdc0 0x559fa332b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a74854740 10 client.0 _ll_register_callbacks cb 0x7f5a5c09bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a70a18640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa3317680 0x559fa332b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a70a18640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa3317680 0x559fa332b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44764/0 (socket says 172.21.15.94:44764) 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a70a18640 1 -- 172.21.15.94:0/2427044411 learned_addr learned my addr 172.21.15.94:0/2427044411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa332af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.273+0000 7f5a63fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330cdc0 0x559fa332b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:43882/0 (socket says 172.21.15.94:43882) 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a6bfff640 1 -- 172.21.15.94:0/2427044411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa3317680 msgr2=0x559fa332b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa3317680 0x559fa332b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a6bfff640 1 -- 172.21.15.94:0/2427044411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330cdc0 msgr2=0x559fa332b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330cdc0 0x559fa332b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a6bfff640 1 -- 172.21.15.94:0/2427044411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa338fb40 con 0x559fa330c4a0 2024-09-19T04:18:07.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a6bfff640 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa332af60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c098d30 tx=0x7f5a5c0982e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9d4350a8e95f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/2427044411 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c098600 con 0x559fa330c4a0 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/2427044411 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f5a5c09a660 con 0x559fa330c4a0 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.274+0000 7f5a74854740 1 -- 172.21.15.94:0/2427044411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a5c0b52a0 con 0x559fa330c4a0 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/2427044411 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a5c09b9c0 con 0x559fa330c4a0 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a74854740 10 client.15708 fetch_fsmap learned FSMap version 30 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a74854740 1 -- 172.21.15.94:0/2427044411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fa338fb40 con 0x559fa330c4a0 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/2427044411 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a5c0b52a0 con 0x559fa330c4a0 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/2427044411 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f5a5c0851b0 con 0x559fa330c4a0 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a74854740 10 client.15708 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a74854740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a74854740 20 client.15708 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:07.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a74854740 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559fa3391050 0x559fa3393440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a74854740 4 client.15708 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:18:07.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.275+0000 7f5a74854740 1 -- 172.21.15.94:0/2427044411 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559fa327f850 con 0x559fa3391050 2024-09-19T04:18:07.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.276+0000 7f5a63fff640 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559fa3391050 0x559fa3393440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.276+0000 7f5a4b7fe640 10 client.15708 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:07.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.276+0000 7f5a63fff640 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559fa3391050 0x559fa3393440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56e92aae7065baa server_cookie=2b8fd1732fc11439 in_seq=0 out_seq=0 2024-09-19T04:18:07.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a4b7fe640 1 -- 172.21.15.94:0/2427044411 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559fa327f850 con 0x559fa3391050 2024-09-19T04:18:07.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a4b7fe640 10 client.15708 handle_command_reply: tid=0 2024-09-19T04:18:07.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 client.15708 shutdown 2024-09-19T04:18:07.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 -- 172.21.15.94:0/2427044411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 msgr2=0x559fa332af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa332af60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c098d30 tx=0x7f5a5c0982e0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 -- 172.21.15.94:0/2427044411 shutdown_connections 2024-09-19T04:18:07.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559fa3391050 0x559fa3393440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fa3317680 0x559fa332b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559fa330cdc0 0x559fa332b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 --2- 172.21.15.94:0/2427044411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fa330c4a0 0x559fa332af60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 -- 172.21.15.94:0/2427044411 >> 172.21.15.94:0/2427044411 conn(0x559fa3308a30 msgr2=0x559fa3307b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a4b7fe640 0 client.15708 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 -- 172.21.15.94:0/2427044411 shutdown_connections 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 1 -- 172.21.15.94:0/2427044411 wait complete. 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.277+0000 7f5a74854740 20 client.15708 trim_cache size 0 max 16384 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:18:07.283 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:18:07.284 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:18:07.284 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 -- 172.21.15.94:0/417249661 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a580a3430 msgr2=0x7f5a580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a580a3430 0x7f5a580a58f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x559fa3309890 tx=0x7f5a54009990 comp rx=0 tx=0).stop 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 -- 172.21.15.94:0/417249661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c100e20 msgr2=0x7f5a6c1a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c100e20 0x7f5a6c1a35c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e9f0 tx=0x7f5a5c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 -- 172.21.15.94:0/417249661 shutdown_connections 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a580a3430 0x7f5a580a58f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c102ef0 0x7f5a6c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6c102570 0x7f5a6c1a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 --2- 172.21.15.94:0/417249661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c100e20 0x7f5a6c1a35c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.278+0000 7f5a7247b640 1 -- 172.21.15.94:0/417249661 >> 172.21.15.94:0/417249661 conn(0x7f5a6c0fc7d0 msgr2=0x7f5a6c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.279+0000 7f5a7247b640 1 -- 172.21.15.94:0/417249661 shutdown_connections 2024-09-19T04:18:07.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:07.279+0000 7f5a7247b640 1 -- 172.21.15.94:0/417249661 wait complete. 2024-09-19T04:18:07.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:07 smithi112 ceph-mon[35051]: pgmap v96: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 19 MiB/s wr, 2.26k op/s 2024-09-19T04:18:07.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:07 smithi094 ceph-mon[30972]: pgmap v96: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 19 MiB/s wr, 2.26k op/s 2024-09-19T04:18:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:07 smithi104 ceph-mon[34145]: pgmap v96: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 19 MiB/s wr, 2.26k op/s 2024-09-19T04:18:08.426 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:18:08.427 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:18:08.840 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:09.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.417+0000 7fe723f0c640 1 -- 172.21.15.94:0/35995950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 msgr2=0x7fe71c115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:09.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.418+0000 7fe723f0c640 1 --2- 172.21.15.94:0/35995950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 0x7fe71c115070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe714009c10 tx=0x7fe71402a6f0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.418+0000 7fe723f0c640 1 -- 172.21.15.94:0/35995950 shutdown_connections 2024-09-19T04:18:09.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.418+0000 7fe723f0c640 1 --2- 172.21.15.94:0/35995950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 0x7fe71c115070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.418+0000 7fe723f0c640 1 --2- 172.21.15.94:0/35995950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c1081f0 0x7fe71c108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.418+0000 7fe723f0c640 1 --2- 172.21.15.94:0/35995950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe71c10e230 0x7fe71c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.418+0000 7fe723f0c640 1 -- 172.21.15.94:0/35995950 >> 172.21.15.94:0/35995950 conn(0x7fe71c077bb0 msgr2=0x7fe71c079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:09.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.418+0000 7fe723f0c640 1 -- 172.21.15.94:0/35995950 shutdown_connections 2024-09-19T04:18:09.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.418+0000 7fe723f0c640 1 -- 172.21.15.94:0/35995950 wait complete. 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.419+0000 7fe723f0c640 1 Processor -- start 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.419+0000 7fe723f0c640 1 -- start start 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe723f0c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c1081f0 0x7fe71c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe723f0c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 0x7fe71c1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe723f0c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe71c10e230 0x7fe71c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe721ca8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c1081f0 0x7fe71c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe7214a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 0x7fe71c1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe7214a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 0x7fe71c1a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44782/0 (socket says 172.21.15.94:44782) 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe7214a7640 1 -- 172.21.15.94:0/1903219551 learned_addr learned my addr 172.21.15.94:0/1903219551 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1183f0 con 0x7fe71c10e230 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c118270 con 0x7fe71c1081f0 2024-09-19T04:18:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.420+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c118570 con 0x7fe71c108b50 2024-09-19T04:18:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.421+0000 7fe7224a9640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe71c10e230 0x7fe71c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.421+0000 7fe7214a7640 1 -- 172.21.15.94:0/1903219551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c1081f0 msgr2=0x7fe71c1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.421+0000 7fe7214a7640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c1081f0 0x7fe71c1a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.421+0000 7fe7214a7640 1 -- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe71c10e230 msgr2=0x7fe71c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.421+0000 7fe7214a7640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe71c10e230 0x7fe71c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.421+0000 7fe7214a7640 1 -- 172.21.15.94:0/1903219551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c19ef80 con 0x7fe71c108b50 2024-09-19T04:18:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.421+0000 7fe7214a7640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 0x7fe71c1a4ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00e9c0 tx=0x7fe70c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.421+0000 7fe712ffd640 1 -- 172.21.15.94:0/1903219551 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00ccc0 con 0x7fe71c108b50 2024-09-19T04:18:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.421+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c19f270 con 0x7fe71c108b50 2024-09-19T04:18:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.422+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1ab4c0 con 0x7fe71c108b50 2024-09-19T04:18:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.422+0000 7fe712ffd640 1 -- 172.21.15.94:0/1903219551 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe70c0054d0 con 0x7fe71c108b50 2024-09-19T04:18:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.422+0000 7fe712ffd640 1 -- 172.21.15.94:0/1903219551 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c010430 con 0x7fe71c108b50 2024-09-19T04:18:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.423+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7000052c0 con 0x7fe71c108b50 2024-09-19T04:18:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.425+0000 7fe712ffd640 1 -- 172.21.15.94:0/1903219551 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe70c0046c0 con 0x7fe71c108b50 2024-09-19T04:18:09.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.426+0000 7fe712ffd640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7080a3430 0x7fe7080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:09.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.427+0000 7fe712ffd640 1 -- 172.21.15.94:0/1903219551 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe70c0d0130 con 0x7fe71c108b50 2024-09-19T04:18:09.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.427+0000 7fe721ca8640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7080a3430 0x7fe7080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:09.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.428+0000 7fe721ca8640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7080a3430 0x7fe7080a58f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7180095f0 tx=0x7fe718005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.430+0000 7fe712ffd640 1 -- 172.21.15.94:0/1903219551 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe70c054210 con 0x7fe71c108b50 2024-09-19T04:18:09.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:09 smithi112 ceph-mon[35051]: pgmap v97: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:18:09.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.809+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7000055b0 con 0x7fe71c108b50 2024-09-19T04:18:09.811 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:09 smithi094 ceph-mon[30972]: pgmap v97: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:18:09.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.810+0000 7fe712ffd640 1 -- 172.21.15.94:0/1903219551 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11922 (secure 0 0 0) 0x7fe70c09caa0 con 0x7fe71c108b50 2024-09-19T04:18:09.813 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:18:09.814 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":30,"btime":"2024-09-19T04:16:28:467883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:16:27.778564+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:18:09.814 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 30 2024-09-19T04:18:09.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7080a3430 msgr2=0x7fe7080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:09.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7080a3430 0x7fe7080a58f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7180095f0 tx=0x7fe718005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:09.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 msgr2=0x7fe71c1a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:09.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 0x7fe71c1a4ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00e9c0 tx=0x7fe70c00ee80 comp rx=0 tx=0).stop 2024-09-19T04:18:09.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 shutdown_connections 2024-09-19T04:18:09.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe7080a3430 0x7fe7080a58f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe71c10e230 0x7fe71c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe71c108b50 0x7fe71c1a4ba0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 --2- 172.21.15.94:0/1903219551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c1081f0 0x7fe71c1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.813+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 >> 172.21.15.94:0/1903219551 conn(0x7fe71c077bb0 msgr2=0x7fe71c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:09.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.814+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 shutdown_connections 2024-09-19T04:18:09.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:09.814+0000 7fe723f0c640 1 -- 172.21.15.94:0/1903219551 wait complete. 2024-09-19T04:18:09.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:09 smithi104 ceph-mon[34145]: pgmap v97: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:18:10.514 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:18:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 115.0 s, 1480.8 tps, lat 67.466 ms stddev 17.914 2024-09-19T04:18:10.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:10 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1903219551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:10.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:10 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1903219551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:10.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:10 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1903219551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:10.924 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59cf7fe640 1 -- 172.21.15.94:0/1444471435 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4031880 con 0x7f59d0109290 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59d71f1640 1 -- 172.21.15.94:0/1444471435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59d0109290 msgr2=0x7f59d0109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59d71f1640 1 --2- 172.21.15.94:0/1444471435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59d0109290 0x7f59d0109670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009900 tx=0x7f59c402a410 comp rx=0 tx=0).stop 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59d71f1640 1 -- 172.21.15.94:0/1444471435 shutdown_connections 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59d71f1640 1 --2- 172.21.15.94:0/1444471435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d010e7b0 0x7f59d0115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59d71f1640 1 --2- 172.21.15.94:0/1444471435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0109c40 0x7f59d010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59d71f1640 1 --2- 172.21.15.94:0/1444471435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59d0109290 0x7f59d0109670 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59d71f1640 1 -- 172.21.15.94:0/1444471435 >> 172.21.15.94:0/1444471435 conn(0x7f59d0077c10 msgr2=0x7f59d007a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59d71f1640 1 -- 172.21.15.94:0/1444471435 shutdown_connections 2024-09-19T04:18:11.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.478+0000 7f59d71f1640 1 -- 172.21.15.94:0/1444471435 wait complete. 2024-09-19T04:18:11.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.479+0000 7f59d71f1640 1 Processor -- start 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.479+0000 7f59d71f1640 1 -- start start 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.479+0000 7f59d71f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0109290 0x7f59d01a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.479+0000 7f59d71f1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d0109c40 0x7f59d01a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.479+0000 7f59d71f1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59d010e7b0 0x7f59d019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.479+0000 7f59d71f1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59d0118460 con 0x7f59d0109c40 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.479+0000 7f59d71f1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01182e0 con 0x7f59d010e7b0 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.479+0000 7f59d71f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01185e0 con 0x7f59d0109290 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d0109c40 0x7f59d01a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d4f8d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0109290 0x7f59d01a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d4f8d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0109290 0x7f59d01a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44800/0 (socket says 172.21.15.94:44800) 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d0109c40 0x7f59d01a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58062/0 (socket says 172.21.15.94:58062) 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d4f8d640 1 -- 172.21.15.94:0/2623855247 learned_addr learned my addr 172.21.15.94:0/2623855247 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d578e640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59d010e7b0 0x7f59d019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d4f8d640 1 -- 172.21.15.94:0/2623855247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59d010e7b0 msgr2=0x7f59d019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59d010e7b0 0x7f59d019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d4f8d640 1 -- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d0109c40 msgr2=0x7f59d01a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d0109c40 0x7f59d01a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d4f8d640 1 -- 172.21.15.94:0/2623855247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d019ef70 con 0x7f59d0109290 2024-09-19T04:18:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59cffff640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d0109c40 0x7f59d01a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.480+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0109290 0x7f59d01a4650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f59c4005ab0 tx=0x7f59c4003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.481+0000 7f59cdffb640 1 -- 172.21.15.94:0/2623855247 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c403f070 con 0x7f59d0109290 2024-09-19T04:18:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.481+0000 7f59cdffb640 1 -- 172.21.15.94:0/2623855247 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59c4031d60 con 0x7f59d0109290 2024-09-19T04:18:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.481+0000 7f59cdffb640 1 -- 172.21.15.94:0/2623855247 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4035030 con 0x7f59d0109290 2024-09-19T04:18:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.481+0000 7f59d71f1640 1 -- 172.21.15.94:0/2623855247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d019f200 con 0x7f59d0109290 2024-09-19T04:18:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.481+0000 7f59d71f1640 1 -- 172.21.15.94:0/2623855247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d01ab4b0 con 0x7f59d0109290 2024-09-19T04:18:11.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.485+0000 7f59cdffb640 1 -- 172.21.15.94:0/2623855247 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f59c4042070 con 0x7f59d0109290 2024-09-19T04:18:11.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.486+0000 7f59d95ca740 1 Processor -- start 2024-09-19T04:18:11.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.486+0000 7f59d95ca740 1 -- start start 2024-09-19T04:18:11.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.486+0000 7f59cdffb640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59c00a3430 0x7f59c00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.486+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 0x5625decbf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.486+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decbfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.486+0000 7f59d578e640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59c00a3430 0x7f59c00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625deccc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decbfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decbfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58078/0 (socket says 172.21.15.94:58078) 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59cffff640 1 -- 172.21.15.94:0/3722860797 learned_addr learned my addr 172.21.15.94:0/3722860797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59cdffb640 1 -- 172.21.15.94:0/2623855247 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f59c40f05f0 con 0x7f59d0109290 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 0x5625decbf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d95ca740 1 -- 172.21.15.94:0/3722860797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5625dec252f0 con 0x5625decbf5d0 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d95ca740 1 -- 172.21.15.94:0/3722860797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5625debcccf0 con 0x5625decc0b40 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d95ca740 1 -- 172.21.15.94:0/3722860797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5625dec212b0 con 0x5625decca370 2024-09-19T04:18:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d578e640 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625deccc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d578e640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59c00a3430 0x7f59c00a58f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f59c80092f0 tx=0x7f59c8010740 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d578e640 1 -- 172.21.15.94:0/3722860797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 msgr2=0x5625decbf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d578e640 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 0x5625decbf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d578e640 1 -- 172.21.15.94:0/3722860797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 msgr2=0x5625decbfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d578e640 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decbfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59d578e640 1 -- 172.21.15.94:0/3722860797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625debb13a0 con 0x5625decca370 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.487+0000 7f59cffff640 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decbfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.488+0000 7f59d578e640 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625deccc760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59c8005fd0 tx=0x7f59c801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b261776d7071816 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.488+0000 7f59baffd640 1 -- 172.21.15.94:0/3722860797 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c801e6e0 con 0x5625decca370 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.488+0000 7f59baffd640 1 -- 172.21.15.94:0/3722860797 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59c801ed30 con 0x5625decca370 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.488+0000 7f59baffd640 1 -- 172.21.15.94:0/3722860797 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8026dd0 con 0x5625decca370 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.489+0000 7f59d95ca740 1 -- 172.21.15.94:0/3722860797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 msgr2=0x5625deccc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.489+0000 7f59d95ca740 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625deccc760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59c8005fd0 tx=0x7f59c801b000 comp rx=0 tx=0).stop 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.489+0000 7f59d95ca740 1 -- 172.21.15.94:0/3722860797 shutdown_connections 2024-09-19T04:18:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.489+0000 7f59d95ca740 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625deccc760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.490+0000 7f59d95ca740 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decbfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.490+0000 7f59d95ca740 1 --2- 172.21.15.94:0/3722860797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 0x5625decbf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.490+0000 7f59d95ca740 1 -- 172.21.15.94:0/3722860797 >> 172.21.15.94:0/3722860797 conn(0x5625dec88310 msgr2=0x5625decbbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.490+0000 7f59d95ca740 1 -- 172.21.15.94:0/3722860797 shutdown_connections 2024-09-19T04:18:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.490+0000 7f59d95ca740 1 -- 172.21.15.94:0/3722860797 wait complete. 2024-09-19T04:18:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d95ca740 1 -- start start 2024-09-19T04:18:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d95ca740 1 Processor -- start 2024-09-19T04:18:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decbf5d0 0x5625decc9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625decc70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decc7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d95ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5625deccd070 con 0x5625decc0b40 2024-09-19T04:18:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d95ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5625debe0060 con 0x5625decbf5d0 2024-09-19T04:18:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d95ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5625deccd1f0 con 0x5625decca370 2024-09-19T04:18:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d95ca740 10 client.0 _ll_register_callbacks cb 0x5625de9338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625decc70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.491+0000 7f59d578e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decc7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.492+0000 7f59cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625decc70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58086/0 (socket says 172.21.15.94:58086) 2024-09-19T04:18:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.492+0000 7f59cffff640 1 -- 172.21.15.94:0/962910128 learned_addr learned my addr 172.21.15.94:0/962910128 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.492+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decbf5d0 0x5625decc9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.492+0000 7f59d578e640 1 -- 172.21.15.94:0/962910128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decbf5d0 msgr2=0x5625decc9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.492+0000 7f59d578e640 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decbf5d0 0x5625decc9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.492+0000 7f59d578e640 1 -- 172.21.15.94:0/962910128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 msgr2=0x5625decc70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.492+0000 7f59d578e640 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625decc70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.492+0000 7f59d578e640 1 -- 172.21.15.94:0/962910128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625de9963c0 con 0x5625decca370 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.493+0000 7f59d578e640 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decc7840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f59c80092c0 tx=0x7f59c8027ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba984cc9df545cfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.493+0000 7f59affff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.493+0000 7f59affff640 1 -- 172.21.15.94:0/962910128 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8036060 con 0x5625decca370 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.493+0000 7f59affff640 1 -- 172.21.15.94:0/962910128 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f59c801e4e0 con 0x5625decca370 2024-09-19T04:18:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.493+0000 7f59d95ca740 1 -- 172.21.15.94:0/962910128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59c801ed30 con 0x5625decca370 2024-09-19T04:18:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.493+0000 7f59affff640 1 -- 172.21.15.94:0/962910128 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f59c801e6e0 con 0x5625decca370 2024-09-19T04:18:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.493+0000 7f59d95ca740 10 client.25276 fetch_fsmap learned FSMap version 30 2024-09-19T04:18:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.493+0000 7f59d95ca740 1 -- 172.21.15.94:0/962910128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625decd8bc0 con 0x5625decca370 2024-09-19T04:18:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.494+0000 7f59affff640 1 -- 172.21.15.94:0/962910128 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59c801ed30 con 0x5625decca370 2024-09-19T04:18:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.494+0000 7f59affff640 1 -- 172.21.15.94:0/962910128 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f59c800f040 con 0x5625decca370 2024-09-19T04:18:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.494+0000 7f59d95ca740 10 client.25276 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:18:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.494+0000 7f59d95ca740 10 client.25276 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:18:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.494+0000 7f59d95ca740 20 client.25276 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.494+0000 7f59d95ca740 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5625decd9f10 0x5625decdc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.494+0000 7f59d95ca740 4 client.25276 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.494+0000 7f59d95ca740 1 -- 172.21.15.94:0/962910128 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625dec32860 con 0x5625decd9f10 2024-09-19T04:18:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.494+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5625decd9f10 0x5625decdc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.495+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5625decd9f10 0x5625decdc300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b66b7725b92f922 server_cookie=c6e3d92415fd8ce8 in_seq=0 out_seq=0 2024-09-19T04:18:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.495+0000 7f59affff640 10 client.25276 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:18:11.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59affff640 1 -- 172.21.15.94:0/962910128 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625dec32860 con 0x5625decd9f10 2024-09-19T04:18:11.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59affff640 10 client.25276 handle_command_reply: tid=0 2024-09-19T04:18:11.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 client.25276 shutdown 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 -- 172.21.15.94:0/962910128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 msgr2=0x5625decc7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decc7840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f59c80092c0 tx=0x7f59c8027ef0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 -- 172.21.15.94:0/962910128 shutdown_connections 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5625decd9f10 0x5625decdc300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59affff640 0 client.25276 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decc7840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625decc70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 --2- 172.21.15.94:0/962910128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decbf5d0 0x5625decc9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 -- 172.21.15.94:0/962910128 >> 172.21.15.94:0/962910128 conn(0x5625dec88310 msgr2=0x5625decc1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 -- 172.21.15.94:0/962910128 shutdown_connections 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 1 -- 172.21.15.94:0/962910128 wait complete. 2024-09-19T04:18:11.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.496+0000 7f59d95ca740 20 client.25276 trim_cache size 0 max 16384 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59d95ca740 1 Processor -- start 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59d95ca740 1 -- start start 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decbf5d0 0x5625decc6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625dece1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decca370 0x5625dece1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59d95ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b66e0 con 0x5625decc0b40 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59d95ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b3f20 con 0x5625decca370 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59d95ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b1d70 con 0x5625decbf5d0 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625dece1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625dece1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58100/0 (socket says 172.21.15.94:58100) 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59cffff640 1 -- 172.21.15.94:0/3724325222 learned_addr learned my addr 172.21.15.94:0/3724325222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.519+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decbf5d0 0x5625decc6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59d578e640 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decca370 0x5625dece1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59cffff640 1 -- 172.21.15.94:0/3724325222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decbf5d0 msgr2=0x5625decc6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59cffff640 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decbf5d0 0x5625decc6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59cffff640 1 -- 172.21.15.94:0/3724325222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decca370 msgr2=0x5625dece1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59cffff640 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decca370 0x5625dece1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59cffff640 1 -- 172.21.15.94:0/3724325222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625decc0760 con 0x5625decc0b40 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59d578e640 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decca370 0x5625dece1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decbf5d0 0x5625decc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59cffff640 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625dece1950 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00bea0 tx=0x7f59bc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d94a9b11fd04866 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59ba7fc640 1 -- 172.21.15.94:0/3724325222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc01f020 con 0x5625decc0b40 2024-09-19T04:18:11.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59ba7fc640 1 -- 172.21.15.94:0/3724325222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59bc0035f0 con 0x5625decc0b40 2024-09-19T04:18:11.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.520+0000 7f59ba7fc640 1 -- 172.21.15.94:0/3724325222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc010710 con 0x5625decc0b40 2024-09-19T04:18:11.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.521+0000 7f59d95ca740 1 -- 172.21.15.94:0/3724325222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 msgr2=0x5625dece1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.521+0000 7f59d95ca740 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625dece1950 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00bea0 tx=0x7f59bc00bf90 comp rx=0 tx=0).stop 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.522+0000 7f59d95ca740 1 -- 172.21.15.94:0/3724325222 shutdown_connections 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.522+0000 7f59d95ca740 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decca370 0x5625dece1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.522+0000 7f59d95ca740 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decc0b40 0x5625dece1950 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.522+0000 7f59d95ca740 1 --2- 172.21.15.94:0/3724325222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decbf5d0 0x5625decc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.522+0000 7f59d95ca740 1 -- 172.21.15.94:0/3724325222 >> 172.21.15.94:0/3724325222 conn(0x5625dec88310 msgr2=0x5625decc5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.522+0000 7f59d95ca740 1 -- 172.21.15.94:0/3724325222 shutdown_connections 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.522+0000 7f59d95ca740 1 -- 172.21.15.94:0/3724325222 wait complete. 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.522+0000 7f59d95ca740 1 -- start start 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.523+0000 7f59d95ca740 1 Processor -- start 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.523+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.523+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 0x5625decce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.523+0000 7f59d95ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.523+0000 7f59d95ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b23e0 con 0x5625decbf5d0 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.523+0000 7f59d95ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b3850 con 0x5625decc0b40 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.523+0000 7f59d95ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b2ab0 con 0x5625decca370 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.523+0000 7f59d95ca740 10 client.0 _ll_register_callbacks cb 0x7f59c4003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.523+0000 7f59d4f8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d578e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d578e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44838/0 (socket says 172.21.15.94:44838) 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d4f8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58108/0 (socket says 172.21.15.94:58108) 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d578e640 1 -- 172.21.15.94:0/84098463 learned_addr learned my addr 172.21.15.94:0/84098463 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59cffff640 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 0x5625decce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d578e640 1 -- 172.21.15.94:0/84098463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 msgr2=0x5625decce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d578e640 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 0x5625decce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d578e640 1 -- 172.21.15.94:0/84098463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 msgr2=0x5625decce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d578e640 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d578e640 1 -- 172.21.15.94:0/84098463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625ded43970 con 0x5625decca370 2024-09-19T04:18:11.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d578e640 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decceca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59c8010cb0 tx=0x7f59c801d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f612bbc83785b001 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59affff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:18:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59affff640 1 -- 172.21.15.94:0/84098463 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8036070 con 0x5625decca370 2024-09-19T04:18:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59affff640 1 -- 172.21.15.94:0/84098463 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f59c8004df0 con 0x5625decca370 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.524+0000 7f59d95ca740 1 -- 172.21.15.94:0/84098463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59bc0035f0 con 0x5625decca370 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.525+0000 7f59affff640 1 -- 172.21.15.94:0/84098463 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f59c8040080 con 0x5625decca370 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.525+0000 7f59d95ca740 10 client.25282 fetch_fsmap learned FSMap version 30 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.525+0000 7f59d95ca740 1 -- 172.21.15.94:0/84098463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625ded44bc0 con 0x5625decca370 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.525+0000 7f59affff640 1 -- 172.21.15.94:0/84098463 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59bc0035f0 con 0x5625decca370 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.525+0000 7f59affff640 1 -- 172.21.15.94:0/84098463 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f59c804c020 con 0x5625decca370 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.525+0000 7f59d95ca740 10 client.25282 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.525+0000 7f59d95ca740 10 client.25282 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.525+0000 7f59d95ca740 20 client.25282 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.525+0000 7f59d95ca740 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5625decd9f10 0x5625ded472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.526+0000 7f59d95ca740 4 client.25282 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.526+0000 7f59d95ca740 1 -- 172.21.15.94:0/84098463 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625dec2cd70 con 0x5625decd9f10 2024-09-19T04:18:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.526+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5625decd9f10 0x5625ded472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59affff640 10 client.25282 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d4f8d640 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5625decd9f10 0x5625ded472f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6daad5fa9ee2c2a5 server_cookie=5fa1e8f1ff0294a5 in_seq=0 out_seq=0 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59affff640 1 -- 172.21.15.94:0/84098463 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625dec2cd70 con 0x5625decd9f10 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59affff640 10 client.25282 handle_command_reply: tid=0 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d95ca740 1 client.25282 shutdown 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d95ca740 1 -- 172.21.15.94:0/84098463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 msgr2=0x5625decceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d95ca740 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decceca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59c8010cb0 tx=0x7f59c801d3b0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d95ca740 1 -- 172.21.15.94:0/84098463 shutdown_connections 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d95ca740 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5625decd9f10 0x5625ded472f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59affff640 0 client.25282 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d95ca740 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625decca370 0x5625decceca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d95ca740 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625decc0b40 0x5625decce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d95ca740 1 --2- 172.21.15.94:0/84098463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625decbf5d0 0x5625decce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.527+0000 7f59d95ca740 1 -- 172.21.15.94:0/84098463 >> 172.21.15.94:0/84098463 conn(0x5625dec88310 msgr2=0x5625decc61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.528+0000 7f59d95ca740 1 -- 172.21.15.94:0/84098463 shutdown_connections 2024-09-19T04:18:11.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.528+0000 7f59d95ca740 1 -- 172.21.15.94:0/84098463 wait complete. 2024-09-19T04:18:11.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.528+0000 7f59d95ca740 20 client.25282 trim_cache size 0 max 16384 2024-09-19T04:18:11.532 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:18:11.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 -- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59c00a3430 msgr2=0x7f59c00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59c00a3430 0x7f59c00a58f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f59c80092f0 tx=0x7f59c8010740 comp rx=0 tx=0).stop 2024-09-19T04:18:11.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 -- 172.21.15.94:0/2623855247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0109290 msgr2=0x7f59d01a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0109290 0x7f59d01a4650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f59c4005ab0 tx=0x7f59c4003cd0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 -- 172.21.15.94:0/2623855247 shutdown_connections 2024-09-19T04:18:11.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59c00a3430 0x7f59c00a58f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59d010e7b0 0x7f59d019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d0109c40 0x7f59d01a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 --2- 172.21.15.94:0/2623855247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0109290 0x7f59d01a4650 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.529+0000 7f59d71f1640 1 -- 172.21.15.94:0/2623855247 >> 172.21.15.94:0/2623855247 conn(0x7f59d0077c10 msgr2=0x7f59d010c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.530+0000 7f59d71f1640 1 -- 172.21.15.94:0/2623855247 shutdown_connections 2024-09-19T04:18:11.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:11.530+0000 7f59d71f1640 1 -- 172.21.15.94:0/2623855247 wait complete. 2024-09-19T04:18:11.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:11 smithi112 ceph-mon[35051]: pgmap v98: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:18:11.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:11 smithi094 ceph-mon[30972]: pgmap v98: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:18:11.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:11 smithi104 ceph-mon[34145]: pgmap v98: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:18:12.564 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:18:12.967 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:13.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.587+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/2553533974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 msgr2=0x7fd8d01021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.587+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/2553533974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 0x7fd8d01021b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009900 tx=0x7fd8c402a410 comp rx=0 tx=0).stop 2024-09-19T04:18:13.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.587+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/2553533974 shutdown_connections 2024-09-19T04:18:13.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.587+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/2553533974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d0106f00 0x7fd8d010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.587+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/2553533974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8d0102780 0x7fd8d01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.587+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/2553533974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 0x7fd8d01021b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.587+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/2553533974 >> 172.21.15.94:0/2553533974 conn(0x7fd8d00fd870 msgr2=0x7fd8d00ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.588+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/2553533974 shutdown_connections 2024-09-19T04:18:13.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.588+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/2553533974 wait complete. 2024-09-19T04:18:13.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.588+0000 7fd8d6a6f640 1 Processor -- start 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d6a6f640 1 -- start start 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d6a6f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 0x7fd8d019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d6a6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8d0102780 0x7fd8d019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d6a6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d0106f00 0x7fd8d019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d6a6f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d00747b0 con 0x7fd8d0101dd0 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d6a6f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0074630 con 0x7fd8d0102780 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d6a6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0074930 con 0x7fd8d0106f00 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d480b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 0x7fd8d019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d480b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 0x7fd8d019f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60094/0 (socket says 172.21.15.94:60094) 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d480b640 1 -- 172.21.15.94:0/3541249837 learned_addr learned my addr 172.21.15.94:0/3541249837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:13.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8cffff640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8d0102780 0x7fd8d019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.589+0000 7fd8d500c640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d0106f00 0x7fd8d019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.590+0000 7fd8d480b640 1 -- 172.21.15.94:0/3541249837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d0106f00 msgr2=0x7fd8d019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.590+0000 7fd8d480b640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d0106f00 0x7fd8d019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.590+0000 7fd8d480b640 1 -- 172.21.15.94:0/3541249837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8d0102780 msgr2=0x7fd8d019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.590+0000 7fd8d480b640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8d0102780 0x7fd8d019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.590+0000 7fd8d480b640 1 -- 172.21.15.94:0/3541249837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d019b410 con 0x7fd8d0101dd0 2024-09-19T04:18:13.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.590+0000 7fd8d500c640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d0106f00 0x7fd8d019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:13.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.590+0000 7fd8d480b640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 0x7fd8d019f150 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4005940 tx=0x7fd8c40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.591+0000 7fd8cdffb640 1 -- 172.21.15.94:0/3541249837 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c403f030 con 0x7fd8d0101dd0 2024-09-19T04:18:13.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.591+0000 7fd8cdffb640 1 -- 172.21.15.94:0/3541249837 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8c40044c0 con 0x7fd8d0101dd0 2024-09-19T04:18:13.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.591+0000 7fd8cdffb640 1 -- 172.21.15.94:0/3541249837 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4035070 con 0x7fd8d0101dd0 2024-09-19T04:18:13.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.591+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/3541249837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d019b640 con 0x7fd8d0101dd0 2024-09-19T04:18:13.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.591+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/3541249837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d01a7080 con 0x7fd8d0101dd0 2024-09-19T04:18:13.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.595+0000 7fd8d8e48740 1 Processor -- start 2024-09-19T04:18:13.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.595+0000 7fd8cdffb640 1 -- 172.21.15.94:0/3541249837 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd8c4042030 con 0x7fd8d0101dd0 2024-09-19T04:18:13.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.595+0000 7fd8d8e48740 1 -- start start 2024-09-19T04:18:13.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.595+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc67090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.595+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc675d0 0x5593ddc67a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.595+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc74760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.595+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5593ddbcd2f0 con 0x5593ddc68b40 2024-09-19T04:18:13.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.595+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5593ddb74cf0 con 0x5593ddc72370 2024-09-19T04:18:13.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.595+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593ddbc92b0 con 0x5593ddc675d0 2024-09-19T04:18:13.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc74760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8d480b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc675d0 0x5593ddc67a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc74760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48358/0 (socket says 172.21.15.94:48358) 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8cffff640 1 -- 172.21.15.94:0/3147986062 learned_addr learned my addr 172.21.15.94:0/3147986062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8d500c640 1 --2- 172.21.15.94:0/3147986062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc67090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8cdffb640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd8c00a3430 0x7fd8c00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8d500c640 1 -- 172.21.15.94:0/3147986062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc675d0 msgr2=0x5593ddc67a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8d500c640 1 --2- 172.21.15.94:0/3147986062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc675d0 0x5593ddc67a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8d500c640 1 -- 172.21.15.94:0/3147986062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 msgr2=0x5593ddc74760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8d500c640 1 --2- 172.21.15.94:0/3147986062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc74760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8d500c640 1 -- 172.21.15.94:0/3147986062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ddb593a0 con 0x5593ddc68b40 2024-09-19T04:18:13.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8cffff640 1 --2- 172.21.15.94:0/3147986062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc74760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:13.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8cdffb640 1 -- 172.21.15.94:0/3541249837 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fd8c40efd50 con 0x7fd8d0101dd0 2024-09-19T04:18:13.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.596+0000 7fd8d500c640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd8c00a3430 0x7fd8c00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.597+0000 7fd8d500c640 1 --2- 172.21.15.94:0/3147986062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc67090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8005fd0 tx=0x7fd8c8011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b008d70d689cd429 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.597+0000 7fd8baffd640 1 -- 172.21.15.94:0/3147986062 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8017050 con 0x5593ddc68b40 2024-09-19T04:18:13.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.597+0000 7fd8baffd640 1 -- 172.21.15.94:0/3147986062 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8c8004470 con 0x5593ddc68b40 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8baffd640 1 -- 172.21.15.94:0/3147986062 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800e6b0 con 0x5593ddc68b40 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d8e48740 1 -- 172.21.15.94:0/3147986062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 msgr2=0x5593ddc67090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/3147986062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc67090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8005fd0 tx=0x7fd8c8011c90 comp rx=0 tx=0).stop 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d500c640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd8c00a3430 0x7fd8c00a58f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8023000 tx=0x7fd8c8024000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d8e48740 1 -- 172.21.15.94:0/3147986062 shutdown_connections 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/3147986062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc74760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/3147986062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc675d0 0x5593ddc67a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/3147986062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc67090 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d8e48740 1 -- 172.21.15.94:0/3147986062 >> 172.21.15.94:0/3147986062 conn(0x5593ddc30310 msgr2=0x5593ddc63c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d8e48740 1 -- 172.21.15.94:0/3147986062 shutdown_connections 2024-09-19T04:18:13.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.598+0000 7fd8d8e48740 1 -- 172.21.15.94:0/3147986062 wait complete. 2024-09-19T04:18:13.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.599+0000 7fd8d8e48740 1 -- start start 2024-09-19T04:18:13.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.599+0000 7fd8d8e48740 1 Processor -- start 2024-09-19T04:18:13.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.600+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 0x5593ddc71d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.600+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc6efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.600+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc72370 0x5593ddc6f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.600+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5593ddc750c0 con 0x5593ddc72370 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.600+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5593ddb88060 con 0x5593ddc675d0 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.600+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593ddc75240 con 0x5593ddc68b40 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.600+0000 7fd8d8e48740 10 client.0 _ll_register_callbacks cb 0x5593dd8db8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc6efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8d500c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc72370 0x5593ddc6f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8d500c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc72370 0x5593ddc6f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60114/0 (socket says 172.21.15.94:60114) 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc6efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55088/0 (socket says 172.21.15.94:55088) 2024-09-19T04:18:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8d500c640 1 -- 172.21.15.94:0/711515679 learned_addr learned my addr 172.21.15.94:0/711515679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.600+0000 7fd8d480b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 0x5593ddc71d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8cffff640 1 -- 172.21.15.94:0/711515679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 msgr2=0x5593ddc71d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8cffff640 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 0x5593ddc71d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8cffff640 1 -- 172.21.15.94:0/711515679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc72370 msgr2=0x5593ddc6f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8cffff640 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc72370 0x5593ddc6f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8cffff640 1 -- 172.21.15.94:0/711515679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593dd93e3c0 con 0x5593ddc68b40 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8cffff640 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc6efb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0126d0 tx=0x7fd8bc012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98bb6aeb0de59123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.601+0000 7fd8affff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.602+0000 7fd8affff640 1 -- 172.21.15.94:0/711515679 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc004050 con 0x5593ddc68b40 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.602+0000 7fd8affff640 1 -- 172.21.15.94:0/711515679 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fd8bc0041f0 con 0x5593ddc68b40 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.602+0000 7fd8d8e48740 1 -- 172.21.15.94:0/711515679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8c8004470 con 0x5593ddc68b40 2024-09-19T04:18:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.602+0000 7fd8affff640 1 -- 172.21.15.94:0/711515679 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8bc0043f0 con 0x5593ddc68b40 2024-09-19T04:18:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.602+0000 7fd8d8e48740 10 client.25294 fetch_fsmap learned FSMap version 30 2024-09-19T04:18:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.602+0000 7fd8d8e48740 1 -- 172.21.15.94:0/711515679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5593ddc89400 con 0x5593ddc68b40 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.603+0000 7fd8affff640 1 -- 172.21.15.94:0/711515679 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8c8004470 con 0x5593ddc68b40 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.603+0000 7fd8affff640 1 -- 172.21.15.94:0/711515679 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd8bc003bc0 con 0x5593ddc68b40 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.603+0000 7fd8d8e48740 10 client.25294 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.603+0000 7fd8d8e48740 10 client.25294 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.603+0000 7fd8d8e48740 20 client.25294 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.603+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5593ddc8a960 0x5593ddc8cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.603+0000 7fd8d8e48740 4 client.25294 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.603+0000 7fd8d8e48740 1 -- 172.21.15.94:0/711515679 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593ddbda860 con 0x5593ddc8a960 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.603+0000 7fd8d480b640 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5593ddc8a960 0x5593ddc8cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.604+0000 7fd8affff640 10 client.25294 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.604+0000 7fd8d480b640 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5593ddc8a960 0x5593ddc8cd90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7b6e7ffb39ae03b server_cookie=47b5fd6a5b49cecb in_seq=0 out_seq=0 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.605+0000 7fd8affff640 1 -- 172.21.15.94:0/711515679 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593ddbda860 con 0x5593ddc8a960 2024-09-19T04:18:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.605+0000 7fd8affff640 10 client.25294 handle_command_reply: tid=0 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.605+0000 7fd8d8e48740 1 client.25294 shutdown 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 -- 172.21.15.94:0/711515679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 msgr2=0x5593ddc6efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc6efb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0126d0 tx=0x7fd8bc012b90 comp rx=0 tx=0).stop 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 -- 172.21.15.94:0/711515679 shutdown_connections 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5593ddc8a960 0x5593ddc8cd90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc72370 0x5593ddc6f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc6efb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8affff640 0 client.25294 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/711515679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 0x5593ddc71d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 -- 172.21.15.94:0/711515679 >> 172.21.15.94:0/711515679 conn(0x5593ddc30310 msgr2=0x5593ddc69960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 -- 172.21.15.94:0/711515679 shutdown_connections 2024-09-19T04:18:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 1 -- 172.21.15.94:0/711515679 wait complete. 2024-09-19T04:18:13.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.606+0000 7fd8d8e48740 20 client.25294 trim_cache size 0 max 16384 2024-09-19T04:18:13.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.628+0000 7fd8d8e48740 1 Processor -- start 2024-09-19T04:18:13.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.628+0000 7fd8d8e48740 1 -- start start 2024-09-19T04:18:13.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.628+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 0x5593ddc86c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.628+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc6e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.628+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc72370 0x5593ddc92660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.628+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01ae010 con 0x5593ddc68b40 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.628+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01ab850 con 0x5593ddc675d0 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.628+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01a96e0 con 0x5593ddc72370 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc6e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8d480b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 0x5593ddc86c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc6e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60116/0 (socket says 172.21.15.94:60116) 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8cffff640 1 -- 172.21.15.94:0/2614631058 learned_addr learned my addr 172.21.15.94:0/2614631058 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8d500c640 1 --2- 172.21.15.94:0/2614631058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc72370 0x5593ddc92660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8d480b640 1 -- 172.21.15.94:0/2614631058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc72370 msgr2=0x5593ddc92660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8d480b640 1 --2- 172.21.15.94:0/2614631058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc72370 0x5593ddc92660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8d480b640 1 -- 172.21.15.94:0/2614631058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 msgr2=0x5593ddc6e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8d480b640 1 --2- 172.21.15.94:0/2614631058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc6e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8d480b640 1 -- 172.21.15.94:0/2614631058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ddc64960 con 0x5593ddc675d0 2024-09-19T04:18:13.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.629+0000 7fd8d480b640 1 --2- 172.21.15.94:0/2614631058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 0x5593ddc86c40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40ed430 tx=0x7fd8c40b8440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8d83a00b4326426 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.630+0000 7fd8ba7fc640 1 -- 172.21.15.94:0/2614631058 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40d4020 con 0x5593ddc675d0 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.630+0000 7fd8ba7fc640 1 -- 172.21.15.94:0/2614631058 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8c40dc0b0 con 0x5593ddc675d0 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.630+0000 7fd8ba7fc640 1 -- 172.21.15.94:0/2614631058 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40f2040 con 0x5593ddc675d0 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 -- 172.21.15.94:0/2614631058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 msgr2=0x5593ddc86c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/2614631058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 0x5593ddc86c40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40ed430 tx=0x7fd8c40b8440 comp rx=0 tx=0).stop 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 -- 172.21.15.94:0/2614631058 shutdown_connections 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/2614631058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc72370 0x5593ddc92660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/2614631058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc68b40 0x5593ddc6e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/2614631058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc675d0 0x5593ddc86c40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 -- 172.21.15.94:0/2614631058 >> 172.21.15.94:0/2614631058 conn(0x5593ddc30310 msgr2=0x5593ddc6d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 -- 172.21.15.94:0/2614631058 shutdown_connections 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 -- 172.21.15.94:0/2614631058 wait complete. 2024-09-19T04:18:13.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 -- start start 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.631+0000 7fd8d8e48740 1 Processor -- start 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc675d0 0x5593ddc83220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc7e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d8e48740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc7eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01a9d30 con 0x5593ddc675d0 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01ab180 con 0x5593ddc72370 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d8e48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01aa3e0 con 0x5593ddc68b40 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d8e48740 10 client.0 _ll_register_callbacks cb 0x7fd8bc009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc7e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d500c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc7eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d480b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc675d0 0x5593ddc83220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d500c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc7eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48398/0 (socket says 172.21.15.94:48398) 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc7e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55108/0 (socket says 172.21.15.94:55108) 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.632+0000 7fd8d500c640 1 -- 172.21.15.94:0/915072360 learned_addr learned my addr 172.21.15.94:0/915072360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8d500c640 1 -- 172.21.15.94:0/915072360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 msgr2=0x5593ddc7e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8d500c640 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc7e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8d500c640 1 -- 172.21.15.94:0/915072360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc675d0 msgr2=0x5593ddc83220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8d500c640 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc675d0 0x5593ddc83220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8d500c640 1 -- 172.21.15.94:0/915072360 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593ddc7f020 con 0x5593ddc72370 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8cffff640 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc7e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8d500c640 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc7eae0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80094f0 tx=0x7fd8c801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f41d28377c1ef4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8affff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8affff640 1 -- 172.21.15.94:0/915072360 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8038060 con 0x5593ddc72370 2024-09-19T04:18:13.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8affff640 1 -- 172.21.15.94:0/915072360 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fd8c8011250 con 0x5593ddc72370 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8d8e48740 1 -- 172.21.15.94:0/915072360 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8c40dc0b0 con 0x5593ddc72370 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8affff640 1 -- 172.21.15.94:0/915072360 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8c8009590 con 0x5593ddc72370 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8d8e48740 10 client.25343 fetch_fsmap learned FSMap version 30 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.633+0000 7fd8d8e48740 1 -- 172.21.15.94:0/915072360 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5593ddcf5340 con 0x5593ddc72370 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8affff640 1 -- 172.21.15.94:0/915072360 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8c40dc0b0 con 0x5593ddc72370 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8affff640 1 -- 172.21.15.94:0/915072360 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd8c8005800 con 0x5593ddc72370 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8d8e48740 10 client.25343 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8d8e48740 10 client.25343 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8d8e48740 20 client.25343 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:13.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5593ddc8a960 0x5593ddcf7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8d8e48740 4 client.25343 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:18:13.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8d8e48740 1 -- 172.21.15.94:0/915072360 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593ddbd4d70 con 0x5593ddc8a960 2024-09-19T04:18:13.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8d480b640 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5593ddc8a960 0x5593ddcf7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.634+0000 7fd8affff640 10 client.25343 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:18:13.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d480b640 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5593ddc8a960 0x5593ddcf7990 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c7111693498743c server_cookie=adebc59e6c49fae8 in_seq=0 out_seq=0 2024-09-19T04:18:13.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8affff640 1 -- 172.21.15.94:0/915072360 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593ddbd4d70 con 0x5593ddc8a960 2024-09-19T04:18:13.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8affff640 10 client.25343 handle_command_reply: tid=0 2024-09-19T04:18:13.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 client.25343 shutdown 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 -- 172.21.15.94:0/915072360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 msgr2=0x5593ddc7eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc7eae0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80094f0 tx=0x7fd8c801e000 comp rx=0 tx=0).stop 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 -- 172.21.15.94:0/915072360 shutdown_connections 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5593ddc8a960 0x5593ddcf7990 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8affff640 0 client.25343 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593ddc72370 0x5593ddc7eae0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593ddc68b40 0x5593ddc7e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 --2- 172.21.15.94:0/915072360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593ddc675d0 0x5593ddc83220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 -- 172.21.15.94:0/915072360 >> 172.21.15.94:0/915072360 conn(0x5593ddc30310 msgr2=0x5593ddc6d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 -- 172.21.15.94:0/915072360 shutdown_connections 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.635+0000 7fd8d8e48740 1 -- 172.21.15.94:0/915072360 wait complete. 2024-09-19T04:18:13.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.636+0000 7fd8d8e48740 20 client.25343 trim_cache size 0 max 16384 2024-09-19T04:18:13.645 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:18:13.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd8c00a3430 msgr2=0x7fd8c00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd8c00a3430 0x7fd8c00a58f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8023000 tx=0x7fd8c8024000 comp rx=0 tx=0).stop 2024-09-19T04:18:13.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 msgr2=0x7fd8d019f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 0x7fd8d019f150 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4005940 tx=0x7fd8c40040c0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/3541249837 shutdown_connections 2024-09-19T04:18:13.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd8c00a3430 0x7fd8c00a58f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d0106f00 0x7fd8d019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8d0102780 0x7fd8d019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 --2- 172.21.15.94:0/3541249837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8d0101dd0 0x7fd8d019f150 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/3541249837 >> 172.21.15.94:0/3541249837 conn(0x7fd8d00fd870 msgr2=0x7fd8d01057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/3541249837 shutdown_connections 2024-09-19T04:18:13.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:13.638+0000 7fd8d6a6f640 1 -- 172.21.15.94:0/3541249837 wait complete. 2024-09-19T04:18:13.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:13 smithi112 ceph-mon[35051]: pgmap v99: 97 pgs: 97 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 2.14k op/s 2024-09-19T04:18:13.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:13 smithi094 ceph-mon[30972]: pgmap v99: 97 pgs: 97 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 2.14k op/s 2024-09-19T04:18:13.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:13 smithi104 ceph-mon[34145]: pgmap v99: 97 pgs: 97 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 2.14k op/s 2024-09-19T04:18:14.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:18:14.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:14 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:18:14.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:14 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:18:14.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:14 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:18:14.910 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:15.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.494+0000 7f5549253640 1 -- 172.21.15.94:0/1792953997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5544076720 msgr2=0x7f5544076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.494+0000 7f5549253640 1 --2- 172.21.15.94:0/1792953997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5544076720 0x7f5544076b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5534009860 tx=0x7f553402a340 comp rx=0 tx=0).stop 2024-09-19T04:18:15.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.494+0000 7f5549253640 1 -- 172.21.15.94:0/1792953997 shutdown_connections 2024-09-19T04:18:15.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.494+0000 7f5549253640 1 --2- 172.21.15.94:0/1792953997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5544112a20 0x7f5544114e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.494+0000 7f5549253640 1 --2- 172.21.15.94:0/1792953997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5544077040 0x7f55441124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.494+0000 7f5549253640 1 --2- 172.21.15.94:0/1792953997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5544076720 0x7f5544076b00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.494+0000 7f5549253640 1 -- 172.21.15.94:0/1792953997 >> 172.21.15.94:0/1792953997 conn(0x7f55440ffac0 msgr2=0x7f5544101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:15.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.494+0000 7f5549253640 1 -- 172.21.15.94:0/1792953997 shutdown_connections 2024-09-19T04:18:15.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.494+0000 7f5549253640 1 -- 172.21.15.94:0/1792953997 wait complete. 2024-09-19T04:18:15.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.495+0000 7f5549253640 1 Processor -- start 2024-09-19T04:18:15.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.495+0000 7f5549253640 1 -- start start 2024-09-19T04:18:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.495+0000 7f5549253640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5544076720 0x7f5544073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.495+0000 7f5549253640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5544077040 0x7f554406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.495+0000 7f5549253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5544112a20 0x7f554406d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.495+0000 7f5549253640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5544118450 con 0x7f5544077040 2024-09-19T04:18:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.495+0000 7f5549253640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f55441182d0 con 0x7f5544076720 2024-09-19T04:18:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.495+0000 7f5549253640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55441185d0 con 0x7f5544112a20 2024-09-19T04:18:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f554259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5544077040 0x7f554406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f554259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5544077040 0x7f554406cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60148/0 (socket says 172.21.15.94:60148) 2024-09-19T04:18:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f554259c640 1 -- 172.21.15.94:0/1271725464 learned_addr learned my addr 172.21.15.94:0/1271725464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:15.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f5542d9d640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5544076720 0x7f5544073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f554359e640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5544112a20 0x7f554406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f5542d9d640 1 -- 172.21.15.94:0/1271725464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5544112a20 msgr2=0x7f554406d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f5542d9d640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5544112a20 0x7f554406d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f5542d9d640 1 -- 172.21.15.94:0/1271725464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5544077040 msgr2=0x7f554406cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f5542d9d640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5544077040 0x7f554406cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.496+0000 7f5542d9d640 1 -- 172.21.15.94:0/1271725464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554406d680 con 0x7f5544076720 2024-09-19T04:18:15.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.497+0000 7f5542d9d640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5544076720 0x7f5544073a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f553402a860 tx=0x7f5534030e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:15.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.497+0000 7f552bfff640 1 -- 172.21.15.94:0/1271725464 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534040020 con 0x7f5544076720 2024-09-19T04:18:15.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.498+0000 7f552bfff640 1 -- 172.21.15.94:0/1271725464 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5534038040 con 0x7f5544076720 2024-09-19T04:18:15.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.498+0000 7f552bfff640 1 -- 172.21.15.94:0/1271725464 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534037070 con 0x7f5544076720 2024-09-19T04:18:15.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.498+0000 7f5549253640 1 -- 172.21.15.94:0/1271725464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554406d910 con 0x7f5544076720 2024-09-19T04:18:15.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.499+0000 7f5549253640 1 -- 172.21.15.94:0/1271725464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5544071830 con 0x7f5544076720 2024-09-19T04:18:15.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.502+0000 7f554b62c740 1 Processor -- start 2024-09-19T04:18:15.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.502+0000 7f554b62c740 1 -- start start 2024-09-19T04:18:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554b62c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fd5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554b62c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd55d0 0x556274fd5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554b62c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274fe2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554b62c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556274f3b2f0 con 0x556274fd6b40 2024-09-19T04:18:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554b62c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556274ee2cf0 con 0x556274fd55d0 2024-09-19T04:18:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554b62c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556274f372b0 con 0x556274fe0370 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274fe2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274fe2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55124/0 (socket says 172.21.15.94:55124) 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fd5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554259c640 1 -- 172.21.15.94:0/1315590822 learned_addr learned my addr 172.21.15.94:0/1315590822 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554259c640 1 -- 172.21.15.94:0/1315590822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd55d0 msgr2=0x556274fd5a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f552bfff640 1 -- 172.21.15.94:0/1271725464 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5534043020 con 0x7f5544076720 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554259c640 1 --2- 172.21.15.94:0/1315590822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd55d0 0x556274fd5a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554259c640 1 -- 172.21.15.94:0/1315590822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 msgr2=0x556274fd5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554259c640 1 --2- 172.21.15.94:0/1315590822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fd5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.503+0000 7f554259c640 1 -- 172.21.15.94:0/1315590822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556274ec73a0 con 0x556274fe0370 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.504+0000 7f554259c640 1 --2- 172.21.15.94:0/1315590822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274fe2760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5538009ab0 tx=0x7f5538009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88dc4bfaca917dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.505+0000 7f55297fa640 1 -- 172.21.15.94:0/1315590822 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538017070 con 0x556274fe0370 2024-09-19T04:18:15.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.505+0000 7f55297fa640 1 -- 172.21.15.94:0/1315590822 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f553800eba0 con 0x556274fe0370 2024-09-19T04:18:15.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.505+0000 7f55297fa640 1 -- 172.21.15.94:0/1315590822 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538012630 con 0x556274fe0370 2024-09-19T04:18:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.505+0000 7f552bfff640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f552c0a34b0 0x7f552c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.505+0000 7f552bfff640 1 -- 172.21.15.94:0/1271725464 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f55340ef9f0 con 0x7f5544076720 2024-09-19T04:18:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.505+0000 7f554359e640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f552c0a34b0 0x7f552c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.506+0000 7f554b62c740 1 -- 172.21.15.94:0/1315590822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 msgr2=0x556274fe2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.506+0000 7f554b62c740 1 --2- 172.21.15.94:0/1315590822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274fe2760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5538009ab0 tx=0x7f5538009f70 comp rx=0 tx=0).stop 2024-09-19T04:18:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.506+0000 7f554b62c740 1 -- 172.21.15.94:0/1315590822 shutdown_connections 2024-09-19T04:18:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.506+0000 7f554b62c740 1 --2- 172.21.15.94:0/1315590822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274fe2760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.506+0000 7f554b62c740 1 --2- 172.21.15.94:0/1315590822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd55d0 0x556274fd5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.506+0000 7f554b62c740 1 --2- 172.21.15.94:0/1315590822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fd5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.506+0000 7f554b62c740 1 -- 172.21.15.94:0/1315590822 >> 172.21.15.94:0/1315590822 conn(0x556274f9e310 msgr2=0x556274fd1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.506+0000 7f554b62c740 1 -- 172.21.15.94:0/1315590822 shutdown_connections 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.507+0000 7f554359e640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f552c0a34b0 0x7f552c0a5970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f553c005fd0 tx=0x7f553c0095c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.507+0000 7f554b62c740 1 -- 172.21.15.94:0/1315590822 wait complete. 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.507+0000 7f554b62c740 1 -- start start 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.507+0000 7f554b62c740 1 Processor -- start 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f554b62c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fdfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f554b62c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 0x556274fdcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f554b62c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 0x556274fdd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f554b62c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556274fe3110 con 0x556274fe0370 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f554b62c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556274ef6060 con 0x556274fd6b40 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f554b62c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556274fe3290 con 0x556274fd55d0 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f554b62c740 10 client.0 _ll_register_callbacks cb 0x556274c498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5542d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fdfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5542d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fdfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55132/0 (socket says 172.21.15.94:55132) 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5542d9d640 1 -- 172.21.15.94:0/238949267 learned_addr learned my addr 172.21.15.94:0/238949267 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f554259c640 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 0x556274fdcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f554359e640 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 0x556274fdd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5542d9d640 1 -- 172.21.15.94:0/238949267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 msgr2=0x556274fdcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5542d9d640 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 0x556274fdcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5542d9d640 1 -- 172.21.15.94:0/238949267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 msgr2=0x556274fdd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5542d9d640 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 0x556274fdd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5542d9d640 1 -- 172.21.15.94:0/238949267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556274cac3c0 con 0x556274fd55d0 2024-09-19T04:18:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5542d9d640 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fdfd40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5534009780 tx=0x7f55340bb8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9d1aeaee303cd90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.508+0000 7f5526ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:18:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.509+0000 7f5526ff5640 1 -- 172.21.15.94:0/238949267 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55340bc470 con 0x556274fd55d0 2024-09-19T04:18:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.509+0000 7f5526ff5640 1 -- 172.21.15.94:0/238949267 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f55340ed350 con 0x556274fd55d0 2024-09-19T04:18:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.509+0000 7f554b62c740 1 -- 172.21.15.94:0/238949267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f553800eba0 con 0x556274fd55d0 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.509+0000 7f5526ff5640 1 -- 172.21.15.94:0/238949267 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f55340ed550 con 0x556274fd55d0 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.509+0000 7f554b62c740 10 client.25312 fetch_fsmap learned FSMap version 30 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.509+0000 7f554b62c740 1 -- 172.21.15.94:0/238949267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556274fe3110 con 0x556274fd55d0 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.509+0000 7f5526ff5640 1 -- 172.21.15.94:0/238949267 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f553800eba0 con 0x556274fd55d0 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.509+0000 7f5526ff5640 1 -- 172.21.15.94:0/238949267 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f55340d2070 con 0x556274fd55d0 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.510+0000 7f554b62c740 10 client.25312 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.510+0000 7f554b62c740 10 client.25312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.510+0000 7f554b62c740 20 client.25312 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.510+0000 7f554b62c740 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556274feff30 0x556274ff2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.510+0000 7f554b62c740 4 client.25312 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.510+0000 7f554b62c740 1 -- 172.21.15.94:0/238949267 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556274f48860 con 0x556274feff30 2024-09-19T04:18:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.510+0000 7f554259c640 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556274feff30 0x556274ff2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.510+0000 7f554259c640 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556274feff30 0x556274ff2350 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45e3bdcdbe22b16 server_cookie=6c2b9bac9545fc90 in_seq=0 out_seq=0 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.510+0000 7f5526ff5640 10 client.25312 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f5526ff5640 1 -- 172.21.15.94:0/238949267 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556274f48860 con 0x556274feff30 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f5526ff5640 10 client.25312 handle_command_reply: tid=0 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f554b62c740 1 client.25312 shutdown 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f554b62c740 1 -- 172.21.15.94:0/238949267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 msgr2=0x556274fdfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f554b62c740 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fdfd40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5534009780 tx=0x7f55340bb8b0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f554b62c740 1 -- 172.21.15.94:0/238949267 shutdown_connections 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f554b62c740 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556274feff30 0x556274ff2350 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f554b62c740 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 0x556274fdd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f554b62c740 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 0x556274fdcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f554b62c740 1 --2- 172.21.15.94:0/238949267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fdfd40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.511+0000 7f554b62c740 1 -- 172.21.15.94:0/238949267 >> 172.21.15.94:0/238949267 conn(0x556274f9e310 msgr2=0x556274fd7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:15.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.512+0000 7f5526ff5640 0 client.25312 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:15.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.512+0000 7f554b62c740 1 -- 172.21.15.94:0/238949267 shutdown_connections 2024-09-19T04:18:15.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.512+0000 7f554b62c740 1 -- 172.21.15.94:0/238949267 wait complete. 2024-09-19T04:18:15.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.512+0000 7f554b62c740 20 client.25312 trim_cache size 0 max 16384 2024-09-19T04:18:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554b62c740 1 Processor -- start 2024-09-19T04:18:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554b62c740 1 -- start start 2024-09-19T04:18:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554b62c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd55d0 0x556274fd1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554b62c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554b62c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274ff7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554b62c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b67e0 con 0x556274fd6b40 2024-09-19T04:18:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554b62c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f554410d520 con 0x556274fd55d0 2024-09-19T04:18:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554b62c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5544076b00 con 0x556274fe0370 2024-09-19T04:18:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.534+0000 7f554259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fec3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60190/0 (socket says 172.21.15.94:60190) 2024-09-19T04:18:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.535+0000 7f554259c640 1 -- 172.21.15.94:0/2317895335 learned_addr learned my addr 172.21.15.94:0/2317895335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.535+0000 7f554359e640 1 --2- 172.21.15.94:0/2317895335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274ff7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.535+0000 7f5542d9d640 1 --2- 172.21.15.94:0/2317895335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd55d0 0x556274fd1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.535+0000 7f554259c640 1 -- 172.21.15.94:0/2317895335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 msgr2=0x556274ff7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:15.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.535+0000 7f554259c640 1 --2- 172.21.15.94:0/2317895335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274ff7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.535+0000 7f554259c640 1 -- 172.21.15.94:0/2317895335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd55d0 msgr2=0x556274fd1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:15.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.535+0000 7f554259c640 1 --2- 172.21.15.94:0/2317895335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd55d0 0x556274fd1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.535+0000 7f554259c640 1 -- 172.21.15.94:0/2317895335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556274fd6760 con 0x556274fd6b40 2024-09-19T04:18:15.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.536+0000 7f554259c640 1 --2- 172.21.15.94:0/2317895335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fec3e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5538010010 tx=0x7f5538000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5360a7940fb288e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:15.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.536+0000 7f5528ff9640 1 -- 172.21.15.94:0/2317895335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538017040 con 0x556274fd6b40 2024-09-19T04:18:15.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.536+0000 7f5528ff9640 1 -- 172.21.15.94:0/2317895335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f553800eba0 con 0x556274fd6b40 2024-09-19T04:18:15.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.536+0000 7f5528ff9640 1 -- 172.21.15.94:0/2317895335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538013e60 con 0x556274fd6b40 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.537+0000 7f554b62c740 1 -- 172.21.15.94:0/2317895335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 msgr2=0x556274fec3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.537+0000 7f554b62c740 1 --2- 172.21.15.94:0/2317895335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fec3e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5538010010 tx=0x7f5538000bc0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.537+0000 7f554b62c740 1 -- 172.21.15.94:0/2317895335 shutdown_connections 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.537+0000 7f554b62c740 1 --2- 172.21.15.94:0/2317895335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fe0370 0x556274ff7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.537+0000 7f554b62c740 1 --2- 172.21.15.94:0/2317895335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fd6b40 0x556274fec3e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.537+0000 7f554b62c740 1 --2- 172.21.15.94:0/2317895335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd55d0 0x556274fd1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.537+0000 7f554b62c740 1 -- 172.21.15.94:0/2317895335 >> 172.21.15.94:0/2317895335 conn(0x556274f9e310 msgr2=0x556274fd16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.538+0000 7f554b62c740 1 -- 172.21.15.94:0/2317895335 shutdown_connections 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.538+0000 7f554b62c740 1 -- 172.21.15.94:0/2317895335 wait complete. 2024-09-19T04:18:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.538+0000 7f554b62c740 1 -- start start 2024-09-19T04:18:15.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.538+0000 7f554b62c740 1 Processor -- start 2024-09-19T04:18:15.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554b62c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fe4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554b62c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 0x556274fe4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554b62c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 0x556274fe4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554b62c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f554410b9e0 con 0x556274fe0370 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554b62c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f554410ce50 con 0x556274fd6b40 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554b62c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f554410c0b0 con 0x556274fd55d0 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 0x556274fe4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f5542d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fe4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554b62c740 10 client.0 _ll_register_callbacks cb 0x7f55340b80e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f5542d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fe4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55154/0 (socket says 172.21.15.94:55154) 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f5542d9d640 1 -- 172.21.15.94:0/2959371833 learned_addr learned my addr 172.21.15.94:0/2959371833 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554259c640 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 0x556274fe4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554259c640 1 -- 172.21.15.94:0/2959371833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 msgr2=0x556274fe4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:15.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554259c640 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fe4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554259c640 1 -- 172.21.15.94:0/2959371833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 msgr2=0x556274fe4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:15.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554259c640 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 0x556274fe4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.539+0000 7f554259c640 1 -- 172.21.15.94:0/2959371833 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556275059db0 con 0x556274fd6b40 2024-09-19T04:18:15.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.540+0000 7f554259c640 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 0x556274fe4960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x556274fd2aa0 tx=0x7f5538009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19fe44e2a0c06eef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:15.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.540+0000 7f5526ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:15.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.540+0000 7f5526ff5640 1 -- 172.21.15.94:0/2959371833 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538034060 con 0x556274fd6b40 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.540+0000 7f554b62c740 1 -- 172.21.15.94:0/2959371833 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f553800eba0 con 0x556274fd6b40 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.540+0000 7f5526ff5640 1 -- 172.21.15.94:0/2959371833 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f5538005d10 con 0x556274fd6b40 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.540+0000 7f5526ff5640 1 -- 172.21.15.94:0/2959371833 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5538017050 con 0x556274fd6b40 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f554b62c740 10 client.25355 fetch_fsmap learned FSMap version 30 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f554b62c740 1 -- 172.21.15.94:0/2959371833 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55627505b050 con 0x556274fd6b40 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f5526ff5640 1 -- 172.21.15.94:0/2959371833 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f553800eba0 con 0x556274fd6b40 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f5526ff5640 1 -- 172.21.15.94:0/2959371833 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f5538013c90 con 0x556274fd6b40 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f554b62c740 10 client.25355 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f554b62c740 10 client.25355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f554b62c740 20 client.25355 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f554b62c740 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55627505b460 0x55627505d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f554b62c740 4 client.25355 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.541+0000 7f554b62c740 1 -- 172.21.15.94:0/2959371833 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556274f42d70 con 0x55627505b460 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.542+0000 7f5542d9d640 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55627505b460 0x55627505d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.542+0000 7f5542d9d640 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55627505b460 0x55627505d880 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3de972d4db3ef360 server_cookie=7965788204012058 in_seq=0 out_seq=0 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.542+0000 7f5526ff5640 10 client.25355 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.542+0000 7f5526ff5640 1 -- 172.21.15.94:0/2959371833 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556274f42d70 con 0x55627505b460 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.542+0000 7f5526ff5640 10 client.25355 handle_command_reply: tid=0 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.542+0000 7f554b62c740 1 client.25355 shutdown 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 -- 172.21.15.94:0/2959371833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 msgr2=0x556274fe4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 0x556274fe4960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x556274fd2aa0 tx=0x7f5538009450 comp rx=0 tx=0).stop 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 -- 172.21.15.94:0/2959371833 shutdown_connections 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55627505b460 0x55627505d880 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f5526ff5640 0 client.25355 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:15.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556274fe0370 0x556274fe4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556274fd6b40 0x556274fe4960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 --2- 172.21.15.94:0/2959371833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556274fd55d0 0x556274fe4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 -- 172.21.15.94:0/2959371833 >> 172.21.15.94:0/2959371833 conn(0x556274f9e310 msgr2=0x556274fd7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:15.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 -- 172.21.15.94:0/2959371833 shutdown_connections 2024-09-19T04:18:15.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 1 -- 172.21.15.94:0/2959371833 wait complete. 2024-09-19T04:18:15.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.543+0000 7f554b62c740 20 client.25355 trim_cache size 0 max 16384 2024-09-19T04:18:15.549 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:18:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 -- 172.21.15.94:0/1271725464 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f552c0a34b0 msgr2=0x7f552c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f552c0a34b0 0x7f552c0a5970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f553c005fd0 tx=0x7f553c0095c0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 -- 172.21.15.94:0/1271725464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5544076720 msgr2=0x7f5544073a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5544076720 0x7f5544073a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f553402a860 tx=0x7f5534030e00 comp rx=0 tx=0).stop 2024-09-19T04:18:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 -- 172.21.15.94:0/1271725464 shutdown_connections 2024-09-19T04:18:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f552c0a34b0 0x7f552c0a5970 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5544112a20 0x7f554406d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5544077040 0x7f554406cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 --2- 172.21.15.94:0/1271725464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5544076720 0x7f5544073a70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.544+0000 7f5549253640 1 -- 172.21.15.94:0/1271725464 >> 172.21.15.94:0/1271725464 conn(0x7f55440ffac0 msgr2=0x7f5544113060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:15.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.545+0000 7f5549253640 1 -- 172.21.15.94:0/1271725464 shutdown_connections 2024-09-19T04:18:15.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:15.545+0000 7f5549253640 1 -- 172.21.15.94:0/1271725464 wait complete. 2024-09-19T04:18:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 120.0 s, 1275.2 tps, lat 78.442 ms stddev 78.211 2024-09-19T04:18:15.755 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:15 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:15.755 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:15 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:18:15.755 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:15 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:18:15.755 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:15 smithi104 ceph-mon[34145]: pgmap v100: 97 pgs: 97 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 1.35k op/s 2024-09-19T04:18:15.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:15 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:15.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:15 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:18:15.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:15 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:18:15.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:15 smithi112 ceph-mon[35051]: pgmap v100: 97 pgs: 97 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 1.35k op/s 2024-09-19T04:18:15.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:15 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:15.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:15 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:18:15.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:15 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:18:15.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:15 smithi094 ceph-mon[30972]: pgmap v100: 97 pgs: 97 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 1.35k op/s 2024-09-19T04:18:17.442 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:18:17.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:18:17.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:17 smithi112 ceph-mon[35051]: pgmap v101: 97 pgs: 97 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 19 MiB/s wr, 2.00k op/s 2024-09-19T04:18:17.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:17 smithi094 ceph-mon[30972]: pgmap v101: 97 pgs: 97 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 19 MiB/s wr, 2.00k op/s 2024-09-19T04:18:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:17 smithi104 ceph-mon[34145]: pgmap v101: 97 pgs: 97 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 19 MiB/s wr, 2.00k op/s 2024-09-19T04:18:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.907+0000 7fd0b0725640 1 -- 172.21.15.94:0/3615838007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8102570 msgr2=0x7fd0a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.907+0000 7fd0b0725640 1 --2- 172.21.15.94:0/3615838007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8102570 0x7fd0a81029b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd09800d960 tx=0x7fd09802c340 comp rx=0 tx=0).stop 2024-09-19T04:18:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.908+0000 7fd0b0725640 1 -- 172.21.15.94:0/3615838007 shutdown_connections 2024-09-19T04:18:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.908+0000 7fd0b0725640 1 --2- 172.21.15.94:0/3615838007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a8102ef0 0x7fd0a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.908+0000 7fd0b0725640 1 --2- 172.21.15.94:0/3615838007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8102570 0x7fd0a81029b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.908+0000 7fd0b0725640 1 --2- 172.21.15.94:0/3615838007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8100e20 0x7fd0a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.908+0000 7fd0b0725640 1 -- 172.21.15.94:0/3615838007 >> 172.21.15.94:0/3615838007 conn(0x7fd0a80fc7d0 msgr2=0x7fd0a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.908+0000 7fd0b0725640 1 -- 172.21.15.94:0/3615838007 shutdown_connections 2024-09-19T04:18:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.908+0000 7fd0b0725640 1 -- 172.21.15.94:0/3615838007 wait complete. 2024-09-19T04:18:18.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.909+0000 7fd0b0725640 1 Processor -- start 2024-09-19T04:18:18.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.909+0000 7fd0b0725640 1 -- start start 2024-09-19T04:18:18.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.909+0000 7fd0b0725640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8100e20 0x7fd0a810c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.909+0000 7fd0b0725640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a8102570 0x7fd0a810cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.909+0000 7fd0b0725640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8102ef0 0x7fd0a81a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0adcc0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a8102570 0x7fd0a810cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0b0725640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a810fc40 con 0x7fd0a8100e20 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0ae4c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8100e20 0x7fd0a810c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0b0725640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a810fac0 con 0x7fd0a8102ef0 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0adcc0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a8102570 0x7fd0a810cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55168/0 (socket says 172.21.15.94:55168) 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0ae4c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8100e20 0x7fd0a810c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60218/0 (socket says 172.21.15.94:60218) 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0adcc0640 1 -- 172.21.15.94:0/2613526134 learned_addr learned my addr 172.21.15.94:0/2613526134 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0aecc2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8102ef0 0x7fd0a81a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0b0725640 1 -- 172.21.15.94:0/2613526134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a810fdc0 con 0x7fd0a8102570 2024-09-19T04:18:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0adcc0640 1 -- 172.21.15.94:0/2613526134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8102ef0 msgr2=0x7fd0a81a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8102ef0 0x7fd0a81a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.910+0000 7fd0adcc0640 1 -- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8100e20 msgr2=0x7fd0a810c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.911+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8100e20 0x7fd0a810c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.911+0000 7fd0adcc0640 1 -- 172.21.15.94:0/2613526134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a81a39f0 con 0x7fd0a8102570 2024-09-19T04:18:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.911+0000 7fd0ae4c1640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8100e20 0x7fd0a810c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.911+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a8102570 0x7fd0a810cea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd098037040 tx=0x7fd098005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.911+0000 7fd09f7fe640 1 -- 172.21.15.94:0/2613526134 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098039070 con 0x7fd0a8102570 2024-09-19T04:18:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.911+0000 7fd09f7fe640 1 -- 172.21.15.94:0/2613526134 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd098042070 con 0x7fd0a8102570 2024-09-19T04:18:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.911+0000 7fd0b0725640 1 -- 172.21.15.94:0/2613526134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81a3c80 con 0x7fd0a8102570 2024-09-19T04:18:18.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.912+0000 7fd0b0725640 1 -- 172.21.15.94:0/2613526134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81ab600 con 0x7fd0a8102570 2024-09-19T04:18:18.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.912+0000 7fd09f7fe640 1 -- 172.21.15.94:0/2613526134 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098004b50 con 0x7fd0a8102570 2024-09-19T04:18:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.916+0000 7fd09f7fe640 1 -- 172.21.15.94:0/2613526134 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd098004cf0 con 0x7fd0a8102570 2024-09-19T04:18:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.916+0000 7fd0b2afe740 1 Processor -- start 2024-09-19T04:18:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.916+0000 7fd0b2afe740 1 -- start start 2024-09-19T04:18:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.916+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f138edf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.917+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 0x558f138f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.917+0000 7fd0aecc2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f138edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.917+0000 7fd0aecc2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f138edf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55170/0 (socket says 172.21.15.94:55170) 2024-09-19T04:18:18.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.917+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f138fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.917+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558f13853280 con 0x558f138ee480 2024-09-19T04:18:18.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.917+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558f137fab10 con 0x558f138f8d40 2024-09-19T04:18:18.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.917+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558f1384f240 con 0x558f138edb60 2024-09-19T04:18:18.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.917+0000 7fd0ae4c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 0x558f138f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.917+0000 7fd0aecc2640 1 -- 172.21.15.94:0/4095893826 learned_addr learned my addr 172.21.15.94:0/4095893826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:18.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/4095893826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f138fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd0aecc2640 1 -- 172.21.15.94:0/4095893826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 msgr2=0x558f138fb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:18.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/4095893826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f138fb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd0aecc2640 1 -- 172.21.15.94:0/4095893826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 msgr2=0x558f138f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:18.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/4095893826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 0x558f138f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd0aecc2640 1 -- 172.21.15.94:0/4095893826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f138071f0 con 0x558f138edb60 2024-09-19T04:18:18.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd09f7fe640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a34b0 0x7fd0940a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/4095893826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f138fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:18.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/4095893826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f138edf40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009f50 tx=0x7fd0a00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=febee0c6d036a5f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:18.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.918+0000 7fd09f7fe640 1 -- 172.21.15.94:0/2613526134 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fd0980efeb0 con 0x7fd0a8102570 2024-09-19T04:18:18.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.919+0000 7fd0ae4c1640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a34b0 0x7fd0940a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.919+0000 7fd09cff9640 1 -- 172.21.15.94:0/4095893826 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0004b20 con 0x558f138edb60 2024-09-19T04:18:18.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.919+0000 7fd09cff9640 1 -- 172.21.15.94:0/4095893826 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0004cc0 con 0x558f138edb60 2024-09-19T04:18:18.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.919+0000 7fd09cff9640 1 -- 172.21.15.94:0/4095893826 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0005420 con 0x558f138edb60 2024-09-19T04:18:18.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0b2afe740 1 -- 172.21.15.94:0/4095893826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 msgr2=0x558f138edf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/4095893826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f138edf40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009f50 tx=0x7fd0a00118d0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0ae4c1640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a34b0 0x7fd0940a5970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x558f138eaf50 tx=0x7fd0a4009260 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0b2afe740 1 -- 172.21.15.94:0/4095893826 shutdown_connections 2024-09-19T04:18:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/4095893826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f138fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/4095893826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 0x558f138f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/4095893826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f138edf40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0b2afe740 1 -- 172.21.15.94:0/4095893826 >> 172.21.15.94:0/4095893826 conn(0x558f138ea0f0 msgr2=0x558f138ea4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0b2afe740 1 -- 172.21.15.94:0/4095893826 shutdown_connections 2024-09-19T04:18:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.920+0000 7fd0b2afe740 1 -- 172.21.15.94:0/4095893826 wait complete. 2024-09-19T04:18:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.921+0000 7fd0b2afe740 1 -- start start 2024-09-19T04:18:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.921+0000 7fd0b2afe740 1 Processor -- start 2024-09-19T04:18:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.921+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.921+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 0x558f138f6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.921+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.922+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558f138fb910 con 0x558f138edb60 2024-09-19T04:18:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.922+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558f1380df10 con 0x558f138ee480 2024-09-19T04:18:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.922+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558f138fba90 con 0x558f138f8d40 2024-09-19T04:18:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.922+0000 7fd0ae4c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.922+0000 7fd0b2afe740 10 client.0 _ll_register_callbacks cb 0x558f137d6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.922+0000 7fd0ae4c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60240/0 (socket says 172.21.15.94:60240) 2024-09-19T04:18:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.922+0000 7fd0ae4c1640 1 -- 172.21.15.94:0/8833672 learned_addr learned my addr 172.21.15.94:0/8833672 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.922+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.922+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 0x558f138f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd0adcc0640 1 -- 172.21.15.94:0/8833672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 msgr2=0x558f138f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd0adcc0640 1 -- 172.21.15.94:0/8833672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 msgr2=0x558f138f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd0adcc0640 1 -- 172.21.15.94:0/8833672 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f135c46a0 con 0x558f138ee480 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd0ae4c1640 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 0x558f138f6170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0980ed6a0 tx=0x7fd0980bbe30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f0f36cbc38e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.923+0000 7fd091ffb640 1 -- 172.21.15.94:0/8833672 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098004460 con 0x558f138ee480 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.924+0000 7fd091ffb640 1 -- 172.21.15.94:0/8833672 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fd0980dc0e0 con 0x558f138ee480 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.924+0000 7fd0b2afe740 1 -- 172.21.15.94:0/8833672 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a0004cc0 con 0x558f138ee480 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.924+0000 7fd091ffb640 1 -- 172.21.15.94:0/8833672 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd0980c05e0 con 0x558f138ee480 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.924+0000 7fd0b2afe740 10 client.25364 fetch_fsmap learned FSMap version 31 2024-09-19T04:18:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.924+0000 7fd0b2afe740 1 -- 172.21.15.94:0/8833672 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558f13907520 con 0x558f138ee480 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.925+0000 7fd091ffb640 1 -- 172.21.15.94:0/8833672 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0004cc0 con 0x558f138ee480 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.925+0000 7fd091ffb640 1 -- 172.21.15.94:0/8833672 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7fd0980edce0 con 0x558f138ee480 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.925+0000 7fd0b2afe740 10 client.25364 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.925+0000 7fd0b2afe740 10 client.25364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.926+0000 7fd0b2afe740 20 client.25364 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.926+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558f13908c20 0x558f1390b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.926+0000 7fd0b2afe740 4 client.25364 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.926+0000 7fd0b2afe740 1 -- 172.21.15.94:0/8833672 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f138607f0 con 0x558f13908c20 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.926+0000 7fd0ae4c1640 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558f13908c20 0x558f1390b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.927+0000 7fd0ae4c1640 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558f13908c20 0x558f1390b0a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=197faf6635aa82f0 server_cookie=866dfbb0a39867f4 in_seq=0 out_seq=0 2024-09-19T04:18:18.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.927+0000 7fd091ffb640 10 client.25364 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:18.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.927+0000 7fd091ffb640 1 -- 172.21.15.94:0/8833672 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f138607f0 con 0x558f13908c20 2024-09-19T04:18:18.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.927+0000 7fd091ffb640 10 client.25364 handle_command_reply: tid=0 2024-09-19T04:18:18.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd0b2afe740 1 client.25364 shutdown 2024-09-19T04:18:18.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd0b2afe740 1 -- 172.21.15.94:0/8833672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 msgr2=0x558f138f6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:18.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 0x558f138f6170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0980ed6a0 tx=0x7fd0980bbe30 comp rx=0 tx=0).stop 2024-09-19T04:18:18.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd0b2afe740 1 -- 172.21.15.94:0/8833672 shutdown_connections 2024-09-19T04:18:18.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558f13908c20 0x558f1390b0a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 0x558f138f6170 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/8833672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd0b2afe740 1 -- 172.21.15.94:0/8833672 >> 172.21.15.94:0/8833672 conn(0x558f138ea0f0 msgr2=0x558f138efe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:18.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.928+0000 7fd091ffb640 0 client.25364 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:18.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.929+0000 7fd0b2afe740 1 -- 172.21.15.94:0/8833672 shutdown_connections 2024-09-19T04:18:18.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.929+0000 7fd0b2afe740 1 -- 172.21.15.94:0/8833672 wait complete. 2024-09-19T04:18:18.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.929+0000 7fd0b2afe740 20 client.25364 trim_cache size 0 max 16384 2024-09-19T04:18:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.962+0000 7fd0b2afe740 1 Processor -- start 2024-09-19T04:18:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.962+0000 7fd0b2afe740 1 -- start start 2024-09-19T04:18:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.962+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.962+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 0x558f138f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.962+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.962+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c011150 con 0x558f138edb60 2024-09-19T04:18:18.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.962+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c00e9f0 con 0x558f138ee480 2024-09-19T04:18:18.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.962+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c00c840 con 0x558f138f8d40 2024-09-19T04:18:18.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0ae4c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0ae4c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60246/0 (socket says 172.21.15.94:60246) 2024-09-19T04:18:18.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0ae4c1640 1 -- 172.21.15.94:0/781273976 learned_addr learned my addr 172.21.15.94:0/781273976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:18.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/781273976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/781273976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 0x558f138f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0aecc2640 1 -- 172.21.15.94:0/781273976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 msgr2=0x558f138f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/781273976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 0x558f138f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0aecc2640 1 -- 172.21.15.94:0/781273976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 msgr2=0x558f138f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/781273976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0aecc2640 1 -- 172.21.15.94:0/781273976 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f138f8500 con 0x558f138f8d40 2024-09-19T04:18:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.963+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/781273976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f33f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0006010 tx=0x7fd0a000fd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9641c76d184f159 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.964+0000 7fd093fff640 1 -- 172.21.15.94:0/781273976 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0029020 con 0x558f138f8d40 2024-09-19T04:18:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.964+0000 7fd093fff640 1 -- 172.21.15.94:0/781273976 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0005bd0 con 0x558f138f8d40 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.964+0000 7fd093fff640 1 -- 172.21.15.94:0/781273976 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00054d0 con 0x558f138f8d40 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 -- 172.21.15.94:0/781273976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 msgr2=0x558f138f33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/781273976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f33f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0006010 tx=0x7fd0a000fd60 comp rx=0 tx=0).stop 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 -- 172.21.15.94:0/781273976 shutdown_connections 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/781273976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138f8d40 0x558f138f33f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/781273976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138ee480 0x558f138f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/781273976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138edb60 0x558f138f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 -- 172.21.15.94:0/781273976 >> 172.21.15.94:0/781273976 conn(0x558f138ea0f0 msgr2=0x558f138f7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 -- 172.21.15.94:0/781273976 shutdown_connections 2024-09-19T04:18:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 -- 172.21.15.94:0/781273976 wait complete. 2024-09-19T04:18:18.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.965+0000 7fd0b2afe740 1 -- start start 2024-09-19T04:18:18.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.966+0000 7fd0b2afe740 1 Processor -- start 2024-09-19T04:18:18.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.966+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f1390e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.966+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 0x558f1390ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.966+0000 7fd0b2afe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f1390f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.966+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c00ceb0 con 0x558f138ee480 2024-09-19T04:18:18.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.966+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c00e320 con 0x558f138f8d40 2024-09-19T04:18:18.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.966+0000 7fd0b2afe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c00d580 con 0x558f138edb60 2024-09-19T04:18:18.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.966+0000 7fd0b2afe740 10 client.0 _ll_register_callbacks cb 0x7fd09800d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:18.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.966+0000 7fd0aecc2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f1390f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0aecc2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f1390f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48518/0 (socket says 172.21.15.94:48518) 2024-09-19T04:18:18.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0aecc2640 1 -- 172.21.15.94:0/1818626633 learned_addr learned my addr 172.21.15.94:0/1818626633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:18.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0ae4c1640 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f1390e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 0x558f1390ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0aecc2640 1 -- 172.21.15.94:0/1818626633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 msgr2=0x558f1390e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f1390e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0aecc2640 1 -- 172.21.15.94:0/1818626633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 msgr2=0x558f1390ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 0x558f1390ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0aecc2640 1 -- 172.21.15.94:0/1818626633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f13972a30 con 0x558f138f8d40 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0adcc0640 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 0x558f1390ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd0aecc2640 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f1390f200 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009540 tx=0x7fd0a00053e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbeb268ee6b7c468 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.967+0000 7fd091ffb640 1 -- 172.21.15.94:0/1818626633 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0029040 con 0x558f138f8d40 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.968+0000 7fd091ffb640 1 -- 172.21.15.94:0/1818626633 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fd0a000f540 con 0x558f138f8d40 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.968+0000 7fd0b2afe740 1 -- 172.21.15.94:0/1818626633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a0005bd0 con 0x558f138f8d40 2024-09-19T04:18:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.968+0000 7fd091ffb640 1 -- 172.21.15.94:0/1818626633 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd0a003e050 con 0x558f138f8d40 2024-09-19T04:18:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.968+0000 7fd0b2afe740 10 client.25370 fetch_fsmap learned FSMap version 31 2024-09-19T04:18:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.968+0000 7fd0b2afe740 1 -- 172.21.15.94:0/1818626633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558f13973d10 con 0x558f138f8d40 2024-09-19T04:18:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.968+0000 7fd091ffb640 1 -- 172.21.15.94:0/1818626633 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0005bd0 con 0x558f138f8d40 2024-09-19T04:18:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd091ffb640 1 -- 172.21.15.94:0/1818626633 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7fd0a00257f0 con 0x558f138f8d40 2024-09-19T04:18:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd0b2afe740 10 client.25370 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:18:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd0b2afe740 10 client.25370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd0b2afe740 20 client.25370 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558f13974110 0x558f13976530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd0b2afe740 4 client.25370 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd0b2afe740 1 -- 172.21.15.94:0/1818626633 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558f1385ad00 con 0x558f13974110 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd0ae4c1640 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558f13974110 0x558f13976530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd091ffb640 10 client.25370 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.969+0000 7fd0ae4c1640 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558f13974110 0x558f13976530 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4d11719e37d36b server_cookie=eb96af1a352caf4e in_seq=0 out_seq=0 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.970+0000 7fd091ffb640 1 -- 172.21.15.94:0/1818626633 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558f1385ad00 con 0x558f13974110 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.970+0000 7fd091ffb640 10 client.25370 handle_command_reply: tid=0 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.970+0000 7fd0b2afe740 1 client.25370 shutdown 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 -- 172.21.15.94:0/1818626633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 msgr2=0x558f1390f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f1390f200 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009540 tx=0x7fd0a00053e0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 -- 172.21.15.94:0/1818626633 shutdown_connections 2024-09-19T04:18:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558f13974110 0x558f13976530 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f138f8d40 0x558f1390f200 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f138ee480 0x558f1390ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 --2- 172.21.15.94:0/1818626633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f138edb60 0x558f1390e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 -- 172.21.15.94:0/1818626633 >> 172.21.15.94:0/1818626633 conn(0x558f138ea0f0 msgr2=0x558f13904590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd091ffb640 0 client.25370 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 -- 172.21.15.94:0/1818626633 shutdown_connections 2024-09-19T04:18:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 1 -- 172.21.15.94:0/1818626633 wait complete. 2024-09-19T04:18:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.971+0000 7fd0b2afe740 20 client.25370 trim_cache size 0 max 16384 2024-09-19T04:18:18.978 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:18:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:18:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "bfb60817-8ea9-42de-9297-a76f2b2e2943", 2024-09-19T04:18:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:18:18.978 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:18:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 -- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a34b0 msgr2=0x7fd0940a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a34b0 0x7fd0940a5970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x558f138eaf50 tx=0x7fd0a4009260 comp rx=0 tx=0).stop 2024-09-19T04:18:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 -- 172.21.15.94:0/2613526134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a8102570 msgr2=0x7fd0a810cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a8102570 0x7fd0a810cea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd098037040 tx=0x7fd098005d80 comp rx=0 tx=0).stop 2024-09-19T04:18:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 -- 172.21.15.94:0/2613526134 shutdown_connections 2024-09-19T04:18:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a34b0 0x7fd0940a5970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8102ef0 0x7fd0a81a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a8102570 0x7fd0a810cea0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 --2- 172.21.15.94:0/2613526134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8100e20 0x7fd0a810c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:18.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.972+0000 7fd0b0725640 1 -- 172.21.15.94:0/2613526134 >> 172.21.15.94:0/2613526134 conn(0x7fd0a80fc7d0 msgr2=0x7fd0a8105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:18.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.973+0000 7fd0b0725640 1 -- 172.21.15.94:0/2613526134 shutdown_connections 2024-09-19T04:18:18.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:18.973+0000 7fd0b0725640 1 -- 172.21.15.94:0/2613526134 wait complete. 2024-09-19T04:18:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:19 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:18:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:19 smithi112 ceph-mon[35051]: pgmap v102: 97 pgs: 97 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:18:19.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:19 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:18:19.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:19 smithi094 ceph-mon[30972]: pgmap v102: 97 pgs: 97 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:18:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:19 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:18:19.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:19 smithi104 ceph-mon[34145]: pgmap v102: 97 pgs: 97 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:18:20.595 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:18:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 125.0 s, 1479.2 tps, lat 67.531 ms stddev 21.427 2024-09-19T04:18:20.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:20 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:18:20.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:20 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:18:20.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:20 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:18:20.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:20 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:18:20.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:20 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:18:20.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:20 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:18:20.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:20 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:18:20.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:20 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:18:20.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:20 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:18:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:20 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:18:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:20 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:18:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:20 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:18:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:18:21.012 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.620+0000 7f20a3fff640 1 -- 172.21.15.94:0/31878229 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098030850 con 0x7f20a4103fd0 2024-09-19T04:18:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.621+0000 7f20ab9fe640 1 -- 172.21.15.94:0/31878229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a4103fd0 msgr2=0x7f20a41043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.621+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/31878229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a4103fd0 0x7f20a41043b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2098009860 tx=0x7f209802a360 comp rx=0 tx=0).stop 2024-09-19T04:18:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.621+0000 7f20ab9fe640 1 -- 172.21.15.94:0/31878229 shutdown_connections 2024-09-19T04:18:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.621+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/31878229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a4109100 0x7f20a4110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.621+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/31878229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4104980 0x7f20a41089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.621+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/31878229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a4103fd0 0x7f20a41043b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.621+0000 7f20ab9fe640 1 -- 172.21.15.94:0/31878229 >> 172.21.15.94:0/31878229 conn(0x7f20a40ffa30 msgr2=0x7f20a4101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.622+0000 7f20ab9fe640 1 -- 172.21.15.94:0/31878229 shutdown_connections 2024-09-19T04:18:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.622+0000 7f20ab9fe640 1 -- 172.21.15.94:0/31878229 wait complete. 2024-09-19T04:18:21.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.622+0000 7f20ab9fe640 1 Processor -- start 2024-09-19T04:18:21.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.622+0000 7f20ab9fe640 1 -- start start 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20ab9fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a4103fd0 0x7f20a4076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20ab9fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a4104980 0x7f20a4076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20ab9fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4109100 0x7f20a4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20ab9fe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4114000 con 0x7f20a4104980 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20ab9fe640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4113e80 con 0x7f20a4103fd0 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20ab9fe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4114180 con 0x7f20a4109100 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a9f9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4109100 0x7f20a4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a9f9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4109100 0x7f20a4077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55226/0 (socket says 172.21.15.94:55226) 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a9f9b640 1 -- 172.21.15.94:0/1967261546 learned_addr learned my addr 172.21.15.94:0/1967261546 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a9f9b640 1 -- 172.21.15.94:0/1967261546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a4103fd0 msgr2=0x7f20a4076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a9f9b640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a4103fd0 0x7f20a4076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a8f99640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a4104980 0x7f20a4076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a9f9b640 1 -- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a4104980 msgr2=0x7f20a4076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a9f9b640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a4104980 0x7f20a4076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a9f9b640 1 -- 172.21.15.94:0/1967261546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a41aacd0 con 0x7f20a4109100 2024-09-19T04:18:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.623+0000 7f20a8f99640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a4104980 0x7f20a4076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.624+0000 7f20a9f9b640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4109100 0x7f20a4077180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f209c00eaa0 tx=0x7f209c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.624+0000 7f20a27fc640 1 -- 172.21.15.94:0/1967261546 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c00cdb0 con 0x7f20a4109100 2024-09-19T04:18:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.624+0000 7f20a27fc640 1 -- 172.21.15.94:0/1967261546 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209c005020 con 0x7f20a4109100 2024-09-19T04:18:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.624+0000 7f20a27fc640 1 -- 172.21.15.94:0/1967261546 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c0116d0 con 0x7f20a4109100 2024-09-19T04:18:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.624+0000 7f20ab9fe640 1 -- 172.21.15.94:0/1967261546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a41aafc0 con 0x7f20a4109100 2024-09-19T04:18:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.625+0000 7f20ab9fe640 1 -- 172.21.15.94:0/1967261546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a41ab480 con 0x7f20a4109100 2024-09-19T04:18:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.629+0000 7f20a27fc640 1 -- 172.21.15.94:0/1967261546 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f209c0051d0 con 0x7f20a4109100 2024-09-19T04:18:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.630+0000 7f20a27fc640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f208c0a34b0 0x7f208c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.630+0000 7f20a27fc640 1 -- 172.21.15.94:0/1967261546 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f209c0cff30 con 0x7f20a4109100 2024-09-19T04:18:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.630+0000 7f20a979a640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f208c0a34b0 0x7f208c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.630+0000 7f20addd7740 1 Processor -- start 2024-09-19T04:18:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20addd7740 1 -- start start 2024-09-19T04:18:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20addd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b495880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20addd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20addd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 0x55f50b4a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20addd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f50b3fb2e0 con 0x55f50b4a0680 2024-09-19T04:18:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20addd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f50b3a2ce0 con 0x55f50b4954a0 2024-09-19T04:18:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20addd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f50b3f72a0 con 0x55f50b495dc0 2024-09-19T04:18:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20a9f9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 0x55f50b4a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20a9f9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 0x55f50b4a2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60298/0 (socket says 172.21.15.94:60298) 2024-09-19T04:18:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20a9f9b640 1 -- 172.21.15.94:0/1240881436 learned_addr learned my addr 172.21.15.94:0/1240881436 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.631+0000 7f20a8f99640 1 --2- 172.21.15.94:0/1240881436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f20a979a640 1 --2- 172.21.15.94:0/1240881436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b495880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f20a8f99640 1 -- 172.21.15.94:0/1240881436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 msgr2=0x55f50b495880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f20a8f99640 1 --2- 172.21.15.94:0/1240881436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b495880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f20a8f99640 1 -- 172.21.15.94:0/1240881436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 msgr2=0x55f50b4a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f20a8f99640 1 --2- 172.21.15.94:0/1240881436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 0x55f50b4a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f20a8f99640 1 -- 172.21.15.94:0/1240881436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50b387440 con 0x55f50b495dc0 2024-09-19T04:18:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f20a8f99640 1 --2- 172.21.15.94:0/1240881436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4a0140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2094009b60 tx=0x7f209400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec34ddbd54c3d5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f208b7fe640 1 -- 172.21.15.94:0/1240881436 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094016070 con 0x55f50b495dc0 2024-09-19T04:18:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f208b7fe640 1 -- 172.21.15.94:0/1240881436 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2094009e50 con 0x55f50b495dc0 2024-09-19T04:18:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f208b7fe640 1 -- 172.21.15.94:0/1240881436 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20940106a0 con 0x55f50b495dc0 2024-09-19T04:18:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.632+0000 7f20a979a640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f208c0a34b0 0x7f208c0a5970 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f209802a880 tx=0x7f2098005b20 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.633+0000 7f20addd7740 1 -- 172.21.15.94:0/1240881436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 msgr2=0x55f50b4a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.633+0000 7f20addd7740 1 --2- 172.21.15.94:0/1240881436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4a0140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2094009b60 tx=0x7f209400e6c0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.634+0000 7f20addd7740 1 -- 172.21.15.94:0/1240881436 shutdown_connections 2024-09-19T04:18:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.634+0000 7f20addd7740 1 --2- 172.21.15.94:0/1240881436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 0x55f50b4a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.634+0000 7f20addd7740 1 --2- 172.21.15.94:0/1240881436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4a0140 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.634+0000 7f20addd7740 1 --2- 172.21.15.94:0/1240881436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b495880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.634+0000 7f20addd7740 1 -- 172.21.15.94:0/1240881436 >> 172.21.15.94:0/1240881436 conn(0x55f50b491a30 msgr2=0x55f50b491e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.634+0000 7f20addd7740 1 -- 172.21.15.94:0/1240881436 shutdown_connections 2024-09-19T04:18:21.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.634+0000 7f20addd7740 1 -- 172.21.15.94:0/1240881436 wait complete. 2024-09-19T04:18:21.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.634+0000 7f20addd7740 1 -- start start 2024-09-19T04:18:21.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.635+0000 7f20addd7740 1 Processor -- start 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.635+0000 7f20addd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b4954a0 0x55f50b49ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.635+0000 7f20addd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b495dc0 0x55f50b49d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.635+0000 7f20addd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b49d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20addd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f50b4a3250 con 0x55f50b495dc0 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a9f9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b49d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a8f99640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b495dc0 0x55f50b49d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a9f9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b49d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48550/0 (socket says 172.21.15.94:48550) 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a9f9b640 1 -- 172.21.15.94:0/1976835313 learned_addr learned my addr 172.21.15.94:0/1976835313 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f50b3b6050 con 0x55f50b4a0680 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f50b4a33d0 con 0x55f50b4954a0 2024-09-19T04:18:21.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20addd7740 10 client.0 _ll_register_callbacks cb 0x55f50b3f6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:21.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a979a640 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b4954a0 0x55f50b49ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a9f9b640 1 -- 172.21.15.94:0/1976835313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b4954a0 msgr2=0x55f50b49ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:21.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a9f9b640 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b4954a0 0x55f50b49ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a9f9b640 1 -- 172.21.15.94:0/1976835313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b495dc0 msgr2=0x55f50b49d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:21.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a9f9b640 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b495dc0 0x55f50b49d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.636+0000 7f20a9f9b640 1 -- 172.21.15.94:0/1976835313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50b16c3c0 con 0x55f50b4a0680 2024-09-19T04:18:21.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.637+0000 7f20a9f9b640 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b49d790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f209c0092f0 tx=0x7f209c0a0610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad843e89d43d7111 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:21.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.637+0000 7f2088ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:21.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.637+0000 7f2088ff9640 1 -- 172.21.15.94:0/1976835313 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c01f050 con 0x55f50b4a0680 2024-09-19T04:18:21.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.637+0000 7f2088ff9640 1 -- 172.21.15.94:0/1976835313 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f209c0bc0b0 con 0x55f50b4a0680 2024-09-19T04:18:21.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.637+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2094009e50 con 0x55f50b4a0680 2024-09-19T04:18:21.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.637+0000 7f2088ff9640 1 -- 172.21.15.94:0/1976835313 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f209c0cd890 con 0x55f50b4a0680 2024-09-19T04:18:21.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.637+0000 7f20addd7740 10 client.25379 fetch_fsmap learned FSMap version 31 2024-09-19T04:18:21.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.637+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f50b4aebd0 con 0x55f50b4a0680 2024-09-19T04:18:21.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.638+0000 7f2088ff9640 1 -- 172.21.15.94:0/1976835313 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209c0b5070 con 0x55f50b4a0680 2024-09-19T04:18:21.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.638+0000 7f2088ff9640 1 -- 172.21.15.94:0/1976835313 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f209c00f040 con 0x55f50b4a0680 2024-09-19T04:18:21.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.638+0000 7f20addd7740 10 client.25379 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:18:21.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.638+0000 7f20addd7740 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:21.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.638+0000 7f20addd7740 20 client.25379 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:21.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.638+0000 7f20addd7740 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f50b4b00e0 0x55f50b4b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.638+0000 7f20addd7740 4 client.25379 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:21.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.638+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2094009e50 con 0x55f50b4b00e0 2024-09-19T04:18:21.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.639+0000 7f20a979a640 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f50b4b00e0 0x55f50b4b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.639+0000 7f20a979a640 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f50b4b00e0 0x55f50b4b2510 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0a4c1775eb58dd server_cookie=e1eb2e95a6d5b196 in_seq=0 out_seq=0 2024-09-19T04:18:21.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.639+0000 7f2088ff9640 10 client.25379 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:21.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.640+0000 7f2088ff9640 1 -- 172.21.15.94:0/1976835313 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2094009e50 con 0x55f50b4b00e0 2024-09-19T04:18:21.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.640+0000 7f2088ff9640 10 client.25379 handle_command_reply: tid=0 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.640+0000 7f20addd7740 1 client.25379 shutdown 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.640+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 msgr2=0x55f50b49d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.640+0000 7f20addd7740 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b49d790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f209c0092f0 tx=0x7f209c0a0610 comp rx=0 tx=0).stop 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.640+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 shutdown_connections 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.640+0000 7f20addd7740 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f50b4b00e0 0x55f50b4b2510 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.640+0000 7f20addd7740 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b49d790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.640+0000 7f2088ff9640 0 client.25379 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.641+0000 7f20addd7740 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b495dc0 0x55f50b49d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.641+0000 7f20addd7740 1 --2- 172.21.15.94:0/1976835313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b4954a0 0x55f50b49ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.641+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 >> 172.21.15.94:0/1976835313 conn(0x55f50b491a30 msgr2=0x55f50b490b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.641+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 shutdown_connections 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.641+0000 7f20addd7740 1 -- 172.21.15.94:0/1976835313 wait complete. 2024-09-19T04:18:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.641+0000 7f20addd7740 20 client.25379 trim_cache size 0 max 16384 2024-09-19T04:18:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.674+0000 7f20addd7740 1 Processor -- start 2024-09-19T04:18:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.674+0000 7f20addd7740 1 -- start start 2024-09-19T04:18:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20addd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4954a0 0x55f50b4ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20addd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20addd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b4b8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20addd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b6850 con 0x55f50b4954a0 2024-09-19T04:18:21.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20addd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b40f0 con 0x55f50b4a0680 2024-09-19T04:18:21.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20addd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b1f40 con 0x55f50b495dc0 2024-09-19T04:18:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20a9f9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b4b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20a9f9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b4b8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48562/0 (socket says 172.21.15.94:48562) 2024-09-19T04:18:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20a9f9b640 1 -- 172.21.15.94:0/583695775 learned_addr learned my addr 172.21.15.94:0/583695775 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20a8f99640 1 --2- 172.21.15.94:0/583695775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20a9f9b640 1 -- 172.21.15.94:0/583695775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 msgr2=0x55f50b4b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20a9f9b640 1 --2- 172.21.15.94:0/583695775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20a9f9b640 1 -- 172.21.15.94:0/583695775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4954a0 msgr2=0x55f50b4ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20a9f9b640 1 --2- 172.21.15.94:0/583695775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4954a0 0x55f50b4ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.675+0000 7f20a9f9b640 1 -- 172.21.15.94:0/583695775 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50b4952d0 con 0x55f50b4a0680 2024-09-19T04:18:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.676+0000 7f20a9f9b640 1 --2- 172.21.15.94:0/583695775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b4b8320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f209c098600 tx=0x7f209c09bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e3a650cbe98546c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.676+0000 7f208affd640 1 -- 172.21.15.94:0/583695775 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c01f070 con 0x55f50b4a0680 2024-09-19T04:18:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.676+0000 7f208affd640 1 -- 172.21.15.94:0/583695775 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209c0b5040 con 0x55f50b4a0680 2024-09-19T04:18:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.676+0000 7f208affd640 1 -- 172.21.15.94:0/583695775 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c0bc0b0 con 0x55f50b4a0680 2024-09-19T04:18:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.677+0000 7f20addd7740 1 -- 172.21.15.94:0/583695775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 msgr2=0x55f50b4b8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.677+0000 7f20addd7740 1 --2- 172.21.15.94:0/583695775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b4b8320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f209c098600 tx=0x7f209c09bc30 comp rx=0 tx=0).stop 2024-09-19T04:18:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.677+0000 7f20addd7740 1 -- 172.21.15.94:0/583695775 shutdown_connections 2024-09-19T04:18:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.677+0000 7f20addd7740 1 --2- 172.21.15.94:0/583695775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4a0680 0x55f50b4b8320 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.677+0000 7f20addd7740 1 --2- 172.21.15.94:0/583695775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.677+0000 7f20addd7740 1 --2- 172.21.15.94:0/583695775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4954a0 0x55f50b4ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.677+0000 7f20addd7740 1 -- 172.21.15.94:0/583695775 >> 172.21.15.94:0/583695775 conn(0x55f50b491a30 msgr2=0x55f50b49f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:21.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.677+0000 7f20addd7740 1 -- 172.21.15.94:0/583695775 shutdown_connections 2024-09-19T04:18:21.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.677+0000 7f20addd7740 1 -- 172.21.15.94:0/583695775 wait complete. 2024-09-19T04:18:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.678+0000 7f20addd7740 1 -- start start 2024-09-19T04:18:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.678+0000 7f20addd7740 1 Processor -- start 2024-09-19T04:18:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20addd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b4a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20addd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20addd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 0x55f50b4a7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20addd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b25b0 con 0x55f50b4a0680 2024-09-19T04:18:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20addd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b3a20 con 0x55f50b4954a0 2024-09-19T04:18:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20addd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b2c80 con 0x55f50b495dc0 2024-09-19T04:18:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20addd7740 10 client.0 _ll_register_callbacks cb 0x7f209c00cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20a979a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b4a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20a9f9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 0x55f50b4a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20a979a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b4a65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48564/0 (socket says 172.21.15.94:48564) 2024-09-19T04:18:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20a979a640 1 -- 172.21.15.94:0/761772615 learned_addr learned my addr 172.21.15.94:0/761772615 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20a8f99640 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20a979a640 1 -- 172.21.15.94:0/761772615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 msgr2=0x55f50b4a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.679+0000 7f20a979a640 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.680+0000 7f20a979a640 1 -- 172.21.15.94:0/761772615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 msgr2=0x55f50b4a7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.680+0000 7f20a979a640 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 0x55f50b4a7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.680+0000 7f20a979a640 1 -- 172.21.15.94:0/761772615 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50b52b020 con 0x55f50b4954a0 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.680+0000 7f20a979a640 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b4a65c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2098009860 tx=0x7f2098005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a51830f11cca51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.680+0000 7f2088ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.680+0000 7f2088ff9640 1 -- 172.21.15.94:0/761772615 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098058060 con 0x55f50b4954a0 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.680+0000 7f2088ff9640 1 -- 172.21.15.94:0/761772615 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f20980334f0 con 0x55f50b4954a0 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.680+0000 7f20addd7740 1 -- 172.21.15.94:0/761772615 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f209c0b5040 con 0x55f50b4954a0 2024-09-19T04:18:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.681+0000 7f2088ff9640 1 -- 172.21.15.94:0/761772615 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f20980336f0 con 0x55f50b4954a0 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.681+0000 7f20addd7740 10 client.25385 fetch_fsmap learned FSMap version 31 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.681+0000 7f20addd7740 1 -- 172.21.15.94:0/761772615 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f50b52c350 con 0x55f50b4954a0 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.681+0000 7f2088ff9640 1 -- 172.21.15.94:0/761772615 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209c0b5040 con 0x55f50b4954a0 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.681+0000 7f2088ff9640 1 -- 172.21.15.94:0/761772615 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f209804a7d0 con 0x55f50b4954a0 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.681+0000 7f20addd7740 10 client.25385 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.681+0000 7f20addd7740 10 client.25385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.682+0000 7f20addd7740 20 client.25385 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.682+0000 7f20addd7740 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f50b4b00e0 0x55f50b52eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.682+0000 7f20addd7740 4 client.25385 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.682+0000 7f20addd7740 1 -- 172.21.15.94:0/761772615 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f50b408850 con 0x55f50b4b00e0 2024-09-19T04:18:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.682+0000 7f20a8f99640 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f50b4b00e0 0x55f50b52eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.683+0000 7f20a8f99640 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f50b4b00e0 0x55f50b52eaa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df676353fbba27ed server_cookie=67c6286165f34360 in_seq=0 out_seq=0 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.683+0000 7f2088ff9640 10 client.25385 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.683+0000 7f2088ff9640 1 -- 172.21.15.94:0/761772615 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f50b408850 con 0x55f50b4b00e0 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.683+0000 7f2088ff9640 10 client.25385 handle_command_reply: tid=0 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.683+0000 7f20addd7740 1 client.25385 shutdown 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.684+0000 7f20addd7740 1 -- 172.21.15.94:0/761772615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 msgr2=0x55f50b4a65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.684+0000 7f20addd7740 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b4a65c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2098009860 tx=0x7f2098005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.684+0000 7f20addd7740 1 -- 172.21.15.94:0/761772615 shutdown_connections 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.684+0000 7f20addd7740 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f50b4b00e0 0x55f50b52eaa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.684+0000 7f20addd7740 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50b4a0680 0x55f50b4a7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.684+0000 7f20addd7740 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f50b495dc0 0x55f50b4a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.684+0000 7f20addd7740 1 --2- 172.21.15.94:0/761772615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f50b4954a0 0x55f50b4a65c0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2098009860 tx=0x7f2098005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.684+0000 7f20addd7740 1 -- 172.21.15.94:0/761772615 >> 172.21.15.94:0/761772615 conn(0x55f50b491a30 msgr2=0x55f50b490350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.684+0000 7f2088ff9640 0 client.25385 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.685+0000 7f20addd7740 1 -- 172.21.15.94:0/761772615 shutdown_connections 2024-09-19T04:18:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.685+0000 7f20addd7740 1 -- 172.21.15.94:0/761772615 wait complete. 2024-09-19T04:18:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.685+0000 7f20addd7740 20 client.25385 trim_cache size 0 max 16384 2024-09-19T04:18:21.688 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:18:21.688 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:18:21.689 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:18:21.689 INFO:teuthology.orchestra.run.smithi094.stdout: "bfb60817-8ea9-42de-9297-a76f2b2e2943": { 2024-09-19T04:18:21.689 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:18:21.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "bfb60817-8ea9-42de-9297-a76f2b2e2943", 2024-09-19T04:18:21.689 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:18:21.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:18:21.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:18:21.689 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:18:21.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.686+0000 7f20ab9fe640 1 -- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f208c0a34b0 msgr2=0x7f208c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.686+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f208c0a34b0 0x7f208c0a5970 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f209802a880 tx=0x7f2098005b20 comp rx=0 tx=0).stop 2024-09-19T04:18:21.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.686+0000 7f20ab9fe640 1 -- 172.21.15.94:0/1967261546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4109100 msgr2=0x7f20a4077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.686+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4109100 0x7f20a4077180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f209c00eaa0 tx=0x7f209c00ef60 comp rx=0 tx=0).stop 2024-09-19T04:18:21.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.687+0000 7f20ab9fe640 1 -- 172.21.15.94:0/1967261546 shutdown_connections 2024-09-19T04:18:21.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.687+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f208c0a34b0 0x7f208c0a5970 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f209802a880 tx=0x7f2098005b20 comp rx=0 tx=0).stop 2024-09-19T04:18:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.687+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4109100 0x7f20a4077180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.687+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a4104980 0x7f20a4076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.687+0000 7f20ab9fe640 1 --2- 172.21.15.94:0/1967261546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20a4103fd0 0x7f20a4076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.687+0000 7f20ab9fe640 1 -- 172.21.15.94:0/1967261546 >> 172.21.15.94:0/1967261546 conn(0x7f20a40ffa30 msgr2=0x7f20a4100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.687+0000 7f20ab9fe640 1 -- 172.21.15.94:0/1967261546 shutdown_connections 2024-09-19T04:18:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:21.688+0000 7f20ab9fe640 1 -- 172.21.15.94:0/1967261546 wait complete. 2024-09-19T04:18:21.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:21 smithi112 ceph-mon[35051]: pgmap v103: 97 pgs: 97 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:18:21.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:21 smithi094 ceph-mon[30972]: pgmap v103: 97 pgs: 97 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:18:21.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:21 smithi104 ceph-mon[34145]: pgmap v103: 97 pgs: 97 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:18:22.675 INFO:tasks.cephfs.filesystem:scrub status for tag:bfb60817-8ea9-42de-9297-a76f2b2e2943 - {'path': '/', 'tag': 'bfb60817-8ea9-42de-9297-a76f2b2e2943', 'options': 'recursive,force'} 2024-09-19T04:18:23.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:23 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:18:23.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:23 smithi112 ceph-mon[35051]: pgmap v104: 97 pgs: 97 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.81k op/s 2024-09-19T04:18:23.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:23 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:18:23.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:23 smithi094 ceph-mon[30972]: pgmap v104: 97 pgs: 97 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.81k op/s 2024-09-19T04:18:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:23 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:18:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:23 smithi104 ceph-mon[34145]: pgmap v104: 97 pgs: 97 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.81k op/s 2024-09-19T04:18:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 130.0 s, 1453.2 tps, lat 68.957 ms stddev 19.215 2024-09-19T04:18:25.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:25 smithi112 ceph-mon[35051]: pgmap v105: 97 pgs: 97 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-19T04:18:25.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:25 smithi094 ceph-mon[30972]: pgmap v105: 97 pgs: 97 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-19T04:18:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:25 smithi104 ceph-mon[34145]: pgmap v105: 97 pgs: 97 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-19T04:18:27.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:26 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:18:27.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:26 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:18:27.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:26 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:18:27.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:26 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:18:27.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:26 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:18:27.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:26 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:18:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:27 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:18:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:27 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:18:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:27 smithi112 ceph-mon[35051]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.94k op/s 2024-09-19T04:18:28.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:27 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:18:28.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:27 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:18:28.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:27 smithi094 ceph-mon[30972]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.94k op/s 2024-09-19T04:18:28.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:27 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:18:28.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:27 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:18:28.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:27 smithi104 ceph-mon[34145]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.94k op/s 2024-09-19T04:18:29.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:29 smithi112 ceph-mon[35051]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:18:29.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:29 smithi094 ceph-mon[30972]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:18:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:29 smithi104 ceph-mon[34145]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:18:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 135.0 s, 1714.4 tps, lat 58.336 ms stddev 14.977 2024-09-19T04:18:31.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:31 smithi112 ceph-mon[35051]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:18:31.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:31 smithi094 ceph-mon[30972]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:18:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:31 smithi104 ceph-mon[34145]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:18:33.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:33 smithi112 ceph-mon[35051]: pgmap v109: 97 pgs: 97 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 2.04k op/s 2024-09-19T04:18:33.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:33 smithi094 ceph-mon[30972]: pgmap v109: 97 pgs: 97 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 2.04k op/s 2024-09-19T04:18:33.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:33 smithi104 ceph-mon[34145]: pgmap v109: 97 pgs: 97 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 2.04k op/s 2024-09-19T04:18:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 140.0 s, 1512.6 tps, lat 66.069 ms stddev 19.311 2024-09-19T04:18:35.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:35 smithi112 ceph-mon[35051]: pgmap v110: 97 pgs: 97 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 1.48k op/s 2024-09-19T04:18:35.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:35 smithi094 ceph-mon[30972]: pgmap v110: 97 pgs: 97 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 1.48k op/s 2024-09-19T04:18:35.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:35 smithi104 ceph-mon[34145]: pgmap v110: 97 pgs: 97 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 1.48k op/s 2024-09-19T04:18:37.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:37 smithi112 ceph-mon[35051]: pgmap v111: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 20 MiB/s wr, 2.13k op/s 2024-09-19T04:18:37.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:37 smithi094 ceph-mon[30972]: pgmap v111: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 20 MiB/s wr, 2.13k op/s 2024-09-19T04:18:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:37 smithi104 ceph-mon[34145]: pgmap v111: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 20 MiB/s wr, 2.13k op/s 2024-09-19T04:18:39.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:39 smithi112 ceph-mon[35051]: pgmap v112: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-19T04:18:39.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:39 smithi094 ceph-mon[30972]: pgmap v112: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-19T04:18:39.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:39 smithi104 ceph-mon[34145]: pgmap v112: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-19T04:18:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 145.0 s, 1469.6 tps, lat 68.074 ms stddev 58.642 2024-09-19T04:18:41.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:41 smithi112 ceph-mon[35051]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-19T04:18:41.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:41 smithi094 ceph-mon[30972]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-19T04:18:41.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:41 smithi104 ceph-mon[34145]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-19T04:18:43.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:43 smithi112 ceph-mon[35051]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 19 MiB/s wr, 2.01k op/s 2024-09-19T04:18:43.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:43 smithi094 ceph-mon[30972]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 19 MiB/s wr, 2.01k op/s 2024-09-19T04:18:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:43 smithi104 ceph-mon[34145]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 19 MiB/s wr, 2.01k op/s 2024-09-19T04:18:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 150.0 s, 1623.2 tps, lat 61.547 ms stddev 15.392 2024-09-19T04:18:45.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:45 smithi112 ceph-mon[35051]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:18:45.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:45 smithi094 ceph-mon[30972]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:18:45.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:45 smithi104 ceph-mon[34145]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:18:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:47 smithi112 ceph-mon[35051]: pgmap v116: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 19 MiB/s wr, 2.00k op/s 2024-09-19T04:18:47.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:47 smithi094 ceph-mon[30972]: pgmap v116: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 19 MiB/s wr, 2.00k op/s 2024-09-19T04:18:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:47 smithi104 ceph-mon[34145]: pgmap v116: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 19 MiB/s wr, 2.00k op/s 2024-09-19T04:18:49.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:49 smithi112 ceph-mon[35051]: pgmap v117: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:18:49.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:49 smithi094 ceph-mon[30972]: pgmap v117: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:18:49.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:49 smithi104 ceph-mon[34145]: pgmap v117: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:18:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 155.0 s, 1673.8 tps, lat 59.728 ms stddev 15.496 2024-09-19T04:18:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:51 smithi112 ceph-mon[35051]: pgmap v118: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:18:51.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:51 smithi094 ceph-mon[30972]: pgmap v118: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:18:51.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:51 smithi104 ceph-mon[34145]: pgmap v118: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:18:52.703 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:18:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:53.541 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:53 smithi094 ceph-mon[30972]: pgmap v119: 97 pgs: 97 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 2.04k op/s 2024-09-19T04:18:53.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.673+0000 7f7e0de01640 1 -- 172.21.15.94:0/1843596519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e0810e540 msgr2=0x7f7e08110930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.673+0000 7f7e0de01640 1 --2- 172.21.15.94:0/1843596519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e0810e540 0x7f7e08110930 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009ca0 tx=0x7f7e0002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.674+0000 7f7e0de01640 1 -- 172.21.15.94:0/1843596519 shutdown_connections 2024-09-19T04:18:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.674+0000 7f7e0de01640 1 --2- 172.21.15.94:0/1843596519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e0810e540 0x7f7e08110930 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.674+0000 7f7e0de01640 1 --2- 172.21.15.94:0/1843596519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e08101480 0x7f7e0810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.674+0000 7f7e0de01640 1 --2- 172.21.15.94:0/1843596519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e08100b60 0x7f7e08100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.674+0000 7f7e0de01640 1 -- 172.21.15.94:0/1843596519 >> 172.21.15.94:0/1843596519 conn(0x7f7e080fc7d0 msgr2=0x7f7e080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:53.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.674+0000 7f7e0de01640 1 -- 172.21.15.94:0/1843596519 shutdown_connections 2024-09-19T04:18:53.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.674+0000 7f7e0de01640 1 -- 172.21.15.94:0/1843596519 wait complete. 2024-09-19T04:18:53.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.675+0000 7f7e0de01640 1 Processor -- start 2024-09-19T04:18:53.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.676+0000 7f7e0de01640 1 -- start start 2024-09-19T04:18:53.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.676+0000 7f7e0de01640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e08100b60 0x7f7e0819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.676+0000 7f7e0de01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e08101480 0x7f7e0819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.676+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e08100b60 0x7f7e0819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.676+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e08100b60 0x7f7e0819bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59494/0 (socket says 172.21.15.94:59494) 2024-09-19T04:18:53.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.676+0000 7f7e077fe640 1 -- 172.21.15.94:0/692711677 learned_addr learned my addr 172.21.15.94:0/692711677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:53.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e0de01640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0810e540 0x7f7e081965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e081053e0 con 0x7f7e08101480 2024-09-19T04:18:53.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7e08105260 con 0x7f7e08100b60 2024-09-19T04:18:53.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e08105560 con 0x7f7e0810e540 2024-09-19T04:18:53.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e07fff640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0810e540 0x7f7e081965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e077fe640 1 -- 172.21.15.94:0/692711677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0810e540 msgr2=0x7f7e081965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:53.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e077fe640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0810e540 0x7f7e081965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e077fe640 1 -- 172.21.15.94:0/692711677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e08101480 msgr2=0x7f7e0819c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e077fe640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e08101480 0x7f7e0819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.677+0000 7f7e077fe640 1 -- 172.21.15.94:0/692711677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e08196d60 con 0x7f7e08100b60 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.678+0000 7f7e077fe640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e08100b60 0x7f7e0819bf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7df800b4a0 tx=0x7f7df800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.678+0000 7f7e04ff9640 1 -- 172.21.15.94:0/692711677 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8014070 con 0x7f7e08100b60 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.678+0000 7f7e04ff9640 1 -- 172.21.15.94:0/692711677 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7df8010ac0 con 0x7f7e08100b60 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.678+0000 7f7e04ff9640 1 -- 172.21.15.94:0/692711677 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8017c30 con 0x7f7e08100b60 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.678+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e08197050 con 0x7f7e08100b60 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.678+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e081a2dd0 con 0x7f7e08100b60 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.681+0000 7f7e04ff9640 1 -- 172.21.15.94:0/692711677 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7df8017070 con 0x7f7e08100b60 2024-09-19T04:18:53.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e101da740 1 Processor -- start 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e101da740 1 -- start start 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e101da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e4865880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e101da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4870140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e101da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 0x55f9e4872a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e101da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f9e47cb2e0 con 0x55f9e48654a0 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e101da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f9e4772ce0 con 0x55f9e4865dc0 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e101da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9e47c72a0 con 0x55f9e4870680 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4870140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4870140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59502/0 (socket says 172.21.15.94:59502) 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.682+0000 7f7e077fe640 1 -- 172.21.15.94:0/580144987 learned_addr learned my addr 172.21.15.94:0/580144987 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 0x55f9e4872a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e07fff640 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e4865880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e06ffd640 1 -- 172.21.15.94:0/580144987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 msgr2=0x55f9e4870140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4870140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e06ffd640 1 -- 172.21.15.94:0/580144987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 msgr2=0x55f9e4865880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e4865880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e06ffd640 1 -- 172.21.15.94:0/580144987 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e4757440 con 0x55f9e4870680 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e077fe640 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4870140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e07fff640 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e4865880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.683+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 0x55f9e4872a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc016460 tx=0x7f7dfc011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed43324ff82f40a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.684+0000 7f7dedffb640 1 -- 172.21.15.94:0/580144987 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc020020 con 0x55f9e4870680 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.684+0000 7f7dedffb640 1 -- 172.21.15.94:0/580144987 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dfc00fde0 con 0x55f9e4870680 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.684+0000 7f7dedffb640 1 -- 172.21.15.94:0/580144987 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc004a00 con 0x55f9e4870680 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.684+0000 7f7e04ff9640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7df00a3430 0x7f7df00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.684+0000 7f7e04ff9640 1 -- 172.21.15.94:0/692711677 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f7df80cf1f0 con 0x7f7e08100b60 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.684+0000 7f7e07fff640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7df00a3430 0x7f7df00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.685+0000 7f7e07fff640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7df00a3430 0x7f7df00a58f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7e00033040 tx=0x7f7e00032fd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.686+0000 7f7e101da740 1 -- 172.21.15.94:0/580144987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 msgr2=0x55f9e4872a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.686+0000 7f7e101da740 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 0x55f9e4872a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc016460 tx=0x7f7dfc011f90 comp rx=0 tx=0).stop 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.687+0000 7f7e101da740 1 -- 172.21.15.94:0/580144987 shutdown_connections 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.687+0000 7f7e101da740 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 0x55f9e4872a70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.687+0000 7f7e101da740 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4870140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.687+0000 7f7e101da740 1 --2- 172.21.15.94:0/580144987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e4865880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.687+0000 7f7e101da740 1 -- 172.21.15.94:0/580144987 >> 172.21.15.94:0/580144987 conn(0x55f9e4861a30 msgr2=0x55f9e4861e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.687+0000 7f7e101da740 1 -- 172.21.15.94:0/580144987 shutdown_connections 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.687+0000 7f7e101da740 1 -- 172.21.15.94:0/580144987 wait complete. 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.688+0000 7f7e101da740 1 -- start start 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.689+0000 7f7e101da740 1 Processor -- start 2024-09-19T04:18:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.689+0000 7f7e101da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e486ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.689+0000 7f7e101da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e486d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.689+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e486ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.689+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e486ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53552/0 (socket says 172.21.15.94:53552) 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e077fe640 1 -- 172.21.15.94:0/619197783 learned_addr learned my addr 172.21.15.94:0/619197783 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e486d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e101da740 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 0x55f9e486d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f9e4873250 con 0x55f9e48654a0 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f9e4786050 con 0x55f9e4865dc0 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9e48733d0 con 0x55f9e4870680 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e07fff640 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 0x55f9e486d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e101da740 10 client.0 _ll_register_callbacks cb 0x55f9e47c6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:53.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e077fe640 1 -- 172.21.15.94:0/619197783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 msgr2=0x55f9e486d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.690+0000 7f7e077fe640 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 0x55f9e486d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.691+0000 7f7e077fe640 1 -- 172.21.15.94:0/619197783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 msgr2=0x55f9e486d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.691+0000 7f7e077fe640 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e486d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.691+0000 7f7e077fe640 1 -- 172.21.15.94:0/619197783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e453c3c0 con 0x55f9e48654a0 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.691+0000 7f7e077fe640 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e486ff40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7df8097b20 tx=0x7f7df809b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508c2b4802abb408 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.691+0000 7f7de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.691+0000 7f7de6ffd640 1 -- 172.21.15.94:0/619197783 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df801e070 con 0x55f9e48654a0 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.691+0000 7f7de6ffd640 1 -- 172.21.15.94:0/619197783 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f7df80ccb50 con 0x55f9e48654a0 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.691+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e486d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.691+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dfc00fde0 con 0x55f9e48654a0 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7de6ffd640 1 -- 172.21.15.94:0/619197783 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7df80ccd50 con 0x55f9e48654a0 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7e101da740 10 client.15804 fetch_fsmap learned FSMap version 33 2024-09-19T04:18:53.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55f9e487eb50 con 0x55f9e48654a0 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7de6ffd640 1 -- 172.21.15.94:0/619197783 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f7df80a3d50 con 0x55f9e48654a0 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7e101da740 10 client.15804 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7e101da740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7e101da740 20 client.15804 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7e101da740 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f9e487ff00 0x55f9e48822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7e101da740 4 client.15804 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.692+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9e47d8850 con 0x55f9e487ff00 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.693+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f9e487ff00 0x55f9e48822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.693+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f9e487ff00 0x55f9e48822f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9150897a37a38bdd server_cookie=c800f4794d63f39e in_seq=0 out_seq=0 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.693+0000 7f7de6ffd640 10 client.15804 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7de6ffd640 1 -- 172.21.15.94:0/619197783 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9e47d8850 con 0x55f9e487ff00 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7de6ffd640 10 client.15804 handle_command_reply: tid=0 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7e101da740 1 client.15804 shutdown 2024-09-19T04:18:53.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 msgr2=0x55f9e486ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7e101da740 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e486ff40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7df8097b20 tx=0x7f7df809b200 comp rx=0 tx=0).stop 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 shutdown_connections 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7e101da740 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f9e487ff00 0x55f9e48822f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7de6ffd640 0 client.15804 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7e101da740 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e4870680 0x55f9e486d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7e101da740 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e486d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7e101da740 1 --2- 172.21.15.94:0/619197783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e48654a0 0x55f9e486ff40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.694+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 >> 172.21.15.94:0/619197783 conn(0x55f9e4861a30 msgr2=0x55f9e4867760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.695+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 shutdown_connections 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.695+0000 7f7e101da740 1 -- 172.21.15.94:0/619197783 wait complete. 2024-09-19T04:18:53.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.695+0000 7f7e101da740 20 client.15804 trim_cache size 0 max 16384 2024-09-19T04:18:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.725+0000 7f7e101da740 1 Processor -- start 2024-09-19T04:18:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.725+0000 7f7e101da740 1 -- start start 2024-09-19T04:18:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e101da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e487c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e487c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e487c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53556/0 (socket says 172.21.15.94:53556) 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e101da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 0x55f9e4887c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e077fe640 1 -- 172.21.15.94:0/2130487262 learned_addr learned my addr 172.21.15.94:0/2130487262 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e101da740 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4888140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 0x55f9e4887c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e101da740 1 -- 172.21.15.94:0/2130487262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4010fb0 con 0x55f9e4861a30 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e101da740 1 -- 172.21.15.94:0/2130487262 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7df400e850 con 0x55f9e4865dc0 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.726+0000 7f7e101da740 1 -- 172.21.15.94:0/2130487262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7df400c6a0 con 0x55f9e48654a0 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.727+0000 7f7e07fff640 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4888140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.727+0000 7f7e077fe640 1 -- 172.21.15.94:0/2130487262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 msgr2=0x55f9e4887c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.727+0000 7f7e077fe640 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 0x55f9e4887c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.727+0000 7f7e077fe640 1 -- 172.21.15.94:0/2130487262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 msgr2=0x55f9e4888140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.727+0000 7f7e077fe640 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4888140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.727+0000 7f7e077fe640 1 -- 172.21.15.94:0/2130487262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e4865bf0 con 0x55f9e4861a30 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.727+0000 7f7e07fff640 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4888140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.727+0000 7f7e077fe640 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e487c540 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7df800b470 tx=0x7f7df801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6478db8e634ad059 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.728+0000 7f7ded7fa640 1 -- 172.21.15.94:0/2130487262 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df80d2990 con 0x55f9e4861a30 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.728+0000 7f7ded7fa640 1 -- 172.21.15.94:0/2130487262 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dfc00fde0 con 0x55f9e4861a30 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.728+0000 7f7ded7fa640 1 -- 172.21.15.94:0/2130487262 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8097410 con 0x55f9e4861a30 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 -- 172.21.15.94:0/2130487262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 msgr2=0x55f9e487c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e487c540 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7df800b470 tx=0x7f7df801e000 comp rx=0 tx=0).stop 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 -- 172.21.15.94:0/2130487262 shutdown_connections 2024-09-19T04:18:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e4888140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 0x55f9e4887c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 --2- 172.21.15.94:0/2130487262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e487c540 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 -- 172.21.15.94:0/2130487262 >> 172.21.15.94:0/2130487262 conn(0x55f9e486d890 msgr2=0x55f9e486bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 -- 172.21.15.94:0/2130487262 shutdown_connections 2024-09-19T04:18:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 -- 172.21.15.94:0/2130487262 wait complete. 2024-09-19T04:18:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.729+0000 7f7e101da740 1 -- start start 2024-09-19T04:18:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e101da740 1 Processor -- start 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e101da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e48747c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e101da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 0x55f9e4874d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e48747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e48747c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53562/0 (socket says 172.21.15.94:53562) 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e077fe640 1 -- 172.21.15.94:0/982365665 learned_addr learned my addr 172.21.15.94:0/982365665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e101da740 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e48e9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e06ffd640 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 0x55f9e4874d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7df400cd10 con 0x55f9e4861a30 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.730+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7df400e180 con 0x55f9e4865dc0 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7df400d3e0 con 0x55f9e48654a0 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e07fff640 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e48e9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e101da740 10 client.0 _ll_register_callbacks cb 0x7f7df80cc980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e07fff640 1 -- 172.21.15.94:0/982365665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 msgr2=0x55f9e4874d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e07fff640 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 0x55f9e4874d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e07fff640 1 -- 172.21.15.94:0/982365665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 msgr2=0x55f9e48747c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e07fff640 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e48747c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e07fff640 1 -- 172.21.15.94:0/982365665 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e48755c0 con 0x55f9e4865dc0 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e077fe640 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e48747c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dfc00fde0 con 0x55f9e4865dc0 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.731+0000 7f7e07fff640 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e48e9bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e0002aed0 tx=0x7f7e0002a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d33c7c41031bc52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.732+0000 7f7de6ffd640 10 client.25403 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.732+0000 7f7de6ffd640 1 -- 172.21.15.94:0/982365665 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00004310 con 0x55f9e4865dc0 2024-09-19T04:18:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.732+0000 7f7de6ffd640 1 -- 172.21.15.94:0/982365665 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f7e00042030 con 0x55f9e4865dc0 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.732+0000 7f7de6ffd640 1 -- 172.21.15.94:0/982365665 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7e000044b0 con 0x55f9e4865dc0 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.732+0000 7f7e101da740 10 client.25403 fetch_fsmap learned FSMap version 33 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.732+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55f9e48eb120 con 0x55f9e4865dc0 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.733+0000 7f7de6ffd640 1 -- 172.21.15.94:0/982365665 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f7e00004650 con 0x55f9e4865dc0 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.733+0000 7f7e101da740 10 client.25403 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.733+0000 7f7e101da740 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.733+0000 7f7e101da740 20 client.25403 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.733+0000 7f7e101da740 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f9e4870680 0x55f9e48ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.733+0000 7f7e101da740 4 client.25403 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.733+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9e47d2d60 con 0x55f9e4870680 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.734+0000 7f7e077fe640 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f9e4870680 0x55f9e48ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.734+0000 7f7de6ffd640 10 client.25403 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.734+0000 7f7e077fe640 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f9e4870680 0x55f9e48ed6f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2d849d640644d1 server_cookie=8a459d9b17c8392e in_seq=0 out_seq=0 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.734+0000 7f7de6ffd640 1 -- 172.21.15.94:0/982365665 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f9e47d2d60 con 0x55f9e4870680 2024-09-19T04:18:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.734+0000 7f7de6ffd640 10 client.25403 handle_command_reply: tid=0 2024-09-19T04:18:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7e101da740 1 client.25403 shutdown 2024-09-19T04:18:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 msgr2=0x55f9e48e9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7e101da740 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e48e9bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e0002aed0 tx=0x7f7e0002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 shutdown_connections 2024-09-19T04:18:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7e101da740 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f9e4870680 0x55f9e48ed6f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7e101da740 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9e4865dc0 0x55f9e48e9bd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7de6ffd640 0 client.25403 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7e101da740 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9e48654a0 0x55f9e4874d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7e101da740 1 --2- 172.21.15.94:0/982365665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e4861a30 0x55f9e48747c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.735+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 >> 172.21.15.94:0/982365665 conn(0x55f9e486d890 msgr2=0x55f9e4860b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.736+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 shutdown_connections 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.736+0000 7f7e101da740 1 -- 172.21.15.94:0/982365665 wait complete. 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.736+0000 7f7e101da740 20 client.25403 trim_cache size 0 max 16384 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:18:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:18:53.742 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:18:53.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7df00a3430 msgr2=0x7f7df00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:53.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7df00a3430 0x7f7df00a58f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7e00033040 tx=0x7f7e00032fd0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e08100b60 msgr2=0x7f7e0819bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:53.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e08100b60 0x7f7e0819bf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7df800b4a0 tx=0x7f7df800b960 comp rx=0 tx=0).stop 2024-09-19T04:18:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 shutdown_connections 2024-09-19T04:18:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7df00a3430 0x7f7df00a58f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0810e540 0x7f7e081965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e08101480 0x7f7e0819c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 --2- 172.21.15.94:0/692711677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e08100b60 0x7f7e0819bf70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 >> 172.21.15.94:0/692711677 conn(0x7f7e080fc7d0 msgr2=0x7f7e080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 shutdown_connections 2024-09-19T04:18:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:53.737+0000 7f7e0de01640 1 -- 172.21.15.94:0/692711677 wait complete. 2024-09-19T04:18:53.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:53 smithi112 ceph-mon[35051]: pgmap v119: 97 pgs: 97 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 2.04k op/s 2024-09-19T04:18:53.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:53 smithi104 ceph-mon[34145]: pgmap v119: 97 pgs: 97 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 2.04k op/s 2024-09-19T04:18:54.606 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:18:54.607 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:18:55.024 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.583+0000 7f3c46781640 1 -- 172.21.15.94:0/1273032173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c40101470 msgr2=0x7f3c40101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.583+0000 7f3c46781640 1 --2- 172.21.15.94:0/1273032173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c40101470 0x7f3c40101850 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009ca0 tx=0x7f3c3802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.584+0000 7f3c46781640 1 -- 172.21.15.94:0/1273032173 shutdown_connections 2024-09-19T04:18:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.584+0000 7f3c46781640 1 --2- 172.21.15.94:0/1273032173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c40105de0 0x7f3c4010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.584+0000 7f3c46781640 1 --2- 172.21.15.94:0/1273032173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c40101e20 0x7f3c401056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.584+0000 7f3c46781640 1 --2- 172.21.15.94:0/1273032173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c40101470 0x7f3c40101850 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.584+0000 7f3c46781640 1 -- 172.21.15.94:0/1273032173 >> 172.21.15.94:0/1273032173 conn(0x7f3c40077230 msgr2=0x7f3c400fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.584+0000 7f3c46781640 1 -- 172.21.15.94:0/1273032173 shutdown_connections 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.584+0000 7f3c46781640 1 -- 172.21.15.94:0/1273032173 wait complete. 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.585+0000 7f3c46781640 1 Processor -- start 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.585+0000 7f3c46781640 1 -- start start 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.585+0000 7f3c46781640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c40101470 0x7f3c4010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.585+0000 7f3c46781640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c40101e20 0x7f3c4010a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.585+0000 7f3c46781640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c40105de0 0x7f3c4010b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.585+0000 7f3c46781640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40113fd0 con 0x7f3c40101470 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.585+0000 7f3c46781640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40113e50 con 0x7f3c40101e20 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c46781640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40114150 con 0x7f3c40105de0 2024-09-19T04:18:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c44d1e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c40105de0 0x7f3c4010b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c44d1e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c40105de0 0x7f3c4010b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37250/0 (socket says 172.21.15.94:37250) 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c44d1e640 1 -- 172.21.15.94:0/1429867139 learned_addr learned my addr 172.21.15.94:0/1429867139 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c3ffff640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c40101470 0x7f3c4010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c3f7fe640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c40101e20 0x7f3c4010a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c44d1e640 1 -- 172.21.15.94:0/1429867139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c40101e20 msgr2=0x7f3c4010a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c44d1e640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c40101e20 0x7f3c4010a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c44d1e640 1 -- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c40101470 msgr2=0x7f3c4010d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c44d1e640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c40101470 0x7f3c4010d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.586+0000 7f3c44d1e640 1 -- 172.21.15.94:0/1429867139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c401aad10 con 0x7f3c40105de0 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.587+0000 7f3c44d1e640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c40105de0 0x7f3c4010b0f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000d950 tx=0x7f3c3000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.587+0000 7f3c3d7fa640 1 -- 172.21.15.94:0/1429867139 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30017030 con 0x7f3c40105de0 2024-09-19T04:18:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.587+0000 7f3c3d7fa640 1 -- 172.21.15.94:0/1429867139 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c30004020 con 0x7f3c40105de0 2024-09-19T04:18:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.588+0000 7f3c3d7fa640 1 -- 172.21.15.94:0/1429867139 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30003b20 con 0x7f3c40105de0 2024-09-19T04:18:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.588+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c401aaf40 con 0x7f3c40105de0 2024-09-19T04:18:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.588+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c401ab2e0 con 0x7f3c40105de0 2024-09-19T04:18:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.589+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c40102220 con 0x7f3c40105de0 2024-09-19T04:18:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.591+0000 7f3c3d7fa640 1 -- 172.21.15.94:0/1429867139 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3c3001a070 con 0x7f3c40105de0 2024-09-19T04:18:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.591+0000 7f3c3d7fa640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3c280a3430 0x7f3c280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.592+0000 7f3c3ffff640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3c280a3430 0x7f3c280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.592+0000 7f3c3d7fa640 1 -- 172.21.15.94:0/1429867139 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3c300cf3b0 con 0x7f3c40105de0 2024-09-19T04:18:55.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.592+0000 7f3c3ffff640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3c280a3430 0x7f3c280a58f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005e30 tx=0x7f3c38031000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.597+0000 7f3c3d7fa640 1 -- 172.21.15.94:0/1429867139 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c30021070 con 0x7f3c40105de0 2024-09-19T04:18:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 160.0 s, 1534.4 tps, lat 65.110 ms stddev 17.820 2024-09-19T04:18:55.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:55 smithi112 ceph-mon[35051]: pgmap v120: 97 pgs: 97 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 1.44k op/s 2024-09-19T04:18:55.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:55 smithi094 ceph-mon[30972]: pgmap v120: 97 pgs: 97 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 1.44k op/s 2024-09-19T04:18:55.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:55 smithi104 ceph-mon[34145]: pgmap v120: 97 pgs: 97 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 1.44k op/s 2024-09-19T04:18:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.961+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c4010bf00 con 0x7f3c40105de0 2024-09-19T04:18:55.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.962+0000 7f3c3d7fa640 1 -- 172.21.15.94:0/1429867139 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11916 (secure 0 0 0) 0x7f3c3009bd80 con 0x7f3c40105de0 2024-09-19T04:18:55.964 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:18:55.964 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 33 2024-09-19T04:18:55.965 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":33,"btime":"2024-09-19T04:18:26:862783+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:18:26.812631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24466,24467,24454]},"id":1}]} 2024-09-19T04:18:55.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3c280a3430 msgr2=0x7f3c280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3c280a3430 0x7f3c280a58f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005e30 tx=0x7f3c38031000 comp rx=0 tx=0).stop 2024-09-19T04:18:55.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c40105de0 msgr2=0x7f3c4010b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c40105de0 0x7f3c4010b0f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000d950 tx=0x7f3c3000de10 comp rx=0 tx=0).stop 2024-09-19T04:18:55.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 shutdown_connections 2024-09-19T04:18:55.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3c280a3430 0x7f3c280a58f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c40105de0 0x7f3c4010b0f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c40101e20 0x7f3c4010a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 --2- 172.21.15.94:0/1429867139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c40101470 0x7f3c4010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 >> 172.21.15.94:0/1429867139 conn(0x7f3c40077230 msgr2=0x7f3c40104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:55.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 shutdown_connections 2024-09-19T04:18:55.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:55.966+0000 7f3c46781640 1 -- 172.21.15.94:0/1429867139 wait complete. 2024-09-19T04:18:56.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:56 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1429867139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:56.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:18:56.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:56 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1429867139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:56.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:56 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1429867139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:57.181 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:57 smithi094 ceph-mon[30972]: pgmap v121: 97 pgs: 97 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 2.10k op/s 2024-09-19T04:18:57.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.754+0000 7f4fbe468640 1 -- 172.21.15.94:0/4290867850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fb81027a0 msgr2=0x7f4fb81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.754+0000 7f4fbe468640 1 --2- 172.21.15.94:0/4290867850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fb81027a0 0x7f4fb81067f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009900 tx=0x7f4fac02a410 comp rx=0 tx=0).stop 2024-09-19T04:18:57.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.754+0000 7f4fbe468640 1 -- 172.21.15.94:0/4290867850 shutdown_connections 2024-09-19T04:18:57.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.754+0000 7f4fbe468640 1 --2- 172.21.15.94:0/4290867850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fb8106f20 0x7f4fb810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.754+0000 7f4fbe468640 1 --2- 172.21.15.94:0/4290867850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fb81027a0 0x7f4fb81067f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.754+0000 7f4fbe468640 1 --2- 172.21.15.94:0/4290867850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb8101df0 0x7f4fb81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.754+0000 7f4fbe468640 1 -- 172.21.15.94:0/4290867850 >> 172.21.15.94:0/4290867850 conn(0x7f4fb80fd8f0 msgr2=0x7f4fb80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.754+0000 7f4fbe468640 1 -- 172.21.15.94:0/4290867850 shutdown_connections 2024-09-19T04:18:57.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.755+0000 7f4fbe468640 1 -- 172.21.15.94:0/4290867850 wait complete. 2024-09-19T04:18:57.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.755+0000 7f4fbe468640 1 Processor -- start 2024-09-19T04:18:57.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.755+0000 7f4fbe468640 1 -- start start 2024-09-19T04:18:57.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fbe468640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb8101df0 0x7f4fb8075b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fbe468640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fb81027a0 0x7f4fb8076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fbe468640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fb8106f20 0x7f4fb806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fb81027a0 0x7f4fb8076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fbe468640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8111e80 con 0x7f4fb8106f20 2024-09-19T04:18:57.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fb81027a0 0x7f4fb8076090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59572/0 (socket says 172.21.15.94:59572) 2024-09-19T04:18:57.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fb77fe640 1 -- 172.21.15.94:0/2064761878 learned_addr learned my addr 172.21.15.94:0/2064761878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:57.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fbca05640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fb8106f20 0x7f4fb806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb8101df0 0x7f4fb8075b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8111d00 con 0x7f4fb81027a0 2024-09-19T04:18:57.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.756+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8112000 con 0x7f4fb8101df0 2024-09-19T04:18:57.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fbca05640 1 -- 172.21.15.94:0/2064761878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb8101df0 msgr2=0x7f4fb8075b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fbca05640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb8101df0 0x7f4fb8075b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fbca05640 1 -- 172.21.15.94:0/2064761878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fb81027a0 msgr2=0x7f4fb8076090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fbca05640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fb81027a0 0x7f4fb8076090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fbca05640 1 -- 172.21.15.94:0/2064761878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb806d600 con 0x7f4fb8106f20 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fbca05640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fb8106f20 0x7f4fb806cd40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000ec50 tx=0x7f4fb000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fb57fa640 1 -- 172.21.15.94:0/2064761878 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0003b70 con 0x7f4fb8106f20 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fb57fa640 1 -- 172.21.15.94:0/2064761878 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fb0003d10 con 0x7f4fb8106f20 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fb57fa640 1 -- 172.21.15.94:0/2064761878 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0010760 con 0x7f4fb8106f20 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.757+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb806d890 con 0x7f4fb8106f20 2024-09-19T04:18:57.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.758+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb8071770 con 0x7f4fb8106f20 2024-09-19T04:18:57.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.761+0000 7f4fb57fa640 1 -- 172.21.15.94:0/2064761878 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4fb0004020 con 0x7f4fb8106f20 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.762+0000 7f4fc0841740 1 Processor -- start 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.762+0000 7f4fc0841740 1 -- start start 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.762+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e119090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.762+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e119a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.762+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e126760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.762+0000 7f4fc0841740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560e7e07f2f0 con 0x560e7e1195d0 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.762+0000 7f4fc0841740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e7e026cf0 con 0x560e7e11ab40 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.762+0000 7f4fc0841740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e7e07b2b0 con 0x560e7e124370 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.763+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e119090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.763+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e119090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59584/0 (socket says 172.21.15.94:59584) 2024-09-19T04:18:57.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.763+0000 7f4fb77fe640 1 -- 172.21.15.94:0/1760120094 learned_addr learned my addr 172.21.15.94:0/1760120094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:57.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.763+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e119a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.763+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e126760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.763+0000 7f4fb7fff640 1 -- 172.21.15.94:0/1760120094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 msgr2=0x560e7e126760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.764+0000 7f4fb57fa640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4fa00a34b0 0x7f4fa00a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.764+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e126760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.764+0000 7f4fbca05640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4fa00a34b0 0x7f4fa00a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.764+0000 7f4fb7fff640 1 -- 172.21.15.94:0/1760120094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 msgr2=0x560e7e119090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.764+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e119090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.764+0000 7f4fb57fa640 1 -- 172.21.15.94:0/2064761878 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f4fb00d0220 con 0x7f4fb8106f20 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.764+0000 7f4fb7fff640 1 -- 172.21.15.94:0/1760120094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e7e00b3a0 con 0x560e7e1195d0 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.764+0000 7f4fbca05640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4fa00a34b0 0x7f4fa00a5970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4fb009c190 tx=0x7f4fb009f040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.765+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e119a10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8005fd0 tx=0x7f4fa80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fefdd5ab9db261 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.765+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e126760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.765+0000 7f4f9e7fc640 1 -- 172.21.15.94:0/1760120094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8004020 con 0x560e7e1195d0 2024-09-19T04:18:57.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.765+0000 7f4f9e7fc640 1 -- 172.21.15.94:0/1760120094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fa80041c0 con 0x560e7e1195d0 2024-09-19T04:18:57.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4f9e7fc640 1 -- 172.21.15.94:0/1760120094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80116a0 con 0x560e7e1195d0 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4fc0841740 1 -- 172.21.15.94:0/1760120094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 msgr2=0x560e7e119a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4fc0841740 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e119a10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8005fd0 tx=0x7f4fa80107a0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4fc0841740 1 -- 172.21.15.94:0/1760120094 shutdown_connections 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4fc0841740 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e126760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4fc0841740 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e119a10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4fc0841740 1 --2- 172.21.15.94:0/1760120094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e119090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4fc0841740 1 -- 172.21.15.94:0/1760120094 >> 172.21.15.94:0/1760120094 conn(0x560e7e0e2310 msgr2=0x560e7e115c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4fc0841740 1 -- 172.21.15.94:0/1760120094 shutdown_connections 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.766+0000 7f4fc0841740 1 -- 172.21.15.94:0/1760120094 wait complete. 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.767+0000 7f4fc0841740 1 -- start start 2024-09-19T04:18:57.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.767+0000 7f4fc0841740 1 Processor -- start 2024-09-19T04:18:57.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.767+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e1195d0 0x560e7e123c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e11ab40 0x560e7e120f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e124370 0x560e7e121470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fc0841740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560e7e1270c0 con 0x560e7e124370 2024-09-19T04:18:57.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fc0841740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e7e03a060 con 0x560e7e1195d0 2024-09-19T04:18:57.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fc0841740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e7e127240 con 0x560e7e11ab40 2024-09-19T04:18:57.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fc0841740 10 client.0 _ll_register_callbacks cb 0x560e7dd8d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:57.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e1195d0 0x560e7e123c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e124370 0x560e7e121470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e11ab40 0x560e7e120f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e124370 0x560e7e121470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53600/0 (socket says 172.21.15.94:53600) 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 -- 172.21.15.94:0/4093002863 learned_addr learned my addr 172.21.15.94:0/4093002863 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 -- 172.21.15.94:0/4093002863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e11ab40 msgr2=0x560e7e120f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e11ab40 0x560e7e120f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 -- 172.21.15.94:0/4093002863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e124370 msgr2=0x560e7e121470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e124370 0x560e7e121470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 -- 172.21.15.94:0/4093002863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7ddf03c0 con 0x560e7e1195d0 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e1195d0 0x560e7e123c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000c00 tx=0x7f4fac004080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a77939cd71a5f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.768+0000 7f4f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.769+0000 7f4f977fe640 1 -- 172.21.15.94:0/4093002863 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac035040 con 0x560e7e1195d0 2024-09-19T04:18:57.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.769+0000 7f4fc0841740 1 -- 172.21.15.94:0/4093002863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa80041c0 con 0x560e7e1195d0 2024-09-19T04:18:57.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.769+0000 7f4f977fe640 1 -- 172.21.15.94:0/4093002863 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f4fac005020 con 0x560e7e1195d0 2024-09-19T04:18:57.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.769+0000 7f4f977fe640 1 -- 172.21.15.94:0/4093002863 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fac005220 con 0x560e7e1195d0 2024-09-19T04:18:57.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.769+0000 7f4fc0841740 10 client.25421 fetch_fsmap learned FSMap version 33 2024-09-19T04:18:57.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.769+0000 7f4fc0841740 1 -- 172.21.15.94:0/4093002863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560e7e132b20 con 0x560e7e1195d0 2024-09-19T04:18:57.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.769+0000 7f4f977fe640 1 -- 172.21.15.94:0/4093002863 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fac004c50 con 0x560e7e1195d0 2024-09-19T04:18:57.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.770+0000 7f4f977fe640 1 -- 172.21.15.94:0/4093002863 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f4fac03d570 con 0x560e7e1195d0 2024-09-19T04:18:57.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.770+0000 7f4fc0841740 10 client.25421 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:18:57.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.770+0000 7f4fc0841740 10 client.25421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:18:57.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.770+0000 7f4fc0841740 20 client.25421 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:57.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.770+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560e7e134350 0x560e7e136780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.770+0000 7f4fc0841740 4 client.25421 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:57.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.770+0000 7f4fc0841740 1 -- 172.21.15.94:0/4093002863 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4fa80041c0 con 0x560e7e134350 2024-09-19T04:18:57.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.770+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560e7e134350 0x560e7e136780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.771+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560e7e134350 0x560e7e136780 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ed54dd5707f1e45 server_cookie=e765e0b885751e53 in_seq=0 out_seq=0 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.771+0000 7f4f977fe640 10 client.25421 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.771+0000 7f4f977fe640 1 -- 172.21.15.94:0/4093002863 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4fa80041c0 con 0x560e7e134350 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.771+0000 7f4f977fe640 10 client.25421 handle_command_reply: tid=0 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.771+0000 7f4fc0841740 1 client.25421 shutdown 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 -- 172.21.15.94:0/4093002863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e1195d0 msgr2=0x560e7e123c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e1195d0 0x560e7e123c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000c00 tx=0x7f4fac004080 comp rx=0 tx=0).stop 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 -- 172.21.15.94:0/4093002863 shutdown_connections 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560e7e134350 0x560e7e136780 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e124370 0x560e7e121470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e11ab40 0x560e7e120f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4f977fe640 0 client.25421 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:18:57.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4093002863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e1195d0 0x560e7e123c30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 -- 172.21.15.94:0/4093002863 >> 172.21.15.94:0/4093002863 conn(0x560e7e0e2310 msgr2=0x560e7e11b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 -- 172.21.15.94:0/4093002863 shutdown_connections 2024-09-19T04:18:57.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 1 -- 172.21.15.94:0/4093002863 wait complete. 2024-09-19T04:18:57.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.772+0000 7f4fc0841740 20 client.25421 trim_cache size 0 max 16384 2024-09-19T04:18:57.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.793+0000 7f4fc0841740 1 Processor -- start 2024-09-19T04:18:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.793+0000 7f4fc0841740 1 -- start start 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e1303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e120ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e121430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e1303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e1303b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53602/0 (socket says 172.21.15.94:53602) 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fb77fe640 1 -- 172.21.15.94:0/3812841062 learned_addr learned my addr 172.21.15.94:0/3812841062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fc0841740 1 -- 172.21.15.94:0/3812841062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81bf060 con 0x560e7e1195d0 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fc0841740 1 -- 172.21.15.94:0/3812841062 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81bc900 con 0x560e7e11ab40 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fc0841740 1 -- 172.21.15.94:0/3812841062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81ba750 con 0x560e7e124370 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/3812841062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e120ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.794+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/3812841062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e121430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.795+0000 7f4fb7fff640 1 -- 172.21.15.94:0/3812841062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 msgr2=0x560e7e121430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.795+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/3812841062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e121430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.795+0000 7f4fb7fff640 1 -- 172.21.15.94:0/3812841062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 msgr2=0x560e7e1303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.795+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/3812841062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e1303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.795+0000 7f4fb7fff640 1 -- 172.21.15.94:0/3812841062 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e7e115510 con 0x560e7e11ab40 2024-09-19T04:18:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.795+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/3812841062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e120ef0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8005d00 tx=0x7f4fa800b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=694beaaec90ad631 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.795+0000 7f4f9dffb640 1 -- 172.21.15.94:0/3812841062 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa802c020 con 0x560e7e11ab40 2024-09-19T04:18:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.795+0000 7f4f9dffb640 1 -- 172.21.15.94:0/3812841062 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fa80035f0 con 0x560e7e11ab40 2024-09-19T04:18:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.795+0000 7f4f9dffb640 1 -- 172.21.15.94:0/3812841062 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80053a0 con 0x560e7e11ab40 2024-09-19T04:18:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.796+0000 7f4fc0841740 1 -- 172.21.15.94:0/3812841062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 msgr2=0x560e7e120ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.796+0000 7f4fc0841740 1 --2- 172.21.15.94:0/3812841062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e120ef0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8005d00 tx=0x7f4fa800b410 comp rx=0 tx=0).stop 2024-09-19T04:18:57.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.796+0000 7f4fc0841740 1 -- 172.21.15.94:0/3812841062 shutdown_connections 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.796+0000 7f4fc0841740 1 --2- 172.21.15.94:0/3812841062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e121430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.796+0000 7f4fc0841740 1 --2- 172.21.15.94:0/3812841062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e120ef0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.796+0000 7f4fc0841740 1 --2- 172.21.15.94:0/3812841062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e1303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.796+0000 7f4fc0841740 1 -- 172.21.15.94:0/3812841062 >> 172.21.15.94:0/3812841062 conn(0x560e7e0e2310 msgr2=0x560e7e11c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.797+0000 7f4fc0841740 1 -- 172.21.15.94:0/3812841062 shutdown_connections 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.797+0000 7f4fc0841740 1 -- 172.21.15.94:0/3812841062 wait complete. 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.797+0000 7f4fc0841740 1 -- start start 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.797+0000 7f4fc0841740 1 Processor -- start 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e12c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e12ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fc0841740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e11e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e12ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e12ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59612/0 (socket says 172.21.15.94:59612) 2024-09-19T04:18:57.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fb7fff640 1 -- 172.21.15.94:0/4003224549 learned_addr learned my addr 172.21.15.94:0/4003224549 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:57.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e12c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81badc0 con 0x560e7e1195d0 2024-09-19T04:18:57.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81bc230 con 0x560e7e11ab40 2024-09-19T04:18:57.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81bb490 con 0x560e7e124370 2024-09-19T04:18:57.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.798+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e11e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fb7fff640 1 -- 172.21.15.94:0/4003224549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 msgr2=0x560e7e11e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e11e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fb7fff640 1 -- 172.21.15.94:0/4003224549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 msgr2=0x560e7e12c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e12c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fb7fff640 1 -- 172.21.15.94:0/4003224549 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7e11e800 con 0x560e7e11ab40 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fc0841740 10 client.0 _ll_register_callbacks cb 0x7f4fac009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fb7fff640 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e12ca60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8003e40 tx=0x7f4fa8003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c5e5322f7d15d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa80035f0 con 0x560e7e11ab40 2024-09-19T04:18:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4f977fe640 1 -- 172.21.15.94:0/4003224549 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8004290 con 0x560e7e11ab40 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4f977fe640 1 -- 172.21.15.94:0/4003224549 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f4fa8004430 con 0x560e7e11ab40 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4f977fe640 1 -- 172.21.15.94:0/4003224549 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fa8005040 con 0x560e7e11ab40 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fc0841740 10 client.25427 fetch_fsmap learned FSMap version 33 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.799+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=33}) -- 0x560e7e13b8a0 con 0x560e7e11ab40 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.800+0000 7f4f977fe640 1 -- 172.21.15.94:0/4003224549 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f4fa80051e0 con 0x560e7e11ab40 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.800+0000 7f4fc0841740 10 client.25427 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.800+0000 7f4fc0841740 10 client.25427 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.800+0000 7f4fc0841740 20 client.25427 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.800+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560e7e134350 0x560e7e1a1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.800+0000 7f4fc0841740 4 client.25427 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.800+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4fa80035f0 con 0x560e7e134350 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.800+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560e7e134350 0x560e7e1a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4fb77fe640 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560e7e134350 0x560e7e1a1070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b38b60db38dd174f server_cookie=2f059dbcf758032a in_seq=0 out_seq=0 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4f977fe640 10 client.25427 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4f977fe640 1 -- 172.21.15.94:0/4003224549 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4fa80035f0 con 0x560e7e134350 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4f977fe640 10 client.25427 handle_command_reply: tid=0 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4fc0841740 1 client.25427 shutdown 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 msgr2=0x560e7e12ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e12ca60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8003e40 tx=0x7f4fa8003e70 comp rx=0 tx=0).stop 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 shutdown_connections 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560e7e134350 0x560e7e1a1070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4f977fe640 0 client.25427 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e7e124370 0x560e7e11e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e7e11ab40 0x560e7e12ca60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.801+0000 7f4fc0841740 1 --2- 172.21.15.94:0/4003224549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e7e1195d0 0x560e7e12c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.802+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 >> 172.21.15.94:0/4003224549 conn(0x560e7e0e2310 msgr2=0x560e7e11c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.802+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 shutdown_connections 2024-09-19T04:18:57.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.802+0000 7f4fc0841740 1 -- 172.21.15.94:0/4003224549 wait complete. 2024-09-19T04:18:57.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.802+0000 7f4fc0841740 20 client.25427 trim_cache size 0 max 16384 2024-09-19T04:18:57.807 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4fa00a34b0 msgr2=0x7f4fa00a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4fa00a34b0 0x7f4fa00a5970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4fb009c190 tx=0x7f4fb009f040 comp rx=0 tx=0).stop 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fb8106f20 msgr2=0x7f4fb806cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fb8106f20 0x7f4fb806cd40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000ec50 tx=0x7f4fb000c6a0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 shutdown_connections 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4fa00a34b0 0x7f4fa00a5970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fb8106f20 0x7f4fb806cd40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fb81027a0 0x7f4fb8076090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 --2- 172.21.15.94:0/2064761878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb8101df0 0x7f4fb8075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 >> 172.21.15.94:0/2064761878 conn(0x7f4fb80fd8f0 msgr2=0x7f4fb80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 shutdown_connections 2024-09-19T04:18:57.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:57.803+0000 7f4fbe468640 1 -- 172.21.15.94:0/2064761878 wait complete. 2024-09-19T04:18:57.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:57 smithi104 ceph-mon[34145]: pgmap v121: 97 pgs: 97 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 2.10k op/s 2024-09-19T04:18:58.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:57 smithi112 ceph-mon[35051]: pgmap v121: 97 pgs: 97 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 2.10k op/s 2024-09-19T04:18:58.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:18:59.044 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:59.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:18:59 smithi112 ceph-mon[35051]: pgmap v122: 97 pgs: 97 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:18:59.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.600+0000 7f3de0b30640 1 -- 172.21.15.94:0/3447473207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ddc103ff0 msgr2=0x7f3ddc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.600+0000 7f3de0b30640 1 --2- 172.21.15.94:0/3447473207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ddc103ff0 0x7f3ddc1043d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009be0 tx=0x7f3dcc02a6c0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.601+0000 7f3de0b30640 1 -- 172.21.15.94:0/3447473207 shutdown_connections 2024-09-19T04:18:59.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.601+0000 7f3de0b30640 1 --2- 172.21.15.94:0/3447473207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ddc109120 0x7f3ddc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.601+0000 7f3de0b30640 1 --2- 172.21.15.94:0/3447473207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ddc1049a0 0x7f3ddc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.601+0000 7f3de0b30640 1 --2- 172.21.15.94:0/3447473207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ddc103ff0 0x7f3ddc1043d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.601+0000 7f3de0b30640 1 -- 172.21.15.94:0/3447473207 >> 172.21.15.94:0/3447473207 conn(0x7f3ddc0ffab0 msgr2=0x7f3ddc101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:59.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.601+0000 7f3de0b30640 1 -- 172.21.15.94:0/3447473207 shutdown_connections 2024-09-19T04:18:59.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.601+0000 7f3de0b30640 1 -- 172.21.15.94:0/3447473207 wait complete. 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.602+0000 7f3de0b30640 1 Processor -- start 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.602+0000 7f3de0b30640 1 -- start start 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.602+0000 7f3de0b30640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ddc103ff0 0x7f3ddc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3de0b30640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ddc1049a0 0x7f3ddc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3de0b30640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ddc109120 0x7f3ddc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3de0b30640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc113fd0 con 0x7f3ddc103ff0 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3de0b30640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc113e50 con 0x7f3ddc1049a0 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3de0b30640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc114150 con 0x7f3ddc109120 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3ddad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ddc109120 0x7f3ddc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3ddad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ddc109120 0x7f3ddc19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37308/0 (socket says 172.21.15.94:37308) 2024-09-19T04:18:59.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3ddad9d640 1 -- 172.21.15.94:0/2147268154 learned_addr learned my addr 172.21.15.94:0/2147268154 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3dd9d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ddc1049a0 0x7f3ddc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3dda59c640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ddc103ff0 0x7f3ddc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3ddad9d640 1 -- 172.21.15.94:0/2147268154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ddc1049a0 msgr2=0x7f3ddc19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ddc1049a0 0x7f3ddc19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3ddad9d640 1 -- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ddc103ff0 msgr2=0x7f3ddc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ddc103ff0 0x7f3ddc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3ddad9d640 1 -- 172.21.15.94:0/2147268154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc19f4a0 con 0x7f3ddc109120 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.603+0000 7f3dd9d9b640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ddc1049a0 0x7f3ddc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.604+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ddc109120 0x7f3ddc19edc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d750 tx=0x7f3dd400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.604+0000 7f3dcb7fe640 1 -- 172.21.15.94:0/2147268154 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4017070 con 0x7f3ddc109120 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.604+0000 7f3de0b30640 1 -- 172.21.15.94:0/2147268154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc19f790 con 0x7f3ddc109120 2024-09-19T04:18:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.604+0000 7f3dcb7fe640 1 -- 172.21.15.94:0/2147268154 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dd40054d0 con 0x7f3ddc109120 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.604+0000 7f3dcb7fe640 1 -- 172.21.15.94:0/2147268154 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4004c60 con 0x7f3ddc109120 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.604+0000 7f3de0b30640 1 -- 172.21.15.94:0/2147268154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc1a7070 con 0x7f3ddc109120 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.608+0000 7f3de2f09740 1 Processor -- start 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.608+0000 7f3dcb7fe640 1 -- 172.21.15.94:0/2147268154 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3dd4004e00 con 0x7f3ddc109120 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.608+0000 7f3de2f09740 1 -- start start 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d5ab40 0x559007d59090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d59a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d64370 0x559007d66760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3de2f09740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559007cbf2f0 con 0x559007d64370 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3de2f09740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559007c66cf0 con 0x559007d595d0 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3de2f09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559007cbb2b0 con 0x559007d5ab40 2024-09-19T04:18:59.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3dda59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d59a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3dda59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d59a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59630/0 (socket says 172.21.15.94:59630) 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3dda59c640 1 -- 172.21.15.94:0/1279327190 learned_addr learned my addr 172.21.15.94:0/1279327190 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3dd9d9b640 1 --2- 172.21.15.94:0/1279327190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d5ab40 0x559007d59090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3dd9d9b640 1 --2- 172.21.15.94:0/1279327190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d64370 0x559007d66760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3dda59c640 1 -- 172.21.15.94:0/1279327190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d5ab40 msgr2=0x559007d59090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3dda59c640 1 --2- 172.21.15.94:0/1279327190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d5ab40 0x559007d59090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.609+0000 7f3dda59c640 1 -- 172.21.15.94:0/1279327190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d64370 msgr2=0x559007d66760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.610+0000 7f3dda59c640 1 --2- 172.21.15.94:0/1279327190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d64370 0x559007d66760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.610+0000 7f3dda59c640 1 -- 172.21.15.94:0/1279327190 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559007c4b3a0 con 0x559007d595d0 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.610+0000 7f3dcb7fe640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3dc40a3430 0x7f3dc40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.610+0000 7f3dda59c640 1 --2- 172.21.15.94:0/1279327190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d59a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc034040 tx=0x7f3dcc030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=222ceaec5d335292 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:59.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.610+0000 7f3dcb7fe640 1 -- 172.21.15.94:0/2147268154 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3dd40cf6d0 con 0x7f3ddc109120 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.610+0000 7f3dc8ff9640 1 -- 172.21.15.94:0/1279327190 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc041030 con 0x559007d595d0 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.610+0000 7f3dd9d9b640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3dc40a3430 0x7f3dc40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.611+0000 7f3dc8ff9640 1 -- 172.21.15.94:0/1279327190 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dcc037040 con 0x559007d595d0 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.611+0000 7f3dc8ff9640 1 -- 172.21.15.94:0/1279327190 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc036040 con 0x559007d595d0 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3de2f09740 1 -- 172.21.15.94:0/1279327190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 msgr2=0x559007d59a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1279327190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d59a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc034040 tx=0x7f3dcc030420 comp rx=0 tx=0).stop 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3dd9d9b640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3dc40a3430 0x7f3dc40a58f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559007d5a070 tx=0x7f3dd0009260 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3de2f09740 1 -- 172.21.15.94:0/1279327190 shutdown_connections 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1279327190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d64370 0x559007d66760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1279327190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d59a10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1279327190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d5ab40 0x559007d59090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3de2f09740 1 -- 172.21.15.94:0/1279327190 >> 172.21.15.94:0/1279327190 conn(0x559007d22310 msgr2=0x559007d55c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3de2f09740 1 -- 172.21.15.94:0/1279327190 shutdown_connections 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.612+0000 7f3de2f09740 1 -- 172.21.15.94:0/1279327190 wait complete. 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3de2f09740 1 -- start start 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3de2f09740 1 Processor -- start 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d63cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d5ab40 0x559007d60fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d61520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3de2f09740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559007d67010 con 0x559007d5ab40 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3de2f09740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559007c7a060 con 0x559007d595d0 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3de2f09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559007d67190 con 0x559007d64370 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3de2f09740 10 client.0 _ll_register_callbacks cb 0x5590079cd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3dda59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d63cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3dda59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d63cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59642/0 (socket says 172.21.15.94:59642) 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.613+0000 7f3dda59c640 1 -- 172.21.15.94:0/2703116390 learned_addr learned my addr 172.21.15.94:0/2703116390 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d61520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3dda59c640 1 -- 172.21.15.94:0/2703116390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 msgr2=0x559007d61520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3dda59c640 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d61520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3dda59c640 1 -- 172.21.15.94:0/2703116390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d5ab40 msgr2=0x559007d60fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3dda59c640 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d5ab40 0x559007d60fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3dda59c640 1 -- 172.21.15.94:0/2703116390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559007a303c0 con 0x559007d595d0 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3dda59c640 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d63cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc02abe0 tx=0x7f3dcc030c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d807c0fdea033181 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3dc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/2703116390 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc04a020 con 0x559007d595d0 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/2703116390 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3dcc003b70 con 0x559007d595d0 2024-09-19T04:18:59.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.614+0000 7f3de2f09740 1 -- 172.21.15.94:0/2703116390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dcc037040 con 0x559007d595d0 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/2703116390 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3dcc055050 con 0x559007d595d0 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3de2f09740 10 client.25439 fetch_fsmap learned FSMap version 33 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3de2f09740 1 -- 172.21.15.94:0/2703116390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559007d72a40 con 0x559007d595d0 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/2703116390 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dcc003d70 con 0x559007d595d0 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/2703116390 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f3dcc03c040 con 0x559007d595d0 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3de2f09740 10 client.25439 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3de2f09740 10 client.25439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3de2f09740 20 client.25439 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3de2f09740 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x559007d73d90 0x559007d76180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3de2f09740 4 client.25439 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.615+0000 7f3de2f09740 1 -- 172.21.15.94:0/2703116390 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3dcc037040 con 0x559007d73d90 2024-09-19T04:18:59.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.616+0000 7f3dd9d9b640 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x559007d73d90 0x559007d76180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.617+0000 7f3dc1ffb640 10 client.25439 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.617+0000 7f3dd9d9b640 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x559007d73d90 0x559007d76180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d95cc8fecfb00629 server_cookie=90379ae5245854ab in_seq=0 out_seq=0 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.618+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/2703116390 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3dcc037040 con 0x559007d73d90 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.618+0000 7f3dc1ffb640 10 client.25439 handle_command_reply: tid=0 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.618+0000 7f3de2f09740 1 client.25439 shutdown 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.618+0000 7f3de2f09740 1 -- 172.21.15.94:0/2703116390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 msgr2=0x559007d63cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.618+0000 7f3de2f09740 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d63cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc02abe0 tx=0x7f3dcc030c30 comp rx=0 tx=0).stop 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3de2f09740 1 -- 172.21.15.94:0/2703116390 shutdown_connections 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3de2f09740 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x559007d73d90 0x559007d76180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3dc1ffb640 0 client.25439 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3de2f09740 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d61520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3de2f09740 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d5ab40 0x559007d60fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3de2f09740 1 --2- 172.21.15.94:0/2703116390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d63cf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3de2f09740 1 -- 172.21.15.94:0/2703116390 >> 172.21.15.94:0/2703116390 conn(0x559007d22310 msgr2=0x559007d5b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3de2f09740 1 -- 172.21.15.94:0/2703116390 shutdown_connections 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3de2f09740 1 -- 172.21.15.94:0/2703116390 wait complete. 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.619+0000 7f3de2f09740 20 client.25439 trim_cache size 0 max 16384 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.641+0000 7f3de2f09740 1 Processor -- start 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.641+0000 7f3de2f09740 1 -- start start 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.641+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d595d0 0x559007d702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.641+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d5ab40 0x559007d60860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.641+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d54bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.641+0000 7f3de2f09740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc1b2160 con 0x559007d595d0 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3de2f09740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc1afa00 con 0x559007d5ab40 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3de2f09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc1ad850 con 0x559007d64370 2024-09-19T04:18:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3ddad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d54bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3ddad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d54bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37346/0 (socket says 172.21.15.94:37346) 2024-09-19T04:18:59.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3ddad9d640 1 -- 172.21.15.94:0/612922862 learned_addr learned my addr 172.21.15.94:0/612922862 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:59.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3dda59c640 1 --2- 172.21.15.94:0/612922862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d595d0 0x559007d702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.846 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:18:59 smithi094 ceph-mon[30972]: pgmap v122: 97 pgs: 97 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:18:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3ddad9d640 1 -- 172.21.15.94:0/612922862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d5ab40 msgr2=0x559007d60860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:18:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/612922862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d5ab40 0x559007d60860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3ddad9d640 1 -- 172.21.15.94:0/612922862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d595d0 msgr2=0x559007d702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/612922862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d595d0 0x559007d702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3ddad9d640 1 -- 172.21.15.94:0/612922862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559007d5ff70 con 0x559007d64370 2024-09-19T04:18:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.642+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/612922862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d54bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009610 tx=0x7f3dd409b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0090ca5c419ffde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.643+0000 7f3dc3fff640 1 -- 172.21.15.94:0/612922862 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40b4070 con 0x559007d64370 2024-09-19T04:18:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.643+0000 7f3dc3fff640 1 -- 172.21.15.94:0/612922862 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dd40a45b0 con 0x559007d64370 2024-09-19T04:18:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.643+0000 7f3dc3fff640 1 -- 172.21.15.94:0/612922862 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40afe60 con 0x559007d64370 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 -- 172.21.15.94:0/612922862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 msgr2=0x559007d54bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 --2- 172.21.15.94:0/612922862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d54bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009610 tx=0x7f3dd409b450 comp rx=0 tx=0).stop 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 -- 172.21.15.94:0/612922862 shutdown_connections 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 --2- 172.21.15.94:0/612922862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d54bf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 --2- 172.21.15.94:0/612922862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d5ab40 0x559007d60860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 --2- 172.21.15.94:0/612922862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d595d0 0x559007d702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 -- 172.21.15.94:0/612922862 >> 172.21.15.94:0/612922862 conn(0x559007d22310 msgr2=0x559007d544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 -- 172.21.15.94:0/612922862 shutdown_connections 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 -- 172.21.15.94:0/612922862 wait complete. 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 -- start start 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.644+0000 7f3de2f09740 1 Processor -- start 2024-09-19T04:18:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d5df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d5ab40 0x559007d5e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3de2f09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d5e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3de2f09740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc1adec0 con 0x559007d5ab40 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3de2f09740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc1af330 con 0x559007d595d0 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3de2f09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc1ae590 con 0x559007d64370 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3de2f09740 10 client.0 _ll_register_callbacks cb 0x7f3dcc009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3ddad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d5e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3dda59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d5df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3dda59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d5df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59668/0 (socket says 172.21.15.94:59668) 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3dda59c640 1 -- 172.21.15.94:0/1642459551 learned_addr learned my addr 172.21.15.94:0/1642459551 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d5e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37356/0 (socket says 172.21.15.94:37356) 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3dd9d9b640 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d5ab40 0x559007d5e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3ddad9d640 1 -- 172.21.15.94:0/1642459551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 msgr2=0x559007d5df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d5df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3ddad9d640 1 -- 172.21.15.94:0/1642459551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d5ab40 msgr2=0x559007d5e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d5ab40 0x559007d5e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3ddad9d640 1 -- 172.21.15.94:0/1642459551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559007ddd890 con 0x559007d64370 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3ddad9d640 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d5e990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009610 tx=0x7f3dd4009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96aa25989e97557 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.645+0000 7f3dc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/1642459551 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40afe60 con 0x559007d64370 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3de2f09740 1 -- 172.21.15.94:0/1642459551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dd40a45b0 con 0x559007d64370 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/1642459551 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3dd40ba2b0 con 0x559007d64370 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/1642459551 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3dd40b4040 con 0x559007d64370 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3de2f09740 10 client.25405 fetch_fsmap learned FSMap version 33 2024-09-19T04:18:59.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3de2f09740 1 -- 172.21.15.94:0/1642459551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559007ddead0 con 0x559007d64370 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/1642459551 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dd40a45b0 con 0x559007d64370 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/1642459551 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f3dd409ae40 con 0x559007d64370 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3de2f09740 10 client.25405 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3de2f09740 10 client.25405 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3de2f09740 20 client.25405 populate_metadata read hostname 'smithi094' 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x559007ddf170 0x559007de1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3de2f09740 4 client.25405 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.646+0000 7f3de2f09740 1 -- 172.21.15.94:0/1642459551 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559007ccc860 con 0x559007ddf170 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.647+0000 7f3dda59c640 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x559007ddf170 0x559007de1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.647+0000 7f3dc1ffb640 10 client.25405 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3dda59c640 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x559007ddf170 0x559007de1590 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9ca9c25c27c1549 server_cookie=bba71538be2b1b27 in_seq=0 out_seq=0 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3dc1ffb640 1 -- 172.21.15.94:0/1642459551 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559007ccc860 con 0x559007ddf170 2024-09-19T04:18:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3dc1ffb640 10 client.25405 handle_command_reply: tid=0 2024-09-19T04:18:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3de2f09740 1 client.25405 shutdown 2024-09-19T04:18:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3de2f09740 1 -- 172.21.15.94:0/1642459551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 msgr2=0x559007d5e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d5e990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009610 tx=0x7f3dd4009e00 comp rx=0 tx=0).stop 2024-09-19T04:18:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3de2f09740 1 -- 172.21.15.94:0/1642459551 shutdown_connections 2024-09-19T04:18:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x559007ddf170 0x559007de1590 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559007d64370 0x559007d5e990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.648+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559007d5ab40 0x559007d5e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.852 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:18:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.649+0000 7f3de2f09740 1 --2- 172.21.15.94:0/1642459551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559007d595d0 0x559007d5df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.649+0000 7f3de2f09740 1 -- 172.21.15.94:0/1642459551 >> 172.21.15.94:0/1642459551 conn(0x559007d22310 msgr2=0x559007d54d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.649+0000 7f3dc1ffb640 0 client.25405 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:18:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.649+0000 7f3de2f09740 1 -- 172.21.15.94:0/1642459551 shutdown_connections 2024-09-19T04:18:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.649+0000 7f3de2f09740 1 -- 172.21.15.94:0/1642459551 wait complete. 2024-09-19T04:18:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.649+0000 7f3de2f09740 20 client.25405 trim_cache size 0 max 16384 2024-09-19T04:18:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 -- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3dc40a3430 msgr2=0x7f3dc40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3dc40a3430 0x7f3dc40a58f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559007d5a070 tx=0x7f3dd0009260 comp rx=0 tx=0).stop 2024-09-19T04:18:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 -- 172.21.15.94:0/2147268154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ddc109120 msgr2=0x7f3ddc19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ddc109120 0x7f3ddc19edc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d750 tx=0x7f3dd400dc10 comp rx=0 tx=0).stop 2024-09-19T04:18:59.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 -- 172.21.15.94:0/2147268154 shutdown_connections 2024-09-19T04:18:59.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3dc40a3430 0x7f3dc40a58f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ddc109120 0x7f3ddc19edc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ddc1049a0 0x7f3ddc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 --2- 172.21.15.94:0/2147268154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ddc103ff0 0x7f3ddc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 -- 172.21.15.94:0/2147268154 >> 172.21.15.94:0/2147268154 conn(0x7f3ddc0ffab0 msgr2=0x7f3ddc100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:59.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 -- 172.21.15.94:0/2147268154 shutdown_connections 2024-09-19T04:18:59.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:18:59.650+0000 7f3de0b30640 1 -- 172.21.15.94:0/2147268154 wait complete. 2024-09-19T04:18:59.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:18:59 smithi104 ceph-mon[34145]: pgmap v122: 97 pgs: 97 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:19:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 165.0 s, 1540.2 tps, lat 62.519 ms stddev 16.815 2024-09-19T04:19:00.945 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:19:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:01.744 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:01 smithi094 ceph-mon[30972]: pgmap v123: 97 pgs: 97 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:19:01.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:01 smithi112 ceph-mon[35051]: pgmap v123: 97 pgs: 97 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:19:01.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.846+0000 7f3ea0852640 1 -- 172.21.15.94:0/1750365132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 msgr2=0x7f3e9c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.846+0000 7f3ea0852640 1 --2- 172.21.15.94:0/1750365132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 0x7f3e9c1029b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009900 tx=0x7f3e8402a480 comp rx=0 tx=0).stop 2024-09-19T04:19:01.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.846+0000 7f3ea0852640 1 -- 172.21.15.94:0/1750365132 shutdown_connections 2024-09-19T04:19:01.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.846+0000 7f3ea0852640 1 --2- 172.21.15.94:0/1750365132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.846+0000 7f3ea0852640 1 --2- 172.21.15.94:0/1750365132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 0x7f3e9c1029b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.846+0000 7f3ea0852640 1 --2- 172.21.15.94:0/1750365132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e9c100e20 0x7f3e9c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.846+0000 7f3ea0852640 1 -- 172.21.15.94:0/1750365132 >> 172.21.15.94:0/1750365132 conn(0x7f3e9c0fc7d0 msgr2=0x7f3e9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:01.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.846+0000 7f3ea0852640 1 -- 172.21.15.94:0/1750365132 shutdown_connections 2024-09-19T04:19:01.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.846+0000 7f3ea0852640 1 -- 172.21.15.94:0/1750365132 wait complete. 2024-09-19T04:19:01.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.847+0000 7f3ea0852640 1 Processor -- start 2024-09-19T04:19:01.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.847+0000 7f3ea0852640 1 -- start start 2024-09-19T04:19:01.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.847+0000 7f3ea0852640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e9c100e20 0x7f3e9c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.847+0000 7f3ea0852640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 0x7f3e9c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.847+0000 7f3ea0852640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.847+0000 7f3ea0852640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c110e30 con 0x7f3e9c102570 2024-09-19T04:19:01.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.847+0000 7f3ea0852640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c110cb0 con 0x7f3e9c100e20 2024-09-19T04:19:01.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.847+0000 7f3ea0852640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c110fb0 con 0x7f3e9c102ef0 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.847+0000 7f3e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 0x7f3e9c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e9a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e9c100e20 0x7f3e9c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 0x7f3e9c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53690/0 (socket says 172.21.15.94:53690) 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e93fff640 1 -- 172.21.15.94:0/2126395220 learned_addr learned my addr 172.21.15.94:0/2126395220 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e9ad9d640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e93fff640 1 -- 172.21.15.94:0/2126395220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e9c102ef0 msgr2=0x7f3e9c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e93fff640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e93fff640 1 -- 172.21.15.94:0/2126395220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e9c100e20 msgr2=0x7f3e9c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e93fff640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e9c100e20 0x7f3e9c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e93fff640 1 -- 172.21.15.94:0/2126395220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9c19f5c0 con 0x7f3e9c102570 2024-09-19T04:19:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e9c100e20 0x7f3e9c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:01.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e93fff640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 0x7f3e9c19ab50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3e84006010 tx=0x7f3e840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:01.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e937fe640 1 -- 172.21.15.94:0/2126395220 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8403f030 con 0x7f3e9c102570 2024-09-19T04:19:01.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3ea0852640 1 -- 172.21.15.94:0/2126395220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9c19f850 con 0x7f3e9c102570 2024-09-19T04:19:01.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e937fe640 1 -- 172.21.15.94:0/2126395220 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e840044c0 con 0x7f3e9c102570 2024-09-19T04:19:01.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3e937fe640 1 -- 172.21.15.94:0/2126395220 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84035070 con 0x7f3e9c102570 2024-09-19T04:19:01.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.848+0000 7f3ea0852640 1 -- 172.21.15.94:0/2126395220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e9c1a7190 con 0x7f3e9c102570 2024-09-19T04:19:01.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.851+0000 7f3e937fe640 1 -- 172.21.15.94:0/2126395220 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3e84042030 con 0x7f3e9c102570 2024-09-19T04:19:01.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.851+0000 7f3ea2c2b740 1 Processor -- start 2024-09-19T04:19:01.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.851+0000 7f3ea2c2b740 1 -- start start 2024-09-19T04:19:01.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.851+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e292090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.851+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e292a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.851+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e29f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.851+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a08e1f82f0 con 0x55a08e293b40 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.851+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a08e19fcf0 con 0x55a08e2925d0 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.851+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a08e1f42b0 con 0x55a08e29d370 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e292090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e292090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53702/0 (socket says 172.21.15.94:53702) 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9ad9d640 1 -- 172.21.15.94:0/2283685567 learned_addr learned my addr 172.21.15.94:0/2283685567 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e292a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e937fe640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3e880a33b0 0x7f3e880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e93fff640 1 --2- 172.21.15.94:0/2283685567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e29f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9ad9d640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3e880a33b0 0x7f3e880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9a59c640 1 -- 172.21.15.94:0/2283685567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 msgr2=0x55a08e29f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2283685567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e29f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9a59c640 1 -- 172.21.15.94:0/2283685567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 msgr2=0x55a08e292090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2283685567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e292090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9a59c640 1 -- 172.21.15.94:0/2283685567 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a08e1843a0 con 0x55a08e2925d0 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e93fff640 1 --2- 172.21.15.94:0/2283685567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e29f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e937fe640 1 -- 172.21.15.94:0/2126395220 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3e840effd0 con 0x7f3e9c102570 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2283685567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e292a10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009fd0 tx=0x7f3e8c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c57d2c0a23bb33f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e9ad9d640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3e880a33b0 0x7f3e880a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fd0 tx=0x7f3e94004ab0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e90ff9640 1 -- 172.21.15.94:0/2283685567 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c004310 con 0x55a08e2925d0 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e90ff9640 1 -- 172.21.15.94:0/2283685567 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e8c0044b0 con 0x55a08e2925d0 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.852+0000 7f3e90ff9640 1 -- 172.21.15.94:0/2283685567 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0035f0 con 0x55a08e2925d0 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.853+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2283685567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 msgr2=0x55a08e292a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.853+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2283685567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e292a10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009fd0 tx=0x7f3e8c0118d0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.853+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2283685567 shutdown_connections 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.853+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2283685567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e29f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.853+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2283685567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e292a10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.853+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2283685567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e292090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.853+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2283685567 >> 172.21.15.94:0/2283685567 conn(0x55a08e25b310 msgr2=0x55a08e28ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.853+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2283685567 shutdown_connections 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.853+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2283685567 wait complete. 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.854+0000 7f3ea2c2b740 1 -- start start 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.854+0000 7f3ea2c2b740 1 Processor -- start 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.854+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 0x55a08e29cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.854+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 0x55a08e299fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.854+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e29a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.854+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a08e2a0070 con 0x55a08e2925d0 2024-09-19T04:19:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.854+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a08e1b3060 con 0x55a08e29d370 2024-09-19T04:19:01.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.854+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a08e2a01f0 con 0x55a08e293b40 2024-09-19T04:19:01.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3ea2c2b740 10 client.0 _ll_register_callbacks cb 0x55a08df068c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:01.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e29a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 0x55a08e29cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e29a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59696/0 (socket says 172.21.15.94:59696) 2024-09-19T04:19:01.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9ad9d640 1 -- 172.21.15.94:0/2911869469 learned_addr learned my addr 172.21.15.94:0/2911869469 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:01.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e93fff640 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 0x55a08e299fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.861 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:01 smithi104 ceph-mon[34145]: pgmap v123: 97 pgs: 97 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9ad9d640 1 -- 172.21.15.94:0/2911869469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 msgr2=0x55a08e299fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9ad9d640 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 0x55a08e299fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9ad9d640 1 -- 172.21.15.94:0/2911869469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 msgr2=0x55a08e29cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9ad9d640 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 0x55a08e29cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9ad9d640 1 -- 172.21.15.94:0/2911869469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a08df693c0 con 0x55a08e29d370 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e9ad9d640 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e29a560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e94004220 tx=0x7f3e94004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70317677bded205e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2911869469 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94026070 con 0x55a08e29d370 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2911869469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e8c0044b0 con 0x55a08e29d370 2024-09-19T04:19:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.855+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2911869469 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3e94010dc0 con 0x55a08e29d370 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2911869469 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e94031050 con 0x55a08e29d370 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3ea2c2b740 10 client.25454 fetch_fsmap learned FSMap version 33 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2911869469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a08e2abb20 con 0x55a08e29d370 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2911869469 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e8c0044b0 con 0x55a08e29d370 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2911869469 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f3e9401f050 con 0x55a08e29d370 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3ea2c2b740 10 client.25454 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3ea2c2b740 10 client.25454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3ea2c2b740 20 client.25454 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a08e2acf30 0x55a08e2af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3ea2c2b740 4 client.25454 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.856+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2911869469 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a08e205860 con 0x55a08e2acf30 2024-09-19T04:19:01.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.857+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a08e2acf30 0x55a08e2af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.857+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a08e2acf30 0x55a08e2af3a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e726c3bba944eb server_cookie=ffb139c999d0eace in_seq=0 out_seq=0 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.857+0000 7f3e81ffb640 10 client.25454 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.858+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2911869469 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a08e205860 con 0x55a08e2acf30 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.858+0000 7f3e81ffb640 10 client.25454 handle_command_reply: tid=0 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.858+0000 7f3ea2c2b740 1 client.25454 shutdown 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.858+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2911869469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 msgr2=0x55a08e29a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.858+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e29a560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e94004220 tx=0x7f3e94004a40 comp rx=0 tx=0).stop 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.858+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2911869469 shutdown_connections 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.858+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a08e2acf30 0x55a08e2af3a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.859+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e29a560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.859+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 0x55a08e299fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.859+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2911869469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 0x55a08e29cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.859+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2911869469 >> 172.21.15.94:0/2911869469 conn(0x55a08e25b310 msgr2=0x55a08e294930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:01.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.859+0000 7f3e81ffb640 0 client.25454 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:01.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.859+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2911869469 shutdown_connections 2024-09-19T04:19:01.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.859+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2911869469 wait complete. 2024-09-19T04:19:01.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.859+0000 7f3ea2c2b740 20 client.25454 trim_cache size 0 max 16384 2024-09-19T04:19:01.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3ea2c2b740 1 Processor -- start 2024-09-19T04:19:01.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3ea2c2b740 1 -- start start 2024-09-19T04:19:01.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e2a9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e299880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e28e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1b22b0 con 0x55a08e293b40 2024-09-19T04:19:01.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1afb50 con 0x55a08e2925d0 2024-09-19T04:19:01.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1ad9a0 con 0x55a08e29d370 2024-09-19T04:19:01.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3e9ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e28e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3e9ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e28e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37402/0 (socket says 172.21.15.94:37402) 2024-09-19T04:19:01.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3e9ad9d640 1 -- 172.21.15.94:0/2248005977 learned_addr learned my addr 172.21.15.94:0/2248005977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:01.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.883+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e2a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.884+0000 7f3e93fff640 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e299880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.884+0000 7f3e9a59c640 1 -- 172.21.15.94:0/2248005977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 msgr2=0x55a08e28e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:01.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.884+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e28e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.884+0000 7f3e9a59c640 1 -- 172.21.15.94:0/2248005977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 msgr2=0x55a08e299880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:01.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.884+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e299880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.884+0000 7f3e9a59c640 1 -- 172.21.15.94:0/2248005977 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a08e298510 con 0x55a08e2925d0 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.884+0000 7f3e93fff640 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e299880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.884+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e2a9310 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c005890 tx=0x7f3e8c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb5e0bbf495854 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.885+0000 7f3e83fff640 1 -- 172.21.15.94:0/2248005977 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c005240 con 0x55a08e2925d0 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.885+0000 7f3e83fff640 1 -- 172.21.15.94:0/2248005977 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e8c0053e0 con 0x55a08e2925d0 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.885+0000 7f3e83fff640 1 -- 172.21.15.94:0/2248005977 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0047a0 con 0x55a08e2925d0 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.885+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2248005977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 msgr2=0x55a08e2a9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.885+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e2a9310 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c005890 tx=0x7f3e8c005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.886+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2248005977 shutdown_connections 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.886+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e29d370 0x55a08e28e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.886+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e293b40 0x55a08e299880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.886+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2248005977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e2925d0 0x55a08e2a9310 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.886+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2248005977 >> 172.21.15.94:0/2248005977 conn(0x55a08e25b310 msgr2=0x55a08e298890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.886+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2248005977 shutdown_connections 2024-09-19T04:19:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.886+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2248005977 wait complete. 2024-09-19T04:19:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.886+0000 7f3ea2c2b740 1 -- start start 2024-09-19T04:19:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.886+0000 7f3ea2c2b740 1 Processor -- start 2024-09-19T04:19:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 0x55a08e2a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 0x55a08e2a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3ea2c2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e296790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1ae010 con 0x55a08e2925d0 2024-09-19T04:19:01.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1af480 con 0x55a08e29d370 2024-09-19T04:19:01.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3ea2c2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1ae6e0 con 0x55a08e293b40 2024-09-19T04:19:01.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3ea2c2b740 10 client.0 _ll_register_callbacks cb 0x7f3e94013e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:01.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 0x55a08e2a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 0x55a08e2a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37410/0 (socket says 172.21.15.94:37410) 2024-09-19T04:19:01.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e9ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e296790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e93fff640 1 -- 172.21.15.94:0/2080849263 learned_addr learned my addr 172.21.15.94:0/2080849263 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 0x55a08e2a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e9ad9d640 1 -- 172.21.15.94:0/2080849263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 msgr2=0x55a08e2a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e9ad9d640 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 0x55a08e2a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e9ad9d640 1 -- 172.21.15.94:0/2080849263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 msgr2=0x55a08e2a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e9ad9d640 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 0x55a08e2a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e9ad9d640 1 -- 172.21.15.94:0/2080849263 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a08e296dd0 con 0x55a08e29d370 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.887+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 0x55a08e2a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.888+0000 7f3e9ad9d640 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e296790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55a08e283380 tx=0x7f3e94010dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fb6cad8d4a56890 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.888+0000 7f3e81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.888+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2080849263 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94039020 con 0x55a08e29d370 2024-09-19T04:19:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.888+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2080849263 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f3e94022890 con 0x55a08e29d370 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.888+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2080849263 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e8c0053e0 con 0x55a08e29d370 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.888+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2080849263 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e94044050 con 0x55a08e29d370 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.888+0000 7f3ea2c2b740 10 client.25460 fetch_fsmap learned FSMap version 33 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2080849263 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a08e317540 con 0x55a08e29d370 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2080849263 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e8c0053e0 con 0x55a08e29d370 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2080849263 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f3e9401f040 con 0x55a08e29d370 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3ea2c2b740 10 client.25460 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3ea2c2b740 10 client.25460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3ea2c2b740 20 client.25460 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a08e317c50 0x55a08e31a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3ea2c2b740 4 client.25460 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2080849263 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a08e1ffd70 con 0x55a08e317c50 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.889+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a08e317c50 0x55a08e31a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.890+0000 7f3e81ffb640 10 client.25460 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.890+0000 7f3e9a59c640 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a08e317c50 0x55a08e31a070 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242a18f7fef2c815 server_cookie=3499bee761225eac in_seq=0 out_seq=0 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.890+0000 7f3e81ffb640 1 -- 172.21.15.94:0/2080849263 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a08e1ffd70 con 0x55a08e317c50 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.890+0000 7f3e81ffb640 10 client.25460 handle_command_reply: tid=0 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.890+0000 7f3ea2c2b740 1 client.25460 shutdown 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2080849263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 msgr2=0x55a08e296790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e296790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55a08e283380 tx=0x7f3e94010dd0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2080849263 shutdown_connections 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a08e317c50 0x55a08e31a070 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a08e29d370 0x55a08e296790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a08e293b40 0x55a08e2a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3e81ffb640 0 client.25460 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 --2- 172.21.15.94:0/2080849263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a08e2925d0 0x55a08e2a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2080849263 >> 172.21.15.94:0/2080849263 conn(0x55a08e25b310 msgr2=0x55a08e29b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2080849263 shutdown_connections 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.891+0000 7f3ea2c2b740 1 -- 172.21.15.94:0/2080849263 wait complete. 2024-09-19T04:19:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.892+0000 7f3ea2c2b740 20 client.25460 trim_cache size 0 max 16384 2024-09-19T04:19:01.899 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:19:01.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 -- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3e880a33b0 msgr2=0x7f3e880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3e880a33b0 0x7f3e880a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fd0 tx=0x7f3e94004ab0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 -- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 msgr2=0x7f3e9c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 0x7f3e9c19ab50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3e84006010 tx=0x7f3e840040c0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 -- 172.21.15.94:0/2126395220 shutdown_connections 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3e880a33b0 0x7f3e880a5870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c102570 0x7f3e9c19ab50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 --2- 172.21.15.94:0/2126395220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e9c100e20 0x7f3e9c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 -- 172.21.15.94:0/2126395220 >> 172.21.15.94:0/2126395220 conn(0x7f3e9c0fc7d0 msgr2=0x7f3e9c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 -- 172.21.15.94:0/2126395220 shutdown_connections 2024-09-19T04:19:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:01.893+0000 7f3ea0852640 1 -- 172.21.15.94:0/2126395220 wait complete. 2024-09-19T04:19:03.746 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:19:03.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:19:03.783 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:03 smithi094 ceph-mon[30972]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 19 MiB/s wr, 2.02k op/s 2024-09-19T04:19:03.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:03 smithi112 ceph-mon[35051]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 19 MiB/s wr, 2.02k op/s 2024-09-19T04:19:03.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:03 smithi104 ceph-mon[34145]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 19 MiB/s wr, 2.02k op/s 2024-09-19T04:19:04.158 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:04.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.676+0000 7f16e2ffd640 1 -- 172.21.15.94:0/455353639 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8037070 con 0x7f16e41002a0 2024-09-19T04:19:04.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.677+0000 7f16ea6a5640 1 -- 172.21.15.94:0/455353639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e41002a0 msgr2=0x7f16e41006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.677+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/455353639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e41002a0 0x7f16e41006c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009860 tx=0x7f16d802a340 comp rx=0 tx=0).stop 2024-09-19T04:19:04.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.677+0000 7f16ea6a5640 1 -- 172.21.15.94:0/455353639 shutdown_connections 2024-09-19T04:19:04.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.677+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/455353639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16e4100c00 0x7f16e410d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.677+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/455353639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e41002a0 0x7f16e41006c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.677+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/455353639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e41062b0 0x7f16e4106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.677+0000 7f16ea6a5640 1 -- 172.21.15.94:0/455353639 >> 172.21.15.94:0/455353639 conn(0x7f16e40fc5c0 msgr2=0x7f16e40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:04.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.677+0000 7f16ea6a5640 1 -- 172.21.15.94:0/455353639 shutdown_connections 2024-09-19T04:19:04.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.677+0000 7f16ea6a5640 1 -- 172.21.15.94:0/455353639 wait complete. 2024-09-19T04:19:04.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.678+0000 7f16ea6a5640 1 Processor -- start 2024-09-19T04:19:04.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.678+0000 7f16ea6a5640 1 -- start start 2024-09-19T04:19:04.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.678+0000 7f16ea6a5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e41002a0 0x7f16e41a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16ea6a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16e4100c00 0x7f16e41a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16ea6a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e41062b0 0x7f16e4109b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e41002a0 0x7f16e41a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16e4100c00 0x7f16e41a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16e4100c00 0x7f16e41a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34970/0 (socket says 172.21.15.94:34970) 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16e37fe640 1 -- 172.21.15.94:0/3222735290 learned_addr learned my addr 172.21.15.94:0/3222735290 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e41002a0 0x7f16e41a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52146/0 (socket says 172.21.15.94:52146) 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16e8c42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e41062b0 0x7f16e4109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410fc40 con 0x7f16e4100c00 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410fac0 con 0x7f16e41002a0 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.679+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410fdc0 con 0x7f16e41062b0 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16e8c42640 1 -- 172.21.15.94:0/3222735290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e41002a0 msgr2=0x7f16e41a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16e8c42640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e41002a0 0x7f16e41a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16e8c42640 1 -- 172.21.15.94:0/3222735290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16e4100c00 msgr2=0x7f16e41a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16e8c42640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16e4100c00 0x7f16e41a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16e8c42640 1 -- 172.21.15.94:0/3222735290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e410a320 con 0x7f16e41062b0 2024-09-19T04:19:04.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16e8c42640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e41062b0 0x7f16e4109b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00efc0 tx=0x7f16dc00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:04.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16e17fa640 1 -- 172.21.15.94:0/3222735290 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc003970 con 0x7f16e41062b0 2024-09-19T04:19:04.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e410a610 con 0x7f16e41062b0 2024-09-19T04:19:04.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16e17fa640 1 -- 172.21.15.94:0/3222735290 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16dc004020 con 0x7f16e41062b0 2024-09-19T04:19:04.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.680+0000 7f16e17fa640 1 -- 172.21.15.94:0/3222735290 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc00b940 con 0x7f16e41062b0 2024-09-19T04:19:04.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.681+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e41ab4d0 con 0x7f16e41062b0 2024-09-19T04:19:04.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.685+0000 7f16e17fa640 1 -- 172.21.15.94:0/3222735290 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f16dc01d020 con 0x7f16e41062b0 2024-09-19T04:19:04.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.686+0000 7f16eca7e740 1 Processor -- start 2024-09-19T04:19:04.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.686+0000 7f16eca7e740 1 -- start start 2024-09-19T04:19:04.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.686+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e21f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.686+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e2c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.686+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 0x55bc83e2f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.686+0000 7f16eca7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc83d87280 con 0x55bc83e21b60 2024-09-19T04:19:04.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.686+0000 7f16eca7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc83d2eb10 con 0x55bc83e22480 2024-09-19T04:19:04.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.686+0000 7f16eca7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc83d83240 con 0x55bc83e2cd40 2024-09-19T04:19:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.687+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e21f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.687+0000 7f16e17fa640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f16cc0a3430 0x7f16cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.687+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e21f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34974/0 (socket says 172.21.15.94:34974) 2024-09-19T04:19:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.687+0000 7f16e37fe640 1 -- 172.21.15.94:0/3775867966 learned_addr learned my addr 172.21.15.94:0/3775867966 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.687+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e2c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.688+0000 7f16e8c42640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f16cc0a3430 0x7f16cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.688+0000 7f16e37fe640 1 --2- 172.21.15.94:0/3775867966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 0x55bc83e2f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.688+0000 7f16e3fff640 1 -- 172.21.15.94:0/3775867966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 msgr2=0x55bc83e2f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.688+0000 7f16e3fff640 1 --2- 172.21.15.94:0/3775867966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 0x55bc83e2f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.688+0000 7f16e8c42640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f16cc0a3430 0x7f16cc0a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0cd300 tx=0x7f16dc0d33b0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.688+0000 7f16e17fa640 1 -- 172.21.15.94:0/3222735290 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f16dc0cfbf0 con 0x7f16e41062b0 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.688+0000 7f16e3fff640 1 -- 172.21.15.94:0/3775867966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 msgr2=0x55bc83e21f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.688+0000 7f16e3fff640 1 --2- 172.21.15.94:0/3775867966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e21f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.688+0000 7f16e3fff640 1 -- 172.21.15.94:0/3775867966 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc83d3b1f0 con 0x55bc83e22480 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.689+0000 7f16e37fe640 1 --2- 172.21.15.94:0/3775867966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 0x55bc83e2f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:04.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.689+0000 7f16e3fff640 1 --2- 172.21.15.94:0/3775867966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e2c800 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009c00 tx=0x7f16d4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f2b870448cf136 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:04.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.689+0000 7f16ca7fc640 1 -- 172.21.15.94:0/3775867966 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4009db0 con 0x55bc83e22480 2024-09-19T04:19:04.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.689+0000 7f16ca7fc640 1 -- 172.21.15.94:0/3775867966 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d4005020 con 0x55bc83e22480 2024-09-19T04:19:04.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.689+0000 7f16ca7fc640 1 -- 172.21.15.94:0/3775867966 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4004a40 con 0x55bc83e22480 2024-09-19T04:19:04.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.690+0000 7f16eca7e740 1 -- 172.21.15.94:0/3775867966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 msgr2=0x55bc83e2c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.690+0000 7f16eca7e740 1 --2- 172.21.15.94:0/3775867966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e2c800 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009c00 tx=0x7f16d4011810 comp rx=0 tx=0).stop 2024-09-19T04:19:04.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.691+0000 7f16eca7e740 1 -- 172.21.15.94:0/3775867966 shutdown_connections 2024-09-19T04:19:04.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.691+0000 7f16eca7e740 1 --2- 172.21.15.94:0/3775867966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 0x55bc83e2f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.691+0000 7f16eca7e740 1 --2- 172.21.15.94:0/3775867966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e2c800 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.691+0000 7f16eca7e740 1 --2- 172.21.15.94:0/3775867966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e21f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.691+0000 7f16eca7e740 1 -- 172.21.15.94:0/3775867966 >> 172.21.15.94:0/3775867966 conn(0x55bc83e1e0f0 msgr2=0x55bc83e1e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:04.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.692+0000 7f16eca7e740 1 -- 172.21.15.94:0/3775867966 shutdown_connections 2024-09-19T04:19:04.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.692+0000 7f16eca7e740 1 -- 172.21.15.94:0/3775867966 wait complete. 2024-09-19T04:19:04.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.692+0000 7f16eca7e740 1 -- start start 2024-09-19T04:19:04.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.693+0000 7f16eca7e740 1 Processor -- start 2024-09-19T04:19:04.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.693+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e29b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.693+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e2a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.693+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e2cd40 0x55bc83e2a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e2a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e29b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e2a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45582/0 (socket says 172.21.15.94:45582) 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16eca7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc83e2f960 con 0x55bc83e21b60 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16eca7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc83d41f10 con 0x55bc83e2cd40 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16eca7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc83e2fae0 con 0x55bc83e22480 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e3fff640 1 -- 172.21.15.94:0/1082598311 learned_addr learned my addr 172.21.15.94:0/1082598311 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16eca7e740 10 client.0 _ll_register_callbacks cb 0x55bc83d0aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e37fe640 1 -- 172.21.15.94:0/1082598311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 msgr2=0x55bc83e2a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e37fe640 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e2a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e37fe640 1 -- 172.21.15.94:0/1082598311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e2cd40 msgr2=0x55bc83e2a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:04.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e37fe640 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e2cd40 0x55bc83e2a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e37fe640 1 -- 172.21.15.94:0/1082598311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc83af86a0 con 0x55bc83e21b60 2024-09-19T04:19:04.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.694+0000 7f16e3fff640 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e2a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:04.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.695+0000 7f16e37fe640 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e29b70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009830 tx=0x7f16d8030570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676017990ddbcf82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.695+0000 7f16c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.695+0000 7f16c37fe640 1 -- 172.21.15.94:0/1082598311 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8044070 con 0x55bc83e21b60 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.695+0000 7f16eca7e740 1 -- 172.21.15.94:0/1082598311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d4005020 con 0x55bc83e21b60 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.695+0000 7f16c37fe640 1 -- 172.21.15.94:0/1082598311 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f16d8030990 con 0x55bc83e21b60 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.695+0000 7f16c37fe640 1 -- 172.21.15.94:0/1082598311 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f16d8050050 con 0x55bc83e21b60 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.695+0000 7f16eca7e740 10 client.15876 fetch_fsmap learned FSMap version 33 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.695+0000 7f16eca7e740 1 -- 172.21.15.94:0/1082598311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bc83e3b450 con 0x55bc83e21b60 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.696+0000 7f16c37fe640 1 -- 172.21.15.94:0/1082598311 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d4005020 con 0x55bc83e21b60 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.696+0000 7f16c37fe640 1 -- 172.21.15.94:0/1082598311 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f16d803a050 con 0x55bc83e21b60 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.696+0000 7f16eca7e740 10 client.15876 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.696+0000 7f16eca7e740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.696+0000 7f16eca7e740 20 client.15876 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.696+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc83e3c910 0x55bc83e3ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.696+0000 7f16eca7e740 4 client.15876 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.696+0000 7f16eca7e740 1 -- 172.21.15.94:0/1082598311 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc83d947f0 con 0x55bc83e3c910 2024-09-19T04:19:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.696+0000 7f16e3fff640 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc83e3c910 0x55bc83e3ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.697+0000 7f16e3fff640 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc83e3c910 0x55bc83e3ed20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db8312198662ea server_cookie=4611c13ed7062609 in_seq=0 out_seq=0 2024-09-19T04:19:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.697+0000 7f16c37fe640 10 client.15876 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.698+0000 7f16c37fe640 1 -- 172.21.15.94:0/1082598311 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc83d947f0 con 0x55bc83e3c910 2024-09-19T04:19:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.698+0000 7f16c37fe640 10 client.15876 handle_command_reply: tid=0 2024-09-19T04:19:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.698+0000 7f16eca7e740 1 client.15876 shutdown 2024-09-19T04:19:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.698+0000 7f16eca7e740 1 -- 172.21.15.94:0/1082598311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 msgr2=0x55bc83e29b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.698+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e29b70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009830 tx=0x7f16d8030570 comp rx=0 tx=0).stop 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.698+0000 7f16eca7e740 1 -- 172.21.15.94:0/1082598311 shutdown_connections 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.699+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc83e3c910 0x55bc83e3ed20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.699+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e2cd40 0x55bc83e2a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.699+0000 7f16c37fe640 0 client.15876 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.699+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e2a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.699+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1082598311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e29b70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.699+0000 7f16eca7e740 1 -- 172.21.15.94:0/1082598311 >> 172.21.15.94:0/1082598311 conn(0x55bc83e1e0f0 msgr2=0x55bc83e1d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.699+0000 7f16eca7e740 1 -- 172.21.15.94:0/1082598311 shutdown_connections 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.699+0000 7f16eca7e740 1 -- 172.21.15.94:0/1082598311 wait complete. 2024-09-19T04:19:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.699+0000 7f16eca7e740 20 client.15876 trim_cache size 0 max 16384 2024-09-19T04:19:04.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16eca7e740 1 Processor -- start 2024-09-19T04:19:04.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16eca7e740 1 -- start start 2024-09-19T04:19:04.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e21b60 0x55bc83e24610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e2cd40 0x55bc83e44b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16eca7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41b6700 con 0x55bc83e2cd40 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16eca7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41b3fa0 con 0x55bc83e21b60 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16eca7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41b1df0 con 0x55bc83e22480 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e21b60 0x55bc83e24610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e21b60 0x55bc83e24610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52168/0 (socket says 172.21.15.94:52168) 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.730+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45584/0 (socket says 172.21.15.94:45584) 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e3fff640 1 -- 172.21.15.94:0/672196614 learned_addr learned my addr 172.21.15.94:0/672196614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e37fe640 1 --2- 172.21.15.94:0/672196614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e2cd40 0x55bc83e44b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e3fff640 1 -- 172.21.15.94:0/672196614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e21b60 msgr2=0x55bc83e24610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:04.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e3fff640 1 --2- 172.21.15.94:0/672196614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e21b60 0x55bc83e24610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e3fff640 1 -- 172.21.15.94:0/672196614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e2cd40 msgr2=0x55bc83e44b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:04.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e3fff640 1 --2- 172.21.15.94:0/672196614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e2cd40 0x55bc83e44b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e3fff640 1 -- 172.21.15.94:0/672196614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc83e222b0 con 0x55bc83e22480 2024-09-19T04:19:04.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e37fe640 1 --2- 172.21.15.94:0/672196614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e2cd40 0x55bc83e44b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:04.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.731+0000 7f16e3fff640 1 --2- 172.21.15.94:0/672196614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e445f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009bd0 tx=0x7f16d4005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb4397d1183d98df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:04.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.732+0000 7f16c9ffb640 1 -- 172.21.15.94:0/672196614 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4031070 con 0x55bc83e22480 2024-09-19T04:19:04.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.732+0000 7f16c9ffb640 1 -- 172.21.15.94:0/672196614 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d4004bb0 con 0x55bc83e22480 2024-09-19T04:19:04.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.732+0000 7f16c9ffb640 1 -- 172.21.15.94:0/672196614 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d402b4f0 con 0x55bc83e22480 2024-09-19T04:19:04.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 -- 172.21.15.94:0/672196614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 msgr2=0x55bc83e445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 --2- 172.21.15.94:0/672196614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e445f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009bd0 tx=0x7f16d4005d70 comp rx=0 tx=0).stop 2024-09-19T04:19:04.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 -- 172.21.15.94:0/672196614 shutdown_connections 2024-09-19T04:19:04.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 --2- 172.21.15.94:0/672196614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e2cd40 0x55bc83e44b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 --2- 172.21.15.94:0/672196614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e22480 0x55bc83e445f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 --2- 172.21.15.94:0/672196614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e21b60 0x55bc83e24610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 -- 172.21.15.94:0/672196614 >> 172.21.15.94:0/672196614 conn(0x55bc83e1e0f0 msgr2=0x55bc83e2bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:04.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 -- 172.21.15.94:0/672196614 shutdown_connections 2024-09-19T04:19:04.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 -- 172.21.15.94:0/672196614 wait complete. 2024-09-19T04:19:04.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.733+0000 7f16eca7e740 1 -- start start 2024-09-19T04:19:04.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16eca7e740 1 Processor -- start 2024-09-19T04:19:04.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e30cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e30cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e30cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35004/0 (socket says 172.21.15.94:35004) 2024-09-19T04:19:04.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16eca7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 0x55bc83e31730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e311f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52176/0 (socket says 172.21.15.94:52176) 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16e3fff640 1 -- 172.21.15.94:0/1119710590 learned_addr learned my addr 172.21.15.94:0/1119710590 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41b2460 con 0x55bc83e21b60 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41b38d0 con 0x55bc83e22480 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.734+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41b2b30 con 0x55bc83e2cd40 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16e3fff640 1 -- 172.21.15.94:0/1119710590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 msgr2=0x55bc83e31730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16e3fff640 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 0x55bc83e31730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16e3fff640 1 -- 172.21.15.94:0/1119710590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 msgr2=0x55bc83e30cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16e3fff640 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e30cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16e3fff640 1 -- 172.21.15.94:0/1119710590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc83ea67c0 con 0x55bc83e22480 2024-09-19T04:19:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16eca7e740 10 client.0 _ll_register_callbacks cb 0x7f16dc009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d4004bb0 con 0x55bc83e22480 2024-09-19T04:19:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16e3fff640 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e311f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f16d400a010 tx=0x7f16d4005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe14a2da01b126 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16c37fe640 10 client.25475 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:19:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16c37fe640 1 -- 172.21.15.94:0/1119710590 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4031070 con 0x55bc83e22480 2024-09-19T04:19:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.735+0000 7f16c37fe640 1 -- 172.21.15.94:0/1119710590 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f16d402a040 con 0x55bc83e22480 2024-09-19T04:19:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16c37fe640 1 -- 172.21.15.94:0/1119710590 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f16d4005600 con 0x55bc83e22480 2024-09-19T04:19:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16eca7e740 10 client.25475 fetch_fsmap learned FSMap version 33 2024-09-19T04:19:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55bc83ea7850 con 0x55bc83e22480 2024-09-19T04:19:04.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16c37fe640 1 -- 172.21.15.94:0/1119710590 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f16d4004700 con 0x55bc83e22480 2024-09-19T04:19:04.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16eca7e740 10 client.25475 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:19:04.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16eca7e740 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:04.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16eca7e740 20 client.25475 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:04.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc83e3c910 0x55bc83ea9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16eca7e740 4 client.25475 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bc83d8ed00 con 0x55bc83e3c910 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.736+0000 7f16e37fe640 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc83e3c910 0x55bc83ea9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.737+0000 7f16c37fe640 10 client.25475 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.737+0000 7f16e37fe640 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc83e3c910 0x55bc83ea9c80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1946119a296d0cd server_cookie=9cf1829973a6b966 in_seq=0 out_seq=0 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.738+0000 7f16c37fe640 1 -- 172.21.15.94:0/1119710590 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc83d8ed00 con 0x55bc83e3c910 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.738+0000 7f16c37fe640 10 client.25475 handle_command_reply: tid=0 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.738+0000 7f16eca7e740 1 client.25475 shutdown 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 msgr2=0x55bc83e311f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e311f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f16d400a010 tx=0x7f16d4005ab0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 shutdown_connections 2024-09-19T04:19:04.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55bc83e3c910 0x55bc83ea9c80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16c37fe640 0 client.25475 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:04.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc83e2cd40 0x55bc83e31730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc83e22480 0x55bc83e311f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 --2- 172.21.15.94:0/1119710590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc83e21b60 0x55bc83e30cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 >> 172.21.15.94:0/1119710590 conn(0x55bc83e1e0f0 msgr2=0x55bc83e37bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:04.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 shutdown_connections 2024-09-19T04:19:04.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.739+0000 7f16eca7e740 1 -- 172.21.15.94:0/1119710590 wait complete. 2024-09-19T04:19:04.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.740+0000 7f16eca7e740 20 client.25475 trim_cache size 0 max 16384 2024-09-19T04:19:04.744 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:19:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:19:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "b3bc9453-ec54-4573-8974-95b8c192c6ae", 2024-09-19T04:19:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:19:04.744 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:19:04.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.741+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f16cc0a3430 msgr2=0x7f16cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.741+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f16cc0a3430 0x7f16cc0a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0cd300 tx=0x7f16dc0d33b0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.741+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e41062b0 msgr2=0x7f16e4109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.741+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e41062b0 0x7f16e4109b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00efc0 tx=0x7f16dc00c800 comp rx=0 tx=0).stop 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.741+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 shutdown_connections 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.742+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f16cc0a3430 0x7f16cc0a58f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.742+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e41062b0 0x7f16e4109b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.742+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16e4100c00 0x7f16e41a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.742+0000 7f16ea6a5640 1 --2- 172.21.15.94:0/3222735290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e41002a0 0x7f16e41a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.742+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 >> 172.21.15.94:0/3222735290 conn(0x7f16e40fc5c0 msgr2=0x7f16e40fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.742+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 shutdown_connections 2024-09-19T04:19:04.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:04.742+0000 7f16ea6a5640 1 -- 172.21.15.94:0/3222735290 wait complete. 2024-09-19T04:19:05.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:04 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:19:05.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:04 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:19:05.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:04 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:05.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:04 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:19:05.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:04 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:19:05.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:04 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:05.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:04 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:19:05.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:04 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:19:05.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:04 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 170.0 s, 1516.2 tps, lat 68.434 ms stddev 75.333 2024-09-19T04:19:05.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:19:06.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:05 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:19:06.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:05 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:06.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:05 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:19:06.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:05 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:06.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:05 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:06.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:05 smithi104 ceph-mon[34145]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.34k op/s 2024-09-19T04:19:06.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:05 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:19:06.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:05 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:06.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:05 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:19:06.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:05 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:06.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:05 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:06.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:05 smithi112 ceph-mon[35051]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.34k op/s 2024-09-19T04:19:06.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:05 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:19:06.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:05 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:06.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:05 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:19:06.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:05 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:06.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:05 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:06.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:05 smithi094 ceph-mon[30972]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.34k op/s 2024-09-19T04:19:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:06.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.921+0000 7f091cba5640 1 -- 172.21.15.94:0/2398892821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918100e20 msgr2=0x7f0918101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:06.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.921+0000 7f091cba5640 1 --2- 172.21.15.94:0/2398892821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918100e20 0x7f0918101200 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0908009900 tx=0x7f090802a480 comp rx=0 tx=0).stop 2024-09-19T04:19:06.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.921+0000 7f091cba5640 1 -- 172.21.15.94:0/2398892821 shutdown_connections 2024-09-19T04:19:06.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.921+0000 7f091cba5640 1 --2- 172.21.15.94:0/2398892821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0918102ef0 0x7f091810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.921+0000 7f091cba5640 1 --2- 172.21.15.94:0/2398892821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0918102570 0x7f09181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.921+0000 7f091cba5640 1 --2- 172.21.15.94:0/2398892821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918100e20 0x7f0918101200 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.921+0000 7f091cba5640 1 -- 172.21.15.94:0/2398892821 >> 172.21.15.94:0/2398892821 conn(0x7f09180fc7d0 msgr2=0x7f09180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.921+0000 7f091cba5640 1 -- 172.21.15.94:0/2398892821 shutdown_connections 2024-09-19T04:19:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.922+0000 7f091cba5640 1 -- 172.21.15.94:0/2398892821 wait complete. 2024-09-19T04:19:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091cba5640 1 Processor -- start 2024-09-19T04:19:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091cba5640 1 -- start start 2024-09-19T04:19:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091cba5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0918100e20 0x7f0918107b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091cba5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918102570 0x7f0918108060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091cba5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0918102ef0 0x7f09181085a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091cba5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0918110e30 con 0x7f0918102570 2024-09-19T04:19:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091cba5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0918110cb0 con 0x7f0918100e20 2024-09-19T04:19:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091cba5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0918110fb0 con 0x7f0918102ef0 2024-09-19T04:19:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0918100e20 0x7f0918107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0918100e20 0x7f0918107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52184/0 (socket says 172.21.15.94:52184) 2024-09-19T04:19:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f091659c640 1 -- 172.21.15.94:0/1409267264 learned_addr learned my addr 172.21.15.94:0/1409267264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.923+0000 7f0916d9d640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0918102ef0 0x7f09181085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.924+0000 7f0916d9d640 1 -- 172.21.15.94:0/1409267264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0918100e20 msgr2=0x7f0918107b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:06.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.924+0000 7f0916d9d640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0918100e20 0x7f0918107b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.924+0000 7f0916d9d640 1 -- 172.21.15.94:0/1409267264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918102570 msgr2=0x7f0918108060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:06.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.924+0000 7f0916d9d640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918102570 0x7f0918108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.924+0000 7f0916d9d640 1 -- 172.21.15.94:0/1409267264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09181aada0 con 0x7f0918102ef0 2024-09-19T04:19:06.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.924+0000 7f0916d9d640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0918102ef0 0x7f09181085a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f091000ec50 tx=0x7f091000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:06.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.925+0000 7f09077fe640 1 -- 172.21.15.94:0/1409267264 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910003b70 con 0x7f0918102ef0 2024-09-19T04:19:06.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.925+0000 7f091cba5640 1 -- 172.21.15.94:0/1409267264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09181ab030 con 0x7f0918102ef0 2024-09-19T04:19:06.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.925+0000 7f091cba5640 1 -- 172.21.15.94:0/1409267264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09181ab540 con 0x7f0918102ef0 2024-09-19T04:19:06.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.925+0000 7f09077fe640 1 -- 172.21.15.94:0/1409267264 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0910003d10 con 0x7f0918102ef0 2024-09-19T04:19:06.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.925+0000 7f09077fe640 1 -- 172.21.15.94:0/1409267264 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910010760 con 0x7f0918102ef0 2024-09-19T04:19:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.928+0000 7f091ef7e740 1 Processor -- start 2024-09-19T04:19:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.928+0000 7f09077fe640 1 -- 172.21.15.94:0/1409267264 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0910010900 con 0x7f0918102ef0 2024-09-19T04:19:06.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.928+0000 7f091ef7e740 1 -- start start 2024-09-19T04:19:06.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c9030880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c903b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 0x5643c903da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091ef7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5643c8f962e0 con 0x5643c90304a0 2024-09-19T04:19:06.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091ef7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5643c8f3dce0 con 0x5643c9030dc0 2024-09-19T04:19:06.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f0915d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c9030880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091ef7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5643c8f922a0 con 0x5643c903b680 2024-09-19T04:19:06.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f0915d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c9030880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35036/0 (socket says 172.21.15.94:35036) 2024-09-19T04:19:06.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f0915d9b640 1 -- 172.21.15.94:0/2060460836 learned_addr learned my addr 172.21.15.94:0/2060460836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:06.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c903b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f0915d9b640 1 --2- 172.21.15.94:0/2060460836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 0x5643c903da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091659c640 1 -- 172.21.15.94:0/2060460836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 msgr2=0x5643c903da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:06.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091659c640 1 --2- 172.21.15.94:0/2060460836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 0x5643c903da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091659c640 1 -- 172.21.15.94:0/2060460836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 msgr2=0x5643c9030880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:06.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091659c640 1 --2- 172.21.15.94:0/2060460836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c9030880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.929+0000 7f091659c640 1 -- 172.21.15.94:0/2060460836 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c8f22440 con 0x5643c9030dc0 2024-09-19T04:19:06.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.930+0000 7f091659c640 1 --2- 172.21.15.94:0/2060460836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c903b140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0908005ab0 tx=0x7f0908032be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b86b9dd46102b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:06.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.930+0000 7f0904ff9640 1 -- 172.21.15.94:0/2060460836 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908040070 con 0x5643c9030dc0 2024-09-19T04:19:06.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.930+0000 7f0904ff9640 1 -- 172.21.15.94:0/2060460836 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0908031630 con 0x5643c9030dc0 2024-09-19T04:19:06.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.930+0000 7f0904ff9640 1 -- 172.21.15.94:0/2060460836 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908035040 con 0x5643c9030dc0 2024-09-19T04:19:06.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f09077fe640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f09000a3430 0x7f09000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f091ef7e740 1 -- 172.21.15.94:0/2060460836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 msgr2=0x5643c903b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2060460836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c903b140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0908005ab0 tx=0x7f0908032be0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f0915d9b640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f09000a3430 0x7f09000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f091ef7e740 1 -- 172.21.15.94:0/2060460836 shutdown_connections 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2060460836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 0x5643c903da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2060460836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c903b140 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2060460836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c9030880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f091ef7e740 1 -- 172.21.15.94:0/2060460836 >> 172.21.15.94:0/2060460836 conn(0x5643c902ca30 msgr2=0x5643c902ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.931+0000 7f09077fe640 1 -- 172.21.15.94:0/1409267264 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f09100d0110 con 0x7f0918102ef0 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.932+0000 7f0915d9b640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f09000a3430 0x7f09000a58f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f090c005fd0 tx=0x7f090c00b2d0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.932+0000 7f091ef7e740 1 -- 172.21.15.94:0/2060460836 shutdown_connections 2024-09-19T04:19:06.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.932+0000 7f091ef7e740 1 -- 172.21.15.94:0/2060460836 wait complete. 2024-09-19T04:19:06.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091ef7e740 1 -- start start 2024-09-19T04:19:06.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091ef7e740 1 Processor -- start 2024-09-19T04:19:06.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c903af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c90381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 0x5643c9038720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091ef7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5643c903e250 con 0x5643c90304a0 2024-09-19T04:19:06.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091ef7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5643c8f51050 con 0x5643c9030dc0 2024-09-19T04:19:06.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091ef7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5643c903e3d0 con 0x5643c903b680 2024-09-19T04:19:06.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091ef7e740 10 client.0 _ll_register_callbacks cb 0x5643c8f91160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:06.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c903af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c903af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35048/0 (socket says 172.21.15.94:35048) 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f091659c640 1 -- 172.21.15.94:0/2791673549 learned_addr learned my addr 172.21.15.94:0/2791673549 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f0916d9d640 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 0x5643c9038720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.933+0000 7f0915d9b640 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c90381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f0915d9b640 1 -- 172.21.15.94:0/2791673549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 msgr2=0x5643c9038720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f0915d9b640 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 0x5643c9038720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f0915d9b640 1 -- 172.21.15.94:0/2791673549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 msgr2=0x5643c903af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f0915d9b640 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c903af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f0915d9b640 1 -- 172.21.15.94:0/2791673549 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c8d073c0 con 0x5643c9030dc0 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f0915d9b640 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c90381e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5643c902ddb0 tx=0x7f090c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bee5c7a4c3b3247 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f08fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:19:06.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f08fdffb640 1 -- 172.21.15.94:0/2791673549 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c026060 con 0x5643c9030dc0 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.934+0000 7f08fdffb640 1 -- 172.21.15.94:0/2791673549 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f090c012710 con 0x5643c9030dc0 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.935+0000 7f091ef7e740 1 -- 172.21.15.94:0/2791673549 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0908031630 con 0x5643c9030dc0 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.935+0000 7f08fdffb640 1 -- 172.21.15.94:0/2791673549 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f090c01c040 con 0x5643c9030dc0 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.935+0000 7f091ef7e740 10 client.25487 fetch_fsmap learned FSMap version 33 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.935+0000 7f091ef7e740 1 -- 172.21.15.94:0/2791673549 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5643c9049c90 con 0x5643c9030dc0 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.935+0000 7f08fdffb640 1 -- 172.21.15.94:0/2791673549 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f090c012dc0 con 0x5643c9030dc0 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.935+0000 7f08fdffb640 1 -- 172.21.15.94:0/2791673549 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f090c021a90 con 0x5643c9030dc0 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.936+0000 7f091ef7e740 10 client.25487 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.936+0000 7f091ef7e740 10 client.25487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.936+0000 7f091ef7e740 20 client.25487 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.936+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5643c904b1a0 0x5643c904d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.936+0000 7f091ef7e740 4 client.25487 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.936+0000 7f091ef7e740 1 -- 172.21.15.94:0/2791673549 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0908031630 con 0x5643c904b1a0 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.936+0000 7f091659c640 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5643c904b1a0 0x5643c904d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.936+0000 7f08fdffb640 10 client.25487 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.936+0000 7f091659c640 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5643c904b1a0 0x5643c904d5c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ff27b8ef1736be server_cookie=14a8a42844c41b3 in_seq=0 out_seq=0 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f08fdffb640 1 -- 172.21.15.94:0/2791673549 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0908031630 con 0x5643c904b1a0 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f08fdffb640 10 client.25487 handle_command_reply: tid=0 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f091ef7e740 1 client.25487 shutdown 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f091ef7e740 1 -- 172.21.15.94:0/2791673549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 msgr2=0x5643c90381e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c90381e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5643c902ddb0 tx=0x7f090c00b260 comp rx=0 tx=0).stop 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f091ef7e740 1 -- 172.21.15.94:0/2791673549 shutdown_connections 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5643c904b1a0 0x5643c904d5c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c903b680 0x5643c9038720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c90381e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f091ef7e740 1 --2- 172.21.15.94:0/2791673549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c90304a0 0x5643c903af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f091ef7e740 1 -- 172.21.15.94:0/2791673549 >> 172.21.15.94:0/2791673549 conn(0x5643c902ca30 msgr2=0x5643c902bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.937+0000 7f08fdffb640 0 client.25487 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.938+0000 7f091ef7e740 1 -- 172.21.15.94:0/2791673549 shutdown_connections 2024-09-19T04:19:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.938+0000 7f091ef7e740 1 -- 172.21.15.94:0/2791673549 wait complete. 2024-09-19T04:19:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.938+0000 7f091ef7e740 20 client.25487 trim_cache size 0 max 16384 2024-09-19T04:19:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.970+0000 7f091ef7e740 1 Processor -- start 2024-09-19T04:19:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.970+0000 7f091ef7e740 1 -- start start 2024-09-19T04:19:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.970+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c90304a0 0x5643c9047540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.970+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c9030dc0 0x5643c9052e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.970+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.970+0000 7f091ef7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8011150 con 0x5643c903b680 2024-09-19T04:19:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.970+0000 7f091ef7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08f800e9f0 con 0x5643c90304a0 2024-09-19T04:19:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.970+0000 7f091ef7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08f800c840 con 0x5643c9030dc0 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0916d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f091659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c90304a0 0x5643c9047540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0916d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35056/0 (socket says 172.21.15.94:35056) 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0916d9d640 1 -- 172.21.15.94:0/1979689906 learned_addr learned my addr 172.21.15.94:0/1979689906 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0915d9b640 1 --2- 172.21.15.94:0/1979689906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c9030dc0 0x5643c9052e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0916d9d640 1 -- 172.21.15.94:0/1979689906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c9030dc0 msgr2=0x5643c9052e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0916d9d640 1 --2- 172.21.15.94:0/1979689906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c9030dc0 0x5643c9052e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0916d9d640 1 -- 172.21.15.94:0/1979689906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c90304a0 msgr2=0x5643c9047540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0916d9d640 1 --2- 172.21.15.94:0/1979689906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c90304a0 0x5643c9047540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0916d9d640 1 -- 172.21.15.94:0/1979689906 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c90391f0 con 0x5643c903b680 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f091659c640 1 --2- 172.21.15.94:0/1979689906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c90304a0 0x5643c9047540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f0916d9d640 1 --2- 172.21.15.94:0/1979689906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90533d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f091009c180 tx=0x7f0910009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f4ebd06b792d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:06.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.971+0000 7f08fffff640 1 -- 172.21.15.94:0/1979689906 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100ba070 con 0x5643c903b680 2024-09-19T04:19:06.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.972+0000 7f08fffff640 1 -- 172.21.15.94:0/1979689906 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f091009cb80 con 0x5643c903b680 2024-09-19T04:19:06.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.972+0000 7f08fffff640 1 -- 172.21.15.94:0/1979689906 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100b5070 con 0x5643c903b680 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.973+0000 7f091ef7e740 1 -- 172.21.15.94:0/1979689906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 msgr2=0x5643c90533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.973+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1979689906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90533d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f091009c180 tx=0x7f0910009960 comp rx=0 tx=0).stop 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.973+0000 7f091ef7e740 1 -- 172.21.15.94:0/1979689906 shutdown_connections 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.973+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1979689906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90533d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.973+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1979689906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c9030dc0 0x5643c9052e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.973+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1979689906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c90304a0 0x5643c9047540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.973+0000 7f091ef7e740 1 -- 172.21.15.94:0/1979689906 >> 172.21.15.94:0/1979689906 conn(0x5643c902ca30 msgr2=0x5643c9031f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.973+0000 7f091ef7e740 1 -- 172.21.15.94:0/1979689906 shutdown_connections 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.973+0000 7f091ef7e740 1 -- 172.21.15.94:0/1979689906 wait complete. 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.974+0000 7f091ef7e740 1 -- start start 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.974+0000 7f091ef7e740 1 Processor -- start 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.974+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c90304a0 0x5643c903fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.974+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c9040240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.974+0000 7f091ef7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.974+0000 7f091ef7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08f800ceb0 con 0x5643c903b680 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.974+0000 7f091ef7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08f800e320 con 0x5643c9030dc0 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.974+0000 7f091ef7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08f800d580 con 0x5643c90304a0 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f091ef7e740 10 client.0 _ll_register_callbacks cb 0x7f090c014f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f091659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c90304a0 0x5643c903fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f091659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c90304a0 0x5643c903fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45666/0 (socket says 172.21.15.94:45666) 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f091659c640 1 -- 172.21.15.94:0/1007034861 learned_addr learned my addr 172.21.15.94:0/1007034861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f0916d9d640 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f0915d9b640 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c9040240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f091659c640 1 -- 172.21.15.94:0/1007034861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 msgr2=0x5643c9040240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f091659c640 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c9040240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f091659c640 1 -- 172.21.15.94:0/1007034861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 msgr2=0x5643c90aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f091659c640 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.975+0000 7f091659c640 1 -- 172.21.15.94:0/1007034861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c90af4e0 con 0x5643c90304a0 2024-09-19T04:19:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.976+0000 7f091659c640 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c90304a0 0x5643c903fd00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5643c904de30 tx=0x7f0908005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da8ddf03c62fb91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.976+0000 7f08fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.976+0000 7f08fdffb640 1 -- 172.21.15.94:0/1007034861 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908035040 con 0x5643c90304a0 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.976+0000 7f08fdffb640 1 -- 172.21.15.94:0/1007034861 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f0908031840 con 0x5643c90304a0 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.976+0000 7f091ef7e740 1 -- 172.21.15.94:0/1007034861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f091009cb80 con 0x5643c90304a0 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.976+0000 7f08fdffb640 1 -- 172.21.15.94:0/1007034861 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0908040040 con 0x5643c90304a0 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.976+0000 7f091ef7e740 10 client.25453 fetch_fsmap learned FSMap version 33 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.976+0000 7f091ef7e740 1 -- 172.21.15.94:0/1007034861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5643c90b68c0 con 0x5643c90304a0 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.976+0000 7f08fdffb640 1 -- 172.21.15.94:0/1007034861 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f091009cb80 con 0x5643c90304a0 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.977+0000 7f08fdffb640 1 -- 172.21.15.94:0/1007034861 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f09080054c0 con 0x5643c90304a0 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.977+0000 7f091ef7e740 10 client.25453 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.977+0000 7f091ef7e740 10 client.25453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.977+0000 7f091ef7e740 20 client.25453 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.977+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5643c90b6f00 0x5643c90b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.977+0000 7f091ef7e740 4 client.25453 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.977+0000 7f091ef7e740 1 -- 172.21.15.94:0/1007034861 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643c8fa3850 con 0x5643c90b6f00 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.977+0000 7f0915d9b640 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5643c90b6f00 0x5643c90b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.977+0000 7f08fdffb640 10 client.25453 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f0915d9b640 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5643c90b6f00 0x5643c90b92f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bb64252e6a15bb9 server_cookie=a3f2f8c8398cf72c in_seq=0 out_seq=0 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f08fdffb640 1 -- 172.21.15.94:0/1007034861 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5643c8fa3850 con 0x5643c90b6f00 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f08fdffb640 10 client.25453 handle_command_reply: tid=0 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 client.25453 shutdown 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 -- 172.21.15.94:0/1007034861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c90304a0 msgr2=0x5643c903fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c90304a0 0x5643c903fd00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5643c904de30 tx=0x7f0908005850 comp rx=0 tx=0).stop 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 -- 172.21.15.94:0/1007034861 shutdown_connections 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5643c90b6f00 0x5643c90b92f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c903b680 0x5643c90aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643c9030dc0 0x5643c9040240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f08fdffb640 0 client.25453 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 --2- 172.21.15.94:0/1007034861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c90304a0 0x5643c903fd00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 -- 172.21.15.94:0/1007034861 >> 172.21.15.94:0/1007034861 conn(0x5643c902ca30 msgr2=0x5643c902b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 -- 172.21.15.94:0/1007034861 shutdown_connections 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 1 -- 172.21.15.94:0/1007034861 wait complete. 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.978+0000 7f091ef7e740 20 client.25453 trim_cache size 0 max 16384 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "b3bc9453-ec54-4573-8974-95b8c192c6ae": { 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:19:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "b3bc9453-ec54-4573-8974-95b8c192c6ae", 2024-09-19T04:19:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:19:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:19:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:19:06.985 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:19:06.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 -- 172.21.15.94:0/1409267264 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f09000a3430 msgr2=0x7f09000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f09000a3430 0x7f09000a58f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f090c005fd0 tx=0x7f090c00b2d0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 -- 172.21.15.94:0/1409267264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0918102ef0 msgr2=0x7f09181085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0918102ef0 0x7f09181085a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f091000ec50 tx=0x7f091000c6a0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 -- 172.21.15.94:0/1409267264 shutdown_connections 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f09000a3430 0x7f09000a58f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0918102ef0 0x7f09181085a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918102570 0x7f0918108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 --2- 172.21.15.94:0/1409267264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0918100e20 0x7f0918107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.979+0000 7f091cba5640 1 -- 172.21.15.94:0/1409267264 >> 172.21.15.94:0/1409267264 conn(0x7f09180fc7d0 msgr2=0x7f09180fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.980+0000 7f091cba5640 1 -- 172.21.15.94:0/1409267264 shutdown_connections 2024-09-19T04:19:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:06.980+0000 7f091cba5640 1 -- 172.21.15.94:0/1409267264 wait complete. 2024-09-19T04:19:07.774 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:07 smithi094 ceph-mon[30972]: pgmap v126: 97 pgs: 97 active+clean; 2.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 2.00k op/s 2024-09-19T04:19:07.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:07 smithi112 ceph-mon[35051]: pgmap v126: 97 pgs: 97 active+clean; 2.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 2.00k op/s 2024-09-19T04:19:07.818 INFO:tasks.cephfs.filesystem:scrub status for tag:b3bc9453-ec54-4573-8974-95b8c192c6ae - {'path': '/', 'tag': 'b3bc9453-ec54-4573-8974-95b8c192c6ae', 'options': 'recursive,force'} 2024-09-19T04:19:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:07 smithi104 ceph-mon[34145]: pgmap v126: 97 pgs: 97 active+clean; 2.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 2.00k op/s 2024-09-19T04:19:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:09 smithi112 ceph-mon[35051]: pgmap v127: 97 pgs: 97 active+clean; 2.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-19T04:19:09.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:09 smithi094 ceph-mon[30972]: pgmap v127: 97 pgs: 97 active+clean; 2.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-19T04:19:09.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:09 smithi104 ceph-mon[34145]: pgmap v127: 97 pgs: 97 active+clean; 2.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-19T04:19:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 175.0 s, 1616.0 tps, lat 61.855 ms stddev 15.737 2024-09-19T04:19:11.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:11 smithi112 ceph-mon[35051]: pgmap v128: 97 pgs: 97 active+clean; 2.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-19T04:19:11.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:11 smithi094 ceph-mon[30972]: pgmap v128: 97 pgs: 97 active+clean; 2.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-19T04:19:11.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:11 smithi104 ceph-mon[34145]: pgmap v128: 97 pgs: 97 active+clean; 2.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-19T04:19:12.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:12 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:19:12.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:12 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:19:12.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:12 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:19:12.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:12 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:19:12.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:12 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:19:12.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:12 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:19:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:13 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:19:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:13 smithi112 ceph-mon[35051]: pgmap v129: 97 pgs: 97 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 18 MiB/s wr, 1.99k op/s 2024-09-19T04:19:13.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:13 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:19:13.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:13 smithi094 ceph-mon[30972]: pgmap v129: 97 pgs: 97 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 18 MiB/s wr, 1.99k op/s 2024-09-19T04:19:13.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:13 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:19:13.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:13 smithi104 ceph-mon[34145]: pgmap v129: 97 pgs: 97 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 18 MiB/s wr, 1.99k op/s 2024-09-19T04:19:15.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:14 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:19:15.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:14 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:19:15.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:14 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:19:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 180.0 s, 1577.6 tps, lat 63.350 ms stddev 17.929 2024-09-19T04:19:16.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:15 smithi112 ceph-mon[35051]: pgmap v130: 97 pgs: 97 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.31k op/s 2024-09-19T04:19:16.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:15 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:19:16.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:15 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:19:16.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:15 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:19:16.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:15 smithi094 ceph-mon[30972]: pgmap v130: 97 pgs: 97 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.31k op/s 2024-09-19T04:19:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:15 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:19:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:15 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:19:16.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:15 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:19:16.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:15 smithi104 ceph-mon[34145]: pgmap v130: 97 pgs: 97 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.31k op/s 2024-09-19T04:19:16.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:15 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:19:16.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:15 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:19:16.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:15 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:19:17.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:17 smithi112 ceph-mon[35051]: pgmap v131: 97 pgs: 97 active+clean; 2.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 1.99k op/s 2024-09-19T04:19:17.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:17 smithi094 ceph-mon[30972]: pgmap v131: 97 pgs: 97 active+clean; 2.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 1.99k op/s 2024-09-19T04:19:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:17 smithi104 ceph-mon[34145]: pgmap v131: 97 pgs: 97 active+clean; 2.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 1.99k op/s 2024-09-19T04:19:19.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:19 smithi112 ceph-mon[35051]: pgmap v132: 97 pgs: 97 active+clean; 2.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:19:19.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:19 smithi094 ceph-mon[30972]: pgmap v132: 97 pgs: 97 active+clean; 2.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:19:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:19 smithi104 ceph-mon[34145]: pgmap v132: 97 pgs: 97 active+clean; 2.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:19:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 185.0 s, 1591.2 tps, lat 62.801 ms stddev 15.635 2024-09-19T04:19:21.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:21 smithi112 ceph-mon[35051]: pgmap v133: 97 pgs: 97 active+clean; 2.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:19:21.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:21 smithi094 ceph-mon[30972]: pgmap v133: 97 pgs: 97 active+clean; 2.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:19:21.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:21 smithi104 ceph-mon[34145]: pgmap v133: 97 pgs: 97 active+clean; 2.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:19:23.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:23 smithi112 ceph-mon[35051]: pgmap v134: 97 pgs: 97 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 1.99k op/s 2024-09-19T04:19:23.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:23 smithi094 ceph-mon[30972]: pgmap v134: 97 pgs: 97 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 1.99k op/s 2024-09-19T04:19:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:23 smithi104 ceph-mon[34145]: pgmap v134: 97 pgs: 97 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 1.99k op/s 2024-09-19T04:19:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 190.0 s, 1481.4 tps, lat 67.603 ms stddev 19.242 2024-09-19T04:19:25.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:25 smithi112 ceph-mon[35051]: pgmap v135: 97 pgs: 97 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 1.34k op/s 2024-09-19T04:19:25.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:25 smithi094 ceph-mon[30972]: pgmap v135: 97 pgs: 97 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 1.34k op/s 2024-09-19T04:19:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:25 smithi104 ceph-mon[34145]: pgmap v135: 97 pgs: 97 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 1.34k op/s 2024-09-19T04:19:27.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:27 smithi112 ceph-mon[35051]: pgmap v136: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 1.97k op/s 2024-09-19T04:19:27.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:27 smithi094 ceph-mon[30972]: pgmap v136: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 1.97k op/s 2024-09-19T04:19:27.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:27 smithi104 ceph-mon[34145]: pgmap v136: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 1.97k op/s 2024-09-19T04:19:29.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:29 smithi112 ceph-mon[35051]: pgmap v137: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-19T04:19:29.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:29 smithi094 ceph-mon[30972]: pgmap v137: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-19T04:19:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:29 smithi104 ceph-mon[34145]: pgmap v137: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-19T04:19:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 195.0 s, 1492.2 tps, lat 66.977 ms stddev 64.972 2024-09-19T04:19:31.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:31 smithi112 ceph-mon[35051]: pgmap v138: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-19T04:19:31.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:31 smithi094 ceph-mon[30972]: pgmap v138: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-19T04:19:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:31 smithi104 ceph-mon[34145]: pgmap v138: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-19T04:19:34.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:33 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:19:34.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:33 smithi094 ceph-mon[30972]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 1.94k op/s 2024-09-19T04:19:34.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:33 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:19:34.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:33 smithi104 ceph-mon[34145]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 1.94k op/s 2024-09-19T04:19:34.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:33 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:19:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:33 smithi112 ceph-mon[35051]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 1.94k op/s 2024-09-19T04:19:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 200.0 s, 1450.8 tps, lat 68.886 ms stddev 20.777 2024-09-19T04:19:35.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:35 smithi112 ceph-mon[35051]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-19T04:19:35.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:35 smithi094 ceph-mon[30972]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-19T04:19:35.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:35 smithi104 ceph-mon[34145]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-19T04:19:37.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:37 smithi112 ceph-mon[35051]: pgmap v141: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 18 MiB/s wr, 1.92k op/s 2024-09-19T04:19:37.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:19:37.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:37 smithi094 ceph-mon[30972]: pgmap v141: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 18 MiB/s wr, 1.92k op/s 2024-09-19T04:19:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:37 smithi104 ceph-mon[34145]: pgmap v141: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 18 MiB/s wr, 1.92k op/s 2024-09-19T04:19:38.245 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:38.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.836+0000 7fadef4d2640 1 -- 172.21.15.94:0/1697496809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade8100e20 msgr2=0x7fade8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.836+0000 7fadef4d2640 1 --2- 172.21.15.94:0/1697496809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade8100e20 0x7fade8101200 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fade400dcc0 tx=0x7fade402e430 comp rx=0 tx=0).stop 2024-09-19T04:19:38.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.836+0000 7fadef4d2640 1 -- 172.21.15.94:0/1697496809 shutdown_connections 2024-09-19T04:19:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.836+0000 7fadef4d2640 1 --2- 172.21.15.94:0/1697496809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade8102ef0 0x7fade810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.836+0000 7fadef4d2640 1 --2- 172.21.15.94:0/1697496809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade8102570 0x7fade81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.836+0000 7fadef4d2640 1 --2- 172.21.15.94:0/1697496809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade8100e20 0x7fade8101200 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.836+0000 7fadef4d2640 1 -- 172.21.15.94:0/1697496809 >> 172.21.15.94:0/1697496809 conn(0x7fade80fc7d0 msgr2=0x7fade80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.837+0000 7fadef4d2640 1 -- 172.21.15.94:0/1697496809 shutdown_connections 2024-09-19T04:19:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.837+0000 7fadef4d2640 1 -- 172.21.15.94:0/1697496809 wait complete. 2024-09-19T04:19:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.837+0000 7fadef4d2640 1 Processor -- start 2024-09-19T04:19:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadef4d2640 1 -- start start 2024-09-19T04:19:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadef4d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade8100e20 0x7fade8196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadef4d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade8102570 0x7fade81965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadef4d2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade8102ef0 0x7fade819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadef4d2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fade810fd30 con 0x7fade8102570 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadef4d2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fade810fbb0 con 0x7fade8102ef0 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadef4d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade810feb0 con 0x7fade8100e20 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadeda6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade8102ef0 0x7fade819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadeda6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade8102ef0 0x7fade819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52320/0 (socket says 172.21.15.94:52320) 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadeda6f640 1 -- 172.21.15.94:0/3770667422 learned_addr learned my addr 172.21.15.94:0/3770667422 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7faded26e640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade8100e20 0x7fade8196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.838+0000 7fadeca6d640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade8102570 0x7fade81965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadeda6f640 1 -- 172.21.15.94:0/3770667422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade8100e20 msgr2=0x7fade8196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadeda6f640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade8100e20 0x7fade8196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadeda6f640 1 -- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade8102570 msgr2=0x7fade81965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadeda6f640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade8102570 0x7fade81965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadeda6f640 1 -- 172.21.15.94:0/3770667422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade819b040 con 0x7fade8102ef0 2024-09-19T04:19:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadeda6f640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade8102ef0 0x7fade819a960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fade000efd0 tx=0x7fade000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadde7fc640 1 -- 172.21.15.94:0/3770667422 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0003b70 con 0x7fade8102ef0 2024-09-19T04:19:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadde7fc640 1 -- 172.21.15.94:0/3770667422 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fade0003d10 con 0x7fade8102ef0 2024-09-19T04:19:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadde7fc640 1 -- 172.21.15.94:0/3770667422 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade00109f0 con 0x7fade8102ef0 2024-09-19T04:19:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadef4d2640 1 -- 172.21.15.94:0/3770667422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade819b330 con 0x7fade8102ef0 2024-09-19T04:19:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.839+0000 7fadef4d2640 1 -- 172.21.15.94:0/3770667422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade81a2c10 con 0x7fade8102ef0 2024-09-19T04:19:38.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.843+0000 7fadf18ab740 1 Processor -- start 2024-09-19T04:19:38.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.843+0000 7fadde7fc640 1 -- 172.21.15.94:0/3770667422 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fade0003ec0 con 0x7fade8102ef0 2024-09-19T04:19:38.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadf18ab740 1 -- start start 2024-09-19T04:19:38.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d87805fcb0 0x55d878060090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8780605d0 0x55d87806a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d87806d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadf18ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d877fbd2e0 con 0x55d87806ae90 2024-09-19T04:19:38.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadf18ab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d877f64ce0 con 0x55d8780605d0 2024-09-19T04:19:38.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadf18ab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d877fb92a0 con 0x55d87805fcb0 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadeca6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d87805fcb0 0x55d878060090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadeca6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d87805fcb0 0x55d878060090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:41716/0 (socket says 172.21.15.94:41716) 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadeca6d640 1 -- 172.21.15.94:0/1617545054 learned_addr learned my addr 172.21.15.94:0/1617545054 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadeca6d640 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d87806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7fadeca6d640 1 -- 172.21.15.94:0/1617545054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8780605d0 msgr2=0x55d87806a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.844+0000 7faded26e640 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8780605d0 0x55d87806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.845+0000 7fadeca6d640 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8780605d0 0x55d87806a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.845+0000 7fadeca6d640 1 -- 172.21.15.94:0/1617545054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 msgr2=0x55d87806d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.845+0000 7fadeca6d640 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d87806d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.845+0000 7fadeca6d640 1 -- 172.21.15.94:0/1617545054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d877f49440 con 0x55d87805fcb0 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.845+0000 7fadde7fc640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fadd40a3430 0x7fadd40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.845+0000 7faded26e640 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8780605d0 0x55d87806a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.846+0000 7fadde7fc640 1 -- 172.21.15.94:0/3770667422 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fade00d0160 con 0x7fade8102ef0 2024-09-19T04:19:38.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.846+0000 7fadeca6d640 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d87805fcb0 0x55d878060090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fadd800add0 tx=0x7fadd800a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eee50907c1d739d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.846+0000 7faded26e640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fadd40a3430 0x7fadd40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.846+0000 7fadd37fe640 1 -- 172.21.15.94:0/1617545054 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd801b070 con 0x55d87805fcb0 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.846+0000 7fadd37fe640 1 -- 172.21.15.94:0/1617545054 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadd800fd70 con 0x55d87805fcb0 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.846+0000 7fadd37fe640 1 -- 172.21.15.94:0/1617545054 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8014650 con 0x55d87805fcb0 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.847+0000 7faded26e640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fadd40a3430 0x7fadd40a58f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55d87805d0a0 tx=0x7fade400d670 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.847+0000 7fadf18ab740 1 -- 172.21.15.94:0/1617545054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d87805fcb0 msgr2=0x55d878060090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.847+0000 7fadf18ab740 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d87805fcb0 0x55d878060090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fadd800add0 tx=0x7fadd800a490 comp rx=0 tx=0).stop 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.847+0000 7fadf18ab740 1 -- 172.21.15.94:0/1617545054 shutdown_connections 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.847+0000 7fadf18ab740 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d87806d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.847+0000 7fadf18ab740 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8780605d0 0x55d87806a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.847+0000 7fadf18ab740 1 --2- 172.21.15.94:0/1617545054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d87805fcb0 0x55d878060090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.847+0000 7fadf18ab740 1 -- 172.21.15.94:0/1617545054 >> 172.21.15.94:0/1617545054 conn(0x55d87805c240 msgr2=0x55d87805c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.848+0000 7fadf18ab740 1 -- 172.21.15.94:0/1617545054 shutdown_connections 2024-09-19T04:19:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.848+0000 7fadf18ab740 1 -- 172.21.15.94:0/1617545054 wait complete. 2024-09-19T04:19:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.848+0000 7fadf18ab740 1 -- start start 2024-09-19T04:19:38.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.848+0000 7fadf18ab740 1 Processor -- start 2024-09-19T04:19:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878058dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d878059300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d878059840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadf18ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d8780536a0 con 0x55d87806ae90 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadf18ab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d877f78050 con 0x55d87805fcb0 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadf18ab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d878053820 con 0x55d8780605d0 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadf18ab740 10 client.0 _ll_register_callbacks cb 0x55d877fb8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadeca6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d878059300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadeca6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d878059300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:41718/0 (socket says 172.21.15.94:41718) 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadeca6d640 1 -- 172.21.15.94:0/2440828201 learned_addr learned my addr 172.21.15.94:0/2440828201 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7faded26e640 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878058dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadeda6f640 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d878059840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadeca6d640 1 -- 172.21.15.94:0/2440828201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 msgr2=0x55d878058dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadeca6d640 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878058dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadeca6d640 1 -- 172.21.15.94:0/2440828201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 msgr2=0x55d878059840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadeca6d640 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d878059840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.849+0000 7fadeca6d640 1 -- 172.21.15.94:0/2440828201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d877d2e3c0 con 0x55d8780605d0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.850+0000 7fadeca6d640 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d878059300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009040 tx=0x7fadd8016870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad5eca5f1ef7db7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.850+0000 7fadd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.850+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2440828201 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8024020 con 0x55d8780605d0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.850+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2440828201 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fadd8020480 con 0x55d8780605d0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.850+0000 7fadf18ab740 1 -- 172.21.15.94:0/2440828201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd800fd70 con 0x55d8780605d0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.850+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2440828201 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fadd802f050 con 0x55d8780605d0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.850+0000 7fadf18ab740 10 client.25465 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.850+0000 7fadf18ab740 1 -- 172.21.15.94:0/2440828201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d87806acb0 con 0x55d8780605d0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.851+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2440828201 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadd800fd70 con 0x55d8780605d0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.851+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2440828201 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fadd8015040 con 0x55d8780605d0 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.851+0000 7fadf18ab740 10 client.25465 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:38.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.851+0000 7fadf18ab740 10 client.25465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.851+0000 7fadf18ab740 20 client.25465 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.851+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d878082fb0 0x55d8780853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.851+0000 7fadf18ab740 4 client.25465 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.851+0000 7fadf18ab740 1 -- 172.21.15.94:0/2440828201 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d877fca850 con 0x55d878082fb0 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.852+0000 7faded26e640 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d878082fb0 0x55d8780853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.852+0000 7faded26e640 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d878082fb0 0x55d8780853a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62dc09faacb7bf8f server_cookie=6b5963d5c5eed2a in_seq=0 out_seq=0 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.852+0000 7fadd0ff9640 10 client.25465 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2440828201 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d877fca850 con 0x55d878082fb0 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadd0ff9640 10 client.25465 handle_command_reply: tid=0 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 client.25465 shutdown 2024-09-19T04:19:38.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 -- 172.21.15.94:0/2440828201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 msgr2=0x55d878059300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d878059300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009040 tx=0x7fadd8016870 comp rx=0 tx=0).stop 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 -- 172.21.15.94:0/2440828201 shutdown_connections 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d878082fb0 0x55d8780853a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d878059840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d878059300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2440828201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878058dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 -- 172.21.15.94:0/2440828201 >> 172.21.15.94:0/2440828201 conn(0x55d87805c240 msgr2=0x55d878052b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadd0ff9640 0 client.25465 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 -- 172.21.15.94:0/2440828201 shutdown_connections 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.853+0000 7fadf18ab740 1 -- 172.21.15.94:0/2440828201 wait complete. 2024-09-19T04:19:38.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.854+0000 7fadf18ab740 20 client.25465 trim_cache size 0 max 16384 2024-09-19T04:19:38.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.885+0000 7fadf18ab740 1 Processor -- start 2024-09-19T04:19:38.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.885+0000 7fadf18ab740 1 -- start start 2024-09-19T04:19:38.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.886+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878052b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.886+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d878066740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.886+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d878066c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.886+0000 7faded26e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878052b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.886+0000 7fadf18ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fade81add40 con 0x55d87806ae90 2024-09-19T04:19:38.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.886+0000 7faded26e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878052b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52348/0 (socket says 172.21.15.94:52348) 2024-09-19T04:19:38.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.886+0000 7fadf18ab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fade81ab610 con 0x55d87805fcb0 2024-09-19T04:19:38.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.886+0000 7faded26e640 1 -- 172.21.15.94:0/1054615144 learned_addr learned my addr 172.21.15.94:0/1054615144 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:38.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.886+0000 7fadf18ab740 1 -- 172.21.15.94:0/1054615144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade81a9460 con 0x55d8780605d0 2024-09-19T04:19:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.887+0000 7faded26e640 1 -- 172.21.15.94:0/1054615144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 msgr2=0x55d878066740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:19:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.887+0000 7faded26e640 1 --2- 172.21.15.94:0/1054615144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d878066740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.887+0000 7faded26e640 1 -- 172.21.15.94:0/1054615144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 msgr2=0x55d878066c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.887+0000 7faded26e640 1 --2- 172.21.15.94:0/1054615144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d878066c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.887+0000 7faded26e640 1 -- 172.21.15.94:0/1054615144 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d87805fae0 con 0x55d87805fcb0 2024-09-19T04:19:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.887+0000 7faded26e640 1 --2- 172.21.15.94:0/1054615144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878052b70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fade402ea20 tx=0x7fade403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59fcad1e4fa20938 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.887+0000 7fadd2ffd640 1 -- 172.21.15.94:0/1054615144 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4048040 con 0x55d87805fcb0 2024-09-19T04:19:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.887+0000 7fadd2ffd640 1 -- 172.21.15.94:0/1054615144 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fade405a070 con 0x55d87805fcb0 2024-09-19T04:19:38.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.887+0000 7fadd2ffd640 1 -- 172.21.15.94:0/1054615144 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4040430 con 0x55d87805fcb0 2024-09-19T04:19:38.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.888+0000 7fadf18ab740 1 -- 172.21.15.94:0/1054615144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 msgr2=0x55d878052b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.888+0000 7fadf18ab740 1 --2- 172.21.15.94:0/1054615144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878052b70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fade402ea20 tx=0x7fade403f000 comp rx=0 tx=0).stop 2024-09-19T04:19:38.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.888+0000 7fadf18ab740 1 -- 172.21.15.94:0/1054615144 shutdown_connections 2024-09-19T04:19:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.888+0000 7fadf18ab740 1 --2- 172.21.15.94:0/1054615144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87806ae90 0x55d878066c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.888+0000 7fadf18ab740 1 --2- 172.21.15.94:0/1054615144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d878066740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.888+0000 7fadf18ab740 1 --2- 172.21.15.94:0/1054615144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87805fcb0 0x55d878052b70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.888+0000 7fadf18ab740 1 -- 172.21.15.94:0/1054615144 >> 172.21.15.94:0/1054615144 conn(0x55d87805c240 msgr2=0x55d87805aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.889+0000 7fadf18ab740 1 -- 172.21.15.94:0/1054615144 shutdown_connections 2024-09-19T04:19:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.889+0000 7fadf18ab740 1 -- 172.21.15.94:0/1054615144 wait complete. 2024-09-19T04:19:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.889+0000 7fadf18ab740 1 -- start start 2024-09-19T04:19:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.889+0000 7fadf18ab740 1 Processor -- start 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87805fcb0 0x55d8780e1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d8780e1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadf18ab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87806ae90 0x55d8780e2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadf18ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fade81a9ad0 con 0x55d87805fcb0 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadf18ab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fade81aaf40 con 0x55d87806ae90 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadf18ab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade81aa1a0 con 0x55d8780605d0 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadf18ab740 10 client.0 _ll_register_callbacks cb 0x7fadd8019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadeca6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d8780e1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadeca6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d8780e1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:41728/0 (socket says 172.21.15.94:41728) 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadeca6d640 1 -- 172.21.15.94:0/2877201891 learned_addr learned my addr 172.21.15.94:0/2877201891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7fadeda6f640 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87806ae90 0x55d8780e2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.890+0000 7faded26e640 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87805fcb0 0x55d8780e1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7fadeda6f640 1 -- 172.21.15.94:0/2877201891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 msgr2=0x55d8780e1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7fadeda6f640 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d8780e1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7fadeda6f640 1 -- 172.21.15.94:0/2877201891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87805fcb0 msgr2=0x55d8780e1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7fadeda6f640 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87805fcb0 0x55d8780e1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7fadeda6f640 1 -- 172.21.15.94:0/2877201891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8780ec980 con 0x55d87806ae90 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7faded26e640 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87805fcb0 0x55d8780e1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7fadeda6f640 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87806ae90 0x55d8780e2300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fade00cd870 tx=0x7fade009c060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d573b6a9ed9424 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7fadd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2877201891 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade00b4020 con 0x55d87806ae90 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.891+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2877201891 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fade0010b90 con 0x55d87806ae90 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadf18ab740 1 -- 172.21.15.94:0/2877201891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade405a070 con 0x55d87806ae90 2024-09-19T04:19:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2877201891 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fade00b5420 con 0x55d87806ae90 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadf18ab740 10 client.25505 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadf18ab740 1 -- 172.21.15.94:0/2877201891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8780edcf0 con 0x55d87806ae90 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2877201891 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fade009cb30 con 0x55d87806ae90 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2877201891 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fade001f020 con 0x55d87806ae90 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadf18ab740 10 client.25505 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadf18ab740 10 client.25505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadf18ab740 20 client.25505 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d8780ee490 0x55d8780f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadf18ab740 4 client.25505 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:19:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.892+0000 7fadf18ab740 1 -- 172.21.15.94:0/2877201891 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fade405a070 con 0x55d8780ee490 2024-09-19T04:19:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.893+0000 7faded26e640 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d8780ee490 0x55d8780f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.893+0000 7faded26e640 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d8780ee490 0x55d8780f08b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bab9288cb982a2 server_cookie=abe5fd766131a545 in_seq=0 out_seq=0 2024-09-19T04:19:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.893+0000 7fadd0ff9640 10 client.25505 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadd0ff9640 1 -- 172.21.15.94:0/2877201891 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fade405a070 con 0x55d8780ee490 2024-09-19T04:19:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadd0ff9640 10 client.25505 handle_command_reply: tid=0 2024-09-19T04:19:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadf18ab740 1 client.25505 shutdown 2024-09-19T04:19:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadf18ab740 1 -- 172.21.15.94:0/2877201891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87806ae90 msgr2=0x55d8780e2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87806ae90 0x55d8780e2300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fade00cd870 tx=0x7fade009c060 comp rx=0 tx=0).stop 2024-09-19T04:19:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadf18ab740 1 -- 172.21.15.94:0/2877201891 shutdown_connections 2024-09-19T04:19:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d8780ee490 0x55d8780f08b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d87806ae90 0x55d8780e2300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8780605d0 0x55d8780e1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadf18ab740 1 --2- 172.21.15.94:0/2877201891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d87805fcb0 0x55d8780e1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadf18ab740 1 -- 172.21.15.94:0/2877201891 >> 172.21.15.94:0/2877201891 conn(0x55d87805c240 msgr2=0x55d878052b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.894+0000 7fadd0ff9640 0 client.25505 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.895+0000 7fadf18ab740 1 -- 172.21.15.94:0/2877201891 shutdown_connections 2024-09-19T04:19:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.895+0000 7fadf18ab740 1 -- 172.21.15.94:0/2877201891 wait complete. 2024-09-19T04:19:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.895+0000 7fadf18ab740 20 client.25505 trim_cache size 0 max 16384 2024-09-19T04:19:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:19:38.898 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:19:38.898 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:19:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 -- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fadd40a3430 msgr2=0x7fadd40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fadd40a3430 0x7fadd40a58f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55d87805d0a0 tx=0x7fade400d670 comp rx=0 tx=0).stop 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 -- 172.21.15.94:0/3770667422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade8102ef0 msgr2=0x7fade819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade8102ef0 0x7fade819a960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fade000efd0 tx=0x7fade000c6a0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 -- 172.21.15.94:0/3770667422 shutdown_connections 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fadd40a3430 0x7fadd40a58f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade8102ef0 0x7fade819a960 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade8102570 0x7fade81965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 --2- 172.21.15.94:0/3770667422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade8100e20 0x7fade8196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 -- 172.21.15.94:0/3770667422 >> 172.21.15.94:0/3770667422 conn(0x7fade80fc7d0 msgr2=0x7fade8104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 -- 172.21.15.94:0/3770667422 shutdown_connections 2024-09-19T04:19:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:38.897+0000 7fadef4d2640 1 -- 172.21.15.94:0/3770667422 wait complete. 2024-09-19T04:19:39.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:39 smithi094 ceph-mon[30972]: pgmap v142: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:19:39.733 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:19:39.734 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:19:39.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:39 smithi112 ceph-mon[35051]: pgmap v142: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:19:39.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:39 smithi104 ceph-mon[34145]: pgmap v142: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:19:40.182 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 205.0 s, 1548.2 tps, lat 64.615 ms stddev 17.185 2024-09-19T04:19:40.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.743+0000 7f65d3788640 1 -- 172.21.15.94:0/2463962377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 msgr2=0x7f65cc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.743+0000 7f65d3788640 1 --2- 172.21.15.94:0/2463962377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 0x7f65cc101200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009900 tx=0x7f65c802a480 comp rx=0 tx=0).stop 2024-09-19T04:19:40.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.743+0000 7f65d3788640 1 -- 172.21.15.94:0/2463962377 shutdown_connections 2024-09-19T04:19:40.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.743+0000 7f65d3788640 1 --2- 172.21.15.94:0/2463962377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65cc102ef0 0x7f65cc069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.743+0000 7f65d3788640 1 --2- 172.21.15.94:0/2463962377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc102570 0x7f65cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.743+0000 7f65d3788640 1 --2- 172.21.15.94:0/2463962377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 0x7f65cc101200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.743+0000 7f65d3788640 1 -- 172.21.15.94:0/2463962377 >> 172.21.15.94:0/2463962377 conn(0x7f65cc0fc7d0 msgr2=0x7f65cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:40.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.743+0000 7f65d3788640 1 -- 172.21.15.94:0/2463962377 shutdown_connections 2024-09-19T04:19:40.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.743+0000 7f65d3788640 1 -- 172.21.15.94:0/2463962377 wait complete. 2024-09-19T04:19:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.744+0000 7f65d3788640 1 Processor -- start 2024-09-19T04:19:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.744+0000 7f65d3788640 1 -- start start 2024-09-19T04:19:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.745+0000 7f65d3788640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 0x7f65cc198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.745+0000 7f65d3788640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc102570 0x7f65cc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.745+0000 7f65d3788640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65cc102ef0 0x7f65cc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.745+0000 7f65d0d23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc102570 0x7f65cc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.745+0000 7f65d1524640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 0x7f65cc198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.745+0000 7f65d1524640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 0x7f65cc198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52384/0 (socket says 172.21.15.94:52384) 2024-09-19T04:19:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.745+0000 7f65d0d23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc102570 0x7f65cc198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49926/0 (socket says 172.21.15.94:49926) 2024-09-19T04:19:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.745+0000 7f65d1524640 1 -- 172.21.15.94:0/3039645198 learned_addr learned my addr 172.21.15.94:0/3039645198 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc111ee0 con 0x7f65cc102570 2024-09-19T04:19:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc111d60 con 0x7f65cc100e20 2024-09-19T04:19:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d1d25640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65cc102ef0 0x7f65cc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc112060 con 0x7f65cc102ef0 2024-09-19T04:19:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d1d25640 1 -- 172.21.15.94:0/3039645198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 msgr2=0x7f65cc198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d1d25640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 0x7f65cc198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d1d25640 1 -- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc102570 msgr2=0x7f65cc198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d1d25640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc102570 0x7f65cc198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d1d25640 1 -- 172.21.15.94:0/3039645198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc19d3f0 con 0x7f65cc102ef0 2024-09-19T04:19:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d1524640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 0x7f65cc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.746+0000 7f65d0d23640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc102570 0x7f65cc198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.747+0000 7f65d1d25640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65cc102ef0 0x7f65cc19cd10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009fd0 tx=0x7f65c400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.747+0000 7f65c27fc640 1 -- 172.21.15.94:0/3039645198 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c400ce20 con 0x7f65cc102ef0 2024-09-19T04:19:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.747+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc19d6e0 con 0x7f65cc102ef0 2024-09-19T04:19:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.747+0000 7f65c27fc640 1 -- 172.21.15.94:0/3039645198 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65c4005020 con 0x7f65cc102ef0 2024-09-19T04:19:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.747+0000 7f65c27fc640 1 -- 172.21.15.94:0/3039645198 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4010690 con 0x7f65cc102ef0 2024-09-19T04:19:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.747+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc1a4fc0 con 0x7f65cc102ef0 2024-09-19T04:19:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.749+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65cc1029b0 con 0x7f65cc102ef0 2024-09-19T04:19:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.751+0000 7f65c27fc640 1 -- 172.21.15.94:0/3039645198 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f65c40051d0 con 0x7f65cc102ef0 2024-09-19T04:19:40.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.752+0000 7f65c27fc640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f65b80a33b0 0x7f65b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.752+0000 7f65d1524640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f65b80a33b0 0x7f65b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.752+0000 7f65c27fc640 1 -- 172.21.15.94:0/3039645198 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f65c40d03d0 con 0x7f65cc102ef0 2024-09-19T04:19:40.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.753+0000 7f65d1524640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f65b80a33b0 0x7f65b80a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65c8031440 tx=0x7f65c80058d0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:40.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:40.758+0000 7f65c27fc640 1 -- 172.21.15.94:0/3039645198 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f65c409cd40 con 0x7f65cc102ef0 2024-09-19T04:19:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.128+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65cc1a51f0 con 0x7f65cc102ef0 2024-09-19T04:19:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.129+0000 7f65c27fc640 1 -- 172.21.15.94:0/3039645198 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11915 (secure 0 0 0) 0x7f65c4098020 con 0x7f65cc102ef0 2024-09-19T04:19:41.131 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:19:41.131 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":34,"btime":"2024-09-19T04:19:32:802034+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:19:32.751814+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:19:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 34 2024-09-19T04:19:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f65b80a33b0 msgr2=0x7f65b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f65b80a33b0 0x7f65b80a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65c8031440 tx=0x7f65c80058d0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65cc102ef0 msgr2=0x7f65cc19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65cc102ef0 0x7f65cc19cd10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009fd0 tx=0x7f65c400ef10 comp rx=0 tx=0).stop 2024-09-19T04:19:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 shutdown_connections 2024-09-19T04:19:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f65b80a33b0 0x7f65b80a5870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65cc102ef0 0x7f65cc19cd10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc102570 0x7f65cc198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 --2- 172.21.15.94:0/3039645198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65cc100e20 0x7f65cc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 >> 172.21.15.94:0/3039645198 conn(0x7f65cc0fc7d0 msgr2=0x7f65cc106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 shutdown_connections 2024-09-19T04:19:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:41.133+0000 7f65d3788640 1 -- 172.21.15.94:0/3039645198 wait complete. 2024-09-19T04:19:41.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:41 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3039645198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:41.545 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:41 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3039645198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:41 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3039645198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:42.026 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:19:42.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:42 smithi094 ceph-mon[30972]: pgmap v143: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:19:42.441 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:42.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:42 smithi112 ceph-mon[35051]: pgmap v143: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:19:42.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:42 smithi104 ceph-mon[34145]: pgmap v143: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:19:43.512 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:43 smithi094 ceph-mon[30972]: pgmap v144: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 17 MiB/s wr, 1.93k op/s 2024-09-19T04:19:43.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:43 smithi112 ceph-mon[35051]: pgmap v144: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 17 MiB/s wr, 1.93k op/s 2024-09-19T04:19:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:43 smithi104 ceph-mon[34145]: pgmap v144: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 17 MiB/s wr, 1.93k op/s 2024-09-19T04:19:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 210.0 s, 1453.4 tps, lat 68.666 ms stddev 34.343 2024-09-19T04:19:45.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:45 smithi112 ceph-mon[35051]: pgmap v145: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:19:45.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:45 smithi094 ceph-mon[30972]: pgmap v145: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:19:45.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:45 smithi104 ceph-mon[34145]: pgmap v145: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:19:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.603+0000 7f26fe65b640 1 -- 172.21.15.94:0/1250397764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f8102790 msgr2=0x7f26f81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.603+0000 7f26fe65b640 1 --2- 172.21.15.94:0/1250397764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f8102790 0x7f26f81067e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009900 tx=0x7f26e002a410 comp rx=0 tx=0).stop 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.603+0000 7f26fe65b640 1 -- 172.21.15.94:0/1250397764 shutdown_connections 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.603+0000 7f26fe65b640 1 --2- 172.21.15.94:0/1250397764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f8106f10 0x7f26f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.603+0000 7f26fe65b640 1 --2- 172.21.15.94:0/1250397764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f8102790 0x7f26f81067e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.603+0000 7f26fe65b640 1 --2- 172.21.15.94:0/1250397764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26f8101de0 0x7f26f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.603+0000 7f26fe65b640 1 -- 172.21.15.94:0/1250397764 >> 172.21.15.94:0/1250397764 conn(0x7f26f80fd880 msgr2=0x7f26f80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.603+0000 7f26fe65b640 1 -- 172.21.15.94:0/1250397764 shutdown_connections 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.603+0000 7f26fe65b640 1 -- 172.21.15.94:0/1250397764 wait complete. 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fe65b640 1 Processor -- start 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fe65b640 1 -- start start 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fe65b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26f8101de0 0x7f26f8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fe65b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f8102790 0x7f26f81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fe65b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f8106f10 0x7f26f819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fe65b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8111e20 con 0x7f26f8106f10 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fe65b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8111ca0 con 0x7f26f8101de0 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fe65b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8111fa0 con 0x7f26f8102790 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26f8101de0 0x7f26f8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fcbf8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f8106f10 0x7f26f819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26f8101de0 0x7f26f8198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50720/0 (socket says 172.21.15.94:50720) 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26f7fff640 1 -- 172.21.15.94:0/702439902 learned_addr learned my addr 172.21.15.94:0/702439902 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26effff640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f8102790 0x7f26f81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fcbf8640 1 -- 172.21.15.94:0/702439902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f8102790 msgr2=0x7f26f81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fcbf8640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f8102790 0x7f26f81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fcbf8640 1 -- 172.21.15.94:0/702439902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26f8101de0 msgr2=0x7f26f8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fcbf8640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26f8101de0 0x7f26f8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26fcbf8640 1 -- 172.21.15.94:0/702439902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f819d320 con 0x7f26f8106f10 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.604+0000 7f26effff640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f8102790 0x7f26f81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.605+0000 7f26fcbf8640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f8106f10 0x7f26f819cc40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f26f000da10 tx=0x7f26f000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.605+0000 7f26f5ffb640 1 -- 172.21.15.94:0/702439902 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f000bd70 con 0x7f26f8106f10 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.605+0000 7f26f5ffb640 1 -- 172.21.15.94:0/702439902 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26f0003e20 con 0x7f26f8106f10 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.605+0000 7f26f5ffb640 1 -- 172.21.15.94:0/702439902 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00106c0 con 0x7f26f8106f10 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.605+0000 7f26fe65b640 1 -- 172.21.15.94:0/702439902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f819d610 con 0x7f26f8106f10 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.605+0000 7f26fe65b640 1 -- 172.21.15.94:0/702439902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f81a4ef0 con 0x7f26f8106f10 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.608+0000 7f2700a34740 1 Processor -- start 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.608+0000 7f26f5ffb640 1 -- 172.21.15.94:0/702439902 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f26f0003970 con 0x7f26f8106f10 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.608+0000 7f2700a34740 1 -- start start 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.608+0000 7f2700a34740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e661e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.608+0000 7f2700a34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e661ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.608+0000 7f2700a34740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f2700a34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5560e65842f0 con 0x5560e661e5d0 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f2700a34740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5560e652bcf0 con 0x5560e662d780 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f2700a34740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5560e65802b0 con 0x5560e661fb40 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e661ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e661ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34362/0 (socket says 172.21.15.94:34362) 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f7fff640 1 -- 172.21.15.94:0/3672266405 learned_addr learned my addr 172.21.15.94:0/3672266405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26effff640 1 --2- 172.21.15.94:0/3672266405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26effff640 1 --2- 172.21.15.94:0/3672266405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e661e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f5ffb640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f26e40a3430 0x7f26e40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f7fff640 1 -- 172.21.15.94:0/3672266405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 msgr2=0x5560e661e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f7fff640 1 --2- 172.21.15.94:0/3672266405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e661e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f7fff640 1 -- 172.21.15.94:0/3672266405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 msgr2=0x5560e662fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f7fff640 1 --2- 172.21.15.94:0/3672266405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f7fff640 1 -- 172.21.15.94:0/3672266405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e65103a0 con 0x5560e661e5d0 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f7fff640 1 --2- 172.21.15.94:0/3672266405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e661ea10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f26e8004650 tx=0x7f26e800bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aedbd5d10a109cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26f5ffb640 1 -- 172.21.15.94:0/702439902 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f26f00d00b0 con 0x7f26f8106f10 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26fcbf8640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f26e40a3430 0x7f26e40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26ee7fc640 1 -- 172.21.15.94:0/3672266405 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8003dd0 con 0x5560e661e5d0 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.609+0000 7f26ee7fc640 1 -- 172.21.15.94:0/3672266405 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e8015070 con 0x5560e661e5d0 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f26ee7fc640 1 -- 172.21.15.94:0/3672266405 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8011680 con 0x5560e661e5d0 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f2700a34740 1 -- 172.21.15.94:0/3672266405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 msgr2=0x5560e661ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f26fcbf8640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f26e40a3430 0x7f26e40a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f26f00cd850 tx=0x7f26f0005f30 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f2700a34740 1 --2- 172.21.15.94:0/3672266405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e661ea10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f26e8004650 tx=0x7f26e800bc70 comp rx=0 tx=0).stop 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f2700a34740 1 -- 172.21.15.94:0/3672266405 shutdown_connections 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f2700a34740 1 --2- 172.21.15.94:0/3672266405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f2700a34740 1 --2- 172.21.15.94:0/3672266405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e661ea10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f2700a34740 1 --2- 172.21.15.94:0/3672266405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e661e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f2700a34740 1 -- 172.21.15.94:0/3672266405 >> 172.21.15.94:0/3672266405 conn(0x5560e65e7310 msgr2=0x5560e661ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f2700a34740 1 -- 172.21.15.94:0/3672266405 shutdown_connections 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.610+0000 7f2700a34740 1 -- 172.21.15.94:0/3672266405 wait complete. 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f2700a34740 1 -- start start 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f2700a34740 1 Processor -- start 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f2700a34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e662d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f2700a34740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e662a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f26effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e662d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f2700a34740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f2700a34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5560e66304d0 con 0x5560e661e5d0 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f2700a34740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5560e653f060 con 0x5560e662d780 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f26effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e662d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34370/0 (socket says 172.21.15.94:34370) 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f2700a34740 1 -- 172.21.15.94:0/1866604409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5560e6630650 con 0x5560e661fb40 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f26effff640 1 -- 172.21.15.94:0/1866604409 learned_addr learned my addr 172.21.15.94:0/1866604409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.611+0000 7f2700a34740 10 client.0 _ll_register_callbacks cb 0x5560e62928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26f7fff640 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e662a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26effff640 1 -- 172.21.15.94:0/1866604409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 msgr2=0x5560e662a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26effff640 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e662a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26effff640 1 -- 172.21.15.94:0/1866604409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 msgr2=0x5560e662a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26effff640 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26effff640 1 -- 172.21.15.94:0/1866604409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e62f53c0 con 0x5560e661e5d0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26f7fff640 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e662a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26effff640 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e662d160 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f26e0005ec0 tx=0x7f26e0032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7524344fe33dcf87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26d77fe640 1 -- 172.21.15.94:0/1866604409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0035040 con 0x5560e661e5d0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26d77fe640 1 -- 172.21.15.94:0/1866604409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f26e00035f0 con 0x5560e661e5d0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f2700a34740 1 -- 172.21.15.94:0/1866604409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e8015070 con 0x5560e661e5d0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26d77fe640 1 -- 172.21.15.94:0/1866604409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f26e0051050 con 0x5560e661e5d0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f2700a34740 10 client.15927 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f2700a34740 1 -- 172.21.15.94:0/1866604409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560e663bf30 con 0x5560e661e5d0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26d77fe640 1 -- 172.21.15.94:0/1866604409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e8015070 con 0x5560e661e5d0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.612+0000 7f26d77fe640 1 -- 172.21.15.94:0/1866604409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f26e0048070 con 0x5560e661e5d0 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.613+0000 7f2700a34740 10 client.15927 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.613+0000 7f2700a34740 10 client.15927 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.613+0000 7f2700a34740 20 client.15927 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.613+0000 7f2700a34740 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5560e663d340 0x5560e663f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.613+0000 7f2700a34740 4 client.15927 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.613+0000 7f2700a34740 1 -- 172.21.15.94:0/1866604409 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560e6591860 con 0x5560e663d340 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.613+0000 7f26f7fff640 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5560e663d340 0x5560e663f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.613+0000 7f26f7fff640 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5560e663d340 0x5560e663f7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ed1ee40559c4e99 server_cookie=48180de1260f7696 in_seq=0 out_seq=0 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.613+0000 7f26d77fe640 10 client.15927 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f26d77fe640 1 -- 172.21.15.94:0/1866604409 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560e6591860 con 0x5560e663d340 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f26d77fe640 10 client.15927 handle_command_reply: tid=0 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 client.15927 shutdown 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 -- 172.21.15.94:0/1866604409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 msgr2=0x5560e662d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e662d160 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f26e0005ec0 tx=0x7f26e0032b00 comp rx=0 tx=0).stop 2024-09-19T04:19:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 -- 172.21.15.94:0/1866604409 shutdown_connections 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5560e663d340 0x5560e663f7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e662a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f26d77fe640 0 client.15927 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 --2- 172.21.15.94:0/1866604409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e662d160 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 -- 172.21.15.94:0/1866604409 >> 172.21.15.94:0/1866604409 conn(0x5560e65e7310 msgr2=0x5560e6620960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 -- 172.21.15.94:0/1866604409 shutdown_connections 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 1 -- 172.21.15.94:0/1866604409 wait complete. 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.614+0000 7f2700a34740 20 client.15927 trim_cache size 0 max 16384 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f2700a34740 1 Processor -- start 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f2700a34740 1 -- start start 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f2700a34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e6639770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f2700a34740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e6619c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f2700a34740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f2700a34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26f81aded0 con 0x5560e661e5d0 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f2700a34740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f26f81ab7a0 con 0x5560e662d780 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f2700a34740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26f81a95f0 con 0x5560e661fb40 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f26effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e6639770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f26effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e6639770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34378/0 (socket says 172.21.15.94:34378) 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f26effff640 1 -- 172.21.15.94:0/3852329711 learned_addr learned my addr 172.21.15.94:0/3852329711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.627+0000 7f26f7fff640 1 --2- 172.21.15.94:0/3852329711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e6619c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f26effff640 1 -- 172.21.15.94:0/3852329711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 msgr2=0x5560e6619c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f26effff640 1 --2- 172.21.15.94:0/3852329711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e6619c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f26effff640 1 -- 172.21.15.94:0/3852329711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 msgr2=0x5560e662eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f26effff640 1 --2- 172.21.15.94:0/3852329711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f26effff640 1 -- 172.21.15.94:0/3852329711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e661aed0 con 0x5560e661e5d0 2024-09-19T04:19:46.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f26effff640 1 --2- 172.21.15.94:0/3852329711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e6639770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f26e002a450 tx=0x7f26e00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326c52d001b11725 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f26edffb640 1 -- 172.21.15.94:0/3852329711 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0035040 con 0x5560e661e5d0 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f26edffb640 1 -- 172.21.15.94:0/3852329711 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e003d460 con 0x5560e661e5d0 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f26edffb640 1 -- 172.21.15.94:0/3852329711 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0000e60 con 0x5560e661e5d0 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f2700a34740 1 -- 172.21.15.94:0/3852329711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 msgr2=0x5560e6639770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f2700a34740 1 --2- 172.21.15.94:0/3852329711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e6639770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f26e002a450 tx=0x7f26e00058b0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f2700a34740 1 -- 172.21.15.94:0/3852329711 shutdown_connections 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f2700a34740 1 --2- 172.21.15.94:0/3852329711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e662eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f2700a34740 1 --2- 172.21.15.94:0/3852329711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e6619c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f2700a34740 1 --2- 172.21.15.94:0/3852329711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e6639770 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.628+0000 7f2700a34740 1 -- 172.21.15.94:0/3852329711 >> 172.21.15.94:0/3852329711 conn(0x5560e65e7310 msgr2=0x5560e6619500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 -- 172.21.15.94:0/3852329711 shutdown_connections 2024-09-19T04:19:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 -- 172.21.15.94:0/3852329711 wait complete. 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 -- start start 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 Processor -- start 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e66360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e6644a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e6645020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26f81a9c60 con 0x5560e661e5d0 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f26f81ab0d0 con 0x5560e662d780 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26f81aa330 con 0x5560e661fb40 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f2700a34740 10 client.0 _ll_register_callbacks cb 0x7f26e0031950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f26effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e66360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f26effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e66360b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34386/0 (socket says 172.21.15.94:34386) 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.629+0000 7f26effff640 1 -- 172.21.15.94:0/365062978 learned_addr learned my addr 172.21.15.94:0/365062978 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26f7fff640 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e6644a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26effff640 1 -- 172.21.15.94:0/365062978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 msgr2=0x5560e6644a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26effff640 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e6644a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26effff640 1 -- 172.21.15.94:0/365062978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 msgr2=0x5560e6645020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26effff640 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e6645020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26effff640 1 -- 172.21.15.94:0/365062978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e6645850 con 0x5560e661e5d0 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26effff640 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e66360b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f26e0040040 tx=0x7f26e0004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1712a76337dad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26d77fe640 1 -- 172.21.15.94:0/365062978 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0063060 con 0x5560e661e5d0 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26d77fe640 1 -- 172.21.15.94:0/365062978 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f26e003db70 con 0x5560e661e5d0 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f2700a34740 1 -- 172.21.15.94:0/365062978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e003d460 con 0x5560e661e5d0 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26d77fe640 1 -- 172.21.15.94:0/365062978 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f26e0031a00 con 0x5560e661e5d0 2024-09-19T04:19:46.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f2700a34740 10 client.15933 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:46.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f2700a34740 1 -- 172.21.15.94:0/365062978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560e66a8170 con 0x5560e661e5d0 2024-09-19T04:19:46.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.630+0000 7f26d77fe640 1 -- 172.21.15.94:0/365062978 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e003d460 con 0x5560e661e5d0 2024-09-19T04:19:46.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f26d77fe640 1 -- 172.21.15.94:0/365062978 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f26e0048040 con 0x5560e661e5d0 2024-09-19T04:19:46.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f2700a34740 10 client.15933 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f2700a34740 10 client.15933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:19:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f2700a34740 20 client.15933 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f2700a34740 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5560e663d340 0x5560e66aa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f2700a34740 4 client.15933 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:19:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f2700a34740 1 -- 172.21.15.94:0/365062978 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560e658bd70 con 0x5560e663d340 2024-09-19T04:19:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:19:46.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f26f7fff640 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5560e663d340 0x5560e66aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f26f7fff640 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5560e663d340 0x5560e66aa730 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20e06846d63adce6 server_cookie=c976c78d8476768b in_seq=0 out_seq=0 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f26d77fe640 10 client.15933 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f26d77fe640 1 -- 172.21.15.94:0/365062978 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560e658bd70 con 0x5560e663d340 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f26d77fe640 10 client.15933 handle_command_reply: tid=0 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.631+0000 7f2700a34740 1 client.15933 shutdown 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 -- 172.21.15.94:0/365062978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 msgr2=0x5560e66360b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e66360b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f26e0040040 tx=0x7f26e0004510 comp rx=0 tx=0).stop 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 -- 172.21.15.94:0/365062978 shutdown_connections 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5560e663d340 0x5560e66aa730 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560e662d780 0x5560e6645020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26d77fe640 0 client.15933 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560e661fb40 0x5560e6644a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 --2- 172.21.15.94:0/365062978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560e661e5d0 0x5560e66360b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 -- 172.21.15.94:0/365062978 >> 172.21.15.94:0/365062978 conn(0x5560e65e7310 msgr2=0x5560e66288a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 -- 172.21.15.94:0/365062978 shutdown_connections 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 1 -- 172.21.15.94:0/365062978 wait complete. 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f2700a34740 20 client.15933 trim_cache size 0 max 16384 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 -- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f26e40a3430 msgr2=0x7f26e40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f26e40a3430 0x7f26e40a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f26f00cd850 tx=0x7f26f0005f30 comp rx=0 tx=0).stop 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 -- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f8106f10 msgr2=0x7f26f819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f8106f10 0x7f26f819cc40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f26f000da10 tx=0x7f26f000ded0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 -- 172.21.15.94:0/702439902 shutdown_connections 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f26e40a3430 0x7f26e40a58f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f8106f10 0x7f26f819cc40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f8102790 0x7f26f81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 --2- 172.21.15.94:0/702439902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26f8101de0 0x7f26f8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 -- 172.21.15.94:0/702439902 >> 172.21.15.94:0/702439902 conn(0x7f26f80fd880 msgr2=0x7f26f80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.632+0000 7f26fe65b640 1 -- 172.21.15.94:0/702439902 shutdown_connections 2024-09-19T04:19:46.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:45.633+0000 7f26fe65b640 1 -- 172.21.15.94:0/702439902 wait complete. 2024-09-19T04:19:47.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:19:47.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:47 smithi112 ceph-mon[35051]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.87k op/s 2024-09-19T04:19:47.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:47 smithi094 ceph-mon[30972]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.87k op/s 2024-09-19T04:19:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:47 smithi104 ceph-mon[34145]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.87k op/s 2024-09-19T04:19:48.051 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.622+0000 7fba0082e640 1 -- 172.21.15.94:0/4001126020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9fc100e20 msgr2=0x7fb9fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.622+0000 7fba0082e640 1 --2- 172.21.15.94:0/4001126020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9fc100e20 0x7fb9fc101200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009890 tx=0x7fb9ec02a400 comp rx=0 tx=0).stop 2024-09-19T04:19:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.622+0000 7fba0082e640 1 -- 172.21.15.94:0/4001126020 shutdown_connections 2024-09-19T04:19:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.622+0000 7fba0082e640 1 --2- 172.21.15.94:0/4001126020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.622+0000 7fba0082e640 1 --2- 172.21.15.94:0/4001126020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9fc102570 0x7fb9fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.622+0000 7fba0082e640 1 --2- 172.21.15.94:0/4001126020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9fc100e20 0x7fb9fc101200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.622+0000 7fba0082e640 1 -- 172.21.15.94:0/4001126020 >> 172.21.15.94:0/4001126020 conn(0x7fb9fc0fc7d0 msgr2=0x7fb9fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.623+0000 7fba0082e640 1 -- 172.21.15.94:0/4001126020 shutdown_connections 2024-09-19T04:19:48.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.623+0000 7fba0082e640 1 -- 172.21.15.94:0/4001126020 wait complete. 2024-09-19T04:19:48.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fba0082e640 1 Processor -- start 2024-09-19T04:19:48.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fba0082e640 1 -- start start 2024-09-19T04:19:48.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fba0082e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9fc100e20 0x7fb9fc10d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fba0082e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9fc102570 0x7fb9fc10a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fba0082e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc10ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fba0082e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc10fd30 con 0x7fb9fc100e20 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fba0082e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc10fbb0 con 0x7fb9fc102ef0 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fba0082e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc10feb0 con 0x7fb9fc102570 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9fc100e20 0x7fb9fc10d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9fc100e20 0x7fb9fc10d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34408/0 (socket says 172.21.15.94:34408) 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.624+0000 7fb9fa59c640 1 -- 172.21.15.94:0/4094298804 learned_addr learned my addr 172.21.15.94:0/4094298804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.625+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9fc102570 0x7fb9fc10a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.625+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc10ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.625+0000 7fb9fa59c640 1 -- 172.21.15.94:0/4094298804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9fc102570 msgr2=0x7fb9fc10a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.625+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9fc102570 0x7fb9fc10a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.625+0000 7fb9fa59c640 1 -- 172.21.15.94:0/4094298804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9fc102ef0 msgr2=0x7fb9fc10ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.625+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc10ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.625+0000 7fb9fa59c640 1 -- 172.21.15.94:0/4094298804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9fc10b6c0 con 0x7fb9fc100e20 2024-09-19T04:19:48.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.625+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:48.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.625+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9fc100e20 0x7fb9fc10d840 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec000c00 tx=0x7fb9ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.626+0000 7fb9f3fff640 1 -- 172.21.15.94:0/4094298804 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec037050 con 0x7fb9fc100e20 2024-09-19T04:19:48.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.626+0000 7fba0082e640 1 -- 172.21.15.94:0/4094298804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9fc1aafa0 con 0x7fb9fc100e20 2024-09-19T04:19:48.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.626+0000 7fb9f3fff640 1 -- 172.21.15.94:0/4094298804 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9ec043070 con 0x7fb9fc100e20 2024-09-19T04:19:48.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.626+0000 7fba0082e640 1 -- 172.21.15.94:0/4094298804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9fc1ab450 con 0x7fb9fc100e20 2024-09-19T04:19:48.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.626+0000 7fb9f3fff640 1 -- 172.21.15.94:0/4094298804 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec030af0 con 0x7fb9fc100e20 2024-09-19T04:19:48.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.629+0000 7fb9f3fff640 1 -- 172.21.15.94:0/4094298804 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fb9ec02bd10 con 0x7fb9fc100e20 2024-09-19T04:19:48.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.629+0000 7fba02c07740 1 Processor -- start 2024-09-19T04:19:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fba02c07740 1 -- start start 2024-09-19T04:19:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fba02c07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caadb40 0x55688caac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fba02c07740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caaca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fba02c07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688cab7370 0x55688cab9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fba02c07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55688ca122f0 con 0x55688cab7370 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fba02c07740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55688c9b9cf0 con 0x55688caac5d0 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fba02c07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55688ca0e2b0 con 0x55688caadb40 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688cab7370 0x55688cab9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688cab7370 0x55688cab9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34414/0 (socket says 172.21.15.94:34414) 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fb9f1d9b640 1 -- 172.21.15.94:0/620140573 learned_addr learned my addr 172.21.15.94:0/620140573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caadb40 0x55688caac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caaca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.630+0000 7fb9f1d9b640 1 -- 172.21.15.94:0/620140573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caadb40 msgr2=0x55688caac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caadb40 0x55688caac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9f1d9b640 1 -- 172.21.15.94:0/620140573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 msgr2=0x55688caaca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caaca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9f3fff640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9dc0a3480 0x7fb9dc0a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9f1d9b640 1 -- 172.21.15.94:0/620140573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688c99e3a0 con 0x55688cab7370 2024-09-19T04:19:48.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caadb40 0x55688caac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:48.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9f3fff640 1 -- 172.21.15.94:0/4094298804 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb9ec0efb50 con 0x7fb9fc100e20 2024-09-19T04:19:48.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688cab7370 0x55688cab9760 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009bf0 tx=0x7fb9e4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472182fb35b79d1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9dc0a3480 0x7fb9dc0a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caaca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:48.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.631+0000 7fb9f0d99640 1 -- 172.21.15.94:0/620140573 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4004020 con 0x55688cab7370 2024-09-19T04:19:48.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.632+0000 7fb9f0d99640 1 -- 172.21.15.94:0/620140573 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9e40041c0 con 0x55688cab7370 2024-09-19T04:19:48.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.632+0000 7fb9f0d99640 1 -- 172.21.15.94:0/620140573 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40126b0 con 0x55688cab7370 2024-09-19T04:19:48.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.632+0000 7fba02c07740 1 -- 172.21.15.94:0/620140573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688cab7370 msgr2=0x55688cab9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.632+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9dc0a3480 0x7fb9dc0a5940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc10df50 tx=0x7fb9f40095c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688cab7370 0x55688cab9760 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009bf0 tx=0x7fb9e4010740 comp rx=0 tx=0).stop 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 -- 172.21.15.94:0/620140573 shutdown_connections 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688cab7370 0x55688cab9760 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caaca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 --2- 172.21.15.94:0/620140573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caadb40 0x55688caac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 -- 172.21.15.94:0/620140573 >> 172.21.15.94:0/620140573 conn(0x55688ca75310 msgr2=0x55688caa8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 -- 172.21.15.94:0/620140573 shutdown_connections 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 -- 172.21.15.94:0/620140573 wait complete. 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 -- start start 2024-09-19T04:19:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.633+0000 7fba02c07740 1 Processor -- start 2024-09-19T04:19:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fba02c07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caac5d0 0x55688cab6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fba02c07740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caadb40 0x55688cab3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fba02c07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688cab44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fba02c07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55688caba110 con 0x55688caac5d0 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fba02c07740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55688c9cd060 con 0x55688caadb40 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fb9fad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688cab44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caac5d0 0x55688cab6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fb9fad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688cab44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:56210/0 (socket says 172.21.15.94:56210) 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caac5d0 0x55688cab6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34418/0 (socket says 172.21.15.94:34418) 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fb9fad9d640 1 -- 172.21.15.94:0/2761895796 learned_addr learned my addr 172.21.15.94:0/2761895796 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.634+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caadb40 0x55688cab3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fba02c07740 1 -- 172.21.15.94:0/2761895796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55688caba290 con 0x55688cab7370 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fba02c07740 10 client.0 _ll_register_callbacks cb 0x55688c7208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fb9fa59c640 1 -- 172.21.15.94:0/2761895796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 msgr2=0x55688cab44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688cab44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fb9fa59c640 1 -- 172.21.15.94:0/2761895796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caadb40 msgr2=0x55688cab3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caadb40 0x55688cab3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fb9fa59c640 1 -- 172.21.15.94:0/2761895796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55688c7833c0 con 0x55688caac5d0 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688cab44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caac5d0 0x55688cab6cb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0ed340 tx=0x7fb9ec0bba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffb0d5f366c2bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fb9e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.635+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/2761895796 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0bbed0 con 0x55688caac5d0 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.636+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/2761895796 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb9ec03bdf0 con 0x55688caac5d0 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.636+0000 7fba02c07740 1 -- 172.21.15.94:0/2761895796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e40041c0 con 0x55688caac5d0 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.636+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/2761895796 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9ec0c0280 con 0x55688caac5d0 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.636+0000 7fba02c07740 10 client.15945 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.636+0000 7fba02c07740 1 -- 172.21.15.94:0/2761895796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55688cac5ae0 con 0x55688caac5d0 2024-09-19T04:19:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.636+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/2761895796 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9e40041c0 con 0x55688caac5d0 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.636+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/2761895796 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fb9ec0c46b0 con 0x55688caac5d0 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.637+0000 7fba02c07740 10 client.15945 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.637+0000 7fba02c07740 10 client.15945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.637+0000 7fba02c07740 20 client.15945 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.637+0000 7fba02c07740 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55688cac7200 0x55688cac9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.637+0000 7fba02c07740 4 client.15945 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.637+0000 7fba02c07740 1 -- 172.21.15.94:0/2761895796 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55688ca1f860 con 0x55688cac7200 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.637+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55688cac7200 0x55688cac9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.638+0000 7fb9e9ffb640 10 client.15945 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:19:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.638+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55688cac7200 0x55688cac9670 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b75a78524be2819 server_cookie=ef4b92e35806a7db in_seq=0 out_seq=0 2024-09-19T04:19:48.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.639+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/2761895796 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55688ca1f860 con 0x55688cac7200 2024-09-19T04:19:48.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.639+0000 7fb9e9ffb640 10 client.15945 handle_command_reply: tid=0 2024-09-19T04:19:48.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.639+0000 7fba02c07740 1 client.15945 shutdown 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.640+0000 7fba02c07740 1 -- 172.21.15.94:0/2761895796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caac5d0 msgr2=0x55688cab6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.640+0000 7fba02c07740 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caac5d0 0x55688cab6cb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0ed340 tx=0x7fb9ec0bba10 comp rx=0 tx=0).stop 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.640+0000 7fba02c07740 1 -- 172.21.15.94:0/2761895796 shutdown_connections 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.640+0000 7fba02c07740 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55688cac7200 0x55688cac9670 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.640+0000 7fba02c07740 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688cab44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.640+0000 7fb9e9ffb640 0 client.15945 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.640+0000 7fba02c07740 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caadb40 0x55688cab3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.640+0000 7fba02c07740 1 --2- 172.21.15.94:0/2761895796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caac5d0 0x55688cab6cb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.641+0000 7fba02c07740 1 -- 172.21.15.94:0/2761895796 >> 172.21.15.94:0/2761895796 conn(0x55688ca75310 msgr2=0x55688caae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.641+0000 7fba02c07740 1 -- 172.21.15.94:0/2761895796 shutdown_connections 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.641+0000 7fba02c07740 1 -- 172.21.15.94:0/2761895796 wait complete. 2024-09-19T04:19:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.641+0000 7fba02c07740 20 client.15945 trim_cache size 0 max 16384 2024-09-19T04:19:48.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.664+0000 7fba02c07740 1 Processor -- start 2024-09-19T04:19:48.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.664+0000 7fba02c07740 1 -- start start 2024-09-19T04:19:48.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.665+0000 7fba02c07740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caa8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.665+0000 7fba02c07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cac3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.665+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caa8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.665+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caa8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50788/0 (socket says 172.21.15.94:50788) 2024-09-19T04:19:48.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.665+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cac3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.665+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cac3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34422/0 (socket says 172.21.15.94:34422) 2024-09-19T04:19:48.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.665+0000 7fb9fa59c640 1 -- 172.21.15.94:0/1012770330 learned_addr learned my addr 172.21.15.94:0/1012770330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:48.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.665+0000 7fba02c07740 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688caaf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fba02c07740 1 -- 172.21.15.94:0/1012770330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0011010 con 0x55688caadb40 2024-09-19T04:19:48.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fba02c07740 1 -- 172.21.15.94:0/1012770330 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e000e8e0 con 0x55688caac5d0 2024-09-19T04:19:48.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fba02c07740 1 -- 172.21.15.94:0/1012770330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e000c730 con 0x55688cab7370 2024-09-19T04:19:48.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688caaf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fb9fa59c640 1 -- 172.21.15.94:0/1012770330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 msgr2=0x55688caaf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688caaf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fb9fa59c640 1 -- 172.21.15.94:0/1012770330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 msgr2=0x55688cac3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cac3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fb9fa59c640 1 -- 172.21.15.94:0/1012770330 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688cab7020 con 0x55688caac5d0 2024-09-19T04:19:48.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688caaf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:48.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.666+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cac3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:48.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.667+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caa8e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0bbe60 tx=0x7fb9ec005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce23f777fe2db19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.667+0000 7fb9ebfff640 1 -- 172.21.15.94:0/1012770330 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0f2070 con 0x55688caac5d0 2024-09-19T04:19:48.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.667+0000 7fb9ebfff640 1 -- 172.21.15.94:0/1012770330 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9ec038040 con 0x55688caac5d0 2024-09-19T04:19:48.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.667+0000 7fb9ebfff640 1 -- 172.21.15.94:0/1012770330 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0c46b0 con 0x55688caac5d0 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.668+0000 7fba02c07740 1 -- 172.21.15.94:0/1012770330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 msgr2=0x55688caa8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.668+0000 7fba02c07740 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caa8e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0bbe60 tx=0x7fb9ec005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.668+0000 7fba02c07740 1 -- 172.21.15.94:0/1012770330 shutdown_connections 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.668+0000 7fba02c07740 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688cab7370 0x55688caaf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.668+0000 7fba02c07740 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cac3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.668+0000 7fba02c07740 1 --2- 172.21.15.94:0/1012770330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688caac5d0 0x55688caa8e10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.668+0000 7fba02c07740 1 -- 172.21.15.94:0/1012770330 >> 172.21.15.94:0/1012770330 conn(0x55688ca75310 msgr2=0x55688caa8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.669+0000 7fba02c07740 1 -- 172.21.15.94:0/1012770330 shutdown_connections 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.669+0000 7fba02c07740 1 -- 172.21.15.94:0/1012770330 wait complete. 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.669+0000 7fba02c07740 1 -- start start 2024-09-19T04:19:48.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.670+0000 7fba02c07740 1 Processor -- start 2024-09-19T04:19:48.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fba02c07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caac5d0 0x55688cabfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fba02c07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cab0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caac5d0 0x55688cabfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caac5d0 0x55688cabfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:56220/0 (socket says 172.21.15.94:56220) 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fb9fa59c640 1 -- 172.21.15.94:0/1998761002 learned_addr learned my addr 172.21.15.94:0/1998761002 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fba02c07740 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688cab7370 0x55688cab0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688cab7370 0x55688cab0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cab0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e000cda0 con 0x55688caadb40 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.671+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e000e210 con 0x55688cab7370 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.672+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e000d470 con 0x55688caac5d0 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.673+0000 7fba02c07740 10 client.0 _ll_register_callbacks cb 0x7fb9ec0b8170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.673+0000 7fb9f1d9b640 1 -- 172.21.15.94:0/1998761002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caac5d0 msgr2=0x55688cabfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.673+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caac5d0 0x55688cabfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.673+0000 7fb9f1d9b640 1 -- 172.21.15.94:0/1998761002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688cab7370 msgr2=0x55688cab0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.673+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688cab7370 0x55688cab0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.673+0000 7fb9f1d9b640 1 -- 172.21.15.94:0/1998761002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55688cab1510 con 0x55688caadb40 2024-09-19T04:19:48.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.673+0000 7fb9fad9d640 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688cab7370 0x55688cab0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:48.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.673+0000 7fb9f1d9b640 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cab0a60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4003e50 tx=0x7fb9e4009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0326728595e9f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.673+0000 7fb9e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:19:48.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.674+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/1998761002 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e402c070 con 0x55688caadb40 2024-09-19T04:19:48.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.674+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/1998761002 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fb9e40037f0 con 0x55688caadb40 2024-09-19T04:19:48.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.674+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9ec038040 con 0x55688caadb40 2024-09-19T04:19:48.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.675+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/1998761002 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9e4003a10 con 0x55688caadb40 2024-09-19T04:19:48.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.675+0000 7fba02c07740 10 client.15951 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:48.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.675+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55688cb53c80 con 0x55688caadb40 2024-09-19T04:19:48.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.675+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/1998761002 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9e4004330 con 0x55688caadb40 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.675+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/1998761002 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fb9e400f040 con 0x55688caadb40 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.675+0000 7fba02c07740 10 client.15951 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.675+0000 7fba02c07740 10 client.15951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.675+0000 7fba02c07740 20 client.15951 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.676+0000 7fba02c07740 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55688cac7200 0x55688cb56130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.676+0000 7fba02c07740 4 client.15951 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.676+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb9ec038040 con 0x55688cac7200 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.676+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55688cac7200 0x55688cb56130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fb9fa59c640 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55688cac7200 0x55688cb56130 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8485e19d5128e1dc server_cookie=b72322daeb00ed59 in_seq=0 out_seq=0 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fb9e9ffb640 10 client.15951 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fb9e9ffb640 1 -- 172.21.15.94:0/1998761002 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb9ec038040 con 0x55688cac7200 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fb9e9ffb640 10 client.15951 handle_command_reply: tid=0 2024-09-19T04:19:48.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fba02c07740 1 client.15951 shutdown 2024-09-19T04:19:48.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 msgr2=0x55688cab0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fba02c07740 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cab0a60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4003e50 tx=0x7fb9e4009e80 comp rx=0 tx=0).stop 2024-09-19T04:19:48.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 shutdown_connections 2024-09-19T04:19:48.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fba02c07740 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55688cac7200 0x55688cb56130 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fba02c07740 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55688cab7370 0x55688cab0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fba02c07740 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55688caadb40 0x55688cab0a60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fb9e9ffb640 0 client.15951 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:19:48.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fba02c07740 1 --2- 172.21.15.94:0/1998761002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55688caac5d0 0x55688cabfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.677+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 >> 172.21.15.94:0/1998761002 conn(0x55688ca75310 msgr2=0x55688caa74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.678+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 shutdown_connections 2024-09-19T04:19:48.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.678+0000 7fba02c07740 1 -- 172.21.15.94:0/1998761002 wait complete. 2024-09-19T04:19:48.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.678+0000 7fba02c07740 20 client.15951 trim_cache size 0 max 16384 2024-09-19T04:19:48.681 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:19:48.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.679+0000 7fba0082e640 1 -- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9dc0a3480 msgr2=0x7fb9dc0a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.679+0000 7fba0082e640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9dc0a3480 0x7fb9dc0a5940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc10df50 tx=0x7fb9f40095c0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.679+0000 7fba0082e640 1 -- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9fc100e20 msgr2=0x7fb9fc10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.679+0000 7fba0082e640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9fc100e20 0x7fb9fc10d840 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec000c00 tx=0x7fb9ec004270 comp rx=0 tx=0).stop 2024-09-19T04:19:48.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.682+0000 7fba0082e640 1 -- 172.21.15.94:0/4094298804 shutdown_connections 2024-09-19T04:19:48.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.683+0000 7fba0082e640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9dc0a3480 0x7fb9dc0a5940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.683+0000 7fba0082e640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.683+0000 7fba0082e640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9fc102570 0x7fb9fc10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.683+0000 7fba0082e640 1 --2- 172.21.15.94:0/4094298804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9fc100e20 0x7fb9fc10d840 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.683+0000 7fba0082e640 1 -- 172.21.15.94:0/4094298804 >> 172.21.15.94:0/4094298804 conn(0x7fb9fc0fc7d0 msgr2=0x7fb9fc0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.684+0000 7fba0082e640 1 -- 172.21.15.94:0/4094298804 shutdown_connections 2024-09-19T04:19:48.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:48.684+0000 7fba0082e640 1 -- 172.21.15.94:0/4094298804 wait complete. 2024-09-19T04:19:49.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:19:49.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:49 smithi112 ceph-mon[35051]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:19:49.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:49 smithi094 ceph-mon[30972]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:19:49.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:49 smithi104 ceph-mon[34145]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:19:49.949 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:50.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.526+0000 7f407159a640 1 -- 172.21.15.94:0/3911679328 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064031880 con 0x7f4074105f40 2024-09-19T04:19:50.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.527+0000 7f4078e83640 1 -- 172.21.15.94:0/3911679328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4074105f40 msgr2=0x7f40741063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.527+0000 7f4078e83640 1 --2- 172.21.15.94:0/3911679328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4074105f40 0x7f40741063d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4064009900 tx=0x7f406402a470 comp rx=0 tx=0).stop 2024-09-19T04:19:50.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.527+0000 7f4078e83640 1 -- 172.21.15.94:0/3911679328 shutdown_connections 2024-09-19T04:19:50.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.527+0000 7f4078e83640 1 --2- 172.21.15.94:0/3911679328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4074105f40 0x7f40741063d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.527+0000 7f4078e83640 1 --2- 172.21.15.94:0/3911679328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4074101450 0x7f4074105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.527+0000 7f4078e83640 1 --2- 172.21.15.94:0/3911679328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074100b30 0x7f4074100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.527+0000 7f4078e83640 1 -- 172.21.15.94:0/3911679328 >> 172.21.15.94:0/3911679328 conn(0x7f40740fc7a0 msgr2=0x7f40740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.528+0000 7f4078e83640 1 -- 172.21.15.94:0/3911679328 shutdown_connections 2024-09-19T04:19:50.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.528+0000 7f4078e83640 1 -- 172.21.15.94:0/3911679328 wait complete. 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4078e83640 1 Processor -- start 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4078e83640 1 -- start start 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4078e83640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4074100b30 0x7f407406d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4078e83640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4074101450 0x7f407406ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4078e83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074105f40 0x7f40740721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4078e83640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f407406c0e0 con 0x7f4074100b30 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4078e83640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f407406bf60 con 0x7f4074101450 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4078e83640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f407406c260 con 0x7f4074105f40 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4072d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074105f40 0x7f40740721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4072d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074105f40 0x7f40740721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:56240/0 (socket says 172.21.15.94:56240) 2024-09-19T04:19:50.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4072d9d640 1 -- 172.21.15.94:0/1862740005 learned_addr learned my addr 172.21.15.94:0/1862740005 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:50.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.529+0000 7f4071d9b640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4074101450 0x7f407406ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f407259c640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4074100b30 0x7f407406d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f4072d9d640 1 -- 172.21.15.94:0/1862740005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4074101450 msgr2=0x7f407406ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4074101450 0x7f407406ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f4072d9d640 1 -- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4074100b30 msgr2=0x7f407406d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4074100b30 0x7f407406d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f4072d9d640 1 -- 172.21.15.94:0/1862740005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40740728c0 con 0x7f4074105f40 2024-09-19T04:19:50.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f4071d9b640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4074101450 0x7f407406ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:50.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074105f40 0x7f40740721c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f406402a9e0 tx=0x7f4064003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f40637fe640 1 -- 172.21.15.94:0/1862740005 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406403f070 con 0x7f4074105f40 2024-09-19T04:19:50.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f40637fe640 1 -- 172.21.15.94:0/1862740005 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4064031b50 con 0x7f4074105f40 2024-09-19T04:19:50.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.530+0000 7f40637fe640 1 -- 172.21.15.94:0/1862740005 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064035030 con 0x7f4074105f40 2024-09-19T04:19:50.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.531+0000 7f4078e83640 1 -- 172.21.15.94:0/1862740005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4074072b50 con 0x7f4074105f40 2024-09-19T04:19:50.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.531+0000 7f4078e83640 1 -- 172.21.15.94:0/1862740005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4074072fe0 con 0x7f4074105f40 2024-09-19T04:19:50.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.535+0000 7f407b25c740 1 Processor -- start 2024-09-19T04:19:50.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.536+0000 7f40637fe640 1 -- 172.21.15.94:0/1862740005 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4064042070 con 0x7f4074105f40 2024-09-19T04:19:50.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.537+0000 7f40637fe640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f405c0a3480 0x7f405c0a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.537+0000 7f40637fe640 1 -- 172.21.15.94:0/1862740005 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f40640f04b0 con 0x7f4074105f40 2024-09-19T04:19:50.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.537+0000 7f407b25c740 1 -- start start 2024-09-19T04:19:50.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.537+0000 7f4071d9b640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f405c0a3480 0x7f405c0a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f407b25c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d078a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f407b25c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d078aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f407b25c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d0797760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f407b25c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5590d06f02f0 con 0x5590d078a5d0 2024-09-19T04:19:50.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f407b25c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5590d0697cf0 con 0x5590d078bb40 2024-09-19T04:19:50.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f407b25c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5590d06ec2b0 con 0x5590d0795370 2024-09-19T04:19:50.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f407259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d078a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f407259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d078a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50814/0 (socket says 172.21.15.94:50814) 2024-09-19T04:19:50.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f407259c640 1 -- 172.21.15.94:0/1951974402 learned_addr learned my addr 172.21.15.94:0/1951974402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:50.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.538+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1951974402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d0797760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f407259c640 1 -- 172.21.15.94:0/1951974402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 msgr2=0x5590d0797760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f407259c640 1 --2- 172.21.15.94:0/1951974402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d0797760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f407259c640 1 -- 172.21.15.94:0/1951974402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 msgr2=0x5590d078aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f407259c640 1 --2- 172.21.15.94:0/1951974402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d078aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f407259c640 1 -- 172.21.15.94:0/1951974402 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d067c3a0 con 0x5590d078bb40 2024-09-19T04:19:50.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1951974402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d0797760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:50.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f407259c640 1 --2- 172.21.15.94:0/1951974402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d078a090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f406c010400 tx=0x7f406c0108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bdce58b1753283f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f4060ff9640 1 -- 172.21.15.94:0/1951974402 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c003b70 con 0x5590d078bb40 2024-09-19T04:19:50.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f4060ff9640 1 -- 172.21.15.94:0/1951974402 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f406c003d10 con 0x5590d078bb40 2024-09-19T04:19:50.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.539+0000 7f4060ff9640 1 -- 172.21.15.94:0/1951974402 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c011a70 con 0x5590d078bb40 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.540+0000 7f407b25c740 1 -- 172.21.15.94:0/1951974402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 msgr2=0x5590d078a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.540+0000 7f407b25c740 1 --2- 172.21.15.94:0/1951974402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d078a090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f406c010400 tx=0x7f406c0108c0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.540+0000 7f4071d9b640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f405c0a3480 0x7f405c0a5940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f407406ef00 tx=0x7f40680117b0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.540+0000 7f407b25c740 1 -- 172.21.15.94:0/1951974402 shutdown_connections 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.540+0000 7f407b25c740 1 --2- 172.21.15.94:0/1951974402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d0797760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.540+0000 7f407b25c740 1 --2- 172.21.15.94:0/1951974402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d078aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.540+0000 7f407b25c740 1 --2- 172.21.15.94:0/1951974402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d078a090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.540+0000 7f407b25c740 1 -- 172.21.15.94:0/1951974402 >> 172.21.15.94:0/1951974402 conn(0x5590d0753310 msgr2=0x5590d0786c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.541+0000 7f407b25c740 1 -- 172.21.15.94:0/1951974402 shutdown_connections 2024-09-19T04:19:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.541+0000 7f407b25c740 1 -- 172.21.15.94:0/1951974402 wait complete. 2024-09-19T04:19:50.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.541+0000 7f407b25c740 1 -- start start 2024-09-19T04:19:50.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.541+0000 7f407b25c740 1 Processor -- start 2024-09-19T04:19:50.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f407b25c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d0794dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f407b25c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d0792070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f407b25c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d0792800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f407b25c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5590d0798070 con 0x5590d078a5d0 2024-09-19T04:19:50.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f407b25c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5590d06ab060 con 0x5590d078bb40 2024-09-19T04:19:50.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f407b25c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5590d07981f0 con 0x5590d0795370 2024-09-19T04:19:50.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f407b25c740 10 client.0 _ll_register_callbacks cb 0x5590d03fe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f4071d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d0792070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f407259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d0794dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f4071d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d0792070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50824/0 (socket says 172.21.15.94:50824) 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f4071d9b640 1 -- 172.21.15.94:0/3447972864 learned_addr learned my addr 172.21.15.94:0/3447972864 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f407259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d0794dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34488/0 (socket says 172.21.15.94:34488) 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f4072d9d640 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d0792800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f4071d9b640 1 -- 172.21.15.94:0/3447972864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 msgr2=0x5590d0792800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.542+0000 7f4071d9b640 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d0792800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.543+0000 7f4071d9b640 1 -- 172.21.15.94:0/3447972864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 msgr2=0x5590d0794dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.543+0000 7f4071d9b640 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d0794dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.543+0000 7f4071d9b640 1 -- 172.21.15.94:0/3447972864 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d04613c0 con 0x5590d078bb40 2024-09-19T04:19:50.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.543+0000 7f4071d9b640 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d0792070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4068012850 tx=0x7f4068004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bbf6149141c30de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.543+0000 7f4059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:19:50.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.543+0000 7f4059ffb640 1 -- 172.21.15.94:0/3447972864 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406800fd60 con 0x5590d078bb40 2024-09-19T04:19:50.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.544+0000 7f4059ffb640 1 -- 172.21.15.94:0/3447972864 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f4068021450 con 0x5590d078bb40 2024-09-19T04:19:50.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.544+0000 7f407b25c740 1 -- 172.21.15.94:0/3447972864 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f406c003d10 con 0x5590d078bb40 2024-09-19T04:19:50.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.544+0000 7f4059ffb640 1 -- 172.21.15.94:0/3447972864 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f406802f050 con 0x5590d078bb40 2024-09-19T04:19:50.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.544+0000 7f407b25c740 10 client.25547 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:50.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.544+0000 7f407b25c740 1 -- 172.21.15.94:0/3447972864 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5590d07a3be0 con 0x5590d078bb40 2024-09-19T04:19:50.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.544+0000 7f4059ffb640 1 -- 172.21.15.94:0/3447972864 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f406c003d10 con 0x5590d078bb40 2024-09-19T04:19:50.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.544+0000 7f4059ffb640 1 -- 172.21.15.94:0/3447972864 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f406802a510 con 0x5590d078bb40 2024-09-19T04:19:50.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.544+0000 7f407b25c740 10 client.25547 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:50.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.544+0000 7f407b25c740 10 client.25547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:50.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.545+0000 7f407b25c740 20 client.25547 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:50.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.545+0000 7f407b25c740 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5590d07a5490 0x5590d07a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.545+0000 7f407b25c740 4 client.25547 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:50.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.545+0000 7f407b25c740 1 -- 172.21.15.94:0/3447972864 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590d06fd860 con 0x5590d07a5490 2024-09-19T04:19:50.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.545+0000 7f407259c640 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5590d07a5490 0x5590d07a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.546+0000 7f407259c640 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5590d07a5490 0x5590d07a7950 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4789c8afaf872135 server_cookie=60b6d3e103470d7d in_seq=0 out_seq=0 2024-09-19T04:19:50.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.546+0000 7f4059ffb640 10 client.25547 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:50.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f4059ffb640 1 -- 172.21.15.94:0/3447972864 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590d06fd860 con 0x5590d07a5490 2024-09-19T04:19:50.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f4059ffb640 10 client.25547 handle_command_reply: tid=0 2024-09-19T04:19:50.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 client.25547 shutdown 2024-09-19T04:19:50.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 -- 172.21.15.94:0/3447972864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 msgr2=0x5590d0792070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d0792070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4068012850 tx=0x7f4068004730 comp rx=0 tx=0).stop 2024-09-19T04:19:50.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 -- 172.21.15.94:0/3447972864 shutdown_connections 2024-09-19T04:19:50.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5590d07a5490 0x5590d07a7950 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d0792800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d0792070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 --2- 172.21.15.94:0/3447972864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d0794dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 -- 172.21.15.94:0/3447972864 >> 172.21.15.94:0/3447972864 conn(0x5590d0753310 msgr2=0x5590d078c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f4059ffb640 0 client.25547 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:50.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 -- 172.21.15.94:0/3447972864 shutdown_connections 2024-09-19T04:19:50.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 1 -- 172.21.15.94:0/3447972864 wait complete. 2024-09-19T04:19:50.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.547+0000 7f407b25c740 20 client.25547 trim_cache size 0 max 16384 2024-09-19T04:19:50.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.569+0000 7f407b25c740 1 Processor -- start 2024-09-19T04:19:50.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.569+0000 7f407b25c740 1 -- start start 2024-09-19T04:19:50.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.570+0000 7f407b25c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078a5d0 0x5590d0786700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.570+0000 7f407b25c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078bb40 0x5590d0786c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.570+0000 7f407b25c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d07ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.570+0000 7f407259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078a5d0 0x5590d0786700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.570+0000 7f407259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078a5d0 0x5590d0786700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50828/0 (socket says 172.21.15.94:50828) 2024-09-19T04:19:50.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.570+0000 7f407259c640 1 -- 172.21.15.94:0/1213254066 learned_addr learned my addr 172.21.15.94:0/1213254066 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:50.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.570+0000 7f407b25c740 1 -- 172.21.15.94:0/1213254066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40741bef50 con 0x5590d078bb40 2024-09-19T04:19:50.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f407b25c740 1 -- 172.21.15.94:0/1213254066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40741bc820 con 0x5590d078a5d0 2024-09-19T04:19:50.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f407b25c740 1 -- 172.21.15.94:0/1213254066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40741ba670 con 0x5590d0795370 2024-09-19T04:19:50.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d07ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f4071d9b640 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078bb40 0x5590d0786c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f4072d9d640 1 -- 172.21.15.94:0/1213254066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078a5d0 msgr2=0x5590d0786700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078a5d0 0x5590d0786700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f4072d9d640 1 -- 172.21.15.94:0/1213254066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078bb40 msgr2=0x5590d0786c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078bb40 0x5590d0786c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f4072d9d640 1 -- 172.21.15.94:0/1213254066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d0792b70 con 0x5590d0795370 2024-09-19T04:19:50.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f4071d9b640 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078bb40 0x5590d0786c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:50.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f407259c640 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078a5d0 0x5590d0786700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:50.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.571+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d07ad260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f40640edbf0 tx=0x7f40640045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ad7951dd3bcab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.572+0000 7f405bfff640 1 -- 172.21.15.94:0/1213254066 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640c0b30 con 0x5590d0795370 2024-09-19T04:19:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.572+0000 7f405bfff640 1 -- 172.21.15.94:0/1213254066 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40640c53f0 con 0x5590d0795370 2024-09-19T04:19:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.572+0000 7f405bfff640 1 -- 172.21.15.94:0/1213254066 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640c51c0 con 0x5590d0795370 2024-09-19T04:19:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.573+0000 7f407b25c740 1 -- 172.21.15.94:0/1213254066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 msgr2=0x5590d07ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.573+0000 7f407b25c740 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d07ad260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f40640edbf0 tx=0x7f40640045e0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.573+0000 7f407b25c740 1 -- 172.21.15.94:0/1213254066 shutdown_connections 2024-09-19T04:19:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.573+0000 7f407b25c740 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d07ad260 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.573+0000 7f407b25c740 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078bb40 0x5590d0786c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.573+0000 7f407b25c740 1 --2- 172.21.15.94:0/1213254066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078a5d0 0x5590d0786700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.573+0000 7f407b25c740 1 -- 172.21.15.94:0/1213254066 >> 172.21.15.94:0/1213254066 conn(0x5590d0753310 msgr2=0x5590d078d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.574+0000 7f407b25c740 1 -- 172.21.15.94:0/1213254066 shutdown_connections 2024-09-19T04:19:50.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.574+0000 7f407b25c740 1 -- 172.21.15.94:0/1213254066 wait complete. 2024-09-19T04:19:50.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.574+0000 7f407b25c740 1 -- start start 2024-09-19T04:19:50.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.575+0000 7f407b25c740 1 Processor -- start 2024-09-19T04:19:50.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.575+0000 7f407b25c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d079d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.575+0000 7f407b25c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d079dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.575+0000 7f407b25c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d078ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.575+0000 7f407b25c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40741bace0 con 0x5590d078a5d0 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.575+0000 7f407b25c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40741bc150 con 0x5590d078bb40 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.575+0000 7f407b25c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40741bb3b0 con 0x5590d0795370 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.575+0000 7f407b25c740 10 client.0 _ll_register_callbacks cb 0x7f407406b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f407259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d079d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f4071d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d079dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f407259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d079d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34510/0 (socket says 172.21.15.94:34510) 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f407259c640 1 -- 172.21.15.94:0/1334650179 learned_addr learned my addr 172.21.15.94:0/1334650179 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f4072d9d640 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d078ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f407259c640 1 -- 172.21.15.94:0/1334650179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 msgr2=0x5590d078ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f407259c640 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d078ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f407259c640 1 -- 172.21.15.94:0/1334650179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 msgr2=0x5590d079dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f407259c640 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d079dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f407259c640 1 -- 172.21.15.94:0/1334650179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d078f440 con 0x5590d078a5d0 2024-09-19T04:19:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f4071d9b640 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d079dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f407259c640 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d079d7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f406c006010 tx=0x7f406c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d28ed2e5f33ba81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.576+0000 7f4059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:19:50.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.577+0000 7f4059ffb640 1 -- 172.21.15.94:0/1334650179 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c011640 con 0x5590d078a5d0 2024-09-19T04:19:50.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.577+0000 7f4059ffb640 1 -- 172.21.15.94:0/1334650179 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f406c0117e0 con 0x5590d078a5d0 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.577+0000 7f407b25c740 1 -- 172.21.15.94:0/1334650179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40640c53f0 con 0x5590d078a5d0 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.577+0000 7f4059ffb640 1 -- 172.21.15.94:0/1334650179 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f406c0119e0 con 0x5590d078a5d0 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.577+0000 7f407b25c740 10 client.15963 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.577+0000 7f407b25c740 1 -- 172.21.15.94:0/1334650179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5590d080fde0 con 0x5590d078a5d0 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.578+0000 7f4059ffb640 1 -- 172.21.15.94:0/1334650179 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40640c53f0 con 0x5590d078a5d0 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.578+0000 7f4059ffb640 1 -- 172.21.15.94:0/1334650179 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f406c027440 con 0x5590d078a5d0 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.578+0000 7f407b25c740 10 client.15963 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.578+0000 7f407b25c740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.578+0000 7f407b25c740 20 client.15963 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.578+0000 7f407b25c740 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5590d0810250 0x5590d0812670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.578+0000 7f407b25c740 4 client.15963 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:19:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.578+0000 7f407b25c740 1 -- 172.21.15.94:0/1334650179 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590d06f7d70 con 0x5590d0810250 2024-09-19T04:19:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.578+0000 7f4071d9b640 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5590d0810250 0x5590d0812670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.579+0000 7f4059ffb640 10 client.15963 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.579+0000 7f4071d9b640 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5590d0810250 0x5590d0812670 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7baf060e50e7c67 server_cookie=a20adcba6619af2b in_seq=0 out_seq=0 2024-09-19T04:19:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.579+0000 7f4059ffb640 1 -- 172.21.15.94:0/1334650179 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590d06f7d70 con 0x5590d0810250 2024-09-19T04:19:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.579+0000 7f4059ffb640 10 client.15963 handle_command_reply: tid=0 2024-09-19T04:19:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.579+0000 7f407b25c740 1 client.15963 shutdown 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 -- 172.21.15.94:0/1334650179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 msgr2=0x5590d079d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d079d7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f406c006010 tx=0x7f406c003970 comp rx=0 tx=0).stop 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 -- 172.21.15.94:0/1334650179 shutdown_connections 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5590d0810250 0x5590d0812670 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590d0795370 0x5590d078ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590d078bb40 0x5590d079dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 --2- 172.21.15.94:0/1334650179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590d078a5d0 0x5590d079d7f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 -- 172.21.15.94:0/1334650179 >> 172.21.15.94:0/1334650179 conn(0x5590d0753310 msgr2=0x5590d07854e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f4059ffb640 0 client.15963 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 -- 172.21.15.94:0/1334650179 shutdown_connections 2024-09-19T04:19:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 1 -- 172.21.15.94:0/1334650179 wait complete. 2024-09-19T04:19:50.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.580+0000 7f407b25c740 20 client.15963 trim_cache size 0 max 16384 2024-09-19T04:19:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:19:50.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.581+0000 7f4078e83640 1 -- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f405c0a3480 msgr2=0x7f405c0a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.581+0000 7f4078e83640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f405c0a3480 0x7f405c0a5940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f407406ef00 tx=0x7f40680117b0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.581+0000 7f4078e83640 1 -- 172.21.15.94:0/1862740005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074105f40 msgr2=0x7f40740721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.581+0000 7f4078e83640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074105f40 0x7f40740721c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f406402a9e0 tx=0x7f4064003b90 comp rx=0 tx=0).stop 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.581+0000 7f4078e83640 1 -- 172.21.15.94:0/1862740005 shutdown_connections 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.581+0000 7f4078e83640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f405c0a3480 0x7f405c0a5940 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.582+0000 7f4078e83640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074105f40 0x7f40740721c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.582+0000 7f4078e83640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4074101450 0x7f407406ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.582+0000 7f4078e83640 1 --2- 172.21.15.94:0/1862740005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4074100b30 0x7f407406d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.582+0000 7f4078e83640 1 -- 172.21.15.94:0/1862740005 >> 172.21.15.94:0/1862740005 conn(0x7f40740fc7a0 msgr2=0x7f40741038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.582+0000 7f4078e83640 1 -- 172.21.15.94:0/1862740005 shutdown_connections 2024-09-19T04:19:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:50.582+0000 7f4078e83640 1 -- 172.21.15.94:0/1862740005 wait complete. 2024-09-19T04:19:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 215.0 s, 1325.6 tps, lat 75.237 ms stddev 43.305 2024-09-19T04:19:51.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:51 smithi112 ceph-mon[35051]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:19:51.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:51 smithi094 ceph-mon[30972]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:19:51.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:51 smithi104 ceph-mon[34145]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:19:52.546 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:19:52.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:19:52.930 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:53.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.528+0000 7f76fc3bd640 1 -- 172.21.15.94:0/414401920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f41049c0 msgr2=0x7f76f4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.528+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/414401920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f41049c0 0x7f76f4108a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009900 tx=0x7f76e402a410 comp rx=0 tx=0).stop 2024-09-19T04:19:53.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.528+0000 7f76fc3bd640 1 -- 172.21.15.94:0/414401920 shutdown_connections 2024-09-19T04:19:53.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.528+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/414401920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76f4109140 0x7f76f4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.528+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/414401920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f41049c0 0x7f76f4108a10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.528+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/414401920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76f4104010 0x7f76f41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.528+0000 7f76fc3bd640 1 -- 172.21.15.94:0/414401920 >> 172.21.15.94:0/414401920 conn(0x7f76f40ffa50 msgr2=0x7f76f4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:53.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.528+0000 7f76fc3bd640 1 -- 172.21.15.94:0/414401920 shutdown_connections 2024-09-19T04:19:53.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.528+0000 7f76fc3bd640 1 -- 172.21.15.94:0/414401920 wait complete. 2024-09-19T04:19:53.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.529+0000 7f76fc3bd640 1 Processor -- start 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.529+0000 7f76fc3bd640 1 -- start start 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fc3bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f4104010 0x7f76f4073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fc3bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76f41049c0 0x7f76f4073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fc3bd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76f4109140 0x7f76f406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fc3bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4114040 con 0x7f76f4104010 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fc3bd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4113ec0 con 0x7f76f4109140 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fc3bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41141c0 con 0x7f76f41049c0 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76f9958640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76f41049c0 0x7f76f4073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fa159640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f4104010 0x7f76f4073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76f9958640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76f41049c0 0x7f76f4073db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:34880/0 (socket says 172.21.15.94:34880) 2024-09-19T04:19:53.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fa159640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f4104010 0x7f76f4073870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56044/0 (socket says 172.21.15.94:56044) 2024-09-19T04:19:53.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76f9958640 1 -- 172.21.15.94:0/1029487523 learned_addr learned my addr 172.21.15.94:0/1029487523 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:53.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fa95a640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76f4109140 0x7f76f406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76f9958640 1 -- 172.21.15.94:0/1029487523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76f4109140 msgr2=0x7f76f406c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76f9958640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76f4109140 0x7f76f406c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76f9958640 1 -- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f4104010 msgr2=0x7f76f4073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76f9958640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f4104010 0x7f76f4073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76f9958640 1 -- 172.21.15.94:0/1029487523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f406d270 con 0x7f76f41049c0 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.530+0000 7f76fa159640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f4104010 0x7f76f4073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.531+0000 7f76f9958640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76f41049c0 0x7f76f4073db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009850 tx=0x7f76e4005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.531+0000 7f76eb7fe640 1 -- 172.21.15.94:0/1029487523 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4035030 con 0x7f76f41049c0 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.531+0000 7f76eb7fe640 1 -- 172.21.15.94:0/1029487523 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76e4045070 con 0x7f76f41049c0 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.531+0000 7f76eb7fe640 1 -- 172.21.15.94:0/1029487523 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003850 con 0x7f76f41049c0 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.531+0000 7f76fc3bd640 1 -- 172.21.15.94:0/1029487523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f406d500 con 0x7f76f41049c0 2024-09-19T04:19:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.531+0000 7f76fc3bd640 1 -- 172.21.15.94:0/1029487523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f40713a0 con 0x7f76f41049c0 2024-09-19T04:19:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.534+0000 7f76eb7fe640 1 -- 172.21.15.94:0/1029487523 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f76e4004050 con 0x7f76f41049c0 2024-09-19T04:19:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.535+0000 7f76fe796740 1 Processor -- start 2024-09-19T04:19:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.535+0000 7f76eb7fe640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76e00a3430 0x7f76e00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.535+0000 7f76fe796740 1 -- start start 2024-09-19T04:19:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.535+0000 7f76eb7fe640 1 -- 172.21.15.94:0/1029487523 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f76e40f8b30 con 0x7f76f41049c0 2024-09-19T04:19:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.535+0000 7f76fa95a640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76e00a3430 0x7f76e00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.535+0000 7f76fe796740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f51cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76fe796740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f527800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76fe796740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 0x56401f52a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76fe796740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56401f482280 con 0x56401f51d480 2024-09-19T04:19:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76fe796740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56401f429b10 con 0x56401f51cb60 2024-09-19T04:19:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76fe796740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56401f47e240 con 0x56401f527d40 2024-09-19T04:19:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76fa95a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 0x56401f52a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76f9958640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f527800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76f9958640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f527800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56050/0 (socket says 172.21.15.94:56050) 2024-09-19T04:19:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76f9958640 1 -- 172.21.15.94:0/1262944427 learned_addr learned my addr 172.21.15.94:0/1262944427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76fa159640 1 --2- 172.21.15.94:0/1262944427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f51cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76f9958640 1 -- 172.21.15.94:0/1262944427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 msgr2=0x56401f52a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76f9958640 1 --2- 172.21.15.94:0/1262944427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 0x56401f52a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76f9958640 1 -- 172.21.15.94:0/1262944427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 msgr2=0x56401f51cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76f9958640 1 --2- 172.21.15.94:0/1262944427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f51cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76f9958640 1 -- 172.21.15.94:0/1262944427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401f4361f0 con 0x56401f51d480 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76fa95a640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76e00a3430 0x7f76e00a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009fd0 tx=0x7f76ec009ac0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.536+0000 7f76fa159640 1 --2- 172.21.15.94:0/1262944427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f51cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.537+0000 7f76f9958640 1 --2- 172.21.15.94:0/1262944427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f527800 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f76e40c18e0 tx=0x7f76e40c4da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc035456ad220642 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.537+0000 7f76e8dea640 1 -- 172.21.15.94:0/1262944427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e40c4f10 con 0x56401f51d480 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.537+0000 7f76e8dea640 1 -- 172.21.15.94:0/1262944427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76e40c92c0 con 0x56401f51d480 2024-09-19T04:19:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.537+0000 7f76e8dea640 1 -- 172.21.15.94:0/1262944427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e40c1580 con 0x56401f51d480 2024-09-19T04:19:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.538+0000 7f76fe796740 1 -- 172.21.15.94:0/1262944427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 msgr2=0x56401f527800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.538+0000 7f76fe796740 1 --2- 172.21.15.94:0/1262944427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f527800 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f76e40c18e0 tx=0x7f76e40c4da0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.538+0000 7f76fe796740 1 -- 172.21.15.94:0/1262944427 shutdown_connections 2024-09-19T04:19:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.538+0000 7f76fe796740 1 --2- 172.21.15.94:0/1262944427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 0x56401f52a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.538+0000 7f76fe796740 1 --2- 172.21.15.94:0/1262944427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f527800 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.538+0000 7f76fe796740 1 --2- 172.21.15.94:0/1262944427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f51cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.538+0000 7f76fe796740 1 -- 172.21.15.94:0/1262944427 >> 172.21.15.94:0/1262944427 conn(0x56401f5190f0 msgr2=0x56401f5194d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.539+0000 7f76fe796740 1 -- 172.21.15.94:0/1262944427 shutdown_connections 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.539+0000 7f76fe796740 1 -- 172.21.15.94:0/1262944427 wait complete. 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.539+0000 7f76fe796740 1 -- start start 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.539+0000 7f76fe796740 1 Processor -- start 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.539+0000 7f76fe796740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f524cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fe796740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 0x56401f525210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fe796740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f527d40 0x56401f525750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fe796740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56401f52a6d0 con 0x56401f527d40 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fe796740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56401f43cf10 con 0x56401f51cb60 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fe796740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56401f52a850 con 0x56401f51d480 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fe796740 10 client.0 _ll_register_callbacks cb 0x56401f405a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa159640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f524cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa95a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f527d40 0x56401f525750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa159640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f524cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:60122/0 (socket says 172.21.15.94:60122) 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa159640 1 -- 172.21.15.94:0/3258689201 learned_addr learned my addr 172.21.15.94:0/3258689201 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76f9958640 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 0x56401f525210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa95a640 1 -- 172.21.15.94:0/3258689201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 msgr2=0x56401f525210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa95a640 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 0x56401f525210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa95a640 1 -- 172.21.15.94:0/3258689201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 msgr2=0x56401f524cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa95a640 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f524cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa95a640 1 -- 172.21.15.94:0/3258689201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56401f1f36a0 con 0x56401f527d40 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa159640 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f524cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:19:53.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.540+0000 7f76fa95a640 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f527d40 0x56401f525750 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x56401f51a470 tx=0x7f76ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7686d524c20d4d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.541+0000 7f76ddffb640 1 -- 172.21.15.94:0/3258689201 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec029670 con 0x56401f527d40 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.541+0000 7f76fe796740 1 -- 172.21.15.94:0/3258689201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e40c92c0 con 0x56401f527d40 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.541+0000 7f76ddffb640 1 -- 172.21.15.94:0/3258689201 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f76ec029470 con 0x56401f527d40 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.541+0000 7f76ddffb640 1 -- 172.21.15.94:0/3258689201 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f76ec029670 con 0x56401f527d40 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.541+0000 7f76fe796740 10 client.15975 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.541+0000 7f76fe796740 1 -- 172.21.15.94:0/3258689201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56401f536300 con 0x56401f527d40 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.542+0000 7f76ddffb640 1 -- 172.21.15.94:0/3258689201 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76e40c92c0 con 0x56401f527d40 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.542+0000 7f76ddffb640 1 -- 172.21.15.94:0/3258689201 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f76ec00f040 con 0x56401f527d40 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.542+0000 7f76fe796740 10 client.15975 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.542+0000 7f76fe796740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.542+0000 7f76fe796740 20 client.15975 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.542+0000 7f76fe796740 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56401f537b40 0x56401f539fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.542+0000 7f76fe796740 4 client.15975 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.542+0000 7f76fe796740 1 -- 172.21.15.94:0/3258689201 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56401f48f7f0 con 0x56401f537b40 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.542+0000 7f76fa159640 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56401f537b40 0x56401f539fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.543+0000 7f76fa159640 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56401f537b40 0x56401f539fc0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47119efe8f9c398 server_cookie=1c02a03970c01b81 in_seq=0 out_seq=0 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.543+0000 7f76ddffb640 10 client.15975 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76ddffb640 1 -- 172.21.15.94:0/3258689201 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56401f48f7f0 con 0x56401f537b40 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76ddffb640 10 client.15975 handle_command_reply: tid=0 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 client.15975 shutdown 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 -- 172.21.15.94:0/3258689201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f527d40 msgr2=0x56401f525750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f527d40 0x56401f525750 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x56401f51a470 tx=0x7f76ec004ab0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 -- 172.21.15.94:0/3258689201 shutdown_connections 2024-09-19T04:19:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56401f537b40 0x56401f539fc0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76ddffb640 0 client.15975 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f527d40 0x56401f525750 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 0x56401f525210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 --2- 172.21.15.94:0/3258689201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f524cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 -- 172.21.15.94:0/3258689201 >> 172.21.15.94:0/3258689201 conn(0x56401f5190f0 msgr2=0x56401f518270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 -- 172.21.15.94:0/3258689201 shutdown_connections 2024-09-19T04:19:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.544+0000 7f76fe796740 1 -- 172.21.15.94:0/3258689201 wait complete. 2024-09-19T04:19:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.545+0000 7f76fe796740 20 client.15975 trim_cache size 0 max 16384 2024-09-19T04:19:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.575+0000 7f76fe796740 1 Processor -- start 2024-09-19T04:19:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.575+0000 7f76fe796740 1 -- start start 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fe796740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51cb60 0x56401f523c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fe796740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 0x56401f51f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fe796740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f527d40 0x56401f53f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fe796740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b6960 con 0x56401f51cb60 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fe796740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b4230 con 0x56401f527d40 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fe796740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b2070 con 0x56401f51d480 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fa159640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51cb60 0x56401f523c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fa159640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51cb60 0x56401f523c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56080/0 (socket says 172.21.15.94:56080) 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76f9958640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 0x56401f51f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fa159640 1 -- 172.21.15.94:0/833791514 learned_addr learned my addr 172.21.15.94:0/833791514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.576+0000 7f76fa95a640 1 --2- 172.21.15.94:0/833791514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f527d40 0x56401f53f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76fa95a640 1 -- 172.21.15.94:0/833791514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 msgr2=0x56401f51f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76fa95a640 1 --2- 172.21.15.94:0/833791514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 0x56401f51f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76fa95a640 1 -- 172.21.15.94:0/833791514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51cb60 msgr2=0x56401f523c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76fa95a640 1 --2- 172.21.15.94:0/833791514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51cb60 0x56401f523c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76fa95a640 1 -- 172.21.15.94:0/833791514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401f53a640 con 0x56401f527d40 2024-09-19T04:19:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76fa159640 1 --2- 172.21.15.94:0/833791514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51cb60 0x56401f523c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76fa95a640 1 --2- 172.21.15.94:0/833791514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f527d40 0x56401f53f8d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f76ec026c00 tx=0x7f76ec009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95bb0160f9c56e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76dffff640 1 -- 172.21.15.94:0/833791514 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec03f020 con 0x56401f527d40 2024-09-19T04:19:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76dffff640 1 -- 172.21.15.94:0/833791514 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76ec024070 con 0x56401f527d40 2024-09-19T04:19:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.577+0000 7f76dffff640 1 -- 172.21.15.94:0/833791514 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec029670 con 0x56401f527d40 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.578+0000 7f76fe796740 1 -- 172.21.15.94:0/833791514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f527d40 msgr2=0x56401f53f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.578+0000 7f76fe796740 1 --2- 172.21.15.94:0/833791514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f527d40 0x56401f53f8d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f76ec026c00 tx=0x7f76ec009600 comp rx=0 tx=0).stop 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.578+0000 7f76fe796740 1 -- 172.21.15.94:0/833791514 shutdown_connections 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.578+0000 7f76fe796740 1 --2- 172.21.15.94:0/833791514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f527d40 0x56401f53f8d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.578+0000 7f76fe796740 1 --2- 172.21.15.94:0/833791514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f51d480 0x56401f51f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.578+0000 7f76fe796740 1 --2- 172.21.15.94:0/833791514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51cb60 0x56401f523c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.578+0000 7f76fe796740 1 -- 172.21.15.94:0/833791514 >> 172.21.15.94:0/833791514 conn(0x56401f5190f0 msgr2=0x56401f526c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.578+0000 7f76fe796740 1 -- 172.21.15.94:0/833791514 shutdown_connections 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.579+0000 7f76fe796740 1 -- 172.21.15.94:0/833791514 wait complete. 2024-09-19T04:19:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.579+0000 7f76fe796740 1 -- start start 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.579+0000 7f76fe796740 1 Processor -- start 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fe796740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f52b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fe796740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f52bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fa159640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f52b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fa159640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f52b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:60134/0 (socket says 172.21.15.94:60134) 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76f9958640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f52bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fa159640 1 -- 172.21.15.94:0/2770720561 learned_addr learned my addr 172.21.15.94:0/2770720561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fe796740 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 0x56401f52c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b26f0 con 0x56401f51d480 2024-09-19T04:19:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b3b60 con 0x56401f51cb60 2024-09-19T04:19:53.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b2dc0 con 0x56401f527d40 2024-09-19T04:19:53.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.580+0000 7f76fe796740 10 client.0 _ll_register_callbacks cb 0x7f76ec009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:53.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76fa95a640 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 0x56401f52c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76fa159640 1 -- 172.21.15.94:0/2770720561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 msgr2=0x56401f52c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:53.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76fa159640 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 0x56401f52c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76fa159640 1 -- 172.21.15.94:0/2770720561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 msgr2=0x56401f52bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:53.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76fa159640 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f52bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76fa159640 1 -- 172.21.15.94:0/2770720561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56401f5a1950 con 0x56401f51cb60 2024-09-19T04:19:53.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76fa159640 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f52b8a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x56401f53a840 tx=0x7f76f0004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c346816986b513d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:53.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:19:53.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76ddffb640 1 -- 172.21.15.94:0/2770720561 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0003d90 con 0x56401f51cb60 2024-09-19T04:19:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76ddffb640 1 -- 172.21.15.94:0/2770720561 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f76f001e410 con 0x56401f51cb60 2024-09-19T04:19:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.581+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ec024070 con 0x56401f51cb60 2024-09-19T04:19:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.582+0000 7f76ddffb640 1 -- 172.21.15.94:0/2770720561 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f76f002e050 con 0x56401f51cb60 2024-09-19T04:19:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.582+0000 7f76fe796740 10 client.25568 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.582+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56401f5a2cb0 con 0x56401f51cb60 2024-09-19T04:19:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.586+0000 7f76ddffb640 1 -- 172.21.15.94:0/2770720561 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76ec024070 con 0x56401f51cb60 2024-09-19T04:19:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.586+0000 7f76ddffb640 1 -- 172.21.15.94:0/2770720561 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f76f00109a0 con 0x56401f51cb60 2024-09-19T04:19:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.586+0000 7f76fe796740 10 client.25568 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.586+0000 7f76fe796740 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:53.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.586+0000 7f76fe796740 20 client.25568 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:53.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.586+0000 7f76fe796740 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56401f5a3400 0x56401f5a5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.586+0000 7f76fe796740 4 client.25568 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:19:53.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.586+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56401f489d00 con 0x56401f5a3400 2024-09-19T04:19:53.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.586+0000 7f76f9958640 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56401f5a3400 0x56401f5a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.587+0000 7f76ddffb640 10 client.25568 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.587+0000 7f76f9958640 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56401f5a3400 0x56401f5a5820 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bcc36c6890fc70b server_cookie=2080f2e8e193d558 in_seq=0 out_seq=0 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.588+0000 7f76ddffb640 1 -- 172.21.15.94:0/2770720561 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56401f489d00 con 0x56401f5a3400 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.588+0000 7f76ddffb640 10 client.25568 handle_command_reply: tid=0 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.588+0000 7f76fe796740 1 client.25568 shutdown 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.588+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 msgr2=0x56401f52b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.588+0000 7f76fe796740 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f52b8a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x56401f53a840 tx=0x7f76f0004410 comp rx=0 tx=0).stop 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 shutdown_connections 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76fe796740 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56401f5a3400 0x56401f5a5820 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76ddffb640 0 client.25568 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76fe796740 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56401f527d40 0x56401f52c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76fe796740 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56401f51d480 0x56401f52bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76fe796740 1 --2- 172.21.15.94:0/2770720561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56401f51cb60 0x56401f52b8a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 >> 172.21.15.94:0/2770720561 conn(0x56401f5190f0 msgr2=0x56401f524c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:53.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 shutdown_connections 2024-09-19T04:19:53.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76fe796740 1 -- 172.21.15.94:0/2770720561 wait complete. 2024-09-19T04:19:53.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.589+0000 7f76fe796740 20 client.25568 trim_cache size 0 max 16384 2024-09-19T04:19:53.592 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:19:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:19:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "5462ed9e-83fa-46fb-b762-558b01123944", 2024-09-19T04:19:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:19:53.593 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:19:53.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 -- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76e00a3430 msgr2=0x7f76e00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76e00a3430 0x7f76e00a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009fd0 tx=0x7f76ec009ac0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 -- 172.21.15.94:0/1029487523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76f41049c0 msgr2=0x7f76f4073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76f41049c0 0x7f76f4073db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009850 tx=0x7f76e4005e10 comp rx=0 tx=0).stop 2024-09-19T04:19:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 -- 172.21.15.94:0/1029487523 shutdown_connections 2024-09-19T04:19:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76e00a3430 0x7f76e00a58f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76f4109140 0x7f76f406c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76f41049c0 0x7f76f4073db0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009850 tx=0x7f76e4005e10 comp rx=0 tx=0).stop 2024-09-19T04:19:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 --2- 172.21.15.94:0/1029487523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f4104010 0x7f76f4073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 -- 172.21.15.94:0/1029487523 >> 172.21.15.94:0/1029487523 conn(0x7f76f40ffa50 msgr2=0x7f76f41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 -- 172.21.15.94:0/1029487523 shutdown_connections 2024-09-19T04:19:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:53.590+0000 7f76fc3bd640 1 -- 172.21.15.94:0/1029487523 wait complete. 2024-09-19T04:19:53.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:53 smithi112 ceph-mon[35051]: pgmap v149: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.85k op/s 2024-09-19T04:19:53.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:53 smithi094 ceph-mon[30972]: pgmap v149: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.85k op/s 2024-09-19T04:19:53.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:53 smithi104 ceph-mon[34145]: pgmap v149: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.85k op/s 2024-09-19T04:19:54.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:54 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:19:54.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:54 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:54 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:19:54.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:54 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:19:54.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:54 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:19:54.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:54 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:54 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:54 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:54 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:19:54.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:54 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:54 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:19:54.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:54 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:19:54.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:54 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:19:54.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:54 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:54 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:54 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:54 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:19:54.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:54 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:54 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:19:54.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:54 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:19:54.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:54 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:19:54.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:54 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:54 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:54.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:54 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:19:55.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:19:55.427 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 220.0 s, 1326.4 tps, lat 75.681 ms stddev 74.726 2024-09-19T04:19:55.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:55 smithi112 ceph-mon[35051]: pgmap v150: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-19T04:19:55.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:55 smithi094 ceph-mon[30972]: pgmap v150: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-19T04:19:55.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:55 smithi104 ceph-mon[34145]: pgmap v150: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-19T04:19:56.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.007+0000 7f714b658640 1 -- 172.21.15.94:0/429556312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71441027a0 msgr2=0x7f71441067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.007+0000 7f714b658640 1 --2- 172.21.15.94:0/429556312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71441027a0 0x7f71441067f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7134009860 tx=0x7f713402a340 comp rx=0 tx=0).stop 2024-09-19T04:19:56.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.007+0000 7f714b658640 1 -- 172.21.15.94:0/429556312 shutdown_connections 2024-09-19T04:19:56.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.007+0000 7f714b658640 1 --2- 172.21.15.94:0/429556312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7144106f20 0x7f714410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.008+0000 7f714b658640 1 --2- 172.21.15.94:0/429556312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71441027a0 0x7f71441067f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.008+0000 7f714b658640 1 --2- 172.21.15.94:0/429556312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7144101df0 0x7f71441021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.008+0000 7f714b658640 1 -- 172.21.15.94:0/429556312 >> 172.21.15.94:0/429556312 conn(0x7f71440fd8f0 msgr2=0x7f71440ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.009+0000 7f714b658640 1 -- 172.21.15.94:0/429556312 shutdown_connections 2024-09-19T04:19:56.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.009+0000 7f714b658640 1 -- 172.21.15.94:0/429556312 wait complete. 2024-09-19T04:19:56.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.009+0000 7f714b658640 1 Processor -- start 2024-09-19T04:19:56.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f714b658640 1 -- start start 2024-09-19T04:19:56.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f714b658640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7144101df0 0x7f7144198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f714b658640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71441027a0 0x7f71441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f714b658640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7144106f20 0x7f714419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f714b658640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7144111df0 con 0x7f7144101df0 2024-09-19T04:19:56.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f714b658640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7144111c70 con 0x7f71441027a0 2024-09-19T04:19:56.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f714b658640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7144111f70 con 0x7f7144106f20 2024-09-19T04:19:56.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f71493f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7144101df0 0x7f7144198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f71493f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7144101df0 0x7f7144198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56094/0 (socket says 172.21.15.94:56094) 2024-09-19T04:19:56.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.010+0000 7f71493f4640 1 -- 172.21.15.94:0/794792073 learned_addr learned my addr 172.21.15.94:0/794792073 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:56.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.011+0000 7f7148bf3640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71441027a0 0x7f71441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.011+0000 7f7149bf5640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7144106f20 0x7f714419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.011+0000 7f71493f4640 1 -- 172.21.15.94:0/794792073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7144106f20 msgr2=0x7f714419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.011+0000 7f71493f4640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7144106f20 0x7f714419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.011+0000 7f71493f4640 1 -- 172.21.15.94:0/794792073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71441027a0 msgr2=0x7f71441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.011+0000 7f71493f4640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71441027a0 0x7f71441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.011+0000 7f71493f4640 1 -- 172.21.15.94:0/794792073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714419d420 con 0x7f7144101df0 2024-09-19T04:19:56.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.011+0000 7f7148bf3640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71441027a0 0x7f71441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:56.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.011+0000 7f7149bf5640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7144106f20 0x7f714419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:56.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.012+0000 7f71493f4640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7144101df0 0x7f7144198470 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7140009c20 tx=0x7f714000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.012+0000 7f713a7fc640 1 -- 172.21.15.94:0/794792073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714001e070 con 0x7f7144101df0 2024-09-19T04:19:56.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.012+0000 7f714b658640 1 -- 172.21.15.94:0/794792073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714419d710 con 0x7f7144101df0 2024-09-19T04:19:56.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.012+0000 7f713a7fc640 1 -- 172.21.15.94:0/794792073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71400198c0 con 0x7f7144101df0 2024-09-19T04:19:56.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.012+0000 7f713a7fc640 1 -- 172.21.15.94:0/794792073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140018710 con 0x7f7144101df0 2024-09-19T04:19:56.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.012+0000 7f714b658640 1 -- 172.21.15.94:0/794792073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7144074420 con 0x7f7144101df0 2024-09-19T04:19:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.015+0000 7f713a7fc640 1 -- 172.21.15.94:0/794792073 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7140021070 con 0x7f7144101df0 2024-09-19T04:19:56.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.016+0000 7f714da31740 1 Processor -- start 2024-09-19T04:19:56.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.016+0000 7f713a7fc640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f71300a3430 0x7f71300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f714da31740 1 -- start start 2024-09-19T04:19:56.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f714da31740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x55601189f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f714da31740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189fdc0 0x5560118aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f714da31740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560118aa680 0x5560118aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f714da31740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5560118052e0 con 0x55601189fdc0 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f7149bf5640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f71300a3430 0x7f71300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f714da31740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5560117acce0 con 0x55601189f4a0 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f714da31740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5560118012a0 con 0x5560118aa680 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f71493f4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x55601189f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f71493f4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x55601189f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:60176/0 (socket says 172.21.15.94:60176) 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f71493f4640 1 -- 172.21.15.94:0/349520395 learned_addr learned my addr 172.21.15.94:0/349520395 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:56.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.017+0000 7f713a7fc640 1 -- 172.21.15.94:0/794792073 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f71400d34d0 con 0x7f7144101df0 2024-09-19T04:19:56.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.018+0000 7f71493f4640 1 -- 172.21.15.94:0/349520395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560118aa680 msgr2=0x5560118aca70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:19:56.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.018+0000 7f71493f4640 1 --2- 172.21.15.94:0/349520395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560118aa680 0x5560118aca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.018+0000 7f71493f4640 1 -- 172.21.15.94:0/349520395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189fdc0 msgr2=0x5560118aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.018+0000 7f71493f4640 1 --2- 172.21.15.94:0/349520395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189fdc0 0x5560118aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.018+0000 7f71493f4640 1 -- 172.21.15.94:0/349520395 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556011791440 con 0x55601189f4a0 2024-09-19T04:19:56.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.018+0000 7f71493f4640 1 --2- 172.21.15.94:0/349520395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x55601189f880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f71400a7ee0 tx=0x7f714009f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100c4452ea84da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.018+0000 7f712f7fe640 1 -- 172.21.15.94:0/349520395 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140028040 con 0x55601189f4a0 2024-09-19T04:19:56.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.018+0000 7f712f7fe640 1 -- 172.21.15.94:0/349520395 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71400a3b50 con 0x55601189f4a0 2024-09-19T04:19:56.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.018+0000 7f712f7fe640 1 -- 172.21.15.94:0/349520395 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140019a70 con 0x55601189f4a0 2024-09-19T04:19:56.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.019+0000 7f714da31740 1 -- 172.21.15.94:0/349520395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 msgr2=0x55601189f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.019+0000 7f714da31740 1 --2- 172.21.15.94:0/349520395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x55601189f880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f71400a7ee0 tx=0x7f714009f270 comp rx=0 tx=0).stop 2024-09-19T04:19:56.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.019+0000 7f7149bf5640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f71300a3430 0x7f71300a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55601189cdb0 tx=0x7f713c010040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.019+0000 7f714da31740 1 -- 172.21.15.94:0/349520395 shutdown_connections 2024-09-19T04:19:56.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.019+0000 7f714da31740 1 --2- 172.21.15.94:0/349520395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560118aa680 0x5560118aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.019+0000 7f714da31740 1 --2- 172.21.15.94:0/349520395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189fdc0 0x5560118aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.019+0000 7f714da31740 1 --2- 172.21.15.94:0/349520395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x55601189f880 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.019+0000 7f714da31740 1 -- 172.21.15.94:0/349520395 >> 172.21.15.94:0/349520395 conn(0x55601189ba30 msgr2=0x55601189be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.020+0000 7f714da31740 1 -- 172.21.15.94:0/349520395 shutdown_connections 2024-09-19T04:19:56.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.020+0000 7f714da31740 1 -- 172.21.15.94:0/349520395 wait complete. 2024-09-19T04:19:56.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.020+0000 7f714da31740 1 -- start start 2024-09-19T04:19:56.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.020+0000 7f714da31740 1 Processor -- start 2024-09-19T04:19:56.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.021+0000 7f714da31740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189f4a0 0x5560118aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.021+0000 7f714da31740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.021+0000 7f714da31740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560118aa680 0x5560118a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.021+0000 7f714da31740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5560118ad250 con 0x55601189f4a0 2024-09-19T04:19:56.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.021+0000 7f714da31740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5560117c0050 con 0x5560118aa680 2024-09-19T04:19:56.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.021+0000 7f714da31740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5560118ad3d0 con 0x55601189fdc0 2024-09-19T04:19:56.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.021+0000 7f714da31740 10 client.0 _ll_register_callbacks cb 0x556011800160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:56.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.021+0000 7f71493f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189f4a0 0x5560118aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.022+0000 7f7149bf5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560118aa680 0x5560118a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.022+0000 7f71493f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189f4a0 0x5560118aa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56104/0 (socket says 172.21.15.94:56104) 2024-09-19T04:19:56.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.022+0000 7f71493f4640 1 -- 172.21.15.94:0/3983240925 learned_addr learned my addr 172.21.15.94:0/3983240925 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:56.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.022+0000 7f7148bf3640 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.022+0000 7f71493f4640 1 -- 172.21.15.94:0/3983240925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 msgr2=0x5560118a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.022+0000 7f71493f4640 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118a7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.022+0000 7f71493f4640 1 -- 172.21.15.94:0/3983240925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560118aa680 msgr2=0x5560118a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.022+0000 7f71493f4640 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560118aa680 0x5560118a7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.022+0000 7f71493f4640 1 -- 172.21.15.94:0/3983240925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560115763c0 con 0x55601189f4a0 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.023+0000 7f71493f4640 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189f4a0 0x5560118aa070 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f714009bcb0 tx=0x7f71400a8480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6395bdf869808014 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.023+0000 7f712cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.023+0000 7f712cff9640 1 -- 172.21.15.94:0/3983240925 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714005c6f0 con 0x55601189f4a0 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.023+0000 7f712cff9640 1 -- 172.21.15.94:0/3983240925 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f714005c890 con 0x55601189f4a0 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.023+0000 7f714da31740 1 -- 172.21.15.94:0/3983240925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71400a3b50 con 0x55601189f4a0 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.023+0000 7f712cff9640 1 -- 172.21.15.94:0/3983240925 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7140059280 con 0x55601189f4a0 2024-09-19T04:19:56.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.023+0000 7f714da31740 10 client.15990 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:56.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.023+0000 7f714da31740 1 -- 172.21.15.94:0/3983240925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560118c14c0 con 0x55601189f4a0 2024-09-19T04:19:56.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.024+0000 7f712cff9640 1 -- 172.21.15.94:0/3983240925 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71400a3b50 con 0x55601189f4a0 2024-09-19T04:19:56.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.024+0000 7f712cff9640 1 -- 172.21.15.94:0/3983240925 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f7140019410 con 0x55601189f4a0 2024-09-19T04:19:56.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.024+0000 7f714da31740 10 client.15990 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:56.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.024+0000 7f714da31740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:56.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.024+0000 7f714da31740 20 client.15990 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:56.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.024+0000 7f714da31740 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5560118c2a10 0x5560118c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.024+0000 7f714da31740 4 client.15990 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:56.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.024+0000 7f714da31740 1 -- 172.21.15.94:0/3983240925 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556011812850 con 0x5560118c2a10 2024-09-19T04:19:56.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.025+0000 7f7148bf3640 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5560118c2a10 0x5560118c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.026+0000 7f712cff9640 10 client.15990 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:56.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.026+0000 7f7148bf3640 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5560118c2a10 0x5560118c4e00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8ee3343591a2d7d server_cookie=4522e61357d38801 in_seq=0 out_seq=0 2024-09-19T04:19:56.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.027+0000 7f712cff9640 1 -- 172.21.15.94:0/3983240925 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556011812850 con 0x5560118c2a10 2024-09-19T04:19:56.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.027+0000 7f712cff9640 10 client.15990 handle_command_reply: tid=0 2024-09-19T04:19:56.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.027+0000 7f714da31740 1 client.15990 shutdown 2024-09-19T04:19:56.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.027+0000 7f714da31740 1 -- 172.21.15.94:0/3983240925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189f4a0 msgr2=0x5560118aa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.027+0000 7f714da31740 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189f4a0 0x5560118aa070 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f714009bcb0 tx=0x7f71400a8480 comp rx=0 tx=0).stop 2024-09-19T04:19:56.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.027+0000 7f714da31740 1 -- 172.21.15.94:0/3983240925 shutdown_connections 2024-09-19T04:19:56.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.027+0000 7f714da31740 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5560118c2a10 0x5560118c4e00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.027+0000 7f714da31740 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560118aa680 0x5560118a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.028+0000 7f714da31740 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.028+0000 7f714da31740 1 --2- 172.21.15.94:0/3983240925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55601189f4a0 0x5560118aa070 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.028+0000 7f714da31740 1 -- 172.21.15.94:0/3983240925 >> 172.21.15.94:0/3983240925 conn(0x55601189ba30 msgr2=0x5560118a1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.028+0000 7f712cff9640 0 client.15990 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:56.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.028+0000 7f714da31740 1 -- 172.21.15.94:0/3983240925 shutdown_connections 2024-09-19T04:19:56.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.028+0000 7f714da31740 1 -- 172.21.15.94:0/3983240925 wait complete. 2024-09-19T04:19:56.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.028+0000 7f714da31740 20 client.15990 trim_cache size 0 max 16384 2024-09-19T04:19:56.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.057+0000 7f714da31740 1 Processor -- start 2024-09-19T04:19:56.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.057+0000 7f714da31740 1 -- start start 2024-09-19T04:19:56.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f714da31740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x5560118bed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f714da31740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118a9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f71493f4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x5560118bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f714da31740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560118aa680 0x5560118ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f714da31740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71441adf90 con 0x5560118aa680 2024-09-19T04:19:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f71493f4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x5560118bed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:60190/0 (socket says 172.21.15.94:60190) 2024-09-19T04:19:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f714da31740 1 -- 172.21.15.94:0/961650300 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f71441ab860 con 0x55601189f4a0 2024-09-19T04:19:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f71493f4640 1 -- 172.21.15.94:0/961650300 learned_addr learned my addr 172.21.15.94:0/961650300 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f714da31740 1 -- 172.21.15.94:0/961650300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71441a96b0 con 0x55601189fdc0 2024-09-19T04:19:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f7149bf5640 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560118aa680 0x5560118ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f7148bf3640 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f71493f4640 1 -- 172.21.15.94:0/961650300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 msgr2=0x5560118a9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f71493f4640 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118a9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f71493f4640 1 -- 172.21.15.94:0/961650300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560118aa680 msgr2=0x5560118ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f71493f4640 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560118aa680 0x5560118ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f71493f4640 1 -- 172.21.15.94:0/961650300 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601189fbf0 con 0x55601189f4a0 2024-09-19T04:19:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.058+0000 7f7148bf3640 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.059+0000 7f71493f4640 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x5560118bed40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71400d1210 tx=0x7f714009f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edef9a54633fa020 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.059+0000 7f712effd640 1 -- 172.21.15.94:0/961650300 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140019410 con 0x55601189f4a0 2024-09-19T04:19:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.059+0000 7f712effd640 1 -- 172.21.15.94:0/961650300 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71400195b0 con 0x55601189f4a0 2024-09-19T04:19:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.059+0000 7f712effd640 1 -- 172.21.15.94:0/961650300 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714009b5a0 con 0x55601189f4a0 2024-09-19T04:19:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.060+0000 7f714da31740 1 -- 172.21.15.94:0/961650300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 msgr2=0x5560118bed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.060+0000 7f714da31740 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x5560118bed40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71400d1210 tx=0x7f714009f270 comp rx=0 tx=0).stop 2024-09-19T04:19:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.060+0000 7f714da31740 1 -- 172.21.15.94:0/961650300 shutdown_connections 2024-09-19T04:19:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.060+0000 7f714da31740 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560118aa680 0x5560118ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.060+0000 7f714da31740 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.060+0000 7f714da31740 1 --2- 172.21.15.94:0/961650300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601189f4a0 0x5560118bed40 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71400d1210 tx=0x7f714009f270 comp rx=0 tx=0).stop 2024-09-19T04:19:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.060+0000 7f714da31740 1 -- 172.21.15.94:0/961650300 >> 172.21.15.94:0/961650300 conn(0x55601189ba30 msgr2=0x5560118a8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.061+0000 7f714da31740 1 -- 172.21.15.94:0/961650300 shutdown_connections 2024-09-19T04:19:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.061+0000 7f714da31740 1 -- 172.21.15.94:0/961650300 wait complete. 2024-09-19T04:19:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.061+0000 7f714da31740 1 -- start start 2024-09-19T04:19:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.062+0000 7f714da31740 1 Processor -- start 2024-09-19T04:19:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.062+0000 7f714da31740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118b6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f714da31740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560118aa680 0x5560118b6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f71493f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f71493f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118b6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:34958/0 (socket says 172.21.15.94:34958) 2024-09-19T04:19:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f714da31740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556011929620 0x55601192ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f71493f4640 1 -- 172.21.15.94:0/556713067 learned_addr learned my addr 172.21.15.94:0/556713067 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:19:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71441a9d20 con 0x5560118aa680 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f71441ab190 con 0x556011929620 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71441aa3f0 con 0x55601189fdc0 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f714da31740 10 client.0 _ll_register_callbacks cb 0x7f714009f390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.063+0000 7f7149bf5640 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556011929620 0x55601192ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.064+0000 7f71493f4640 1 -- 172.21.15.94:0/556713067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556011929620 msgr2=0x55601192ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.064+0000 7f71493f4640 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556011929620 0x55601192ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.064+0000 7f71493f4640 1 -- 172.21.15.94:0/556713067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560118aa680 msgr2=0x5560118b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.064+0000 7f71493f4640 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560118aa680 0x5560118b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.064+0000 7f71493f4640 1 -- 172.21.15.94:0/556713067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601192bf50 con 0x55601189fdc0 2024-09-19T04:19:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.064+0000 7f71493f4640 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118b6410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f714009bcb0 tx=0x7f714009baf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d661fc4e1de829 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f712cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:19:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f712cff9640 1 -- 172.21.15.94:0/556713067 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140019410 con 0x55601189fdc0 2024-09-19T04:19:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f712cff9640 1 -- 172.21.15.94:0/556713067 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f714009b5a0 con 0x55601189fdc0 2024-09-19T04:19:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71400195b0 con 0x55601189fdc0 2024-09-19T04:19:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f712cff9640 1 -- 172.21.15.94:0/556713067 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f71400a8410 con 0x55601189fdc0 2024-09-19T04:19:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f714da31740 10 client.25555 fetch_fsmap learned FSMap version 34 2024-09-19T04:19:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55601192bf50 con 0x55601189fdc0 2024-09-19T04:19:56.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f712cff9640 1 -- 172.21.15.94:0/556713067 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71400a3b50 con 0x55601189fdc0 2024-09-19T04:19:56.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f712cff9640 1 -- 172.21.15.94:0/556713067 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f71400a4ce0 con 0x55601189fdc0 2024-09-19T04:19:56.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f714da31740 10 client.25555 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:19:56.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f714da31740 10 client.25555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:19:56.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.065+0000 7f714da31740 20 client.25555 populate_metadata read hostname 'smithi094' 2024-09-19T04:19:56.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.066+0000 7f714da31740 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55601192d4b0 0x55601192f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.066+0000 7f714da31740 4 client.25555 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:19:56.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.066+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f71400195b0 con 0x55601192d4b0 2024-09-19T04:19:56.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.066+0000 7f7148bf3640 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55601192d4b0 0x55601192f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.066+0000 7f712cff9640 10 client.25555 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:56.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.066+0000 7f7148bf3640 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55601192d4b0 0x55601192f8a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a56007b9f43b41 server_cookie=da6b8597287494d4 in_seq=0 out_seq=0 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f712cff9640 1 -- 172.21.15.94:0/556713067 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f71400195b0 con 0x55601192d4b0 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f712cff9640 10 client.25555 handle_command_reply: tid=0 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 client.25555 shutdown 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 msgr2=0x5560118b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118b6410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f714009bcb0 tx=0x7f714009baf0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 shutdown_connections 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55601192d4b0 0x55601192f8a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556011929620 0x55601192ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f712cff9640 0 client.25555 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560118aa680 0x5560118b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 --2- 172.21.15.94:0/556713067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601189fdc0 0x5560118b6410 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 >> 172.21.15.94:0/556713067 conn(0x55601189ba30 msgr2=0x5560118a1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 shutdown_connections 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.067+0000 7f714da31740 1 -- 172.21.15.94:0/556713067 wait complete. 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.068+0000 7f714da31740 20 client.25555 trim_cache size 0 max 16384 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stdout: "5462ed9e-83fa-46fb-b762-558b01123944": { 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "5462ed9e-83fa-46fb-b762-558b01123944", 2024-09-19T04:19:56.072 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:19:56.072 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:19:56.072 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:19:56.072 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 -- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f71300a3430 msgr2=0x7f71300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f71300a3430 0x7f71300a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55601189cdb0 tx=0x7f713c010040 comp rx=0 tx=0).stop 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 -- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7144101df0 msgr2=0x7f7144198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7144101df0 0x7f7144198470 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7140009c20 tx=0x7f714000b690 comp rx=0 tx=0).stop 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 -- 172.21.15.94:0/794792073 shutdown_connections 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f71300a3430 0x7f71300a58f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7144106f20 0x7f714419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71441027a0 0x7f71441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 --2- 172.21.15.94:0/794792073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7144101df0 0x7f7144198470 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.069+0000 7f714b658640 1 -- 172.21.15.94:0/794792073 >> 172.21.15.94:0/794792073 conn(0x7f71440fd8f0 msgr2=0x7f71440fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.070+0000 7f714b658640 1 -- 172.21.15.94:0/794792073 shutdown_connections 2024-09-19T04:19:56.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:19:56.070+0000 7f714b658640 1 -- 172.21.15.94:0/794792073 wait complete. 2024-09-19T04:19:56.856 INFO:tasks.cephfs.filesystem:scrub status for tag:5462ed9e-83fa-46fb-b762-558b01123944 - {'path': '/', 'tag': '5462ed9e-83fa-46fb-b762-558b01123944', 'options': 'recursive,force'} 2024-09-19T04:19:57.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:57 smithi112 ceph-mon[35051]: pgmap v151: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 16 MiB/s wr, 1.74k op/s 2024-09-19T04:19:57.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:57 smithi094 ceph-mon[30972]: pgmap v151: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 16 MiB/s wr, 1.74k op/s 2024-09-19T04:19:57.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:57 smithi104 ceph-mon[34145]: pgmap v151: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 16 MiB/s wr, 1.74k op/s 2024-09-19T04:19:59.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:19:59 smithi112 ceph-mon[35051]: pgmap v152: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-09-19T04:19:59.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:19:59 smithi094 ceph-mon[30972]: pgmap v152: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-09-19T04:19:59.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:19:59 smithi104 ceph-mon[34145]: pgmap v152: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-09-19T04:20:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 225.0 s, 1605.6 tps, lat 62.292 ms stddev 16.403 2024-09-19T04:20:00.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:00 smithi112 ceph-mon[35051]: overall HEALTH_OK 2024-09-19T04:20:00.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:00 smithi094 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T04:20:00.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:00 smithi104 ceph-mon[34145]: overall HEALTH_OK 2024-09-19T04:20:01.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:01 smithi112 ceph-mon[35051]: pgmap v153: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-09-19T04:20:01.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:01 smithi094 ceph-mon[30972]: pgmap v153: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-09-19T04:20:01.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:01 smithi104 ceph-mon[34145]: pgmap v153: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-09-19T04:20:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:02 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:20:02.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:02 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:20:02.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:02 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:20:02.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:02 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:20:02.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:02 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:20:02.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:02 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:20:03.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:03 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:20:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:03 smithi112 ceph-mon[35051]: pgmap v154: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 17 MiB/s wr, 1.87k op/s 2024-09-19T04:20:03.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:03 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:20:03.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:03 smithi094 ceph-mon[30972]: pgmap v154: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 17 MiB/s wr, 1.87k op/s 2024-09-19T04:20:03.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:03 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:20:03.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:03 smithi104 ceph-mon[34145]: pgmap v154: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 17 MiB/s wr, 1.87k op/s 2024-09-19T04:20:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 230.0 s, 1687.2 tps, lat 59.209 ms stddev 18.140 2024-09-19T04:20:05.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:05 smithi112 ceph-mon[35051]: pgmap v155: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:20:05.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:05 smithi094 ceph-mon[30972]: pgmap v155: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:20:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:05 smithi104 ceph-mon[34145]: pgmap v155: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-19T04:20:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:07 smithi112 ceph-mon[35051]: pgmap v156: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 1.94k op/s 2024-09-19T04:20:07.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:07 smithi094 ceph-mon[30972]: pgmap v156: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 1.94k op/s 2024-09-19T04:20:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:07 smithi104 ceph-mon[34145]: pgmap v156: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 1.94k op/s 2024-09-19T04:20:09.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:09 smithi112 ceph-mon[35051]: pgmap v157: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:20:09.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:09 smithi094 ceph-mon[30972]: pgmap v157: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:20:09.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:09 smithi104 ceph-mon[34145]: pgmap v157: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:20:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 235.0 s, 1585.0 tps, lat 63.114 ms stddev 16.013 2024-09-19T04:20:11.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:11 smithi112 ceph-mon[35051]: pgmap v158: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:20:11.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:11 smithi094 ceph-mon[30972]: pgmap v158: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:20:11.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:11 smithi104 ceph-mon[34145]: pgmap v158: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:20:13.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:13 smithi112 ceph-mon[35051]: pgmap v159: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 2.06k op/s 2024-09-19T04:20:13.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:13 smithi094 ceph-mon[30972]: pgmap v159: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 2.06k op/s 2024-09-19T04:20:13.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:13 smithi104 ceph-mon[34145]: pgmap v159: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 2.06k op/s 2024-09-19T04:20:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 240.0 s, 1572.4 tps, lat 62.611 ms stddev 15.952 2024-09-19T04:20:15.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:15 smithi112 ceph-mon[35051]: pgmap v160: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:20:15.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:15 smithi094 ceph-mon[30972]: pgmap v160: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:20:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:15 smithi104 ceph-mon[34145]: pgmap v160: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-19T04:20:16.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:16 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:16.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:16 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:16.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:16 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:16.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:16 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:20:16.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:16 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:16.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:16 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:16.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:16 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:16.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:16 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:20:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:16 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:16 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:16 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:16 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:20:17.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:17 smithi112 ceph-mon[35051]: pgmap v161: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 2.02k op/s 2024-09-19T04:20:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:17 smithi094 ceph-mon[30972]: pgmap v161: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 2.02k op/s 2024-09-19T04:20:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:17 smithi104 ceph-mon[34145]: pgmap v161: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 2.02k op/s 2024-09-19T04:20:19.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:19 smithi112 ceph-mon[35051]: pgmap v162: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:20:19.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:19 smithi094 ceph-mon[30972]: pgmap v162: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:20:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:19 smithi104 ceph-mon[34145]: pgmap v162: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:20:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 245.0 s, 1429.0 tps, lat 70.999 ms stddev 65.462 2024-09-19T04:20:21.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:21 smithi112 ceph-mon[35051]: pgmap v163: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:20:21.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:21 smithi094 ceph-mon[30972]: pgmap v163: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:20:21.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:21 smithi104 ceph-mon[34145]: pgmap v163: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-09-19T04:20:23.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:23 smithi112 ceph-mon[35051]: pgmap v164: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 1.87k op/s 2024-09-19T04:20:23.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:23 smithi094 ceph-mon[30972]: pgmap v164: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 1.87k op/s 2024-09-19T04:20:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:23 smithi104 ceph-mon[34145]: pgmap v164: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 1.87k op/s 2024-09-19T04:20:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 250.0 s, 1597.6 tps, lat 62.610 ms stddev 16.054 2024-09-19T04:20:25.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:25 smithi112 ceph-mon[35051]: pgmap v165: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-09-19T04:20:25.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:25 smithi094 ceph-mon[30972]: pgmap v165: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-09-19T04:20:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:25 smithi104 ceph-mon[34145]: pgmap v165: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-09-19T04:20:26.857 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:20:27.282 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:27.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:27 smithi094 ceph-mon[30972]: pgmap v166: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 1.86k op/s 2024-09-19T04:20:27.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:27 smithi112 ceph-mon[35051]: pgmap v166: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 1.86k op/s 2024-09-19T04:20:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.834+0000 7fcc6a83c640 1 -- 172.21.15.94:0/4275581635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc64105de0 msgr2=0x7fcc6410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.834+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/4275581635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc64105de0 0x7fcc6410d960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009be0 tx=0x7fcc5c02a6d0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.834+0000 7fcc6a83c640 1 -- 172.21.15.94:0/4275581635 shutdown_connections 2024-09-19T04:20:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.834+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/4275581635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc64105de0 0x7fcc6410d960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.834+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/4275581635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc64101e20 0x7fcc641056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.834+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/4275581635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc64101470 0x7fcc64101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.834+0000 7fcc6a83c640 1 -- 172.21.15.94:0/4275581635 >> 172.21.15.94:0/4275581635 conn(0x7fcc64077230 msgr2=0x7fcc640fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.834+0000 7fcc6a83c640 1 -- 172.21.15.94:0/4275581635 shutdown_connections 2024-09-19T04:20:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.834+0000 7fcc6a83c640 1 -- 172.21.15.94:0/4275581635 wait complete. 2024-09-19T04:20:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.835+0000 7fcc6a83c640 1 Processor -- start 2024-09-19T04:20:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.835+0000 7fcc6a83c640 1 -- start start 2024-09-19T04:20:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc6a83c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc64101470 0x7fcc641960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc6a83c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc64101e20 0x7fcc64196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc6a83c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc64105de0 0x7fcc6419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc6a83c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410fbf0 con 0x7fcc64101e20 2024-09-19T04:20:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc6a83c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410fa70 con 0x7fcc64101470 2024-09-19T04:20:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc6a83c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410fd70 con 0x7fcc64105de0 2024-09-19T04:20:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc64101e20 0x7fcc64196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc64101e20 0x7fcc64196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34234/0 (socket says 172.21.15.94:34234) 2024-09-19T04:20:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc637fe640 1 -- 172.21.15.94:0/1280125098 learned_addr learned my addr 172.21.15.94:0/1280125098 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc63fff640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc64101470 0x7fcc641960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.836+0000 7fcc68dd9640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc64105de0 0x7fcc6419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.837+0000 7fcc637fe640 1 -- 172.21.15.94:0/1280125098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc64105de0 msgr2=0x7fcc6419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.837+0000 7fcc637fe640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc64105de0 0x7fcc6419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.837+0000 7fcc637fe640 1 -- 172.21.15.94:0/1280125098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc64101470 msgr2=0x7fcc641960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.837+0000 7fcc637fe640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc64101470 0x7fcc641960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.837+0000 7fcc637fe640 1 -- 172.21.15.94:0/1280125098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6419b0a0 con 0x7fcc64101e20 2024-09-19T04:20:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.837+0000 7fcc637fe640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc64101e20 0x7fcc64196630 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e9c0 tx=0x7fcc5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.837+0000 7fcc617fa640 1 -- 172.21.15.94:0/1280125098 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800cde0 con 0x7fcc64101e20 2024-09-19T04:20:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.837+0000 7fcc617fa640 1 -- 172.21.15.94:0/1280125098 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc580137d0 con 0x7fcc64101e20 2024-09-19T04:20:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.837+0000 7fcc617fa640 1 -- 172.21.15.94:0/1280125098 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58003890 con 0x7fcc64101e20 2024-09-19T04:20:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.838+0000 7fcc6a83c640 1 -- 172.21.15.94:0/1280125098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6419b390 con 0x7fcc64101e20 2024-09-19T04:20:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.839+0000 7fcc6a83c640 1 -- 172.21.15.94:0/1280125098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc641a2c70 con 0x7fcc64101e20 2024-09-19T04:20:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.841+0000 7fcc617fa640 1 -- 172.21.15.94:0/1280125098 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fcc58022020 con 0x7fcc64101e20 2024-09-19T04:20:27.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.842+0000 7fcc6cc15740 1 Processor -- start 2024-09-19T04:20:27.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.842+0000 7fcc6cc15740 1 -- start start 2024-09-19T04:20:27.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.842+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x56234179c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417a6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc6cc15740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5623416f92e0 con 0x56234179bcb0 2024-09-19T04:20:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc6cc15740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623416a0ce0 con 0x5623417a6e90 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc6cc15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623416f52a0 con 0x56234179c5d0 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc637fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc68dd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x56234179c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc68dd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x56234179c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34236/0 (socket says 172.21.15.94:34236) 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc68dd9640 1 -- 172.21.15.94:0/3673434609 learned_addr learned my addr 172.21.15.94:0/3673434609 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc63fff640 1 --2- 172.21.15.94:0/3673434609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.843+0000 7fcc617fa640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc4c0a3580 0x7fcc4c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.844+0000 7fcc68dd9640 1 -- 172.21.15.94:0/3673434609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 msgr2=0x5623417a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.844+0000 7fcc68dd9640 1 --2- 172.21.15.94:0/3673434609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.844+0000 7fcc68dd9640 1 -- 172.21.15.94:0/3673434609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 msgr2=0x5623417a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.844+0000 7fcc68dd9640 1 --2- 172.21.15.94:0/3673434609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.844+0000 7fcc68dd9640 1 -- 172.21.15.94:0/3673434609 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562341685440 con 0x56234179bcb0 2024-09-19T04:20:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.844+0000 7fcc617fa640 1 -- 172.21.15.94:0/1280125098 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fcc580d39d0 con 0x7fcc64101e20 2024-09-19T04:20:27.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.844+0000 7fcc637fe640 1 --2- 172.21.15.94:0/3673434609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:27.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.845+0000 7fcc68dd9640 1 --2- 172.21.15.94:0/3673434609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x56234179c090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009be0 tx=0x7fcc5c030480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d38bf0b5c2c783a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.845+0000 7fcc68dd9640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc4c0a3580 0x7fcc4c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.845+0000 7fcc4a7fc640 1 -- 172.21.15.94:0/3673434609 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c041030 con 0x56234179bcb0 2024-09-19T04:20:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.845+0000 7fcc4a7fc640 1 -- 172.21.15.94:0/3673434609 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc5c037040 con 0x56234179bcb0 2024-09-19T04:20:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.845+0000 7fcc4a7fc640 1 -- 172.21.15.94:0/3673434609 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c036040 con 0x56234179bcb0 2024-09-19T04:20:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.846+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3673434609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 msgr2=0x56234179c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.846+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3673434609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x56234179c090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009be0 tx=0x7fcc5c030480 comp rx=0 tx=0).stop 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.847+0000 7fcc68dd9640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc4c0a3580 0x7fcc4c0a5a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c044000 tx=0x7fcc5c004380 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.847+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3673434609 shutdown_connections 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.847+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3673434609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.847+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3673434609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.847+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3673434609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x56234179c090 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.847+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3673434609 >> 172.21.15.94:0/3673434609 conn(0x562341798240 msgr2=0x562341798620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.847+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3673434609 shutdown_connections 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.847+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3673434609 wait complete. 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.847+0000 7fcc6cc15740 1 -- start start 2024-09-19T04:20:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.848+0000 7fcc6cc15740 1 Processor -- start 2024-09-19T04:20:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.848+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179bcb0 0x5623417a6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.848+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179c5d0 0x5623417a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.848+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.848+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179bcb0 0x5623417a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.848+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179bcb0 0x5623417a6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:52046/0 (socket says 172.21.15.94:52046) 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179c5d0 0x5623417a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179c5d0 0x5623417a3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34244/0 (socket says 172.21.15.94:34244) 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc63fff640 1 -- 172.21.15.94:0/2638327638 learned_addr learned my addr 172.21.15.94:0/2638327638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5623417a9a60 con 0x56234179c5d0 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623416b4050 con 0x5623417a6e90 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc68dd9640 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623417a9be0 con 0x56234179bcb0 2024-09-19T04:20:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc6cc15740 10 client.0 _ll_register_callbacks cb 0x5623416f4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc637fe640 1 -- 172.21.15.94:0/2638327638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179bcb0 msgr2=0x5623417a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc637fe640 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179bcb0 0x5623417a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc637fe640 1 -- 172.21.15.94:0/2638327638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 msgr2=0x5623417a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc637fe640 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.849+0000 7fcc637fe640 1 -- 172.21.15.94:0/2638327638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234146a3c0 con 0x56234179c5d0 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.850+0000 7fcc637fe640 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179c5d0 0x5623417a3a40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcc580d11c0 tx=0x7fcc58013270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b64ef53b1b80f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.850+0000 7fcc437fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.850+0000 7fcc437fe640 1 -- 172.21.15.94:0/2638327638 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5809f6b0 con 0x56234179c5d0 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.851+0000 7fcc437fe640 1 -- 172.21.15.94:0/2638327638 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fcc58011110 con 0x56234179c5d0 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.851+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc5c037040 con 0x56234179c5d0 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.851+0000 7fcc437fe640 1 -- 172.21.15.94:0/2638327638 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcc58013d10 con 0x56234179c5d0 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.851+0000 7fcc6cc15740 10 client.16005 fetch_fsmap learned FSMap version 34 2024-09-19T04:20:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.851+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5623417b53b0 con 0x56234179c5d0 2024-09-19T04:20:27.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:27 smithi104 ceph-mon[34145]: pgmap v166: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 1.86k op/s 2024-09-19T04:20:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.852+0000 7fcc437fe640 1 -- 172.21.15.94:0/2638327638 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc580a0450 con 0x56234179c5d0 2024-09-19T04:20:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.852+0000 7fcc437fe640 1 -- 172.21.15.94:0/2638327638 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fcc5809ffb0 con 0x56234179c5d0 2024-09-19T04:20:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.852+0000 7fcc6cc15740 10 client.16005 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:20:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.852+0000 7fcc6cc15740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.852+0000 7fcc6cc15740 20 client.16005 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.852+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5623417b6c30 0x5623417b9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.852+0000 7fcc6cc15740 4 client.16005 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.852+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcc5c037040 con 0x5623417b6c30 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.853+0000 7fcc63fff640 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5623417b6c30 0x5623417b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.853+0000 7fcc63fff640 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5623417b6c30 0x5623417b9040 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69cf224b146dff75 server_cookie=7ad2f887762af3e6 in_seq=0 out_seq=0 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.853+0000 7fcc437fe640 10 client.16005 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.854+0000 7fcc437fe640 1 -- 172.21.15.94:0/2638327638 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcc5c037040 con 0x5623417b6c30 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.854+0000 7fcc437fe640 10 client.16005 handle_command_reply: tid=0 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.854+0000 7fcc6cc15740 1 client.16005 shutdown 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179c5d0 msgr2=0x5623417a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179c5d0 0x5623417a3a40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcc580d11c0 tx=0x7fcc58013270 comp rx=0 tx=0).stop 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 shutdown_connections 2024-09-19T04:20:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5623417b6c30 0x5623417b9040 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc437fe640 0 client.16005 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179c5d0 0x5623417a3a40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/2638327638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179bcb0 0x5623417a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 >> 172.21.15.94:0/2638327638 conn(0x562341798240 msgr2=0x56234178eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 shutdown_connections 2024-09-19T04:20:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 1 -- 172.21.15.94:0/2638327638 wait complete. 2024-09-19T04:20:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.855+0000 7fcc6cc15740 20 client.16005 trim_cache size 0 max 16384 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.884+0000 7fcc6cc15740 1 Processor -- start 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.885+0000 7fcc6cc15740 1 -- start start 2024-09-19T04:20:27.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.885+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x5623417a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.885+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.885+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417bee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.885+0000 7fcc6cc15740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641adf30 con 0x56234179bcb0 2024-09-19T04:20:27.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.885+0000 7fcc6cc15740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641ab800 con 0x5623417a6e90 2024-09-19T04:20:27.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.885+0000 7fcc6cc15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641a9650 con 0x56234179c5d0 2024-09-19T04:20:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.886+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x5623417a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.886+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x5623417a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34252/0 (socket says 172.21.15.94:34252) 2024-09-19T04:20:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.886+0000 7fcc63fff640 1 -- 172.21.15.94:0/834461643 learned_addr learned my addr 172.21.15.94:0/834461643 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.887+0000 7fcc63fff640 1 -- 172.21.15.94:0/834461643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 msgr2=0x5623417be950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:20:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.887+0000 7fcc63fff640 1 --2- 172.21.15.94:0/834461643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417be950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.887+0000 7fcc63fff640 1 -- 172.21.15.94:0/834461643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 msgr2=0x5623417bee90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:20:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.887+0000 7fcc63fff640 1 --2- 172.21.15.94:0/834461643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417bee90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.887+0000 7fcc63fff640 1 -- 172.21.15.94:0/834461643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623417a29f0 con 0x56234179bcb0 2024-09-19T04:20:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.887+0000 7fcc63fff640 1 --2- 172.21.15.94:0/834461643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x5623417a2310 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400bee0 tx=0x7fcc5400d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec04c595aa8061f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.888+0000 7fcc49ffb640 1 -- 172.21.15.94:0/834461643 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54025030 con 0x56234179bcb0 2024-09-19T04:20:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.888+0000 7fcc49ffb640 1 -- 172.21.15.94:0/834461643 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc54003b90 con 0x56234179bcb0 2024-09-19T04:20:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.888+0000 7fcc49ffb640 1 -- 172.21.15.94:0/834461643 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54017460 con 0x56234179bcb0 2024-09-19T04:20:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.889+0000 7fcc6cc15740 1 -- 172.21.15.94:0/834461643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 msgr2=0x5623417a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.889+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/834461643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x5623417a2310 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400bee0 tx=0x7fcc5400d040 comp rx=0 tx=0).stop 2024-09-19T04:20:27.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.890+0000 7fcc6cc15740 1 -- 172.21.15.94:0/834461643 shutdown_connections 2024-09-19T04:20:27.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.890+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/834461643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623417a6e90 0x5623417bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.890+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/834461643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417be950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.890+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/834461643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234179bcb0 0x5623417a2310 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.890+0000 7fcc6cc15740 1 -- 172.21.15.94:0/834461643 >> 172.21.15.94:0/834461643 conn(0x562341798240 msgr2=0x56234178eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:27.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.891+0000 7fcc6cc15740 1 -- 172.21.15.94:0/834461643 shutdown_connections 2024-09-19T04:20:27.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.891+0000 7fcc6cc15740 1 -- 172.21.15.94:0/834461643 wait complete. 2024-09-19T04:20:27.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.892+0000 7fcc6cc15740 1 -- start start 2024-09-19T04:20:27.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.892+0000 7fcc6cc15740 1 Processor -- start 2024-09-19T04:20:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56234179bcb0 0x5623417acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc6cc15740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623417a6e90 0x5623417ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc6cc15740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641a9cc0 con 0x5623417a6e90 2024-09-19T04:20:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc6cc15740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641ab130 con 0x56234179bcb0 2024-09-19T04:20:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc6cc15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641aa390 con 0x56234179c5d0 2024-09-19T04:20:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc6cc15740 10 client.0 _ll_register_callbacks cb 0x7fcc58013f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:52074/0 (socket says 172.21.15.94:52074) 2024-09-19T04:20:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc637fe640 1 -- 172.21.15.94:0/3162437381 learned_addr learned my addr 172.21.15.94:0/3162437381 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:27.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.893+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56234179bcb0 0x5623417acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.894+0000 7fcc68dd9640 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623417a6e90 0x5623417ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.894+0000 7fcc637fe640 1 -- 172.21.15.94:0/3162437381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56234179bcb0 msgr2=0x5623417acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.894+0000 7fcc637fe640 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56234179bcb0 0x5623417acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.894+0000 7fcc637fe640 1 -- 172.21.15.94:0/3162437381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623417a6e90 msgr2=0x5623417ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.894+0000 7fcc637fe640 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623417a6e90 0x5623417ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.894+0000 7fcc637fe640 1 -- 172.21.15.94:0/3162437381 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562341820930 con 0x56234179c5d0 2024-09-19T04:20:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.894+0000 7fcc637fe640 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417ad090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcc58013fb0 tx=0x7fcc58013370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf1b1a197c5b9b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.894+0000 7fcc437fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:20:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.895+0000 7fcc437fe640 1 -- 172.21.15.94:0/3162437381 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5809f6b0 con 0x56234179c5d0 2024-09-19T04:20:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.895+0000 7fcc437fe640 1 -- 172.21.15.94:0/3162437381 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fcc58013980 con 0x56234179c5d0 2024-09-19T04:20:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.895+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3162437381 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc54003b90 con 0x56234179c5d0 2024-09-19T04:20:27.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.896+0000 7fcc437fe640 1 -- 172.21.15.94:0/3162437381 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcc580a0450 con 0x56234179c5d0 2024-09-19T04:20:27.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.896+0000 7fcc6cc15740 10 client.25570 fetch_fsmap learned FSMap version 34 2024-09-19T04:20:27.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.896+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3162437381 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562341821bc0 con 0x56234179c5d0 2024-09-19T04:20:27.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.896+0000 7fcc437fe640 1 -- 172.21.15.94:0/3162437381 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc54003b90 con 0x56234179c5d0 2024-09-19T04:20:27.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.896+0000 7fcc437fe640 1 -- 172.21.15.94:0/3162437381 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fcc5809f1a0 con 0x56234179c5d0 2024-09-19T04:20:27.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.896+0000 7fcc6cc15740 10 client.25570 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:20:27.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.896+0000 7fcc6cc15740 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:27.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.896+0000 7fcc6cc15740 20 client.25570 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:27.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.896+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5623417b6c30 0x5623418242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.897+0000 7fcc6cc15740 4 client.25570 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:27.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.897+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3162437381 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562341706850 con 0x5623417b6c30 2024-09-19T04:20:27.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.897+0000 7fcc63fff640 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5623417b6c30 0x5623418242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.897+0000 7fcc437fe640 10 client.25570 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:27.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.897+0000 7fcc63fff640 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5623417b6c30 0x5623418242a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f2e62101f9c3c8 server_cookie=a345ded7f33da71b in_seq=0 out_seq=0 2024-09-19T04:20:27.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc437fe640 1 -- 172.21.15.94:0/3162437381 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562341706850 con 0x5623417b6c30 2024-09-19T04:20:27.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc437fe640 10 client.25570 handle_command_reply: tid=0 2024-09-19T04:20:27.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 client.25570 shutdown 2024-09-19T04:20:27.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3162437381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 msgr2=0x5623417ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417ad090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcc58013fb0 tx=0x7fcc58013370 comp rx=0 tx=0).stop 2024-09-19T04:20:27.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3162437381 shutdown_connections 2024-09-19T04:20:27.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5623417b6c30 0x5623418242a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623417a6e90 0x5623417ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56234179c5d0 0x5623417ad090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 --2- 172.21.15.94:0/3162437381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56234179bcb0 0x5623417acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3162437381 >> 172.21.15.94:0/3162437381 conn(0x562341798240 msgr2=0x5623417a52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:27.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc437fe640 0 client.25570 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:27.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3162437381 shutdown_connections 2024-09-19T04:20:27.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.898+0000 7fcc6cc15740 1 -- 172.21.15.94:0/3162437381 wait complete. 2024-09-19T04:20:27.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.899+0000 7fcc6cc15740 20 client.25570 trim_cache size 0 max 16384 2024-09-19T04:20:27.902 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:20:27.903 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:20:27.903 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:20:27.903 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:20:27.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.899+0000 7fcc6a83c640 1 -- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc4c0a3580 msgr2=0x7fcc4c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.899+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc4c0a3580 0x7fcc4c0a5a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c044000 tx=0x7fcc5c004380 comp rx=0 tx=0).stop 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.899+0000 7fcc6a83c640 1 -- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc64101e20 msgr2=0x7fcc64196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.900+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc64101e20 0x7fcc64196630 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e9c0 tx=0x7fcc5800ee80 comp rx=0 tx=0).stop 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.900+0000 7fcc6a83c640 1 -- 172.21.15.94:0/1280125098 shutdown_connections 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.900+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcc4c0a3580 0x7fcc4c0a5a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.900+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc64105de0 0x7fcc6419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.900+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc64101e20 0x7fcc64196630 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.900+0000 7fcc6a83c640 1 --2- 172.21.15.94:0/1280125098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc64101470 0x7fcc641960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.900+0000 7fcc6a83c640 1 -- 172.21.15.94:0/1280125098 >> 172.21.15.94:0/1280125098 conn(0x7fcc64077230 msgr2=0x7fcc640fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.900+0000 7fcc6a83c640 1 -- 172.21.15.94:0/1280125098 shutdown_connections 2024-09-19T04:20:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:27.900+0000 7fcc6a83c640 1 -- 172.21.15.94:0/1280125098 wait complete. 2024-09-19T04:20:28.778 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:28.778 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:20:29.194 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:29.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:29 smithi094 ceph-mon[30972]: pgmap v167: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:20:29.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:29 smithi112 ceph-mon[35051]: pgmap v167: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:20:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:29 smithi104 ceph-mon[34145]: pgmap v167: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:20:29.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.880+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/2616493702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481008c0 msgr2=0x7f1e48100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.880+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/2616493702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481008c0 0x7f1e48100ca0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009890 tx=0x7f1e3802a380 comp rx=0 tx=0).stop 2024-09-19T04:20:29.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.880+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/2616493702 shutdown_connections 2024-09-19T04:20:29.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.880+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/2616493702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e48105ec0 0x7f1e48106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.880+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/2616493702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481011e0 0x7f1e48105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.880+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/2616493702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e481008c0 0x7f1e48100ca0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.880+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/2616493702 >> 172.21.15.94:0/2616493702 conn(0x7f1e480fc7d0 msgr2=0x7f1e480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.881+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/2616493702 shutdown_connections 2024-09-19T04:20:29.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.881+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/2616493702 wait complete. 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.881+0000 7f1e4e8a8640 1 Processor -- start 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.882+0000 7f1e4e8a8640 1 -- start start 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.882+0000 7f1e4e8a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481008c0 0x7f1e481a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.882+0000 7f1e4e8a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e481011e0 0x7f1e481a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.882+0000 7f1e4e8a8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e48105ec0 0x7f1e4819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.882+0000 7f1e4e8a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481184c0 con 0x7f1e481008c0 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.882+0000 7f1e4e8a8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48118340 con 0x7f1e48105ec0 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.882+0000 7f1e4e8a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48118640 con 0x7f1e481011e0 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.882+0000 7f1e47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481008c0 0x7f1e481a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.883+0000 7f1e47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481008c0 0x7f1e481a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34292/0 (socket says 172.21.15.94:34292) 2024-09-19T04:20:29.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.883+0000 7f1e47fff640 1 -- 172.21.15.94:0/3858177178 learned_addr learned my addr 172.21.15.94:0/3858177178 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:29.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.883+0000 7f1e4ce45640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e48105ec0 0x7f1e4819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.883+0000 7f1e47fff640 1 -- 172.21.15.94:0/3858177178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e481011e0 msgr2=0x7f1e481a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.883+0000 7f1e47fff640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e481011e0 0x7f1e481a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.883+0000 7f1e47fff640 1 -- 172.21.15.94:0/3858177178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e48105ec0 msgr2=0x7f1e4819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.883+0000 7f1e47fff640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e48105ec0 0x7f1e4819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.883+0000 7f1e47fff640 1 -- 172.21.15.94:0/3858177178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4819f020 con 0x7f1e481008c0 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.883+0000 7f1e47fff640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481008c0 0x7f1e481a4730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1e38030e00 tx=0x7f1e38030e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.884+0000 7f1e457fa640 1 -- 172.21.15.94:0/3858177178 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38040020 con 0x7f1e481008c0 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.884+0000 7f1e457fa640 1 -- 172.21.15.94:0/3858177178 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e38038040 con 0x7f1e481008c0 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.884+0000 7f1e457fa640 1 -- 172.21.15.94:0/3858177178 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38037070 con 0x7f1e481008c0 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.884+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4819f2b0 con 0x7f1e481008c0 2024-09-19T04:20:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.884+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481ab600 con 0x7f1e481008c0 2024-09-19T04:20:29.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.885+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4806a850 con 0x7f1e481008c0 2024-09-19T04:20:29.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.887+0000 7f1e457fa640 1 -- 172.21.15.94:0/3858177178 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1e38043020 con 0x7f1e481008c0 2024-09-19T04:20:29.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.888+0000 7f1e457fa640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e300a3580 0x7f1e300a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.889+0000 7f1e457fa640 1 -- 172.21.15.94:0/3858177178 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f1e380efa70 con 0x7f1e481008c0 2024-09-19T04:20:29.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.889+0000 7f1e477fe640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e300a3580 0x7f1e300a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.889+0000 7f1e477fe640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e300a3580 0x7f1e300a5a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009a60 tx=0x7f1e40009280 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:29.897+0000 7f1e457fa640 1 -- 172.21.15.94:0/3858177178 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e3804a070 con 0x7f1e481008c0 2024-09-19T04:20:30.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.274+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e48105980 con 0x7f1e481008c0 2024-09-19T04:20:30.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.276+0000 7f1e457fa640 1 -- 172.21.15.94:0/3858177178 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11915 (secure 0 0 0) 0x7f1e380bc4f0 con 0x7f1e481008c0 2024-09-19T04:20:30.278 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:20:30.278 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":34,"btime":"2024-09-19T04:19:32:802034+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:19:32.751814+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:20:30.278 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 34 2024-09-19T04:20:30.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.280+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e300a3580 msgr2=0x7f1e300a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.280+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e300a3580 0x7f1e300a5a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009a60 tx=0x7f1e40009280 comp rx=0 tx=0).stop 2024-09-19T04:20:30.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.280+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481008c0 msgr2=0x7f1e481a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.280+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481008c0 0x7f1e481a4730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1e38030e00 tx=0x7f1e38030e30 comp rx=0 tx=0).stop 2024-09-19T04:20:30.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.281+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 shutdown_connections 2024-09-19T04:20:30.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.281+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e300a3580 0x7f1e300a5a40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.281+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e48105ec0 0x7f1e4819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.281+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e481011e0 0x7f1e481a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.281+0000 7f1e4e8a8640 1 --2- 172.21.15.94:0/3858177178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481008c0 0x7f1e481a4730 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.281+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 >> 172.21.15.94:0/3858177178 conn(0x7f1e480fc7d0 msgr2=0x7f1e48103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:30.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.282+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 shutdown_connections 2024-09-19T04:20:30.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:30.282+0000 7f1e4e8a8640 1 -- 172.21.15.94:0/3858177178 wait complete. 2024-09-19T04:20:30.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:30 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3858177178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 255.0 s, 1472.6 tps, lat 67.902 ms stddev 17.771 2024-09-19T04:20:30.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:30 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3858177178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:30 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3858177178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.990 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:20:31.403 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:31.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:31 smithi112 ceph-mon[35051]: pgmap v168: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:20:31.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:31 smithi094 ceph-mon[30972]: pgmap v168: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:20:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:31 smithi104 ceph-mon[34145]: pgmap v168: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:20:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.981+0000 7eff32811640 1 -- 172.21.15.94:0/446347570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff2c101430 msgr2=0x7eff2c10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.981+0000 7eff32811640 1 --2- 172.21.15.94:0/446347570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff2c101430 0x7eff2c10d960 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7eff24009c10 tx=0x7eff2402a6f0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.981+0000 7eff32811640 1 -- 172.21.15.94:0/446347570 shutdown_connections 2024-09-19T04:20:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.981+0000 7eff32811640 1 --2- 172.21.15.94:0/446347570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff2c101430 0x7eff2c10d960 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.981+0000 7eff32811640 1 --2- 172.21.15.94:0/446347570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c100a90 0x7eff2c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.981+0000 7eff32811640 1 --2- 172.21.15.94:0/446347570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c106a90 0x7eff2c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.981+0000 7eff32811640 1 -- 172.21.15.94:0/446347570 >> 172.21.15.94:0/446347570 conn(0x7eff2c0fc7d0 msgr2=0x7eff2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.981+0000 7eff32811640 1 -- 172.21.15.94:0/446347570 shutdown_connections 2024-09-19T04:20:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.981+0000 7eff32811640 1 -- 172.21.15.94:0/446347570 wait complete. 2024-09-19T04:20:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.982+0000 7eff32811640 1 Processor -- start 2024-09-19T04:20:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.982+0000 7eff32811640 1 -- start start 2024-09-19T04:20:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.983+0000 7eff32811640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c100a90 0x7eff2c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.983+0000 7eff32811640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c101430 0x7eff2c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.983+0000 7eff32811640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff2c106a90 0x7eff2c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.983+0000 7eff32811640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c110d40 con 0x7eff2c106a90 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.983+0000 7eff30dae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff2c106a90 0x7eff2c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.983+0000 7eff32811640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c110bc0 con 0x7eff2c100a90 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.983+0000 7eff32811640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c110ec0 con 0x7eff2c101430 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.983+0000 7eff2b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c101430 0x7eff2c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.984+0000 7eff2b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c101430 0x7eff2c19acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54182/0 (socket says 172.21.15.94:54182) 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.984+0000 7eff2b7fe640 1 -- 172.21.15.94:0/1471487638 learned_addr learned my addr 172.21.15.94:0/1471487638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.984+0000 7eff2b7fe640 1 -- 172.21.15.94:0/1471487638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c100a90 msgr2=0x7eff2c19a790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.984+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c100a90 0x7eff2c19a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.984+0000 7eff2b7fe640 1 -- 172.21.15.94:0/1471487638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff2c106a90 msgr2=0x7eff2c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.984+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff2c106a90 0x7eff2c19ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.984+0000 7eff2b7fe640 1 -- 172.21.15.94:0/1471487638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c19f2e0 con 0x7eff2c101430 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.984+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c101430 0x7eff2c19acd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7eff2000d950 tx=0x7eff2000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.984+0000 7eff297fa640 1 -- 172.21.15.94:0/1471487638 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20017030 con 0x7eff2c101430 2024-09-19T04:20:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.985+0000 7eff297fa640 1 -- 172.21.15.94:0/1471487638 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff20004020 con 0x7eff2c101430 2024-09-19T04:20:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.985+0000 7eff32811640 1 -- 172.21.15.94:0/1471487638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c19f5d0 con 0x7eff2c101430 2024-09-19T04:20:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.985+0000 7eff297fa640 1 -- 172.21.15.94:0/1471487638 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20003c70 con 0x7eff2c101430 2024-09-19T04:20:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.985+0000 7eff32811640 1 -- 172.21.15.94:0/1471487638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c1a6eb0 con 0x7eff2c101430 2024-09-19T04:20:31.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.988+0000 7eff297fa640 1 -- 172.21.15.94:0/1471487638 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7eff2001a070 con 0x7eff2c101430 2024-09-19T04:20:31.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.989+0000 7eff297fa640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7eff140a3430 0x7eff140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.989+0000 7eff297fa640 1 -- 172.21.15.94:0/1471487638 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7eff200cf260 con 0x7eff2c101430 2024-09-19T04:20:31.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff34bea740 1 Processor -- start 2024-09-19T04:20:31.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff34bea740 1 -- start start 2024-09-19T04:20:31.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff34bea740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 0x561868d3e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff34bea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d3ede0 0x561868d3f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff34bea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d4bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff34bea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561868c9c2f0 con 0x561868d3ede0 2024-09-19T04:20:31.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff2bfff640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7eff140a3430 0x7eff140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff34bea740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561868c43cf0 con 0x561868d40350 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff34bea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561868c982b0 con 0x561868d49b80 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d3ede0 0x561868d3f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d3ede0 0x561868d3f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46816/0 (socket says 172.21.15.94:46816) 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff2b7fe640 1 -- 172.21.15.94:0/1984515908 learned_addr learned my addr 172.21.15.94:0/1984515908 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.990+0000 7eff2bfff640 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 0x561868d3e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff30dae640 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d4bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff2bfff640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7eff140a3430 0x7eff140a58f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009e10 tx=0x7eff1c0127c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff2bfff640 1 -- 172.21.15.94:0/1984515908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 msgr2=0x561868d4bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff2bfff640 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d4bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff2bfff640 1 -- 172.21.15.94:0/1984515908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d3ede0 msgr2=0x561868d3f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff2bfff640 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d3ede0 0x561868d3f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff2bfff640 1 -- 172.21.15.94:0/1984515908 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561868c283a0 con 0x561868d40350 2024-09-19T04:20:31.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d3ede0 0x561868d3f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:31.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff2bfff640 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 0x561868d3e8a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff1c01d610 tx=0x7eff1c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a9b0eb3098285b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.991+0000 7eff127fc640 1 -- 172.21.15.94:0/1984515908 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c012230 con 0x561868d40350 2024-09-19T04:20:31.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.992+0000 7eff127fc640 1 -- 172.21.15.94:0/1984515908 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff1c0054d0 con 0x561868d40350 2024-09-19T04:20:31.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.992+0000 7eff127fc640 1 -- 172.21.15.94:0/1984515908 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c01e450 con 0x561868d40350 2024-09-19T04:20:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.993+0000 7eff34bea740 1 -- 172.21.15.94:0/1984515908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 msgr2=0x561868d3e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.993+0000 7eff34bea740 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 0x561868d3e8a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff1c01d610 tx=0x7eff1c01d000 comp rx=0 tx=0).stop 2024-09-19T04:20:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.993+0000 7eff34bea740 1 -- 172.21.15.94:0/1984515908 shutdown_connections 2024-09-19T04:20:31.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.993+0000 7eff34bea740 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d4bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.993+0000 7eff34bea740 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d3ede0 0x561868d3f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.993+0000 7eff34bea740 1 --2- 172.21.15.94:0/1984515908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 0x561868d3e8a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.993+0000 7eff34bea740 1 -- 172.21.15.94:0/1984515908 >> 172.21.15.94:0/1984515908 conn(0x561868cff310 msgr2=0x561868d3b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.993+0000 7eff34bea740 1 -- 172.21.15.94:0/1984515908 shutdown_connections 2024-09-19T04:20:31.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.993+0000 7eff34bea740 1 -- 172.21.15.94:0/1984515908 wait complete. 2024-09-19T04:20:31.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.994+0000 7eff34bea740 1 -- start start 2024-09-19T04:20:31.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.994+0000 7eff34bea740 1 Processor -- start 2024-09-19T04:20:31.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff34bea740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d49470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff34bea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d40350 0x561868d46720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff34bea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d46c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff34bea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561868d4c7c0 con 0x561868d40350 2024-09-19T04:20:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff34bea740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561868c57060 con 0x561868d3ede0 2024-09-19T04:20:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d49470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff34bea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561868d4c940 con 0x561868d49b80 2024-09-19T04:20:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d49470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37064/0 (socket says 172.21.15.94:37064) 2024-09-19T04:20:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff2bfff640 1 -- 172.21.15.94:0/3862663711 learned_addr learned my addr 172.21.15.94:0/3862663711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.995+0000 7eff34bea740 10 client.0 _ll_register_callbacks cb 0x5618689aa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff2bfff640 1 -- 172.21.15.94:0/3862663711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 msgr2=0x561868d46c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff2bfff640 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d46c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff2bfff640 1 -- 172.21.15.94:0/3862663711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d40350 msgr2=0x561868d46720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff2bfff640 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d40350 0x561868d46720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff2bfff640 1 -- 172.21.15.94:0/3862663711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561868a0d3c0 con 0x561868d3ede0 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff2bfff640 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d49470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0096e0 tx=0x7eff1c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85fcde7420effe83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff0b7fe640 1 -- 172.21.15.94:0/3862663711 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c02c020 con 0x561868d3ede0 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff0b7fe640 1 -- 172.21.15.94:0/3862663711 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7eff1c0046c0 con 0x561868d3ede0 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.996+0000 7eff34bea740 1 -- 172.21.15.94:0/3862663711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1c0054d0 con 0x561868d3ede0 2024-09-19T04:20:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff0b7fe640 1 -- 172.21.15.94:0/3862663711 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7eff1c0048c0 con 0x561868d3ede0 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff34bea740 10 client.25613 fetch_fsmap learned FSMap version 34 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff34bea740 1 -- 172.21.15.94:0/3862663711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561868d58150 con 0x561868d3ede0 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff0b7fe640 1 -- 172.21.15.94:0/3862663711 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff1c0054d0 con 0x561868d3ede0 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff0b7fe640 1 -- 172.21.15.94:0/3862663711 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7eff1c027a50 con 0x561868d3ede0 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff34bea740 10 client.25613 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff34bea740 10 client.25613 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff34bea740 20 client.25613 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff34bea740 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561868d59500 0x561868d5b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff34bea740 4 client.25613 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff34bea740 1 -- 172.21.15.94:0/3862663711 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561868ca9860 con 0x561868d59500 2024-09-19T04:20:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.997+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561868d59500 0x561868d5b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.998+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561868d59500 0x561868d5b8f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=492f12ed3e4c112c server_cookie=c5b946b576c0ac45 in_seq=0 out_seq=0 2024-09-19T04:20:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.998+0000 7eff0b7fe640 10 client.25613 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff0b7fe640 1 -- 172.21.15.94:0/3862663711 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561868ca9860 con 0x561868d59500 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff0b7fe640 10 client.25613 handle_command_reply: tid=0 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff34bea740 1 client.25613 shutdown 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff34bea740 1 -- 172.21.15.94:0/3862663711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 msgr2=0x561868d49470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff34bea740 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d49470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0096e0 tx=0x7eff1c009b80 comp rx=0 tx=0).stop 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff34bea740 1 -- 172.21.15.94:0/3862663711 shutdown_connections 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff34bea740 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561868d59500 0x561868d5b8f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff0b7fe640 0 client.25613 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff34bea740 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d46c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:31.999+0000 7eff34bea740 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d40350 0x561868d46720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.000+0000 7eff34bea740 1 --2- 172.21.15.94:0/3862663711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d49470 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.000+0000 7eff34bea740 1 -- 172.21.15.94:0/3862663711 >> 172.21.15.94:0/3862663711 conn(0x561868cff310 msgr2=0x561868d41170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:32.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.000+0000 7eff34bea740 1 -- 172.21.15.94:0/3862663711 shutdown_connections 2024-09-19T04:20:32.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.000+0000 7eff34bea740 1 -- 172.21.15.94:0/3862663711 wait complete. 2024-09-19T04:20:32.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.001+0000 7eff34bea740 20 client.25613 trim_cache size 0 max 16384 2024-09-19T04:20:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.025+0000 7eff34bea740 1 Processor -- start 2024-09-19T04:20:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.025+0000 7eff34bea740 1 -- start start 2024-09-19T04:20:32.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.025+0000 7eff34bea740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d55a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.025+0000 7eff34bea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d40350 0x561868d60ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.025+0000 7eff34bea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d61430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.025+0000 7eff34bea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1adfa0 con 0x561868d40350 2024-09-19T04:20:32.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.025+0000 7eff34bea740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1ab870 con 0x561868d3ede0 2024-09-19T04:20:32.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.025+0000 7eff34bea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1a9700 con 0x561868d49b80 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d55a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d55a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37078/0 (socket says 172.21.15.94:37078) 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff2bfff640 1 -- 172.21.15.94:0/4227323184 learned_addr learned my addr 172.21.15.94:0/4227323184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d40350 0x561868d60ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff30dae640 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d61430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff30dae640 1 -- 172.21.15.94:0/4227323184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 msgr2=0x561868d55a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff30dae640 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d55a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff30dae640 1 -- 172.21.15.94:0/4227323184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d40350 msgr2=0x561868d60ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff30dae640 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d40350 0x561868d60ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff30dae640 1 -- 172.21.15.94:0/4227323184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561868d3c170 con 0x561868d49b80 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff2bfff640 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d55a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.026+0000 7eff30dae640 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d61430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x561868d3f880 tx=0x7eff2400b180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc0419871242d7c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:32.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.027+0000 7eff11ffb640 1 -- 172.21.15.94:0/4227323184 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24041070 con 0x561868d49b80 2024-09-19T04:20:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.027+0000 7eff11ffb640 1 -- 172.21.15.94:0/4227323184 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff24037040 con 0x561868d49b80 2024-09-19T04:20:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.027+0000 7eff11ffb640 1 -- 172.21.15.94:0/4227323184 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24036030 con 0x561868d49b80 2024-09-19T04:20:32.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 -- 172.21.15.94:0/4227323184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 msgr2=0x561868d61430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d61430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x561868d3f880 tx=0x7eff2400b180 comp rx=0 tx=0).stop 2024-09-19T04:20:32.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 -- 172.21.15.94:0/4227323184 shutdown_connections 2024-09-19T04:20:32.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d49b80 0x561868d61430 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d40350 0x561868d60ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 --2- 172.21.15.94:0/4227323184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d3ede0 0x561868d55a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 -- 172.21.15.94:0/4227323184 >> 172.21.15.94:0/4227323184 conn(0x561868cff310 msgr2=0x561868d45810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 -- 172.21.15.94:0/4227323184 shutdown_connections 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 -- 172.21.15.94:0/4227323184 wait complete. 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 -- start start 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.028+0000 7eff34bea740 1 Processor -- start 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff34bea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d3ede0 0x561868d4d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff34bea740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 0x561868d4de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff34bea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d49b80 0x561868d4e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff34bea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1a9d50 con 0x561868d49b80 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff34bea740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1ab1a0 con 0x561868d40350 2024-09-19T04:20:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff34bea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1aa400 con 0x561868d3ede0 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff34bea740 10 client.0 _ll_register_callbacks cb 0x7eff1c005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d3ede0 0x561868d4d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff30dae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d49b80 0x561868d4e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d3ede0 0x561868d4d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54214/0 (socket says 172.21.15.94:54214) 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff30dae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d49b80 0x561868d4e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46836/0 (socket says 172.21.15.94:46836) 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff2bfff640 1 -- 172.21.15.94:0/4002521499 learned_addr learned my addr 172.21.15.94:0/4002521499 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.029+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 0x561868d4de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.030+0000 7eff2bfff640 1 -- 172.21.15.94:0/4002521499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 msgr2=0x561868d4de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.030+0000 7eff2bfff640 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 0x561868d4de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.030+0000 7eff2bfff640 1 -- 172.21.15.94:0/4002521499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d49b80 msgr2=0x561868d4e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.030+0000 7eff2bfff640 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d49b80 0x561868d4e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.030+0000 7eff2bfff640 1 -- 172.21.15.94:0/4002521499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561868dc2ec0 con 0x561868d3ede0 2024-09-19T04:20:32.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.030+0000 7eff30dae640 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d49b80 0x561868d4e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:32.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.030+0000 7eff2bfff640 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d3ede0 0x561868d4d8c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0123c0 tx=0x7eff1c009a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69791ec377c2a372 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:32.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.030+0000 7eff0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:20:32.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.031+0000 7eff0b7fe640 1 -- 172.21.15.94:0/4002521499 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c02c040 con 0x561868d3ede0 2024-09-19T04:20:32.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.031+0000 7eff0b7fe640 1 -- 172.21.15.94:0/4002521499 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7eff1c004680 con 0x561868d3ede0 2024-09-19T04:20:32.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.031+0000 7eff34bea740 1 -- 172.21.15.94:0/4002521499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff24037040 con 0x561868d3ede0 2024-09-19T04:20:32.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.031+0000 7eff0b7fe640 1 -- 172.21.15.94:0/4002521499 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7eff1c004f90 con 0x561868d3ede0 2024-09-19T04:20:32.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.031+0000 7eff34bea740 10 client.25585 fetch_fsmap learned FSMap version 34 2024-09-19T04:20:32.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.031+0000 7eff34bea740 1 -- 172.21.15.94:0/4002521499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561868dc4140 con 0x561868d3ede0 2024-09-19T04:20:32.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.032+0000 7eff0b7fe640 1 -- 172.21.15.94:0/4002521499 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff24037040 con 0x561868d3ede0 2024-09-19T04:20:32.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.032+0000 7eff0b7fe640 1 -- 172.21.15.94:0/4002521499 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7eff1c0276b0 con 0x561868d3ede0 2024-09-19T04:20:32.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.032+0000 7eff34bea740 10 client.25585 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:20:32.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.032+0000 7eff34bea740 10 client.25585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:20:32.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.032+0000 7eff34bea740 20 client.25585 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:32.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.032+0000 7eff34bea740 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561868d59500 0x561868dc6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.032+0000 7eff34bea740 4 client.25585 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:20:32.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.032+0000 7eff34bea740 1 -- 172.21.15.94:0/4002521499 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561868ca3d70 con 0x561868d59500 2024-09-19T04:20:32.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.033+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561868d59500 0x561868dc6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.033+0000 7eff0b7fe640 10 client.25585 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:20:32.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.033+0000 7eff2b7fe640 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561868d59500 0x561868dc6890 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=688688248e8fbb83 server_cookie=93d1628edff80d81 in_seq=0 out_seq=0 2024-09-19T04:20:32.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff0b7fe640 1 -- 172.21.15.94:0/4002521499 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561868ca3d70 con 0x561868d59500 2024-09-19T04:20:32.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff0b7fe640 10 client.25585 handle_command_reply: tid=0 2024-09-19T04:20:32.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 client.25585 shutdown 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 -- 172.21.15.94:0/4002521499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d3ede0 msgr2=0x561868d4d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d3ede0 0x561868d4d8c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0123c0 tx=0x7eff1c009a60 comp rx=0 tx=0).stop 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 -- 172.21.15.94:0/4002521499 shutdown_connections 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561868d59500 0x561868dc6890 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561868d49b80 0x561868d4e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561868d40350 0x561868d4de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 --2- 172.21.15.94:0/4002521499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561868d3ede0 0x561868d4d8c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 -- 172.21.15.94:0/4002521499 >> 172.21.15.94:0/4002521499 conn(0x561868cff310 msgr2=0x561868d40920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff0b7fe640 0 client.25585 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 -- 172.21.15.94:0/4002521499 shutdown_connections 2024-09-19T04:20:32.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.034+0000 7eff34bea740 1 -- 172.21.15.94:0/4002521499 wait complete. 2024-09-19T04:20:32.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.035+0000 7eff34bea740 20 client.25585 trim_cache size 0 max 16384 2024-09-19T04:20:32.040 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.036+0000 7eff32811640 1 -- 172.21.15.94:0/1471487638 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7eff140a3430 msgr2=0x7eff140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.036+0000 7eff32811640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7eff140a3430 0x7eff140a58f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009e10 tx=0x7eff1c0127c0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.036+0000 7eff32811640 1 -- 172.21.15.94:0/1471487638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c101430 msgr2=0x7eff2c19acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.036+0000 7eff32811640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c101430 0x7eff2c19acd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7eff2000d950 tx=0x7eff2000de10 comp rx=0 tx=0).stop 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.037+0000 7eff32811640 1 -- 172.21.15.94:0/1471487638 shutdown_connections 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.037+0000 7eff32811640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7eff140a3430 0x7eff140a58f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.037+0000 7eff32811640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff2c106a90 0x7eff2c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.037+0000 7eff32811640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c101430 0x7eff2c19acd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.037+0000 7eff32811640 1 --2- 172.21.15.94:0/1471487638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c100a90 0x7eff2c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.037+0000 7eff32811640 1 -- 172.21.15.94:0/1471487638 >> 172.21.15.94:0/1471487638 conn(0x7eff2c0fc7d0 msgr2=0x7eff2c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:32.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.037+0000 7eff32811640 1 -- 172.21.15.94:0/1471487638 shutdown_connections 2024-09-19T04:20:32.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:32.037+0000 7eff32811640 1 -- 172.21.15.94:0/1471487638 wait complete. 2024-09-19T04:20:32.843 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:20:33.182 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:33.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:33 smithi094 ceph-mon[30972]: pgmap v169: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 1.78k op/s 2024-09-19T04:20:33.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:33 smithi112 ceph-mon[35051]: pgmap v169: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 1.78k op/s 2024-09-19T04:20:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.799+0000 7fe515f17640 1 -- 172.21.15.94:0/3066149134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe510074eb0 msgr2=0x7fe510112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.799+0000 7fe515f17640 1 --2- 172.21.15.94:0/3066149134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe510074eb0 0x7fe510112640 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe500009ca0 tx=0x7fe50002a950 comp rx=0 tx=0).stop 2024-09-19T04:20:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.800+0000 7fe515f17640 1 -- 172.21.15.94:0/3066149134 shutdown_connections 2024-09-19T04:20:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.800+0000 7fe515f17640 1 --2- 172.21.15.94:0/3066149134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe510112b80 0x7fe510114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.800+0000 7fe515f17640 1 --2- 172.21.15.94:0/3066149134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe510074eb0 0x7fe510112640 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.800+0000 7fe515f17640 1 --2- 172.21.15.94:0/3066149134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe510074590 0x7fe510074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.800+0000 7fe515f17640 1 -- 172.21.15.94:0/3066149134 >> 172.21.15.94:0/3066149134 conn(0x7fe5100fd8b0 msgr2=0x7fe5100ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.800+0000 7fe515f17640 1 -- 172.21.15.94:0/3066149134 shutdown_connections 2024-09-19T04:20:33.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.800+0000 7fe515f17640 1 -- 172.21.15.94:0/3066149134 wait complete. 2024-09-19T04:20:33.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.801+0000 7fe515f17640 1 Processor -- start 2024-09-19T04:20:33.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.801+0000 7fe515f17640 1 -- start start 2024-09-19T04:20:33.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.801+0000 7fe515f17640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe510074590 0x7fe5101a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.801+0000 7fe515f17640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe510074eb0 0x7fe5101a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.801+0000 7fe515f17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe510112b80 0x7fe51019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.801+0000 7fe515f17640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101183e0 con 0x7fe510112b80 2024-09-19T04:20:33.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.801+0000 7fe515f17640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe510118260 con 0x7fe510074590 2024-09-19T04:20:33.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.801+0000 7fe515f17640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe510118560 con 0x7fe510074eb0 2024-09-19T04:20:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.801+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe510112b80 0x7fe51019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.802+0000 7fe50effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe510074eb0 0x7fe5101a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.802+0000 7fe50effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe510074eb0 0x7fe5101a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54228/0 (socket says 172.21.15.94:54228) 2024-09-19T04:20:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.802+0000 7fe50effd640 1 -- 172.21.15.94:0/2419458490 learned_addr learned my addr 172.21.15.94:0/2419458490 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.802+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe510074590 0x7fe5101a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.802+0000 7fe50effd640 1 -- 172.21.15.94:0/2419458490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe510074590 msgr2=0x7fe5101a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.802+0000 7fe50effd640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe510074590 0x7fe5101a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.802+0000 7fe50effd640 1 -- 172.21.15.94:0/2419458490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe510112b80 msgr2=0x7fe51019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.802+0000 7fe50effd640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe510112b80 0x7fe51019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.802+0000 7fe50effd640 1 -- 172.21.15.94:0/2419458490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51019efb0 con 0x7fe510074eb0 2024-09-19T04:20:33.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.803+0000 7fe50effd640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe510074eb0 0x7fe5101a4bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe500005cc0 tx=0x7fe50002f650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.803+0000 7fe50cff9640 1 -- 172.21.15.94:0/2419458490 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500034040 con 0x7fe510074eb0 2024-09-19T04:20:33.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.803+0000 7fe515f17640 1 -- 172.21.15.94:0/2419458490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51019f240 con 0x7fe510074eb0 2024-09-19T04:20:33.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.803+0000 7fe515f17640 1 -- 172.21.15.94:0/2419458490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5101ab4f0 con 0x7fe510074eb0 2024-09-19T04:20:33.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.803+0000 7fe50cff9640 1 -- 172.21.15.94:0/2419458490 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe50002fe80 con 0x7fe510074eb0 2024-09-19T04:20:33.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.803+0000 7fe50cff9640 1 -- 172.21.15.94:0/2419458490 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5000046a0 con 0x7fe510074eb0 2024-09-19T04:20:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.807+0000 7fe5182f0740 1 Processor -- start 2024-09-19T04:20:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.807+0000 7fe5182f0740 1 -- start start 2024-09-19T04:20:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86833090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db86833a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86840760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe5182f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db867992f0 con 0x55db86834b40 2024-09-19T04:20:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe5182f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db86740cf0 con 0x55db8683e370 2024-09-19T04:20:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe5182f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db867952b0 con 0x55db868335d0 2024-09-19T04:20:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86833090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86833090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46870/0 (socket says 172.21.15.94:46870) 2024-09-19T04:20:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50ffff640 1 -- 172.21.15.94:0/857315587 learned_addr learned my addr 172.21.15.94:0/857315587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50cff9640 1 -- 172.21.15.94:0/2419458490 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe500037020 con 0x7fe510074eb0 2024-09-19T04:20:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db86833a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50effd640 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86840760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50ffff640 1 -- 172.21.15.94:0/857315587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 msgr2=0x55db86833a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50ffff640 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db86833a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50ffff640 1 -- 172.21.15.94:0/857315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 msgr2=0x55db86840760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50ffff640 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86840760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.808+0000 7fe50ffff640 1 -- 172.21.15.94:0/857315587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db867253a0 con 0x55db86834b40 2024-09-19T04:20:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.809+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db86833a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.809+0000 7fe50effd640 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86840760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:33.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.809+0000 7fe50ffff640 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86833090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe504009ba0 tx=0x7fe504011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367e5afae91578c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.809+0000 7fe4f5ffb640 1 -- 172.21.15.94:0/857315587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504004230 con 0x55db86834b40 2024-09-19T04:20:33.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.809+0000 7fe4f5ffb640 1 -- 172.21.15.94:0/857315587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe504009e10 con 0x55db86834b40 2024-09-19T04:20:33.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.809+0000 7fe50cff9640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe4f80a34b0 0x7fe4f80a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.810+0000 7fe4f5ffb640 1 -- 172.21.15.94:0/857315587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504004cf0 con 0x55db86834b40 2024-09-19T04:20:33.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.810+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe4f80a34b0 0x7fe4f80a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.810+0000 7fe50cff9640 1 -- 172.21.15.94:0/2419458490 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe5000f0aa0 con 0x7fe510074eb0 2024-09-19T04:20:33.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.810+0000 7fe5182f0740 1 -- 172.21.15.94:0/857315587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 msgr2=0x55db86833090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.810+0000 7fe5182f0740 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86833090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe504009ba0 tx=0x7fe504011810 comp rx=0 tx=0).stop 2024-09-19T04:20:33.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.810+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe4f80a34b0 0x7fe4f80a5970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55db86830ac0 tx=0x7fe50800b710 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.811+0000 7fe5182f0740 1 -- 172.21.15.94:0/857315587 shutdown_connections 2024-09-19T04:20:33.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.811+0000 7fe5182f0740 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86840760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.811+0000 7fe5182f0740 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db86833a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.811+0000 7fe5182f0740 1 --2- 172.21.15.94:0/857315587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86833090 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.811+0000 7fe5182f0740 1 -- 172.21.15.94:0/857315587 >> 172.21.15.94:0/857315587 conn(0x55db867fc310 msgr2=0x55db8682fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.811+0000 7fe5182f0740 1 -- 172.21.15.94:0/857315587 shutdown_connections 2024-09-19T04:20:33.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.811+0000 7fe5182f0740 1 -- 172.21.15.94:0/857315587 wait complete. 2024-09-19T04:20:33.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.812+0000 7fe5182f0740 1 -- start start 2024-09-19T04:20:33.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.812+0000 7fe5182f0740 1 Processor -- start 2024-09-19T04:20:33.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.812+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 0x55db8683dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.812+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db8683afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.812+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db8683b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe5182f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db868410c0 con 0x55db868335d0 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe5182f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db86754060 con 0x55db86834b40 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe5182f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db86841240 con 0x55db8683e370 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe5182f0740 10 client.0 _ll_register_callbacks cb 0x55db864a78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db8683afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db8683afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37122/0 (socket says 172.21.15.94:37122) 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50effd640 1 -- 172.21.15.94:0/456439123 learned_addr learned my addr 172.21.15.94:0/456439123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50ffff640 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db8683b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50effd640 1 -- 172.21.15.94:0/456439123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 msgr2=0x55db8683b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50effd640 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db8683b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50effd640 1 -- 172.21.15.94:0/456439123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 msgr2=0x55db8683dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 0x55db8683dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50effd640 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 0x55db8683dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50effd640 1 -- 172.21.15.94:0/456439123 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8650a3c0 con 0x55db86834b40 2024-09-19T04:20:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.813+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 0x55db8683dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.814+0000 7fe50effd640 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db8683afb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe500044000 tx=0x7fe500004100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b37b01f7d39b73c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.814+0000 7fe4eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:20:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.814+0000 7fe4eeffd640 1 -- 172.21.15.94:0/456439123 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5000bd520 con 0x55db86834b40 2024-09-19T04:20:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.814+0000 7fe4eeffd640 1 -- 172.21.15.94:0/456439123 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe5000bd080 con 0x55db86834b40 2024-09-19T04:20:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.814+0000 7fe5182f0740 1 -- 172.21.15.94:0/456439123 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe504009e10 con 0x55db86834b40 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.814+0000 7fe4eeffd640 1 -- 172.21.15.94:0/456439123 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe5000bce20 con 0x55db86834b40 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.814+0000 7fe5182f0740 10 client.25628 fetch_fsmap learned FSMap version 34 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.814+0000 7fe5182f0740 1 -- 172.21.15.94:0/456439123 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55db8684caf0 con 0x55db86834b40 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.815+0000 7fe4eeffd640 1 -- 172.21.15.94:0/456439123 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe504009e10 con 0x55db86834b40 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.815+0000 7fe4eeffd640 1 -- 172.21.15.94:0/456439123 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe5000c5600 con 0x55db86834b40 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.815+0000 7fe5182f0740 10 client.25628 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.815+0000 7fe5182f0740 10 client.25628 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.815+0000 7fe5182f0740 20 client.25628 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.815+0000 7fe5182f0740 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db8684dfa0 0x55db868503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.815+0000 7fe5182f0740 4 client.25628 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.815+0000 7fe5182f0740 1 -- 172.21.15.94:0/456439123 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db867a6860 con 0x55db8684dfa0 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.815+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db8684dfa0 0x55db868503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.816+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db8684dfa0 0x55db868503e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6e3451d2a12dbd1 server_cookie=300088b7a61e2014 in_seq=0 out_seq=0 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.816+0000 7fe4eeffd640 10 client.25628 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe4eeffd640 1 -- 172.21.15.94:0/456439123 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db867a6860 con 0x55db8684dfa0 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe4eeffd640 10 client.25628 handle_command_reply: tid=0 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe5182f0740 1 client.25628 shutdown 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe5182f0740 1 -- 172.21.15.94:0/456439123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 msgr2=0x55db8683afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe5182f0740 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db8683afb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe500044000 tx=0x7fe500004100 comp rx=0 tx=0).stop 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe5182f0740 1 -- 172.21.15.94:0/456439123 shutdown_connections 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe5182f0740 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db8684dfa0 0x55db868503e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe4eeffd640 0 client.25628 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:20:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe5182f0740 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db8683b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe5182f0740 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db8683afb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe5182f0740 1 --2- 172.21.15.94:0/456439123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 0x55db8683dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.817+0000 7fe5182f0740 1 -- 172.21.15.94:0/456439123 >> 172.21.15.94:0/456439123 conn(0x55db867fc310 msgr2=0x55db86835930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.818+0000 7fe5182f0740 1 -- 172.21.15.94:0/456439123 shutdown_connections 2024-09-19T04:20:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.818+0000 7fe5182f0740 1 -- 172.21.15.94:0/456439123 wait complete. 2024-09-19T04:20:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.818+0000 7fe5182f0740 20 client.25628 trim_cache size 0 max 16384 2024-09-19T04:20:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe5182f0740 1 Processor -- start 2024-09-19T04:20:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe5182f0740 1 -- start start 2024-09-19T04:20:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 0x55db8684a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db86836120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db86836660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe5182f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc011010 con 0x55db868335d0 2024-09-19T04:20:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe5182f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00e8e0 con 0x55db86834b40 2024-09-19T04:20:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe5182f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00c730 con 0x55db8683e370 2024-09-19T04:20:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db86836660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.840+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db86836660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54266/0 (socket says 172.21.15.94:54266) 2024-09-19T04:20:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50ffff640 1 -- 172.21.15.94:0/3954007194 learned_addr learned my addr 172.21.15.94:0/3954007194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db86836120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/3954007194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 0x55db8684a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50ffff640 1 -- 172.21.15.94:0/3954007194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 msgr2=0x55db86836120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50ffff640 1 --2- 172.21.15.94:0/3954007194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db86836120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50ffff640 1 -- 172.21.15.94:0/3954007194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 msgr2=0x55db8684a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50ffff640 1 --2- 172.21.15.94:0/3954007194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 0x55db8684a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50ffff640 1 -- 172.21.15.94:0/3954007194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db86850a60 con 0x55db8683e370 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50effd640 1 --2- 172.21.15.94:0/3954007194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db86836120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.841+0000 7fe50ffff640 1 --2- 172.21.15.94:0/3954007194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db86836660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe504009b40 tx=0x7fe504009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8253bcd805d96faa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.842+0000 7fe4f57fa640 1 -- 172.21.15.94:0/3954007194 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50401e040 con 0x55db8683e370 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.842+0000 7fe4f57fa640 1 -- 172.21.15.94:0/3954007194 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5040035f0 con 0x55db8683e370 2024-09-19T04:20:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.842+0000 7fe4f57fa640 1 -- 172.21.15.94:0/3954007194 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504023650 con 0x55db8683e370 2024-09-19T04:20:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.843+0000 7fe5182f0740 1 -- 172.21.15.94:0/3954007194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 msgr2=0x55db86836660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.843+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3954007194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db86836660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe504009b40 tx=0x7fe504009ef0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.843+0000 7fe5182f0740 1 -- 172.21.15.94:0/3954007194 shutdown_connections 2024-09-19T04:20:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.843+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3954007194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db8683e370 0x55db86836660 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.843+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3954007194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db86834b40 0x55db86836120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.843+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3954007194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db868335d0 0x55db8684a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.843+0000 7fe5182f0740 1 -- 172.21.15.94:0/3954007194 >> 172.21.15.94:0/3954007194 conn(0x55db867fc310 msgr2=0x55db8683a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.843+0000 7fe5182f0740 1 -- 172.21.15.94:0/3954007194 shutdown_connections 2024-09-19T04:20:33.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.844+0000 7fe5182f0740 1 -- 172.21.15.94:0/3954007194 wait complete. 2024-09-19T04:20:33.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.844+0000 7fe5182f0740 1 -- start start 2024-09-19T04:20:33.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.845+0000 7fe5182f0740 1 Processor -- start 2024-09-19T04:20:33.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.845+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db868425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86842ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe5182f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86843040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db868425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db868425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54282/0 (socket says 172.21.15.94:54282) 2024-09-19T04:20:33.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe5182f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00cda0 con 0x55db86834b40 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe5182f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00e210 con 0x55db8683e370 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe5182f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00d470 con 0x55db868335d0 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86843040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86842ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86843040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37142/0 (socket says 172.21.15.94:37142) 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50ffff640 1 -- 172.21.15.94:0/3782637760 learned_addr learned my addr 172.21.15.94:0/3782637760 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe5182f0740 10 client.0 _ll_register_callbacks cb 0x7fe5000bc780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50f7fe640 1 -- 172.21.15.94:0/3782637760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 msgr2=0x55db86843040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86843040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50f7fe640 1 -- 172.21.15.94:0/3782637760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 msgr2=0x55db86842ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.846+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86842ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.847+0000 7fe50f7fe640 1 -- 172.21.15.94:0/3782637760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db868b7f90 con 0x55db868335d0 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.847+0000 7fe50f7fe640 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db868425a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55db86850c60 tx=0x7fe5080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3334e7b2dfdc4a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.847+0000 7fe4eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:20:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.847+0000 7fe4eeffd640 1 -- 172.21.15.94:0/3782637760 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508004230 con 0x55db868335d0 2024-09-19T04:20:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.847+0000 7fe4eeffd640 1 -- 172.21.15.94:0/3782637760 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe5080043d0 con 0x55db868335d0 2024-09-19T04:20:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.847+0000 7fe5182f0740 1 -- 172.21.15.94:0/3782637760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5040035f0 con 0x55db868335d0 2024-09-19T04:20:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe4eeffd640 1 -- 172.21.15.94:0/3782637760 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe508012e40 con 0x55db868335d0 2024-09-19T04:20:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe5182f0740 10 client.25603 fetch_fsmap learned FSMap version 34 2024-09-19T04:20:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe5182f0740 1 -- 172.21.15.94:0/3782637760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55db868b92b0 con 0x55db868335d0 2024-09-19T04:20:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe4eeffd640 1 -- 172.21.15.94:0/3782637760 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5040035f0 con 0x55db868335d0 2024-09-19T04:20:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe4eeffd640 1 -- 172.21.15.94:0/3782637760 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe50802f020 con 0x55db868335d0 2024-09-19T04:20:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe5182f0740 10 client.25603 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:20:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe5182f0740 10 client.25603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:20:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe5182f0740 20 client.25603 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db868b9930 0x55db868bbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe5182f0740 4 client.25603 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:20:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.848+0000 7fe5182f0740 1 -- 172.21.15.94:0/3782637760 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db867a0d70 con 0x55db868b9930 2024-09-19T04:20:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.849+0000 7fe50effd640 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db868b9930 0x55db868bbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.849+0000 7fe50effd640 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db868b9930 0x55db868bbd80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c01e15feedf6ec9e server_cookie=dc78a27df608e50f in_seq=0 out_seq=0 2024-09-19T04:20:33.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.849+0000 7fe4eeffd640 10 client.25603 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:20:33.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.849+0000 7fe4eeffd640 1 -- 172.21.15.94:0/3782637760 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db867a0d70 con 0x55db868b9930 2024-09-19T04:20:33.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.849+0000 7fe4eeffd640 10 client.25603 handle_command_reply: tid=0 2024-09-19T04:20:33.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.849+0000 7fe5182f0740 1 client.25603 shutdown 2024-09-19T04:20:33.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 -- 172.21.15.94:0/3782637760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 msgr2=0x55db868425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db868425a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55db86850c60 tx=0x7fe5080046c0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 -- 172.21.15.94:0/3782637760 shutdown_connections 2024-09-19T04:20:33.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db868b9930 0x55db868bbd80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db8683e370 0x55db86843040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db86834b40 0x55db86842ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 --2- 172.21.15.94:0/3782637760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db868335d0 0x55db868425a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 -- 172.21.15.94:0/3782637760 >> 172.21.15.94:0/3782637760 conn(0x55db867fc310 msgr2=0x55db8683c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe4eeffd640 0 client.25603 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:20:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 -- 172.21.15.94:0/3782637760 shutdown_connections 2024-09-19T04:20:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 1 -- 172.21.15.94:0/3782637760 wait complete. 2024-09-19T04:20:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.850+0000 7fe5182f0740 20 client.25603 trim_cache size 0 max 16384 2024-09-19T04:20:33.862 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:20:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 -- 172.21.15.94:0/2419458490 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe4f80a34b0 msgr2=0x7fe4f80a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe4f80a34b0 0x7fe4f80a5970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55db86830ac0 tx=0x7fe50800b710 comp rx=0 tx=0).stop 2024-09-19T04:20:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 -- 172.21.15.94:0/2419458490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe510074eb0 msgr2=0x7fe5101a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe510074eb0 0x7fe5101a4bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe500005cc0 tx=0x7fe50002f650 comp rx=0 tx=0).stop 2024-09-19T04:20:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 -- 172.21.15.94:0/2419458490 shutdown_connections 2024-09-19T04:20:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe4f80a34b0 0x7fe4f80a5970 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe510112b80 0x7fe51019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe510074eb0 0x7fe5101a4bd0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe500005cc0 tx=0x7fe50002f650 comp rx=0 tx=0).stop 2024-09-19T04:20:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 --2- 172.21.15.94:0/2419458490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe510074590 0x7fe5101a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.851+0000 7fe515f17640 1 -- 172.21.15.94:0/2419458490 >> 172.21.15.94:0/2419458490 conn(0x7fe5100fd8b0 msgr2=0x7fe5100ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.852+0000 7fe515f17640 1 -- 172.21.15.94:0/2419458490 shutdown_connections 2024-09-19T04:20:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:33.852+0000 7fe515f17640 1 -- 172.21.15.94:0/2419458490 wait complete. 2024-09-19T04:20:33.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:33 smithi104 ceph-mon[34145]: pgmap v169: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 1.78k op/s 2024-09-19T04:20:34.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:20:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 260.0 s, 1113.2 tps, lat 89.906 ms stddev 128.308 2024-09-19T04:20:35.697 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:35 smithi094 ceph-mon[30972]: pgmap v170: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 1.24k op/s 2024-09-19T04:20:35.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.701+0000 7f245b508640 1 -- 172.21.15.94:0/864770850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24541008e0 msgr2=0x7f2454100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.701+0000 7f245b508640 1 --2- 172.21.15.94:0/864770850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24541008e0 0x7f2454100cc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2450009900 tx=0x7f245002a410 comp rx=0 tx=0).stop 2024-09-19T04:20:35.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.701+0000 7f245b508640 1 -- 172.21.15.94:0/864770850 shutdown_connections 2024-09-19T04:20:35.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.701+0000 7f245b508640 1 --2- 172.21.15.94:0/864770850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2454108180 0x7f245410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.701+0000 7f245b508640 1 --2- 172.21.15.94:0/864770850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2454101290 0x7f2454107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.701+0000 7f245b508640 1 --2- 172.21.15.94:0/864770850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24541008e0 0x7f2454100cc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.701+0000 7f245b508640 1 -- 172.21.15.94:0/864770850 >> 172.21.15.94:0/864770850 conn(0x7f2454077210 msgr2=0x7f2454077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:35.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.702+0000 7f245b508640 1 -- 172.21.15.94:0/864770850 shutdown_connections 2024-09-19T04:20:35.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.702+0000 7f245b508640 1 -- 172.21.15.94:0/864770850 wait complete. 2024-09-19T04:20:35.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.703+0000 7f245b508640 1 Processor -- start 2024-09-19T04:20:35.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.703+0000 7f245b508640 1 -- start start 2024-09-19T04:20:35.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f245b508640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24541008e0 0x7f2454198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f245b508640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2454101290 0x7f2454198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f245b508640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2454108180 0x7f245419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f245b508640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2454111f70 con 0x7f2454108180 2024-09-19T04:20:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f245b508640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2454111df0 con 0x7f2454101290 2024-09-19T04:20:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f245b508640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24541120f0 con 0x7f24541008e0 2024-09-19T04:20:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f2459aa5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2454108180 0x7f245419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f2459aa5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2454108180 0x7f245419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46932/0 (socket says 172.21.15.94:46932) 2024-09-19T04:20:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f2459aa5640 1 -- 172.21.15.94:0/2852552793 learned_addr learned my addr 172.21.15.94:0/2852552793 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f2458aa3640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2454101290 0x7f2454198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.704+0000 7f24592a4640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24541008e0 0x7f2454198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f2458aa3640 1 -- 172.21.15.94:0/2852552793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24541008e0 msgr2=0x7f2454198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f2458aa3640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24541008e0 0x7f2454198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f2458aa3640 1 -- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2454108180 msgr2=0x7f245419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f2458aa3640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2454108180 0x7f245419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f2458aa3640 1 -- 172.21.15.94:0/2852552793 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245419d3d0 con 0x7f2454101290 2024-09-19T04:20:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f2459aa5640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2454108180 0x7f245419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f2458aa3640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2454101290 0x7f2454198960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f244400e9c0 tx=0x7f244400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f244a7fc640 1 -- 172.21.15.94:0/2852552793 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244400cd80 con 0x7f2454101290 2024-09-19T04:20:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f244a7fc640 1 -- 172.21.15.94:0/2852552793 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24440035f0 con 0x7f2454101290 2024-09-19T04:20:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f245b508640 1 -- 172.21.15.94:0/2852552793 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245419d6c0 con 0x7f2454101290 2024-09-19T04:20:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f245b508640 1 -- 172.21.15.94:0/2852552793 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2454101790 con 0x7f2454101290 2024-09-19T04:20:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.705+0000 7f244a7fc640 1 -- 172.21.15.94:0/2852552793 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444010690 con 0x7f2454101290 2024-09-19T04:20:35.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.708+0000 7f244a7fc640 1 -- 172.21.15.94:0/2852552793 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2444004020 con 0x7f2454101290 2024-09-19T04:20:35.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f245d8e1740 1 Processor -- start 2024-09-19T04:20:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f245d8e1740 1 -- start start 2024-09-19T04:20:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 0x55c182d3b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f245d8e1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c182c942f0 con 0x55c182d39370 2024-09-19T04:20:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f245d8e1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c182c3bcf0 con 0x55c182d2fb40 2024-09-19T04:20:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f245d8e1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c182c902b0 con 0x55c182d2e5d0 2024-09-19T04:20:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f2458aa3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 0x55c182d3b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f24592a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f24592a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d2ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54316/0 (socket says 172.21.15.94:54316) 2024-09-19T04:20:35.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.709+0000 7f24592a4640 1 -- 172.21.15.94:0/3365951455 learned_addr learned my addr 172.21.15.94:0/3365951455 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:35.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.710+0000 7f2459aa5640 1 --2- 172.21.15.94:0/3365951455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.710+0000 7f24592a4640 1 -- 172.21.15.94:0/3365951455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 msgr2=0x55c182d2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.710+0000 7f24592a4640 1 --2- 172.21.15.94:0/3365951455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.710+0000 7f24592a4640 1 -- 172.21.15.94:0/3365951455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 msgr2=0x55c182d3b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.710+0000 7f24592a4640 1 --2- 172.21.15.94:0/3365951455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 0x55c182d3b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.710+0000 7f24592a4640 1 -- 172.21.15.94:0/3365951455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c182c203a0 con 0x55c182d2e5d0 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.710+0000 7f2458aa3640 1 --2- 172.21.15.94:0/3365951455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 0x55c182d3b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.710+0000 7f24592a4640 1 --2- 172.21.15.94:0/3365951455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d2ea10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2450000c00 tx=0x7f2450004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d11385586ce43c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.711+0000 7f243f7fe640 1 -- 172.21.15.94:0/3365951455 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450035040 con 0x55c182d2e5d0 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.711+0000 7f243f7fe640 1 -- 172.21.15.94:0/3365951455 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24500054f0 con 0x55c182d2e5d0 2024-09-19T04:20:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.711+0000 7f243f7fe640 1 -- 172.21.15.94:0/3365951455 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500035f0 con 0x55c182d2e5d0 2024-09-19T04:20:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.711+0000 7f244a7fc640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24400a3430 0x7f24400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f2459aa5640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24400a3430 0x7f24400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f245d8e1740 1 -- 172.21.15.94:0/3365951455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 msgr2=0x55c182d2ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3365951455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d2ea10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2450000c00 tx=0x7f2450004040 comp rx=0 tx=0).stop 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f244a7fc640 1 -- 172.21.15.94:0/2852552793 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f24440cfeb0 con 0x7f2454101290 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f2459aa5640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24400a3430 0x7f24400a58f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f244c0049d0 tx=0x7f244c009500 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f245d8e1740 1 -- 172.21.15.94:0/3365951455 shutdown_connections 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3365951455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 0x55c182d3b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3365951455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d2ea10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3365951455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f245d8e1740 1 -- 172.21.15.94:0/3365951455 >> 172.21.15.94:0/3365951455 conn(0x55c182cf7310 msgr2=0x55c182d2ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.712+0000 7f245d8e1740 1 -- 172.21.15.94:0/3365951455 shutdown_connections 2024-09-19T04:20:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.713+0000 7f245d8e1740 1 -- 172.21.15.94:0/3365951455 wait complete. 2024-09-19T04:20:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.713+0000 7f245d8e1740 1 -- start start 2024-09-19T04:20:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.713+0000 7f245d8e1740 1 Processor -- start 2024-09-19T04:20:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d38cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2fb40 0x55c182d35f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d39370 0x55c182d364e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f245d8e1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c182d4d020 con 0x55c182d2e5d0 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f245d8e1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c182c4f060 con 0x55c182d39370 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f245d8e1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c182d4d1a0 con 0x55c182d2fb40 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f245d8e1740 10 client.0 _ll_register_callbacks cb 0x55c1829a28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f2459aa5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d39370 0x55c182d364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f2459aa5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d39370 0x55c182d364e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37192/0 (socket says 172.21.15.94:37192) 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f2459aa5640 1 -- 172.21.15.94:0/674387300 learned_addr learned my addr 172.21.15.94:0/674387300 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f2458aa3640 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2fb40 0x55c182d35f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f24592a4640 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d38cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f2458aa3640 1 -- 172.21.15.94:0/674387300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d39370 msgr2=0x55c182d364e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f2458aa3640 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d39370 0x55c182d364e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.714+0000 7f2458aa3640 1 -- 172.21.15.94:0/674387300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 msgr2=0x55c182d38cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.715+0000 7f2458aa3640 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d38cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.715+0000 7f2458aa3640 1 -- 172.21.15.94:0/674387300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c182a053c0 con 0x55c182d2fb40 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.715+0000 7f24592a4640 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d38cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.715+0000 7f2458aa3640 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2fb40 0x55c182d35f80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24440cd590 tx=0x7f2444098480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f431a28f7c8c3899 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.715+0000 7f243cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.715+0000 7f243cff9640 1 -- 172.21.15.94:0/674387300 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244409c820 con 0x55c182d2fb40 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.715+0000 7f245d8e1740 1 -- 172.21.15.94:0/674387300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24500054f0 con 0x55c182d2fb40 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.715+0000 7f243cff9640 1 -- 172.21.15.94:0/674387300 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f24440a01d0 con 0x55c182d2fb40 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f243cff9640 1 -- 172.21.15.94:0/674387300 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f244409c120 con 0x55c182d2fb40 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f245d8e1740 10 client.25615 fetch_fsmap learned FSMap version 34 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f245d8e1740 1 -- 172.21.15.94:0/674387300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c182d58ac0 con 0x55c182d2fb40 2024-09-19T04:20:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f243cff9640 1 -- 172.21.15.94:0/674387300 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24500054f0 con 0x55c182d2fb40 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f243cff9640 1 -- 172.21.15.94:0/674387300 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f24440cdce0 con 0x55c182d2fb40 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f245d8e1740 10 client.25615 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f245d8e1740 10 client.25615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f245d8e1740 20 client.25615 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f245d8e1740 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c182d59f80 0x55c182d5c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f245d8e1740 4 client.25615 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.716+0000 7f245d8e1740 1 -- 172.21.15.94:0/674387300 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c182ca1860 con 0x55c182d59f80 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.717+0000 7f24592a4640 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c182d59f80 0x55c182d5c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.717+0000 7f24592a4640 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c182d59f80 0x55c182d5c3c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feca4777290ea688 server_cookie=e29869f9fe4d4ad5 in_seq=0 out_seq=0 2024-09-19T04:20:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.717+0000 7f243cff9640 10 client.25615 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.718+0000 7f243cff9640 1 -- 172.21.15.94:0/674387300 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c182ca1860 con 0x55c182d59f80 2024-09-19T04:20:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.718+0000 7f243cff9640 10 client.25615 handle_command_reply: tid=0 2024-09-19T04:20:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.718+0000 7f245d8e1740 1 client.25615 shutdown 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.718+0000 7f245d8e1740 1 -- 172.21.15.94:0/674387300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2fb40 msgr2=0x55c182d35f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.718+0000 7f245d8e1740 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2fb40 0x55c182d35f80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24440cd590 tx=0x7f2444098480 comp rx=0 tx=0).stop 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f245d8e1740 1 -- 172.21.15.94:0/674387300 shutdown_connections 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f245d8e1740 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c182d59f80 0x55c182d5c3c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f245d8e1740 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d39370 0x55c182d364e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f245d8e1740 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2fb40 0x55c182d35f80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f243cff9640 0 client.25615 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f245d8e1740 1 --2- 172.21.15.94:0/674387300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d38cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f245d8e1740 1 -- 172.21.15.94:0/674387300 >> 172.21.15.94:0/674387300 conn(0x55c182cf7310 msgr2=0x55c182d30930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f245d8e1740 1 -- 172.21.15.94:0/674387300 shutdown_connections 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f245d8e1740 1 -- 172.21.15.94:0/674387300 wait complete. 2024-09-19T04:20:35.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.719+0000 7f245d8e1740 20 client.25615 trim_cache size 0 max 16384 2024-09-19T04:20:35.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.744+0000 7f245d8e1740 1 Processor -- start 2024-09-19T04:20:35.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.744+0000 7f245d8e1740 1 -- start start 2024-09-19T04:20:35.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.744+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d35f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d39370 0x55c182d31120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f245d8e1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24380110d0 con 0x55c182d2e5d0 2024-09-19T04:20:35.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f245d8e1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f243800e9a0 con 0x55c182d2fb40 2024-09-19T04:20:35.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f245d8e1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f243800c7f0 con 0x55c182d39370 2024-09-19T04:20:35.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f24592a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d35f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f24592a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d35f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46954/0 (socket says 172.21.15.94:46954) 2024-09-19T04:20:35.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f24592a4640 1 -- 172.21.15.94:0/3269037103 learned_addr learned my addr 172.21.15.94:0/3269037103 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f2459aa5640 1 --2- 172.21.15.94:0/3269037103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d39370 0x55c182d31120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.745+0000 7f2458aa3640 1 --2- 172.21.15.94:0/3269037103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.746+0000 7f24592a4640 1 -- 172.21.15.94:0/3269037103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d39370 msgr2=0x55c182d31120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.746+0000 7f24592a4640 1 --2- 172.21.15.94:0/3269037103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d39370 0x55c182d31120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.746+0000 7f24592a4640 1 -- 172.21.15.94:0/3269037103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 msgr2=0x55c182d56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.746+0000 7f24592a4640 1 --2- 172.21.15.94:0/3269037103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.746+0000 7f24592a4640 1 -- 172.21.15.94:0/3269037103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c182d39000 con 0x55c182d2e5d0 2024-09-19T04:20:35.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.746+0000 7f24592a4640 1 --2- 172.21.15.94:0/3269037103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d35f50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2450009900 tx=0x7f2450005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa0303e9d01a516 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.746+0000 7f243effd640 1 -- 172.21.15.94:0/3269037103 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450035040 con 0x55c182d2e5d0 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.746+0000 7f243effd640 1 -- 172.21.15.94:0/3269037103 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2450003aa0 con 0x55c182d2e5d0 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.746+0000 7f243effd640 1 -- 172.21.15.94:0/3269037103 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450004f10 con 0x55c182d2e5d0 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.747+0000 7f245d8e1740 1 -- 172.21.15.94:0/3269037103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 msgr2=0x55c182d35f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.747+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3269037103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d35f50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2450009900 tx=0x7f2450005a20 comp rx=0 tx=0).stop 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.748+0000 7f245d8e1740 1 -- 172.21.15.94:0/3269037103 shutdown_connections 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.748+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3269037103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d39370 0x55c182d31120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.748+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3269037103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.748+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3269037103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d2e5d0 0x55c182d35f50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.748+0000 7f245d8e1740 1 -- 172.21.15.94:0/3269037103 >> 172.21.15.94:0/3269037103 conn(0x55c182cf7310 msgr2=0x55c182d35840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.748+0000 7f245d8e1740 1 -- 172.21.15.94:0/3269037103 shutdown_connections 2024-09-19T04:20:35.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.748+0000 7f245d8e1740 1 -- 172.21.15.94:0/3269037103 wait complete. 2024-09-19T04:20:35.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.749+0000 7f245d8e1740 1 -- start start 2024-09-19T04:20:35.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.749+0000 7f245d8e1740 1 Processor -- start 2024-09-19T04:20:35.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d4dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d4e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f245d8e1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 0x55c182dbcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f24592a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d4dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f2458aa3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d4e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f24592a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d4dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54342/0 (socket says 172.21.15.94:54342) 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f2458aa3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d4e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37204/0 (socket says 172.21.15.94:37204) 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f24592a4640 1 -- 172.21.15.94:0/3033469165 learned_addr learned my addr 172.21.15.94:0/3033469165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f243800ce60 con 0x55c182d39370 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f243800e2d0 con 0x55c182d2fb40 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f243800d530 con 0x55c182d2e5d0 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.750+0000 7f2459aa5640 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 0x55c182dbcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f24592a4640 1 -- 172.21.15.94:0/3033469165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 msgr2=0x55c182d4e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f24592a4640 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d4e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f24592a4640 1 -- 172.21.15.94:0/3033469165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 msgr2=0x55c182dbcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f24592a4640 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 0x55c182dbcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f24592a4640 1 -- 172.21.15.94:0/3033469165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c182dbd260 con 0x55c182d2e5d0 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f245d8e1740 10 client.0 _ll_register_callbacks cb 0x7f24440d3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:35.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2450003aa0 con 0x55c182d2e5d0 2024-09-19T04:20:35.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f24592a4640 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d4dc50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2450009850 tx=0x7f2450004520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60cea168b854cd40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:35.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f243cff9640 10 client.25621 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:20:35.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f243cff9640 1 -- 172.21.15.94:0/3033469165 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450035070 con 0x55c182d2e5d0 2024-09-19T04:20:35.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.751+0000 7f243cff9640 1 -- 172.21.15.94:0/3033469165 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f24500035f0 con 0x55c182d2e5d0 2024-09-19T04:20:35.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.752+0000 7f243cff9640 1 -- 172.21.15.94:0/3033469165 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2450003850 con 0x55c182d2e5d0 2024-09-19T04:20:35.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.752+0000 7f245d8e1740 10 client.25621 fetch_fsmap learned FSMap version 34 2024-09-19T04:20:35.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.752+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55c182dc43e0 con 0x55c182d2e5d0 2024-09-19T04:20:35.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.752+0000 7f243cff9640 1 -- 172.21.15.94:0/3033469165 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f24500314a0 con 0x55c182d2e5d0 2024-09-19T04:20:35.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.752+0000 7f245d8e1740 10 client.25621 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:20:35.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.752+0000 7f245d8e1740 10 client.25621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:35.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.752+0000 7f245d8e1740 20 client.25621 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:35.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.752+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c182d59f80 0x55c182dc65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.753+0000 7f245d8e1740 4 client.25621 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:20:35.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.753+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c182c9bd70 con 0x55c182d59f80 2024-09-19T04:20:35.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.753+0000 7f2458aa3640 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c182d59f80 0x55c182dc65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.753+0000 7f243cff9640 10 client.25621 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.753+0000 7f2458aa3640 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c182d59f80 0x55c182dc65c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b05d9eeee36fb84 server_cookie=3b457af73002f224 in_seq=0 out_seq=0 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.753+0000 7f243cff9640 1 -- 172.21.15.94:0/3033469165 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c182c9bd70 con 0x55c182d59f80 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.753+0000 7f243cff9640 10 client.25621 handle_command_reply: tid=0 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.753+0000 7f245d8e1740 1 client.25621 shutdown 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 msgr2=0x55c182d4dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d4dc50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2450009850 tx=0x7f2450004520 comp rx=0 tx=0).stop 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 shutdown_connections 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c182d59f80 0x55c182dc65c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c182d39370 0x55c182dbcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c182d2fb40 0x55c182d4e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 --2- 172.21.15.94:0/3033469165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c182d2e5d0 0x55c182d4dc50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 >> 172.21.15.94:0/3033469165 conn(0x55c182cf7310 msgr2=0x55c182d377e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:35.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f243cff9640 0 client.25621 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:35.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 shutdown_connections 2024-09-19T04:20:35.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 1 -- 172.21.15.94:0/3033469165 wait complete. 2024-09-19T04:20:35.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.754+0000 7f245d8e1740 20 client.25621 trim_cache size 0 max 16384 2024-09-19T04:20:35.759 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:20:35.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 -- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24400a3430 msgr2=0x7f24400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24400a3430 0x7f24400a58f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f244c0049d0 tx=0x7f244c009500 comp rx=0 tx=0).stop 2024-09-19T04:20:35.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 -- 172.21.15.94:0/2852552793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2454101290 msgr2=0x7f2454198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2454101290 0x7f2454198960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f244400e9c0 tx=0x7f244400ee80 comp rx=0 tx=0).stop 2024-09-19T04:20:35.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 -- 172.21.15.94:0/2852552793 shutdown_connections 2024-09-19T04:20:35.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24400a3430 0x7f24400a58f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2454108180 0x7f245419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2454101290 0x7f2454198960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 --2- 172.21.15.94:0/2852552793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24541008e0 0x7f2454198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 -- 172.21.15.94:0/2852552793 >> 172.21.15.94:0/2852552793 conn(0x7f2454077210 msgr2=0x7f2454105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:35.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 -- 172.21.15.94:0/2852552793 shutdown_connections 2024-09-19T04:20:35.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:35.755+0000 7f245b508640 1 -- 172.21.15.94:0/2852552793 wait complete. 2024-09-19T04:20:35.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:35 smithi112 ceph-mon[35051]: pgmap v170: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 1.24k op/s 2024-09-19T04:20:35.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:35 smithi104 ceph-mon[34145]: pgmap v170: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 1.24k op/s 2024-09-19T04:20:38.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:37 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:20:38.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:37 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:20:38.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:37 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:20:38.458 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:20:38.458 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:20:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:39.133 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:38 smithi094 ceph-mon[30972]: pgmap v171: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 24 MiB/s wr, 1.82k op/s 2024-09-19T04:20:39.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:38 smithi112 ceph-mon[35051]: pgmap v171: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 24 MiB/s wr, 1.82k op/s 2024-09-19T04:20:39.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:38 smithi104 ceph-mon[34145]: pgmap v171: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 24 MiB/s wr, 1.82k op/s 2024-09-19T04:20:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.392+0000 7f56e5e61640 1 -- 172.21.15.94:0/3921105152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0102570 msgr2=0x7f56e01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.392+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3921105152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0102570 0x7f56e01029b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009900 tx=0x7f56d402a480 comp rx=0 tx=0).stop 2024-09-19T04:20:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.392+0000 7f56e5e61640 1 -- 172.21.15.94:0/3921105152 shutdown_connections 2024-09-19T04:20:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.392+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3921105152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e0102ef0 0x7f56e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.392+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3921105152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0102570 0x7f56e01029b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.392+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3921105152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56e0100e20 0x7f56e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.392+0000 7f56e5e61640 1 -- 172.21.15.94:0/3921105152 >> 172.21.15.94:0/3921105152 conn(0x7f56e00fc7d0 msgr2=0x7f56e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.393+0000 7f56e5e61640 1 -- 172.21.15.94:0/3921105152 shutdown_connections 2024-09-19T04:20:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.393+0000 7f56e5e61640 1 -- 172.21.15.94:0/3921105152 wait complete. 2024-09-19T04:20:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.393+0000 7f56e5e61640 1 Processor -- start 2024-09-19T04:20:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.394+0000 7f56e5e61640 1 -- start start 2024-09-19T04:20:39.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.394+0000 7f56e5e61640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0100e20 0x7f56e01092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.394+0000 7f56e5e61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e0102570 0x7f56e0109800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.394+0000 7f56e5e61640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56e0102ef0 0x7f56e01aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.394+0000 7f56df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0100e20 0x7f56e01092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.394+0000 7f56df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0100e20 0x7f56e01092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46994/0 (socket says 172.21.15.94:46994) 2024-09-19T04:20:39.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56df7fe640 1 -- 172.21.15.94:0/3538794456 learned_addr learned my addr 172.21.15.94:0/3538794456 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:39.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56e010fd40 con 0x7f56e0100e20 2024-09-19T04:20:39.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56dffff640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56e0102ef0 0x7f56e01aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f56e010fbc0 con 0x7f56e0102ef0 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56deffd640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e0102570 0x7f56e0109800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56e010fec0 con 0x7f56e0102570 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56deffd640 1 -- 172.21.15.94:0/3538794456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56e0102ef0 msgr2=0x7f56e01aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56deffd640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56e0102ef0 0x7f56e01aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56deffd640 1 -- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0100e20 msgr2=0x7f56e01092c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56deffd640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0100e20 0x7f56e01092c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56deffd640 1 -- 172.21.15.94:0/3538794456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e01aac60 con 0x7f56e0102570 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.395+0000 7f56df7fe640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0100e20 0x7f56e01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.396+0000 7f56deffd640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e0102570 0x7f56e0109800 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005940 tx=0x7f56d4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.396+0000 7f56dcff9640 1 -- 172.21.15.94:0/3538794456 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d403f030 con 0x7f56e0102570 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.396+0000 7f56dcff9640 1 -- 172.21.15.94:0/3538794456 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56d4004440 con 0x7f56e0102570 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.396+0000 7f56dcff9640 1 -- 172.21.15.94:0/3538794456 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4035070 con 0x7f56e0102570 2024-09-19T04:20:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.396+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e01aaef0 con 0x7f56e0102570 2024-09-19T04:20:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.396+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e01ab290 con 0x7f56e0102570 2024-09-19T04:20:39.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.399+0000 7f56e823a740 1 Processor -- start 2024-09-19T04:20:39.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.399+0000 7f56e823a740 1 -- start start 2024-09-19T04:20:39.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.399+0000 7f56dcff9640 1 -- 172.21.15.94:0/3538794456 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f56d4042030 con 0x7f56e0102570 2024-09-19T04:20:39.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.399+0000 7f56e823a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62eb60 0x561b6e62ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56e823a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62f480 0x561b6e639800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56e823a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e639d40 0x561b6e63c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56e823a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561b6e594280 con 0x561b6e62f480 2024-09-19T04:20:39.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56e823a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561b6e53bb10 con 0x561b6e639d40 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56e823a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b6e590240 con 0x561b6e62eb60 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62f480 0x561b6e639800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62f480 0x561b6e639800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47006/0 (socket says 172.21.15.94:47006) 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56df7fe640 1 -- 172.21.15.94:0/3278401636 learned_addr learned my addr 172.21.15.94:0/3278401636 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56dffff640 1 --2- 172.21.15.94:0/3278401636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62eb60 0x561b6e62ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56deffd640 1 --2- 172.21.15.94:0/3278401636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e639d40 0x561b6e63c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56df7fe640 1 -- 172.21.15.94:0/3278401636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62eb60 msgr2=0x561b6e62ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56df7fe640 1 --2- 172.21.15.94:0/3278401636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62eb60 0x561b6e62ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56df7fe640 1 -- 172.21.15.94:0/3278401636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e639d40 msgr2=0x561b6e63c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56df7fe640 1 --2- 172.21.15.94:0/3278401636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e639d40 0x561b6e63c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.400+0000 7f56df7fe640 1 -- 172.21.15.94:0/3278401636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6e5481f0 con 0x561b6e62f480 2024-09-19T04:20:39.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.401+0000 7f56df7fe640 1 --2- 172.21.15.94:0/3278401636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62f480 0x561b6e639800 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009fd0 tx=0x7f56d0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5a2bc6e2782b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:39.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.401+0000 7f56dcff9640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f56c80a3430 0x7f56c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.401+0000 7f56c5ffb640 1 -- 172.21.15.94:0/3278401636 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d001e070 con 0x561b6e62f480 2024-09-19T04:20:39.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.401+0000 7f56c5ffb640 1 -- 172.21.15.94:0/3278401636 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56d000fba0 con 0x561b6e62f480 2024-09-19T04:20:39.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.401+0000 7f56c5ffb640 1 -- 172.21.15.94:0/3278401636 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0019c70 con 0x561b6e62f480 2024-09-19T04:20:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.401+0000 7f56dcff9640 1 -- 172.21.15.94:0/3538794456 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f56d40f0030 con 0x7f56e0102570 2024-09-19T04:20:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.401+0000 7f56dffff640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f56c80a3430 0x7f56c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56e823a740 1 -- 172.21.15.94:0/3278401636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62f480 msgr2=0x561b6e639800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56dffff640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f56c80a3430 0x7f56c80a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005fd0 tx=0x7f56d8009500 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56e823a740 1 --2- 172.21.15.94:0/3278401636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62f480 0x561b6e639800 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009fd0 tx=0x7f56d0011ae0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56e823a740 1 -- 172.21.15.94:0/3278401636 shutdown_connections 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56e823a740 1 --2- 172.21.15.94:0/3278401636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e639d40 0x561b6e63c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56e823a740 1 --2- 172.21.15.94:0/3278401636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62f480 0x561b6e639800 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56e823a740 1 --2- 172.21.15.94:0/3278401636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62eb60 0x561b6e62ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56e823a740 1 -- 172.21.15.94:0/3278401636 >> 172.21.15.94:0/3278401636 conn(0x561b6e62b0f0 msgr2=0x561b6e62b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56e823a740 1 -- 172.21.15.94:0/3278401636 shutdown_connections 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.402+0000 7f56e823a740 1 -- 172.21.15.94:0/3278401636 wait complete. 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.403+0000 7f56e823a740 1 -- start start 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.403+0000 7f56e823a740 1 Processor -- start 2024-09-19T04:20:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.403+0000 7f56e823a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62eb60 0x561b6e636be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.403+0000 7f56e823a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62f480 0x561b6e637120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.403+0000 7f56e823a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e639d40 0x561b6e637660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.403+0000 7f56e823a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561b6e63c910 con 0x561b6e639d40 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.403+0000 7f56e823a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561b6e54ef10 con 0x561b6e62eb60 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.403+0000 7f56e823a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b6e63ca90 con 0x561b6e62f480 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56dffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e639d40 0x561b6e637660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56deffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62f480 0x561b6e637120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56dffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e639d40 0x561b6e637660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47008/0 (socket says 172.21.15.94:47008) 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56dffff640 1 -- 172.21.15.94:0/2454242023 learned_addr learned my addr 172.21.15.94:0/2454242023 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56e823a740 10 client.0 _ll_register_callbacks cb 0x561b6e517a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56df7fe640 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62eb60 0x561b6e636be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56deffd640 1 -- 172.21.15.94:0/2454242023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62eb60 msgr2=0x561b6e636be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56deffd640 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62eb60 0x561b6e636be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56deffd640 1 -- 172.21.15.94:0/2454242023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e639d40 msgr2=0x561b6e637660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56deffd640 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e639d40 0x561b6e637660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56deffd640 1 -- 172.21.15.94:0/2454242023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6e3056a0 con 0x561b6e62f480 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56deffd640 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62f480 0x561b6e637120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f56d40ed710 tx=0x7f56d40bc120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2882de52b113c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56beffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56beffd640 1 -- 172.21.15.94:0/2454242023 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d40ede60 con 0x561b6e62f480 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56beffd640 1 -- 172.21.15.94:0/2454242023 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f56d40b89c0 con 0x561b6e62f480 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.404+0000 7f56e823a740 1 -- 172.21.15.94:0/2454242023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56d000fba0 con 0x561b6e62f480 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56beffd640 1 -- 172.21.15.94:0/2454242023 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f56d4049050 con 0x561b6e62f480 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56e823a740 10 client.25633 fetch_fsmap learned FSMap version 35 2024-09-19T04:20:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56e823a740 1 -- 172.21.15.94:0/2454242023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561b6e63c970 con 0x561b6e62f480 2024-09-19T04:20:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56beffd640 1 -- 172.21.15.94:0/2454242023 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56d000fba0 con 0x561b6e62f480 2024-09-19T04:20:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56beffd640 1 -- 172.21.15.94:0/2454242023 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f56d40c4a80 con 0x561b6e62f480 2024-09-19T04:20:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56e823a740 10 client.25633 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:20:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56e823a740 10 client.25633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56e823a740 20 client.25633 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56e823a740 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561b6e6499c0 0x561b6e64bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56e823a740 4 client.25633 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.405+0000 7f56e823a740 1 -- 172.21.15.94:0/2454242023 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b6e5a17f0 con 0x561b6e6499c0 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.406+0000 7f56df7fe640 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561b6e6499c0 0x561b6e64bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.406+0000 7f56beffd640 10 client.25633 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.406+0000 7f56df7fe640 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561b6e6499c0 0x561b6e64bde0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb0826edbc5d96d server_cookie=b0f0dc396653d7cf in_seq=0 out_seq=0 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56beffd640 1 -- 172.21.15.94:0/2454242023 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b6e5a17f0 con 0x561b6e6499c0 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56beffd640 10 client.25633 handle_command_reply: tid=0 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56e823a740 1 client.25633 shutdown 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56e823a740 1 -- 172.21.15.94:0/2454242023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62f480 msgr2=0x561b6e637120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56e823a740 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62f480 0x561b6e637120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f56d40ed710 tx=0x7f56d40bc120 comp rx=0 tx=0).stop 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56e823a740 1 -- 172.21.15.94:0/2454242023 shutdown_connections 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56e823a740 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561b6e6499c0 0x561b6e64bde0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56e823a740 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e639d40 0x561b6e637660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56e823a740 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e62f480 0x561b6e637120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56e823a740 1 --2- 172.21.15.94:0/2454242023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62eb60 0x561b6e636be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56e823a740 1 -- 172.21.15.94:0/2454242023 >> 172.21.15.94:0/2454242023 conn(0x561b6e62b0f0 msgr2=0x561b6e62a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.407+0000 7f56beffd640 0 client.25633 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.408+0000 7f56e823a740 1 -- 172.21.15.94:0/2454242023 shutdown_connections 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.408+0000 7f56e823a740 1 -- 172.21.15.94:0/2454242023 wait complete. 2024-09-19T04:20:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.408+0000 7f56e823a740 20 client.25633 trim_cache size 0 max 16384 2024-09-19T04:20:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56e823a740 1 Processor -- start 2024-09-19T04:20:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56e823a740 1 -- start start 2024-09-19T04:20:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56e823a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 0x561b6e638a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56e823a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e6516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56e823a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 0x561b6e651c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 0x561b6e638a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 0x561b6e638a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47018/0 (socket says 172.21.15.94:47018) 2024-09-19T04:20:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56df7fe640 1 -- 172.21.15.94:0/2720194314 learned_addr learned my addr 172.21.15.94:0/2720194314 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:39.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56e823a740 1 -- 172.21.15.94:0/2720194314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01b6650 con 0x561b6e62eb60 2024-09-19T04:20:39.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56e823a740 1 -- 172.21.15.94:0/2720194314 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01b3f20 con 0x561b6e62f480 2024-09-19T04:20:39.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56e823a740 1 -- 172.21.15.94:0/2720194314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01b1d70 con 0x561b6e639d40 2024-09-19T04:20:39.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56deffd640 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e6516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56dffff640 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 0x561b6e651c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56df7fe640 1 -- 172.21.15.94:0/2720194314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 msgr2=0x561b6e651c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.434+0000 7f56df7fe640 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 0x561b6e651c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.435+0000 7f56df7fe640 1 -- 172.21.15.94:0/2720194314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 msgr2=0x561b6e6516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.435+0000 7f56df7fe640 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e6516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.435+0000 7f56df7fe640 1 -- 172.21.15.94:0/2720194314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6e635130 con 0x561b6e62eb60 2024-09-19T04:20:39.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.435+0000 7f56deffd640 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e6516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:39.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.435+0000 7f56df7fe640 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 0x561b6e638a40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f56d001c040 tx=0x7f56d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13034b822ee563 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.435+0000 7f56c57fa640 1 -- 172.21.15.94:0/2720194314 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d001e040 con 0x561b6e62eb60 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.435+0000 7f56c57fa640 1 -- 172.21.15.94:0/2720194314 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56d000fba0 con 0x561b6e62eb60 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.435+0000 7f56c57fa640 1 -- 172.21.15.94:0/2720194314 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0019cf0 con 0x561b6e62eb60 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.436+0000 7f56e823a740 1 -- 172.21.15.94:0/2720194314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 msgr2=0x561b6e638a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.436+0000 7f56e823a740 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 0x561b6e638a40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f56d001c040 tx=0x7f56d0014000 comp rx=0 tx=0).stop 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.436+0000 7f56e823a740 1 -- 172.21.15.94:0/2720194314 shutdown_connections 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.436+0000 7f56e823a740 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 0x561b6e651c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.436+0000 7f56e823a740 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e6516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.436+0000 7f56e823a740 1 --2- 172.21.15.94:0/2720194314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 0x561b6e638a40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.436+0000 7f56e823a740 1 -- 172.21.15.94:0/2720194314 >> 172.21.15.94:0/2720194314 conn(0x561b6e62b0f0 msgr2=0x561b6e638330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.437+0000 7f56e823a740 1 -- 172.21.15.94:0/2720194314 shutdown_connections 2024-09-19T04:20:39.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.437+0000 7f56e823a740 1 -- 172.21.15.94:0/2720194314 wait complete. 2024-09-19T04:20:39.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.437+0000 7f56e823a740 1 -- start start 2024-09-19T04:20:39.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.438+0000 7f56e823a740 1 Processor -- start 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.438+0000 7f56e823a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 0x561b6e63db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.438+0000 7f56e823a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e63e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.438+0000 7f56e823a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 0x561b6e63e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56e823a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01b23e0 con 0x561b6e62eb60 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56e823a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01b3850 con 0x561b6e62f480 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56e823a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01b2ab0 con 0x561b6e639d40 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56deffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e63e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56deffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e63e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37250/0 (socket says 172.21.15.94:37250) 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56deffd640 1 -- 172.21.15.94:0/1613638209 learned_addr learned my addr 172.21.15.94:0/1613638209 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56e823a740 10 client.0 _ll_register_callbacks cb 0x7f56d4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56dffff640 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 0x561b6e63e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56deffd640 1 -- 172.21.15.94:0/1613638209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 msgr2=0x561b6e63e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56deffd640 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 0x561b6e63e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56deffd640 1 -- 172.21.15.94:0/1613638209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 msgr2=0x561b6e63db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56deffd640 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 0x561b6e63db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.439+0000 7f56deffd640 1 -- 172.21.15.94:0/1613638209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6e6b3790 con 0x561b6e62f480 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.440+0000 7f56deffd640 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e63e080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56d40d5260 tx=0x7f56d402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2508717e06b6b19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.440+0000 7f56beffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.440+0000 7f56beffd640 1 -- 172.21.15.94:0/1613638209 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d40d9060 con 0x561b6e62f480 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.440+0000 7f56beffd640 1 -- 172.21.15.94:0/1613638209 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f56d40f30f0 con 0x561b6e62f480 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.440+0000 7f56e823a740 1 -- 172.21.15.94:0/1613638209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56d000fba0 con 0x561b6e62f480 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.441+0000 7f56beffd640 1 -- 172.21.15.94:0/1613638209 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f56d40ede60 con 0x561b6e62f480 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.441+0000 7f56e823a740 10 client.25670 fetch_fsmap learned FSMap version 35 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.441+0000 7f56e823a740 1 -- 172.21.15.94:0/1613638209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561b6e6b3790 con 0x561b6e62f480 2024-09-19T04:20:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.441+0000 7f56beffd640 1 -- 172.21.15.94:0/1613638209 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56d000fba0 con 0x561b6e62f480 2024-09-19T04:20:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.441+0000 7f56beffd640 1 -- 172.21.15.94:0/1613638209 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f56d40c0150 con 0x561b6e62f480 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.441+0000 7f56e823a740 10 client.25670 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.441+0000 7f56e823a740 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.441+0000 7f56e823a740 20 client.25670 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.442+0000 7f56e823a740 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561b6e6b4e50 0x561b6e6b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.442+0000 7f56e823a740 4 client.25670 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.442+0000 7f56e823a740 1 -- 172.21.15.94:0/1613638209 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561b6e62bca0 con 0x561b6e6b4e50 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.442+0000 7f56df7fe640 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561b6e6b4e50 0x561b6e6b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.442+0000 7f56df7fe640 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561b6e6b4e50 0x561b6e6b7270 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8089e0b735dae94a server_cookie=d5cd0e943bbfa039 in_seq=0 out_seq=0 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.442+0000 7f56beffd640 10 client.25670 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.443+0000 7f56beffd640 1 -- 172.21.15.94:0/1613638209 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561b6e62bca0 con 0x561b6e6b4e50 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.443+0000 7f56beffd640 10 client.25670 handle_command_reply: tid=0 2024-09-19T04:20:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.443+0000 7f56e823a740 1 client.25670 shutdown 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.443+0000 7f56e823a740 1 -- 172.21.15.94:0/1613638209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 msgr2=0x561b6e63e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.443+0000 7f56e823a740 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e63e080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56d40d5260 tx=0x7f56d402a480 comp rx=0 tx=0).stop 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56e823a740 1 -- 172.21.15.94:0/1613638209 shutdown_connections 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56e823a740 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x561b6e6b4e50 0x561b6e6b7270 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56e823a740 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6e639d40 0x561b6e63e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56beffd640 0 client.25670 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56e823a740 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b6e62f480 0x561b6e63e080 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56e823a740 1 --2- 172.21.15.94:0/1613638209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b6e62eb60 0x561b6e63db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56e823a740 1 -- 172.21.15.94:0/1613638209 >> 172.21.15.94:0/1613638209 conn(0x561b6e62b0f0 msgr2=0x561b6e62a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56e823a740 1 -- 172.21.15.94:0/1613638209 shutdown_connections 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56e823a740 1 -- 172.21.15.94:0/1613638209 wait complete. 2024-09-19T04:20:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.444+0000 7f56e823a740 20 client.25670 trim_cache size 0 max 16384 2024-09-19T04:20:39.448 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:20:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:20:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "def71a9f-becc-40b2-ba4d-32772d2d6bc5", 2024-09-19T04:20:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:20:39.448 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:20:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.445+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f56c80a3430 msgr2=0x7f56c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.445+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f56c80a3430 0x7f56c80a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005fd0 tx=0x7f56d8009500 comp rx=0 tx=0).stop 2024-09-19T04:20:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.445+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e0102570 msgr2=0x7f56e0109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.445+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e0102570 0x7f56e0109800 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005940 tx=0x7f56d4004040 comp rx=0 tx=0).stop 2024-09-19T04:20:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.446+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 shutdown_connections 2024-09-19T04:20:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.446+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f56c80a3430 0x7f56c80a58f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.446+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56e0102ef0 0x7f56e01aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.446+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e0102570 0x7f56e0109800 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.446+0000 7f56e5e61640 1 --2- 172.21.15.94:0/3538794456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56e0100e20 0x7f56e01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.446+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 >> 172.21.15.94:0/3538794456 conn(0x7f56e00fc7d0 msgr2=0x7f56e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.446+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 shutdown_connections 2024-09-19T04:20:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:39.446+0000 7f56e5e61640 1 -- 172.21.15.94:0/3538794456 wait complete. 2024-09-19T04:20:40.158 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:20:40.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:20:40.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:20:40.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:20:40.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:20:40.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.159 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:39 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.194 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:39 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:20:40.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:20:40.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:20:40.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:20:40.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:20:40.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:39 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:40.546 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 265.0 s, 1011.4 tps, lat 96.323 ms stddev 153.783 2024-09-19T04:20:40.986 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:40 smithi094 ceph-mon[30972]: pgmap v172: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-19T04:20:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.114+0000 7fe206081640 1 -- 172.21.15.94:0/4121931419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe200102d90 msgr2=0x7fe200103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.114+0000 7fe206081640 1 --2- 172.21.15.94:0/4121931419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe200102d90 0x7fe200103170 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009930 tx=0x7fe1f002a420 comp rx=0 tx=0).stop 2024-09-19T04:20:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.115+0000 7fe206081640 1 -- 172.21.15.94:0/4121931419 shutdown_connections 2024-09-19T04:20:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.115+0000 7fe206081640 1 --2- 172.21.15.94:0/4121931419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe200108290 0x7fe20010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.115+0000 7fe206081640 1 --2- 172.21.15.94:0/4121931419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe200103740 0x7fe200107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.115+0000 7fe206081640 1 --2- 172.21.15.94:0/4121931419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe200102d90 0x7fe200103170 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.115+0000 7fe206081640 1 -- 172.21.15.94:0/4121931419 >> 172.21.15.94:0/4121931419 conn(0x7fe2000fc7d0 msgr2=0x7fe2000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:41.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.115+0000 7fe206081640 1 -- 172.21.15.94:0/4121931419 shutdown_connections 2024-09-19T04:20:41.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.115+0000 7fe206081640 1 -- 172.21.15.94:0/4121931419 wait complete. 2024-09-19T04:20:41.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.115+0000 7fe206081640 1 Processor -- start 2024-09-19T04:20:41.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.116+0000 7fe206081640 1 -- start start 2024-09-19T04:20:41.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe206081640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe200102d90 0x7fe2001009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe206081640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe200103740 0x7fe200100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe206081640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe200108290 0x7fe200101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe200103740 0x7fe200100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe206081640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe200111f40 con 0x7fe200108290 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe206081640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe200111dc0 con 0x7fe200103740 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe206081640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001120c0 con 0x7fe200102d90 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe200103740 0x7fe200100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37276/0 (socket says 172.21.15.94:37276) 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe1feffd640 1 -- 172.21.15.94:0/1256805520 learned_addr learned my addr 172.21.15.94:0/1256805520 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe200102d90 0x7fe2001009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.117+0000 7fe1fffff640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe200108290 0x7fe200101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe1ff7fe640 1 -- 172.21.15.94:0/1256805520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe200103740 msgr2=0x7fe200100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:41.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe200103740 0x7fe200100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe1ff7fe640 1 -- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe200108290 msgr2=0x7fe200101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:41.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe200108290 0x7fe200101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe1ff7fe640 1 -- 172.21.15.94:0/1256805520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2001aaeb0 con 0x7fe200102d90 2024-09-19T04:20:41.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe200102d90 0x7fe2001009f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1f002a940 tx=0x7fe1f0032ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:41.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe1fcff9640 1 -- 172.21.15.94:0/1256805520 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f003f070 con 0x7fe200102d90 2024-09-19T04:20:41.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe206081640 1 -- 172.21.15.94:0/1256805520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2001ab080 con 0x7fe200102d90 2024-09-19T04:20:41.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe1fcff9640 1 -- 172.21.15.94:0/1256805520 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1f0003ca0 con 0x7fe200102d90 2024-09-19T04:20:41.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe1fcff9640 1 -- 172.21.15.94:0/1256805520 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0035070 con 0x7fe200102d90 2024-09-19T04:20:41.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.118+0000 7fe206081640 1 -- 172.21.15.94:0/1256805520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2001ab610 con 0x7fe200102d90 2024-09-19T04:20:41.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.122+0000 7fe1fcff9640 1 -- 172.21.15.94:0/1256805520 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe1f00037f0 con 0x7fe200102d90 2024-09-19T04:20:41.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.122+0000 7fe20845a740 1 Processor -- start 2024-09-19T04:20:41.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe20845a740 1 -- start start 2024-09-19T04:20:41.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe20845a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41525880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe20845a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41530140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe20845a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c41532a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe20845a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555c4148b2e0 con 0x555c415254a0 2024-09-19T04:20:41.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe20845a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555c41432ce0 con 0x555c41530680 2024-09-19T04:20:41.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe20845a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555c414872a0 con 0x555c41525dc0 2024-09-19T04:20:41.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe1fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41525880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe1fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41525880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47042/0 (socket says 172.21.15.94:47042) 2024-09-19T04:20:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe1fffff640 1 -- 172.21.15.94:0/1696508959 learned_addr learned my addr 172.21.15.94:0/1696508959 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.123+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/1696508959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41530140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.124+0000 7fe1feffd640 1 --2- 172.21.15.94:0/1696508959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c41532a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.124+0000 7fe1fcff9640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe1e80a34b0 0x7fe1e80a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.124+0000 7fe1ff7fe640 1 -- 172.21.15.94:0/1696508959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 msgr2=0x555c41532a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.124+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/1696508959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c41532a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.124+0000 7fe1ff7fe640 1 -- 172.21.15.94:0/1696508959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 msgr2=0x555c41525880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.124+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/1696508959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41525880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.124+0000 7fe1ff7fe640 1 -- 172.21.15.94:0/1696508959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c41417440 con 0x555c41525dc0 2024-09-19T04:20:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.124+0000 7fe1fffff640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe1e80a34b0 0x7fe1e80a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.124+0000 7fe1fcff9640 1 -- 172.21.15.94:0/1256805520 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe1f00f0570 con 0x7fe200102d90 2024-09-19T04:20:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.125+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/1696508959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41530140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00edc50 tx=0x7fe1f004a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=426668b919323b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.125+0000 7fe1e5ffb640 1 -- 172.21.15.94:0/1696508959 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0048070 con 0x555c41525dc0 2024-09-19T04:20:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.125+0000 7fe1e5ffb640 1 -- 172.21.15.94:0/1696508959 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1f0003ae0 con 0x555c41525dc0 2024-09-19T04:20:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.125+0000 7fe1e5ffb640 1 -- 172.21.15.94:0/1696508959 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00bc140 con 0x555c41525dc0 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe1fffff640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe1e80a34b0 0x7fe1e80a5970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8005c40 tx=0x7fe1f8004ab0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe20845a740 1 -- 172.21.15.94:0/1696508959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 msgr2=0x555c41530140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe20845a740 1 --2- 172.21.15.94:0/1696508959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41530140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00edc50 tx=0x7fe1f004a000 comp rx=0 tx=0).stop 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe20845a740 1 -- 172.21.15.94:0/1696508959 shutdown_connections 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe20845a740 1 --2- 172.21.15.94:0/1696508959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c41532a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe20845a740 1 --2- 172.21.15.94:0/1696508959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41530140 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe20845a740 1 --2- 172.21.15.94:0/1696508959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41525880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe20845a740 1 -- 172.21.15.94:0/1696508959 >> 172.21.15.94:0/1696508959 conn(0x555c41521a30 msgr2=0x555c41521e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe20845a740 1 -- 172.21.15.94:0/1696508959 shutdown_connections 2024-09-19T04:20:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.126+0000 7fe20845a740 1 -- 172.21.15.94:0/1696508959 wait complete. 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.127+0000 7fe20845a740 1 -- start start 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.127+0000 7fe20845a740 1 Processor -- start 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.127+0000 7fe20845a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c4152ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.127+0000 7fe20845a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c4152d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe20845a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41530680 0x555c4152d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe20845a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555c41533010 con 0x555c41530680 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe20845a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555c41446050 con 0x555c415254a0 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe20845a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555c41533190 con 0x555c41525dc0 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe20845a740 10 client.0 _ll_register_callbacks cb 0x555c41486160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c4152d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1ff7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c4152ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c4152d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54420/0 (socket says 172.21.15.94:54420) 2024-09-19T04:20:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1ff7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c4152ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37290/0 (socket says 172.21.15.94:37290) 2024-09-19T04:20:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1feffd640 1 -- 172.21.15.94:0/82299057 learned_addr learned my addr 172.21.15.94:0/82299057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1fffff640 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41530680 0x555c4152d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1ff7fe640 1 -- 172.21.15.94:0/82299057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 msgr2=0x555c4152d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c4152d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1ff7fe640 1 -- 172.21.15.94:0/82299057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41530680 msgr2=0x555c4152d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41530680 0x555c4152d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1ff7fe640 1 -- 172.21.15.94:0/82299057 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c411fc3c0 con 0x555c415254a0 2024-09-19T04:20:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c4152ff30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00baa90 tx=0x7fe1f0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554cb70be9d96b90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.128+0000 7fe1deffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:20:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe1deffd640 1 -- 172.21.15.94:0/82299057 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0048040 con 0x555c415254a0 2024-09-19T04:20:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe1deffd640 1 -- 172.21.15.94:0/82299057 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe1f00d09c0 con 0x555c415254a0 2024-09-19T04:20:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe20845a740 1 -- 172.21.15.94:0/82299057 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c41498850 con 0x555c415254a0 2024-09-19T04:20:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe1deffd640 1 -- 172.21.15.94:0/82299057 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe1f00364c0 con 0x555c415254a0 2024-09-19T04:20:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe20845a740 10 client.25679 fetch_fsmap learned FSMap version 35 2024-09-19T04:20:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe20845a740 1 -- 172.21.15.94:0/82299057 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555c41533010 con 0x555c415254a0 2024-09-19T04:20:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe1deffd640 1 -- 172.21.15.94:0/82299057 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555c41498850 con 0x555c415254a0 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe1deffd640 1 -- 172.21.15.94:0/82299057 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fe1f00b8e50 con 0x555c415254a0 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe20845a740 10 client.25679 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.129+0000 7fe20845a740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.130+0000 7fe20845a740 20 client.25679 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.130+0000 7fe20845a740 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555c4153fa70 0x555c41541ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.130+0000 7fe20845a740 4 client.25679 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.130+0000 7fe20845a740 1 -- 172.21.15.94:0/82299057 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c41492d60 con 0x555c4153fa70 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.130+0000 7fe1feffd640 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555c4153fa70 0x555c41541ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.131+0000 7fe1feffd640 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555c4153fa70 0x555c41541ea0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce8fc87949503cf server_cookie=a1e798edd036706a in_seq=0 out_seq=0 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.131+0000 7fe1deffd640 10 client.25679 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe1deffd640 1 -- 172.21.15.94:0/82299057 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c41492d60 con 0x555c4153fa70 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe1deffd640 10 client.25679 handle_command_reply: tid=0 2024-09-19T04:20:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 client.25679 shutdown 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 -- 172.21.15.94:0/82299057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 msgr2=0x555c4152ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c4152ff30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00baa90 tx=0x7fe1f0000bc0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 -- 172.21.15.94:0/82299057 shutdown_connections 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555c4153fa70 0x555c41541ea0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41530680 0x555c4152d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c4152d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 --2- 172.21.15.94:0/82299057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c4152ff30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 -- 172.21.15.94:0/82299057 >> 172.21.15.94:0/82299057 conn(0x555c41521a30 msgr2=0x555c41527760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe1deffd640 0 client.25679 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 -- 172.21.15.94:0/82299057 shutdown_connections 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.132+0000 7fe20845a740 1 -- 172.21.15.94:0/82299057 wait complete. 2024-09-19T04:20:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.133+0000 7fe20845a740 20 client.25679 trim_cache size 0 max 16384 2024-09-19T04:20:41.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.162+0000 7fe20845a740 1 Processor -- start 2024-09-19T04:20:41.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.162+0000 7fe20845a740 1 -- start start 2024-09-19T04:20:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe20845a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41547770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe20845a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41547cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe20845a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c4152a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe20845a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001adfc0 con 0x555c415254a0 2024-09-19T04:20:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe20845a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001ab8c0 con 0x555c41530680 2024-09-19T04:20:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe20845a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe200072290 con 0x555c41525dc0 2024-09-19T04:20:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41547cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41547770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1fffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c4152a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41547cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54430/0 (socket says 172.21.15.94:54430) 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1feffd640 1 -- 172.21.15.94:0/2034097525 learned_addr learned my addr 172.21.15.94:0/2034097525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41547770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47066/0 (socket says 172.21.15.94:47066) 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1fffff640 1 --2- 172.21.15.94:0/2034097525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c4152a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:37294/0 (socket says 172.21.15.94:37294) 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1fffff640 1 -- 172.21.15.94:0/2034097525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 msgr2=0x555c41547cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1fffff640 1 --2- 172.21.15.94:0/2034097525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41547cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1fffff640 1 -- 172.21.15.94:0/2034097525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 msgr2=0x555c41547770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1fffff640 1 --2- 172.21.15.94:0/2034097525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41547770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1fffff640 1 -- 172.21.15.94:0/2034097525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c41525bf0 con 0x555c41530680 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.163+0000 7fe1feffd640 1 --2- 172.21.15.94:0/2034097525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41547cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.164+0000 7fe1fffff640 1 --2- 172.21.15.94:0/2034097525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c4152a490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8014580 tx=0x7fe1f8009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=469cd7feabdeb88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:41.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.164+0000 7fe1e57fa640 1 -- 172.21.15.94:0/2034097525 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f801a040 con 0x555c41530680 2024-09-19T04:20:41.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.164+0000 7fe1e57fa640 1 -- 172.21.15.94:0/2034097525 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1f8012dc0 con 0x555c41530680 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.164+0000 7fe1e57fa640 1 -- 172.21.15.94:0/2034097525 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8023d50 con 0x555c41530680 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.165+0000 7fe20845a740 1 -- 172.21.15.94:0/2034097525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 msgr2=0x555c4152a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.165+0000 7fe20845a740 1 --2- 172.21.15.94:0/2034097525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c4152a490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8014580 tx=0x7fe1f8009b10 comp rx=0 tx=0).stop 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.165+0000 7fe20845a740 1 -- 172.21.15.94:0/2034097525 shutdown_connections 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.165+0000 7fe20845a740 1 --2- 172.21.15.94:0/2034097525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c41530680 0x555c4152a490 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.165+0000 7fe20845a740 1 --2- 172.21.15.94:0/2034097525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41525dc0 0x555c41547cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.165+0000 7fe20845a740 1 --2- 172.21.15.94:0/2034097525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c415254a0 0x555c41547770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.165+0000 7fe20845a740 1 -- 172.21.15.94:0/2034097525 >> 172.21.15.94:0/2034097525 conn(0x555c41521a30 msgr2=0x555c4152c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.165+0000 7fe20845a740 1 -- 172.21.15.94:0/2034097525 shutdown_connections 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.165+0000 7fe20845a740 1 -- 172.21.15.94:0/2034097525 wait complete. 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.166+0000 7fe20845a740 1 -- start start 2024-09-19T04:20:41.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.166+0000 7fe20845a740 1 Processor -- start 2024-09-19T04:20:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.166+0000 7fe20845a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c41536470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.166+0000 7fe20845a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41525dc0 0x555c415369b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.166+0000 7fe20845a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41530680 0x555c415a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.166+0000 7fe20845a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe200072900 con 0x555c41525dc0 2024-09-19T04:20:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.166+0000 7fe20845a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe200073d70 con 0x555c415254a0 2024-09-19T04:20:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.166+0000 7fe20845a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe200072fd0 con 0x555c41530680 2024-09-19T04:20:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe20845a740 10 client.0 _ll_register_callbacks cb 0x7fe1f8009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41525dc0 0x555c415369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41525dc0 0x555c415369b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47074/0 (socket says 172.21.15.94:47074) 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1feffd640 1 -- 172.21.15.94:0/520837827 learned_addr learned my addr 172.21.15.94:0/520837827 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1fffff640 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41530680 0x555c415a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c41536470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1feffd640 1 -- 172.21.15.94:0/520837827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41530680 msgr2=0x555c415a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1feffd640 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41530680 0x555c415a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1feffd640 1 -- 172.21.15.94:0/520837827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 msgr2=0x555c41536470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1feffd640 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c41536470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1feffd640 1 -- 172.21.15.94:0/520837827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c415372b0 con 0x555c41525dc0 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1feffd640 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41525dc0 0x555c415369b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400bea0 tx=0x7fe1f400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb4765204b98c2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.167+0000 7fe1deffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:20:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.168+0000 7fe1deffd640 1 -- 172.21.15.94:0/520837827 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4024470 con 0x555c41525dc0 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.168+0000 7fe1deffd640 1 -- 172.21.15.94:0/520837827 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fe1f4024610 con 0x555c41525dc0 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.168+0000 7fe20845a740 1 -- 172.21.15.94:0/520837827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1f8012dc0 con 0x555c41525dc0 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.168+0000 7fe1deffd640 1 -- 172.21.15.94:0/520837827 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe1f4024810 con 0x555c41525dc0 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.168+0000 7fe20845a740 10 client.16095 fetch_fsmap learned FSMap version 35 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.168+0000 7fe20845a740 1 -- 172.21.15.94:0/520837827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555c415aae60 con 0x555c41525dc0 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.168+0000 7fe1deffd640 1 -- 172.21.15.94:0/520837827 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1f8012dc0 con 0x555c41525dc0 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.168+0000 7fe1deffd640 1 -- 172.21.15.94:0/520837827 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fe1f4012aa0 con 0x555c41525dc0 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.169+0000 7fe20845a740 10 client.16095 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.169+0000 7fe20845a740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.169+0000 7fe20845a740 20 client.16095 populate_metadata read hostname 'smithi094' 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.169+0000 7fe20845a740 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555c415ab5f0 0x555c415ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.169+0000 7fe20845a740 4 client.16095 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.169+0000 7fe20845a740 1 -- 172.21.15.94:0/520837827 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c4152db00 con 0x555c415ab5f0 2024-09-19T04:20:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.169+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555c415ab5f0 0x555c415ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.169+0000 7fe1deffd640 10 client.16095 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.169+0000 7fe1ff7fe640 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555c415ab5f0 0x555c415ada10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd2485cefd1e7b1 server_cookie=f70b6d22422a0304 in_seq=0 out_seq=0 2024-09-19T04:20:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe1deffd640 1 -- 172.21.15.94:0/520837827 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555c4152db00 con 0x555c415ab5f0 2024-09-19T04:20:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe1deffd640 10 client.16095 handle_command_reply: tid=0 2024-09-19T04:20:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 client.16095 shutdown 2024-09-19T04:20:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 -- 172.21.15.94:0/520837827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41525dc0 msgr2=0x555c415369b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41525dc0 0x555c415369b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400bea0 tx=0x7fe1f400bf90 comp rx=0 tx=0).stop 2024-09-19T04:20:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 -- 172.21.15.94:0/520837827 shutdown_connections 2024-09-19T04:20:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555c415ab5f0 0x555c415ada10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe1deffd640 0 client.16095 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c41530680 0x555c415a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c41525dc0 0x555c415369b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 --2- 172.21.15.94:0/520837827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c415254a0 0x555c41536470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 -- 172.21.15.94:0/520837827 >> 172.21.15.94:0/520837827 conn(0x555c41521a30 msgr2=0x555c41520350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 -- 172.21.15.94:0/520837827 shutdown_connections 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.170+0000 7fe20845a740 1 -- 172.21.15.94:0/520837827 wait complete. 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.171+0000 7fe20845a740 20 client.16095 trim_cache size 0 max 16384 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:20:41.175 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:20:41.176 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:20:41.176 INFO:teuthology.orchestra.run.smithi094.stdout: "def71a9f-becc-40b2-ba4d-32772d2d6bc5": { 2024-09-19T04:20:41.176 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:20:41.176 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "def71a9f-becc-40b2-ba4d-32772d2d6bc5", 2024-09-19T04:20:41.176 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:20:41.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:20:41.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:20:41.176 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:20:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.171+0000 7fe206081640 1 -- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe1e80a34b0 msgr2=0x7fe1e80a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.171+0000 7fe206081640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe1e80a34b0 0x7fe1e80a5970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8005c40 tx=0x7fe1f8004ab0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.171+0000 7fe206081640 1 -- 172.21.15.94:0/1256805520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe200102d90 msgr2=0x7fe2001009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.171+0000 7fe206081640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe200102d90 0x7fe2001009f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1f002a940 tx=0x7fe1f0032ad0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.172+0000 7fe206081640 1 -- 172.21.15.94:0/1256805520 shutdown_connections 2024-09-19T04:20:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.172+0000 7fe206081640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe1e80a34b0 0x7fe1e80a5970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.172+0000 7fe206081640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe200108290 0x7fe200101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.172+0000 7fe206081640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe200103740 0x7fe200100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.172+0000 7fe206081640 1 --2- 172.21.15.94:0/1256805520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe200102d90 0x7fe2001009f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.172+0000 7fe206081640 1 -- 172.21.15.94:0/1256805520 >> 172.21.15.94:0/1256805520 conn(0x7fe2000fc7d0 msgr2=0x7fe200105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.172+0000 7fe206081640 1 -- 172.21.15.94:0/1256805520 shutdown_connections 2024-09-19T04:20:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:20:41.172+0000 7fe206081640 1 -- 172.21.15.94:0/1256805520 wait complete. 2024-09-19T04:20:41.185 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:40 smithi112 ceph-mon[35051]: pgmap v172: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-19T04:20:41.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:40 smithi104 ceph-mon[34145]: pgmap v172: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-19T04:20:42.405 INFO:tasks.cephfs.filesystem:scrub status for tag:def71a9f-becc-40b2-ba4d-32772d2d6bc5 - {'path': '/', 'tag': 'def71a9f-becc-40b2-ba4d-32772d2d6bc5', 'options': 'recursive,force'} 2024-09-19T04:20:43.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:42 smithi112 ceph-mon[35051]: pgmap v173: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-19T04:20:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:42 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:20:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:42 smithi094 ceph-mon[30972]: pgmap v173: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-19T04:20:43.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:42 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:20:43.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:42 smithi104 ceph-mon[34145]: pgmap v173: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-19T04:20:43.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:42 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:20:45.166 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:44 smithi104 ceph-mon[34145]: pgmap v174: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-09-19T04:20:45.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:44 smithi112 ceph-mon[35051]: pgmap v174: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-09-19T04:20:45.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:44 smithi094 ceph-mon[30972]: pgmap v174: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-09-19T04:20:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 270.0 s, 1192.8 tps, lat 81.060 ms stddev 117.324 2024-09-19T04:20:47.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:46 smithi112 ceph-mon[35051]: pgmap v175: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 26 MiB/s wr, 1.20k op/s 2024-09-19T04:20:47.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:46 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:20:47.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:46 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:20:47.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:46 smithi094 ceph-mon[30972]: pgmap v175: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 26 MiB/s wr, 1.20k op/s 2024-09-19T04:20:47.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:46 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:20:47.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:46 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:20:47.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:46 smithi104 ceph-mon[34145]: pgmap v175: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 26 MiB/s wr, 1.20k op/s 2024-09-19T04:20:47.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:46 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:20:47.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:46 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:20:48.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:47 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:20:48.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:47 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:20:48.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:47 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:20:49.242 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:48 smithi094 ceph-mon[30972]: pgmap v176: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 38 MiB/s wr, 1.79k op/s 2024-09-19T04:20:49.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:48 smithi112 ceph-mon[35051]: pgmap v176: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 38 MiB/s wr, 1.79k op/s 2024-09-19T04:20:49.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:48 smithi104 ceph-mon[34145]: pgmap v176: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 38 MiB/s wr, 1.79k op/s 2024-09-19T04:20:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 275.0 s, 973.2 tps, lat 108.764 ms stddev 162.306 2024-09-19T04:20:51.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:50 smithi112 ceph-mon[35051]: pgmap v177: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 25 MiB/s wr, 1.20k op/s 2024-09-19T04:20:51.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:50 smithi094 ceph-mon[30972]: pgmap v177: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 25 MiB/s wr, 1.20k op/s 2024-09-19T04:20:51.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:50 smithi104 ceph-mon[34145]: pgmap v177: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 25 MiB/s wr, 1.20k op/s 2024-09-19T04:20:53.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:52 smithi112 ceph-mon[35051]: pgmap v178: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 25 MiB/s wr, 1.20k op/s 2024-09-19T04:20:53.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:52 smithi094 ceph-mon[30972]: pgmap v178: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 25 MiB/s wr, 1.20k op/s 2024-09-19T04:20:53.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:52 smithi104 ceph-mon[34145]: pgmap v178: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 25 MiB/s wr, 1.20k op/s 2024-09-19T04:20:55.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:54 smithi112 ceph-mon[35051]: pgmap v179: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 38 MiB/s wr, 1.80k op/s 2024-09-19T04:20:55.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:54 smithi094 ceph-mon[30972]: pgmap v179: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 38 MiB/s wr, 1.80k op/s 2024-09-19T04:20:55.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:54 smithi104 ceph-mon[34145]: pgmap v179: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 38 MiB/s wr, 1.80k op/s 2024-09-19T04:20:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 280.0 s, 1315.4 tps, lat 74.697 ms stddev 102.006 2024-09-19T04:20:57.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:56 smithi104 ceph-mon[34145]: pgmap v180: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 25 MiB/s wr, 1.19k op/s 2024-09-19T04:20:57.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:56 smithi112 ceph-mon[35051]: pgmap v180: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 25 MiB/s wr, 1.19k op/s 2024-09-19T04:20:57.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:56 smithi094 ceph-mon[30972]: pgmap v180: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 25 MiB/s wr, 1.19k op/s 2024-09-19T04:20:59.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:20:58 smithi112 ceph-mon[35051]: pgmap v181: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 37 MiB/s wr, 1.81k op/s 2024-09-19T04:20:59.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:20:58 smithi094 ceph-mon[30972]: pgmap v181: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 37 MiB/s wr, 1.81k op/s 2024-09-19T04:20:59.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:20:58 smithi104 ceph-mon[34145]: pgmap v181: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 37 MiB/s wr, 1.81k op/s 2024-09-19T04:21:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 285.0 s, 1111.6 tps, lat 91.478 ms stddev 139.482 2024-09-19T04:21:01.210 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:00 smithi094 ceph-mon[30972]: pgmap v182: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 MiB/s wr, 1.22k op/s 2024-09-19T04:21:01.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:00 smithi112 ceph-mon[35051]: pgmap v182: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 MiB/s wr, 1.22k op/s 2024-09-19T04:21:01.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:00 smithi104 ceph-mon[34145]: pgmap v182: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 MiB/s wr, 1.22k op/s 2024-09-19T04:21:03.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:02 smithi112 ceph-mon[35051]: pgmap v183: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 MiB/s wr, 1.22k op/s 2024-09-19T04:21:03.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:02 smithi094 ceph-mon[30972]: pgmap v183: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 MiB/s wr, 1.22k op/s 2024-09-19T04:21:03.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:02 smithi104 ceph-mon[34145]: pgmap v183: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 MiB/s wr, 1.22k op/s 2024-09-19T04:21:05.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:04 smithi112 ceph-mon[35051]: pgmap v184: 97 pgs: 97 active+clean; 3.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 37 MiB/s wr, 1.93k op/s 2024-09-19T04:21:05.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:04 smithi094 ceph-mon[30972]: pgmap v184: 97 pgs: 97 active+clean; 3.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 37 MiB/s wr, 1.93k op/s 2024-09-19T04:21:05.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:04 smithi104 ceph-mon[34145]: pgmap v184: 97 pgs: 97 active+clean; 3.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 37 MiB/s wr, 1.93k op/s 2024-09-19T04:21:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 290.0 s, 1260.0 tps, lat 79.294 ms stddev 157.334 2024-09-19T04:21:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:06 smithi112 ceph-mon[35051]: pgmap v185: 97 pgs: 97 active+clean; 3.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 24 MiB/s wr, 1.34k op/s 2024-09-19T04:21:07.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:06 smithi094 ceph-mon[30972]: pgmap v185: 97 pgs: 97 active+clean; 3.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 24 MiB/s wr, 1.34k op/s 2024-09-19T04:21:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:06 smithi104 ceph-mon[34145]: pgmap v185: 97 pgs: 97 active+clean; 3.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 24 MiB/s wr, 1.34k op/s 2024-09-19T04:21:09.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:08 smithi112 ceph-mon[35051]: pgmap v186: 97 pgs: 97 active+clean; 3.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 36 MiB/s wr, 2.02k op/s 2024-09-19T04:21:09.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:08 smithi094 ceph-mon[30972]: pgmap v186: 97 pgs: 97 active+clean; 3.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 36 MiB/s wr, 2.02k op/s 2024-09-19T04:21:09.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:08 smithi104 ceph-mon[34145]: pgmap v186: 97 pgs: 97 active+clean; 3.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 36 MiB/s wr, 2.02k op/s 2024-09-19T04:21:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 295.0 s, 1240.6 tps, lat 80.650 ms stddev 114.586 2024-09-19T04:21:11.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:10 smithi112 ceph-mon[35051]: pgmap v187: 97 pgs: 97 active+clean; 3.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-09-19T04:21:11.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:10 smithi094 ceph-mon[30972]: pgmap v187: 97 pgs: 97 active+clean; 3.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-09-19T04:21:11.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:10 smithi104 ceph-mon[34145]: pgmap v187: 97 pgs: 97 active+clean; 3.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-09-19T04:21:12.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:21:12.811 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:13.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:12 smithi112 ceph-mon[35051]: pgmap v188: 97 pgs: 97 active+clean; 3.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-09-19T04:21:13.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:12 smithi094 ceph-mon[30972]: pgmap v188: 97 pgs: 97 active+clean; 3.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-09-19T04:21:13.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:12 smithi104 ceph-mon[34145]: pgmap v188: 97 pgs: 97 active+clean; 3.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-09-19T04:21:15.242 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:14 smithi094 ceph-mon[30972]: pgmap v189: 97 pgs: 97 active+clean; 3.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 35 MiB/s wr, 2.06k op/s 2024-09-19T04:21:15.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:14 smithi112 ceph-mon[35051]: pgmap v189: 97 pgs: 97 active+clean; 3.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 35 MiB/s wr, 2.06k op/s 2024-09-19T04:21:15.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:14 smithi104 ceph-mon[34145]: pgmap v189: 97 pgs: 97 active+clean; 3.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 35 MiB/s wr, 2.06k op/s 2024-09-19T04:21:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 300.0 s, 1008.0 tps, lat 99.058 ms stddev 124.632 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.240+0000 7f692658b640 1 -- 172.21.15.94:0/2266012554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6920101430 msgr2=0x7f692010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.240+0000 7f692658b640 1 --2- 172.21.15.94:0/2266012554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6920101430 0x7f692010d960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6918009ca0 tx=0x7f691802a8b0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.240+0000 7f692658b640 1 -- 172.21.15.94:0/2266012554 shutdown_connections 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.240+0000 7f692658b640 1 --2- 172.21.15.94:0/2266012554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6920101430 0x7f692010d960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.240+0000 7f692658b640 1 --2- 172.21.15.94:0/2266012554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6920100a90 0x7f6920100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.240+0000 7f692658b640 1 --2- 172.21.15.94:0/2266012554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6920106a90 0x7f6920106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.240+0000 7f692658b640 1 -- 172.21.15.94:0/2266012554 >> 172.21.15.94:0/2266012554 conn(0x7f69200fc7d0 msgr2=0x7f69200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.241+0000 7f692658b640 1 -- 172.21.15.94:0/2266012554 shutdown_connections 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.241+0000 7f692658b640 1 -- 172.21.15.94:0/2266012554 wait complete. 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.242+0000 7f692658b640 1 Processor -- start 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.243+0000 7f692658b640 1 -- start start 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.243+0000 7f692658b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6920100a90 0x7f692019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.244+0000 7f692658b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6920101430 0x7f692019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.244+0000 7f692658b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6920106a90 0x7f692019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.244+0000 7f692658b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f692010fc90 con 0x7f6920101430 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.244+0000 7f692658b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f692010fb10 con 0x7f6920106a90 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.244+0000 7f692658b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f692010fe10 con 0x7f6920100a90 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.244+0000 7f691ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6920100a90 0x7f692019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.244+0000 7f691ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6920100a90 0x7f692019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47118/0 (socket says 172.21.15.94:47118) 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.244+0000 7f691ffff640 1 -- 172.21.15.94:0/1447238365 learned_addr learned my addr 172.21.15.94:0/1447238365 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.245+0000 7f6924b28640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6920106a90 0x7f692019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.245+0000 7f69177fe640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6920101430 0x7f692019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.245+0000 7f691ffff640 1 -- 172.21.15.94:0/1447238365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6920106a90 msgr2=0x7f692019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.245+0000 7f691ffff640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6920106a90 0x7f692019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.245+0000 7f691ffff640 1 -- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6920101430 msgr2=0x7f692019b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.245+0000 7f691ffff640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6920101430 0x7f692019b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.245+0000 7f691ffff640 1 -- 172.21.15.94:0/1447238365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692019f290 con 0x7f6920100a90 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.246+0000 7f691ffff640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6920100a90 0x7f692019aae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f691000e9f0 tx=0x7f691000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.246+0000 7f691dffb640 1 -- 172.21.15.94:0/1447238365 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691000cd80 con 0x7f6920100a90 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.246+0000 7f691dffb640 1 -- 172.21.15.94:0/1447238365 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69100108e0 con 0x7f6920100a90 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.246+0000 7f691dffb640 1 -- 172.21.15.94:0/1447238365 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691000cd80 con 0x7f6920100a90 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.246+0000 7f692658b640 1 -- 172.21.15.94:0/1447238365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692019f580 con 0x7f6920100a90 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.247+0000 7f692658b640 1 -- 172.21.15.94:0/1447238365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69201a6e60 con 0x7f6920100a90 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f6928964740 1 Processor -- start 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f6928964740 1 -- start start 2024-09-19T04:21:16.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f6928964740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc2cb0 0x563cf9cc3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f6928964740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc35d0 0x563cf9ccd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f6928964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cd0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f6928964740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563cf9c202e0 con 0x563cf9cc35d0 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f6928964740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563cf9bc7ce0 con 0x563cf9cc2cb0 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f6928964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563cf9c1c2a0 con 0x563cf9ccde90 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f691dffb640 1 -- 172.21.15.94:0/1447238365 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f69100186c0 con 0x7f6920100a90 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.251+0000 7f6924b28640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc2cb0 0x563cf9cc3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.252+0000 7f6924b28640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc2cb0 0x563cf9cc3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:55862/0 (socket says 172.21.15.94:55862) 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.252+0000 7f6924b28640 1 -- 172.21.15.94:0/3180784758 learned_addr learned my addr 172.21.15.94:0/3180784758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.252+0000 7f69177fe640 1 --2- 172.21.15.94:0/3180784758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cd0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.252+0000 7f6924b28640 1 -- 172.21.15.94:0/3180784758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 msgr2=0x563cf9cd0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.252+0000 7f6924b28640 1 --2- 172.21.15.94:0/3180784758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cd0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.252+0000 7f6924b28640 1 -- 172.21.15.94:0/3180784758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc35d0 msgr2=0x563cf9ccd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.252+0000 7f6924b28640 1 --2- 172.21.15.94:0/3180784758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc35d0 0x563cf9ccd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.252+0000 7f6924b28640 1 -- 172.21.15.94:0/3180784758 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf9bac440 con 0x563cf9cc2cb0 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.252+0000 7f6924b28640 1 --2- 172.21.15.94:0/3180784758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc2cb0 0x563cf9cc3090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6918035000 tx=0x7f6918001140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f6145a758c547f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.253+0000 7f691dffb640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f690c0a37b0 0x7f690c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.253+0000 7f69167fc640 1 -- 172.21.15.94:0/3180784758 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69180305a0 con 0x563cf9cc2cb0 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.253+0000 7f69167fc640 1 -- 172.21.15.94:0/3180784758 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6918040070 con 0x563cf9cc2cb0 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.253+0000 7f691dffb640 1 -- 172.21.15.94:0/1447238365 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f691000cb80 con 0x7f6920100a90 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.253+0000 7f691ffff640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f690c0a37b0 0x7f690c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.253+0000 7f69167fc640 1 -- 172.21.15.94:0/3180784758 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69180043c0 con 0x563cf9cc2cb0 2024-09-19T04:21:16.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 -- 172.21.15.94:0/3180784758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc2cb0 msgr2=0x563cf9cc3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 --2- 172.21.15.94:0/3180784758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc2cb0 0x563cf9cc3090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6918035000 tx=0x7f6918001140 comp rx=0 tx=0).stop 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f691ffff640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f690c0a37b0 0x7f690c0a5c70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x563cf9cc00a0 tx=0x7f69100cfa60 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 -- 172.21.15.94:0/3180784758 shutdown_connections 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 --2- 172.21.15.94:0/3180784758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cd0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 --2- 172.21.15.94:0/3180784758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc35d0 0x563cf9ccd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 --2- 172.21.15.94:0/3180784758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc2cb0 0x563cf9cc3090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 -- 172.21.15.94:0/3180784758 >> 172.21.15.94:0/3180784758 conn(0x563cf9cbf240 msgr2=0x563cf9cbf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 -- 172.21.15.94:0/3180784758 shutdown_connections 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 -- 172.21.15.94:0/3180784758 wait complete. 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.254+0000 7f6928964740 1 -- start start 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.255+0000 7f6928964740 1 Processor -- start 2024-09-19T04:21:16.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.255+0000 7f6928964740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9ccd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.255+0000 7f6928964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9cc35d0 0x563cf9cca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.255+0000 7f6928964740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9ccde90 0x563cf9ccaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.255+0000 7f6928964740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563cf9cd0b00 con 0x563cf9cc2cb0 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.255+0000 7f6928964740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563cf9bdb050 con 0x563cf9ccde90 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.255+0000 7f6928964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563cf9cd0c80 con 0x563cf9cc35d0 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.255+0000 7f6928964740 10 client.0 _ll_register_callbacks cb 0x563cf9c1b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f6924b28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9ccd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f69177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9cc35d0 0x563cf9cca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f69177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9cc35d0 0x563cf9cca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47144/0 (socket says 172.21.15.94:47144) 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f69177fe640 1 -- 172.21.15.94:0/1315126584 learned_addr learned my addr 172.21.15.94:0/1315126584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f69177fe640 1 -- 172.21.15.94:0/1315126584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9ccde90 msgr2=0x563cf9ccaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f69177fe640 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9ccde90 0x563cf9ccaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f69177fe640 1 -- 172.21.15.94:0/1315126584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 msgr2=0x563cf9ccd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f69177fe640 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9ccd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f69177fe640 1 -- 172.21.15.94:0/1315126584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf99913c0 con 0x563cf9cc35d0 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.256+0000 7f69177fe640 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9cc35d0 0x563cf9cca9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6908012700 tx=0x7f6908012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c7bd692da9b076d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.257+0000 7f68ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.257+0000 7f68ff7fe640 1 -- 172.21.15.94:0/1315126584 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908011bb0 con 0x563cf9cc35d0 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.257+0000 7f68ff7fe640 1 -- 172.21.15.94:0/1315126584 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f6908011d50 con 0x563cf9cc35d0 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.257+0000 7f6928964740 1 -- 172.21.15.94:0/1315126584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6918040070 con 0x563cf9cc35d0 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.257+0000 7f68ff7fe640 1 -- 172.21.15.94:0/1315126584 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6908005020 con 0x563cf9cc35d0 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.257+0000 7f6928964740 10 client.25669 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.257+0000 7f6928964740 1 -- 172.21.15.94:0/1315126584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563cf9cdc540 con 0x563cf9cc35d0 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.257+0000 7f68ff7fe640 1 -- 172.21.15.94:0/1315126584 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6918040070 con 0x563cf9cc35d0 2024-09-19T04:21:16.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.258+0000 7f68ff7fe640 1 -- 172.21.15.94:0/1315126584 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f690801c470 con 0x563cf9cc35d0 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.258+0000 7f6928964740 10 client.25669 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.258+0000 7f6928964740 10 client.25669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.258+0000 7f6928964740 20 client.25669 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.258+0000 7f6928964740 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563cf9cddaf0 0x563cf9cdffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.258+0000 7f6928964740 4 client.25669 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.258+0000 7f6928964740 1 -- 172.21.15.94:0/1315126584 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cf9c2d850 con 0x563cf9cddaf0 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.258+0000 7f6924b28640 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563cf9cddaf0 0x563cf9cdffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.259+0000 7f6924b28640 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563cf9cddaf0 0x563cf9cdffb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ed9a7c14bd7eaf server_cookie=184978b71e02e476 in_seq=0 out_seq=0 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.259+0000 7f68ff7fe640 10 client.25669 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.260+0000 7f68ff7fe640 1 -- 172.21.15.94:0/1315126584 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563cf9c2d850 con 0x563cf9cddaf0 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.260+0000 7f68ff7fe640 10 client.25669 handle_command_reply: tid=0 2024-09-19T04:21:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.260+0000 7f6928964740 1 client.25669 shutdown 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 -- 172.21.15.94:0/1315126584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9cc35d0 msgr2=0x563cf9cca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9cc35d0 0x563cf9cca9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6908012700 tx=0x7f6908012bc0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 -- 172.21.15.94:0/1315126584 shutdown_connections 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563cf9cddaf0 0x563cf9cdffb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9ccde90 0x563cf9ccaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9cc35d0 0x563cf9cca9f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 --2- 172.21.15.94:0/1315126584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9ccd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 -- 172.21.15.94:0/1315126584 >> 172.21.15.94:0/1315126584 conn(0x563cf9cbf240 msgr2=0x563cf9cc4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f68ff7fe640 0 client.25669 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 -- 172.21.15.94:0/1315126584 shutdown_connections 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 1 -- 172.21.15.94:0/1315126584 wait complete. 2024-09-19T04:21:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.261+0000 7f6928964740 20 client.25669 trim_cache size 0 max 16384 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.293+0000 7f6928964740 1 Processor -- start 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.293+0000 7f6928964740 1 -- start start 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f6928964740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f6928964740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 0x563cf9cc6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f6928964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cc99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f6928964740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6900011120 con 0x563cf9cc2cb0 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f6928964740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f690000e9f0 con 0x563cf9cc35d0 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f6928964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f690000c840 con 0x563cf9ccde90 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f69177fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 0x563cf9cc6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f6924b28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f69177fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 0x563cf9cc6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:55892/0 (socket says 172.21.15.94:55892) 2024-09-19T04:21:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f69177fe640 1 -- 172.21.15.94:0/4217137043 learned_addr learned my addr 172.21.15.94:0/4217137043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.294+0000 7f6924b28640 1 --2- 172.21.15.94:0/4217137043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cc99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.295+0000 7f6924b28640 1 -- 172.21.15.94:0/4217137043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 msgr2=0x563cf9cc99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.295+0000 7f6924b28640 1 --2- 172.21.15.94:0/4217137043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cc99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.295+0000 7f6924b28640 1 -- 172.21.15.94:0/4217137043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 msgr2=0x563cf9cc6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.295+0000 7f6924b28640 1 --2- 172.21.15.94:0/4217137043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 0x563cf9cc6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.295+0000 7f6924b28640 1 -- 172.21.15.94:0/4217137043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf9cbf070 con 0x563cf9cc2cb0 2024-09-19T04:21:16.906 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:21:16.906 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.295+0000 7f6924b28640 1 --2- 172.21.15.94:0/4217137043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd9df0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6918009bf0 tx=0x7f691802a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801f5ffb9731c02a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.295+0000 7f6915ffb640 1 -- 172.21.15.94:0/4217137043 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918003b00 con 0x563cf9cc2cb0 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.296+0000 7f6915ffb640 1 -- 172.21.15.94:0/4217137043 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6918040040 con 0x563cf9cc2cb0 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.296+0000 7f6915ffb640 1 -- 172.21.15.94:0/4217137043 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918004690 con 0x563cf9cc2cb0 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.296+0000 7f6928964740 1 -- 172.21.15.94:0/4217137043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 msgr2=0x563cf9cd9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.296+0000 7f6928964740 1 --2- 172.21.15.94:0/4217137043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd9df0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6918009bf0 tx=0x7f691802a8b0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.297+0000 7f6928964740 1 -- 172.21.15.94:0/4217137043 shutdown_connections 2024-09-19T04:21:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.297+0000 7f6928964740 1 --2- 172.21.15.94:0/4217137043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cc99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.297+0000 7f6928964740 1 --2- 172.21.15.94:0/4217137043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 0x563cf9cc6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.297+0000 7f6928964740 1 --2- 172.21.15.94:0/4217137043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd9df0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.297+0000 7f6928964740 1 -- 172.21.15.94:0/4217137043 >> 172.21.15.94:0/4217137043 conn(0x563cf9cbf240 msgr2=0x563cf9cc92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.297+0000 7f6928964740 1 -- 172.21.15.94:0/4217137043 shutdown_connections 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.297+0000 7f6928964740 1 -- 172.21.15.94:0/4217137043 wait complete. 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.298+0000 7f6928964740 1 -- start start 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.298+0000 7f6928964740 1 Processor -- start 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6928964740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6928964740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 0x563cf9cd1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6928964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cd1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6928964740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f690000ceb0 con 0x563cf9cc2cb0 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6928964740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f690000e320 con 0x563cf9cc35d0 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6928964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f690000d580 con 0x563cf9ccde90 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6928964740 10 client.0 _ll_register_callbacks cb 0x7f69080096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6924b28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6924b28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59586/0 (socket says 172.21.15.94:59586) 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6924b28640 1 -- 172.21.15.94:0/2349783927 learned_addr learned my addr 172.21.15.94:0/2349783927 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6924b28640 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cd1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6924b28640 1 -- 172.21.15.94:0/2349783927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 msgr2=0x563cf9cd1790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6924b28640 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 0x563cf9cd1790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6924b28640 1 -- 172.21.15.94:0/2349783927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 msgr2=0x563cf9cd6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6924b28640 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.299+0000 7f6924b28640 1 -- 172.21.15.94:0/2349783927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf9cd2240 con 0x563cf9ccde90 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.300+0000 7f6924b28640 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cd1cd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6918001110 tx=0x7f6918000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ad08487b4236c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.300+0000 7f68ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.300+0000 7f68ff7fe640 1 -- 172.21.15.94:0/2349783927 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918004100 con 0x563cf9ccde90 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.300+0000 7f68ff7fe640 1 -- 172.21.15.94:0/2349783927 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f6918004410 con 0x563cf9ccde90 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.300+0000 7f6928964740 1 -- 172.21.15.94:0/2349783927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6918040040 con 0x563cf9ccde90 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.300+0000 7f68ff7fe640 1 -- 172.21.15.94:0/2349783927 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6918069050 con 0x563cf9ccde90 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.300+0000 7f6928964740 10 client.25675 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.300+0000 7f6928964740 1 -- 172.21.15.94:0/2349783927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563cf9cb71c0 con 0x563cf9ccde90 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.301+0000 7f68ff7fe640 1 -- 172.21.15.94:0/2349783927 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6918040040 con 0x563cf9ccde90 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.301+0000 7f68ff7fe640 1 -- 172.21.15.94:0/2349783927 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f6918052430 con 0x563cf9ccde90 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.301+0000 7f6928964740 10 client.25675 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.301+0000 7f6928964740 10 client.25675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.301+0000 7f6928964740 20 client.25675 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.301+0000 7f6928964740 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563cf9cb74b0 0x563cf9cb98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.301+0000 7f6928964740 4 client.25675 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.301+0000 7f6928964740 1 -- 172.21.15.94:0/2349783927 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563cf9c27d60 con 0x563cf9cb74b0 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.302+0000 7f69177fe640 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563cf9cb74b0 0x563cf9cb98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.302+0000 7f69177fe640 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563cf9cb74b0 0x563cf9cb98d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323db54f199e8cab server_cookie=e0e911e4c78ad263 in_seq=0 out_seq=0 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.302+0000 7f68ff7fe640 10 client.25675 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.302+0000 7f68ff7fe640 1 -- 172.21.15.94:0/2349783927 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563cf9c27d60 con 0x563cf9cb74b0 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.302+0000 7f68ff7fe640 10 client.25675 handle_command_reply: tid=0 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.302+0000 7f6928964740 1 client.25675 shutdown 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 -- 172.21.15.94:0/2349783927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 msgr2=0x563cf9cd1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cd1cd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6918001110 tx=0x7f6918000f40 comp rx=0 tx=0).stop 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 -- 172.21.15.94:0/2349783927 shutdown_connections 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563cf9cb74b0 0x563cf9cb98d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf9ccde90 0x563cf9cd1cd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563cf9cc35d0 0x563cf9cd1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 --2- 172.21.15.94:0/2349783927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cf9cc2cb0 0x563cf9cd6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 -- 172.21.15.94:0/2349783927 >> 172.21.15.94:0/2349783927 conn(0x563cf9cbf240 msgr2=0x563cf9cc9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f68ff7fe640 0 client.25675 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 -- 172.21.15.94:0/2349783927 shutdown_connections 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 1 -- 172.21.15.94:0/2349783927 wait complete. 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.303+0000 7f6928964740 20 client.25675 trim_cache size 0 max 16384 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 -- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f690c0a37b0 msgr2=0x7f690c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f690c0a37b0 0x7f690c0a5c70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x563cf9cc00a0 tx=0x7f69100cfa60 comp rx=0 tx=0).stop 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 -- 172.21.15.94:0/1447238365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6920100a90 msgr2=0x7f692019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6920100a90 0x7f692019aae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f691000e9f0 tx=0x7f691000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 -- 172.21.15.94:0/1447238365 shutdown_connections 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f690c0a37b0 0x7f690c0a5c70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6920106a90 0x7f692019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6920101430 0x7f692019b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 --2- 172.21.15.94:0/1447238365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6920100a90 0x7f692019aae0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 -- 172.21.15.94:0/1447238365 >> 172.21.15.94:0/1447238365 conn(0x7f69200fc7d0 msgr2=0x7f6920102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 -- 172.21.15.94:0/1447238365 shutdown_connections 2024-09-19T04:21:16.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:16.304+0000 7f692658b640 1 -- 172.21.15.94:0/1447238365 wait complete. 2024-09-19T04:21:17.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:16 smithi112 ceph-mon[35051]: pgmap v190: 97 pgs: 97 active+clean; 3.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 23 MiB/s wr, 1.35k op/s 2024-09-19T04:21:17.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:16 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:21:17.315 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:16 smithi094 ceph-mon[30972]: pgmap v190: 97 pgs: 97 active+clean; 3.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 23 MiB/s wr, 1.35k op/s 2024-09-19T04:21:17.316 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:16 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:21:17.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:16 smithi104 ceph-mon[34145]: pgmap v190: 97 pgs: 97 active+clean; 3.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 23 MiB/s wr, 1.35k op/s 2024-09-19T04:21:17.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:16 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:21:17.785 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:21:17.786 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:21:18.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:17 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:21:18.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:17 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:21:18.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:17 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:21:18.195 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:18.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:17 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:21:18.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:17 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:21:18.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:17 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:21:18.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:17 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:21:18.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:17 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:21:18.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:17 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:21:18.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.828+0000 7f11e6dbe640 1 -- 172.21.15.94:0/1174207848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e010e230 msgr2=0x7f11e010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:18.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.828+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/1174207848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e010e230 0x7f11e010e610 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009930 tx=0x7f11d802a420 comp rx=0 tx=0).stop 2024-09-19T04:21:18.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.828+0000 7f11e6dbe640 1 -- 172.21.15.94:0/1174207848 shutdown_connections 2024-09-19T04:21:18.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.828+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/1174207848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0108b50 0x7f11e0115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:18.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.828+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/1174207848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e01081f0 0x7f11e0108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:18.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.828+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/1174207848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e010e230 0x7f11e010e610 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:18.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.828+0000 7f11e6dbe640 1 -- 172.21.15.94:0/1174207848 >> 172.21.15.94:0/1174207848 conn(0x7f11e0077bb0 msgr2=0x7f11e0079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:18.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.828+0000 7f11e6dbe640 1 -- 172.21.15.94:0/1174207848 shutdown_connections 2024-09-19T04:21:18.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.828+0000 7f11e6dbe640 1 -- 172.21.15.94:0/1174207848 wait complete. 2024-09-19T04:21:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.829+0000 7f11e6dbe640 1 Processor -- start 2024-09-19T04:21:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.829+0000 7f11e6dbe640 1 -- start start 2024-09-19T04:21:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.829+0000 7f11e6dbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e01081f0 0x7f11e01a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.829+0000 7f11e6dbe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0108b50 0x7f11e01a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11e4b5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e01081f0 0x7f11e01a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11dffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0108b50 0x7f11e01a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11e4b5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e01081f0 0x7f11e01a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47198/0 (socket says 172.21.15.94:47198) 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11dffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0108b50 0x7f11e01a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:55912/0 (socket says 172.21.15.94:55912) 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11e6dbe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e010e230 0x7f11e019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11e4b5a640 1 -- 172.21.15.94:0/279269277 learned_addr learned my addr 172.21.15.94:0/279269277 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0118430 con 0x7f11e010e230 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01182b0 con 0x7f11e0108b50 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01185b0 con 0x7f11e01081f0 2024-09-19T04:21:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.830+0000 7f11e535b640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e010e230 0x7f11e019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:18.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.831+0000 7f11e4b5a640 1 -- 172.21.15.94:0/279269277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0108b50 msgr2=0x7f11e01a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:18.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.831+0000 7f11e4b5a640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0108b50 0x7f11e01a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:18.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.831+0000 7f11e4b5a640 1 -- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e010e230 msgr2=0x7f11e019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:18.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.831+0000 7f11e4b5a640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e010e230 0x7f11e019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:18.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.831+0000 7f11e4b5a640 1 -- 172.21.15.94:0/279269277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e019efc0 con 0x7f11e01081f0 2024-09-19T04:21:18.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.831+0000 7f11e4b5a640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e01081f0 0x7f11e01a46a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11d8005ab0 tx=0x7f11d8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:18.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.831+0000 7f11ddffb640 1 -- 172.21.15.94:0/279269277 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d803f070 con 0x7f11e01081f0 2024-09-19T04:21:18.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.831+0000 7f11ddffb640 1 -- 172.21.15.94:0/279269277 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d8031b50 con 0x7f11e01081f0 2024-09-19T04:21:18.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.831+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e019f250 con 0x7f11e01081f0 2024-09-19T04:21:18.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.832+0000 7f11ddffb640 1 -- 172.21.15.94:0/279269277 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8035030 con 0x7f11e01081f0 2024-09-19T04:21:18.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.832+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01ab500 con 0x7f11e01081f0 2024-09-19T04:21:18.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.833+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c0005290 con 0x7f11e01081f0 2024-09-19T04:21:18.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.834+0000 7f11ddffb640 1 -- 172.21.15.94:0/279269277 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f11d8042070 con 0x7f11e01081f0 2024-09-19T04:21:18.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.835+0000 7f11ddffb640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11cc0a3430 0x7f11cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:18.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.836+0000 7f11dffff640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11cc0a3430 0x7f11cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:18.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.836+0000 7f11ddffb640 1 -- 172.21.15.94:0/279269277 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f11d80f02d0 con 0x7f11e01081f0 2024-09-19T04:21:18.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.836+0000 7f11dffff640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11cc0a3430 0x7f11cc0a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11e010d990 tx=0x7f11c8009500 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:18.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:18.842+0000 7f11ddffb640 1 -- 172.21.15.94:0/279269277 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f11d80bcca0 con 0x7f11e01081f0 2024-09-19T04:21:19.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.207+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11c0005850 con 0x7f11e01081f0 2024-09-19T04:21:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.208+0000 7f11ddffb640 1 -- 172.21.15.94:0/279269277 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11918 (secure 0 0 0) 0x7f11d80bc5a0 con 0x7f11e01081f0 2024-09-19T04:21:19.210 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:21:19.210 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":36,"btime":"2024-09-19T04:20:41:895027+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:20:41.814448+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:21:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 36 2024-09-19T04:21:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.211+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11cc0a3430 msgr2=0x7f11cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.211+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11cc0a3430 0x7f11cc0a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11e010d990 tx=0x7f11c8009500 comp rx=0 tx=0).stop 2024-09-19T04:21:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e01081f0 msgr2=0x7f11e01a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e01081f0 0x7f11e01a46a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11d8005ab0 tx=0x7f11d8003b90 comp rx=0 tx=0).stop 2024-09-19T04:21:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 shutdown_connections 2024-09-19T04:21:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11cc0a3430 0x7f11cc0a58f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e010e230 0x7f11e019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0108b50 0x7f11e01a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 --2- 172.21.15.94:0/279269277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e01081f0 0x7f11e01a46a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 >> 172.21.15.94:0/279269277 conn(0x7f11e0077bb0 msgr2=0x7f11e0079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 shutdown_connections 2024-09-19T04:21:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:19.212+0000 7f11e6dbe640 1 -- 172.21.15.94:0/279269277 wait complete. 2024-09-19T04:21:19.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:19 smithi112 ceph-mon[35051]: pgmap v191: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 33 MiB/s wr, 1.96k op/s 2024-09-19T04:21:19.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:19 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/279269277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:21:19.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:19 smithi094 ceph-mon[30972]: pgmap v191: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 33 MiB/s wr, 1.96k op/s 2024-09-19T04:21:19.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:19 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/279269277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:21:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:19 smithi104 ceph-mon[34145]: pgmap v191: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 33 MiB/s wr, 1.96k op/s 2024-09-19T04:21:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:19 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/279269277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:21:20.353 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:21:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 305.0 s, 1117.4 tps, lat 88.370 ms stddev 106.851 2024-09-19T04:21:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:21.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.283+0000 7f21f8d56640 1 -- 172.21.15.94:0/1122060070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4101470 msgr2=0x7f21f4101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.283+0000 7f21f8d56640 1 --2- 172.21.15.94:0/1122060070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4101470 0x7f21f4101850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009900 tx=0x7f21e402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.283+0000 7f21f8d56640 1 -- 172.21.15.94:0/1122060070 shutdown_connections 2024-09-19T04:21:21.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.283+0000 7f21f8d56640 1 --2- 172.21.15.94:0/1122060070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4105de0 0x7f21f410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.283+0000 7f21f8d56640 1 --2- 172.21.15.94:0/1122060070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4101e20 0x7f21f41056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.283+0000 7f21f8d56640 1 --2- 172.21.15.94:0/1122060070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4101470 0x7f21f4101850 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.283+0000 7f21f8d56640 1 -- 172.21.15.94:0/1122060070 >> 172.21.15.94:0/1122060070 conn(0x7f21f4077230 msgr2=0x7f21f40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.283+0000 7f21f8d56640 1 -- 172.21.15.94:0/1122060070 shutdown_connections 2024-09-19T04:21:21.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.283+0000 7f21f8d56640 1 -- 172.21.15.94:0/1122060070 wait complete. 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.284+0000 7f21f8d56640 1 Processor -- start 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.284+0000 7f21f8d56640 1 -- start start 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f8d56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101470 0x7f21f410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f8d56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4101e20 0x7f21f410e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f8d56640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4105de0 0x7f21f410e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f8d56640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111e00 con 0x7f21f4101470 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f8d56640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111c80 con 0x7f21f4105de0 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f8d56640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111f80 con 0x7f21f4101e20 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f2d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4105de0 0x7f21f410e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4101e20 0x7f21f410e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f2d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4105de0 0x7f21f410e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:55924/0 (socket says 172.21.15.94:55924) 2024-09-19T04:21:21.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f2d9d640 1 -- 172.21.15.94:0/3060796163 learned_addr learned my addr 172.21.15.94:0/3060796163 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:21.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4101e20 0x7f21f410e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47220/0 (socket says 172.21.15.94:47220) 2024-09-19T04:21:21.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.285+0000 7f21f259c640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101470 0x7f21f410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.286+0000 7f21f1d9b640 1 -- 172.21.15.94:0/3060796163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4105de0 msgr2=0x7f21f410e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.286+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4105de0 0x7f21f410e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.286+0000 7f21f1d9b640 1 -- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101470 msgr2=0x7f21f410df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.286+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101470 0x7f21f410df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.286+0000 7f21f1d9b640 1 -- 172.21.15.94:0/3060796163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f41a6800 con 0x7f21f4101e20 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.286+0000 7f21f2d9d640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4105de0 0x7f21f410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.286+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4101e20 0x7f21f410e4b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21e800d6d0 tx=0x7f21e800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.286+0000 7f21e37fe640 1 -- 172.21.15.94:0/3060796163 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8017070 con 0x7f21f4101e20 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.287+0000 7f21f8d56640 1 -- 172.21.15.94:0/3060796163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f41a6af0 con 0x7f21f4101e20 2024-09-19T04:21:21.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.287+0000 7f21f8d56640 1 -- 172.21.15.94:0/3060796163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41a7080 con 0x7f21f4101e20 2024-09-19T04:21:21.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.287+0000 7f21e37fe640 1 -- 172.21.15.94:0/3060796163 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21e8009820 con 0x7f21f4101e20 2024-09-19T04:21:21.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.287+0000 7f21e37fe640 1 -- 172.21.15.94:0/3060796163 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8004c40 con 0x7f21f4101e20 2024-09-19T04:21:21.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.290+0000 7f21e37fe640 1 -- 172.21.15.94:0/3060796163 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f21e8005220 con 0x7f21f4101e20 2024-09-19T04:21:21.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.291+0000 7f21e37fe640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f21dc0a34b0 0x7f21dc0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.291+0000 7f21e37fe640 1 -- 172.21.15.94:0/3060796163 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f21e80cf490 con 0x7f21f4101e20 2024-09-19T04:21:21.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.292+0000 7f21f259c640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f21dc0a34b0 0x7f21dc0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.293+0000 7f21fb12f740 1 Processor -- start 2024-09-19T04:21:21.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.294+0000 7f21f259c640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f21dc0a34b0 0x7f21dc0a5970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005ab0 tx=0x7f21e4005a20 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.295+0000 7f21fb12f740 1 -- start start 2024-09-19T04:21:21.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.296+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afbb40 0x557240afa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.296+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.296+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 0x557240b07760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.296+0000 7f21fb12f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557240a602f0 con 0x557240afbb40 2024-09-19T04:21:21.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.296+0000 7f21f2d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 0x557240b07760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47222/0 (socket says 172.21.15.94:47222) 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21f1d9b640 1 -- 172.21.15.94:0/3417741238 learned_addr learned my addr 172.21.15.94:0/3417741238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21fb12f740 1 -- 172.21.15.94:0/3417741238 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557240a07cf0 con 0x557240b05370 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21fb12f740 1 -- 172.21.15.94:0/3417741238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557240a5c2b0 con 0x557240afa5d0 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21f259c640 1 --2- 172.21.15.94:0/3417741238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afbb40 0x557240afa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21f1d9b640 1 -- 172.21.15.94:0/3417741238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 msgr2=0x557240b07760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/3417741238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 0x557240b07760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21f1d9b640 1 -- 172.21.15.94:0/3417741238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afbb40 msgr2=0x557240afa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/3417741238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afbb40 0x557240afa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.297+0000 7f21f1d9b640 1 -- 172.21.15.94:0/3417741238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572409ec3a0 con 0x557240afa5d0 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.298+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/3417741238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afaa10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f21e8099a60 tx=0x7f21e809fab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10bd106d160ed954 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.298+0000 7f21e0ff9640 1 -- 172.21.15.94:0/3417741238 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8020070 con 0x557240afa5d0 2024-09-19T04:21:21.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.298+0000 7f21e0ff9640 1 -- 172.21.15.94:0/3417741238 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21e8097e20 con 0x557240afa5d0 2024-09-19T04:21:21.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.298+0000 7f21e0ff9640 1 -- 172.21.15.94:0/3417741238 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8097500 con 0x557240afa5d0 2024-09-19T04:21:21.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.299+0000 7f21fb12f740 1 -- 172.21.15.94:0/3417741238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 msgr2=0x557240afaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.299+0000 7f21fb12f740 1 --2- 172.21.15.94:0/3417741238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afaa10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f21e8099a60 tx=0x7f21e809fab0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.300+0000 7f21fb12f740 1 -- 172.21.15.94:0/3417741238 shutdown_connections 2024-09-19T04:21:21.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.300+0000 7f21fb12f740 1 --2- 172.21.15.94:0/3417741238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 0x557240b07760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.300+0000 7f21fb12f740 1 --2- 172.21.15.94:0/3417741238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afaa10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.300+0000 7f21fb12f740 1 --2- 172.21.15.94:0/3417741238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afbb40 0x557240afa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.300+0000 7f21fb12f740 1 -- 172.21.15.94:0/3417741238 >> 172.21.15.94:0/3417741238 conn(0x557240ac3310 msgr2=0x557240af6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.301+0000 7f21fb12f740 1 -- 172.21.15.94:0/3417741238 shutdown_connections 2024-09-19T04:21:21.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.301+0000 7f21fb12f740 1 -- 172.21.15.94:0/3417741238 wait complete. 2024-09-19T04:21:21.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.302+0000 7f21fb12f740 1 -- start start 2024-09-19T04:21:21.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.302+0000 7f21fb12f740 1 Processor -- start 2024-09-19T04:21:21.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.303+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afa5d0 0x557240b04de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.303+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.303+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 0x557240b027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.303+0000 7f21fb12f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557240b08070 con 0x557240b05370 2024-09-19T04:21:21.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.303+0000 7f21fb12f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557240a1b060 con 0x557240afa5d0 2024-09-19T04:21:21.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.303+0000 7f21fb12f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557240b081f0 con 0x557240afbb40 2024-09-19T04:21:21.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.303+0000 7f21fb12f740 10 client.0 _ll_register_callbacks cb 0x55724076e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b02090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47236/0 (socket says 172.21.15.94:47236) 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afa5d0 0x557240b04de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f1d9b640 1 -- 172.21.15.94:0/74370263 learned_addr learned my addr 172.21.15.94:0/74370263 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f2d9d640 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 0x557240b027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f259c640 1 -- 172.21.15.94:0/74370263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 msgr2=0x557240b02090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f259c640 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b02090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f259c640 1 -- 172.21.15.94:0/74370263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 msgr2=0x557240b027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f259c640 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 0x557240b027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.304+0000 7f21f259c640 1 -- 172.21.15.94:0/74370263 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572407d13c0 con 0x557240afa5d0 2024-09-19T04:21:21.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.305+0000 7f21f259c640 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afa5d0 0x557240b04de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f21e402a9e0 tx=0x7f21e4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5364f04142205393 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.306+0000 7f21d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:21:21.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.306+0000 7f21d9ffb640 1 -- 172.21.15.94:0/74370263 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e404e070 con 0x557240afa5d0 2024-09-19T04:21:21.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.306+0000 7f21d9ffb640 1 -- 172.21.15.94:0/74370263 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f21e4004b80 con 0x557240afa5d0 2024-09-19T04:21:21.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.306+0000 7f21fb12f740 1 -- 172.21.15.94:0/74370263 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e8097e20 con 0x557240afa5d0 2024-09-19T04:21:21.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.307+0000 7f21d9ffb640 1 -- 172.21.15.94:0/74370263 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f21e405a050 con 0x557240afa5d0 2024-09-19T04:21:21.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.307+0000 7f21fb12f740 10 client.25712 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:21.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.307+0000 7f21fb12f740 1 -- 172.21.15.94:0/74370263 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557240b1c410 con 0x557240afa5d0 2024-09-19T04:21:21.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.308+0000 7f21d9ffb640 1 -- 172.21.15.94:0/74370263 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21e4004d80 con 0x557240afa5d0 2024-09-19T04:21:21.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.308+0000 7f21d9ffb640 1 -- 172.21.15.94:0/74370263 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f21e4032510 con 0x557240afa5d0 2024-09-19T04:21:21.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.308+0000 7f21fb12f740 10 client.25712 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:21.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.308+0000 7f21fb12f740 10 client.25712 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:21:21.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.308+0000 7f21fb12f740 20 client.25712 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:21.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.308+0000 7f21fb12f740 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557240b1d870 0x557240b1fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.308+0000 7f21fb12f740 4 client.25712 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:21.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.308+0000 7f21fb12f740 1 -- 172.21.15.94:0/74370263 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f21e8097e20 con 0x557240b1d870 2024-09-19T04:21:21.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.308+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557240b1d870 0x557240b1fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.309+0000 7f21d9ffb640 10 client.25712 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:21:21.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.309+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557240b1d870 0x557240b1fc60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d238d004d6e21892 server_cookie=4d5eb62554972d4d in_seq=0 out_seq=0 2024-09-19T04:21:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.309+0000 7f21d9ffb640 1 -- 172.21.15.94:0/74370263 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f21e8097e20 con 0x557240b1d870 2024-09-19T04:21:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.309+0000 7f21d9ffb640 10 client.25712 handle_command_reply: tid=0 2024-09-19T04:21:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.309+0000 7f21fb12f740 1 client.25712 shutdown 2024-09-19T04:21:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 -- 172.21.15.94:0/74370263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afa5d0 msgr2=0x557240b04de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afa5d0 0x557240b04de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f21e402a9e0 tx=0x7f21e4009260 comp rx=0 tx=0).stop 2024-09-19T04:21:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 -- 172.21.15.94:0/74370263 shutdown_connections 2024-09-19T04:21:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557240b1d870 0x557240b1fc60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 0x557240b027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21d9ffb640 0 client.25712 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:21:21.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 --2- 172.21.15.94:0/74370263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afa5d0 0x557240b04de0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 -- 172.21.15.94:0/74370263 >> 172.21.15.94:0/74370263 conn(0x557240ac3310 msgr2=0x557240afc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 -- 172.21.15.94:0/74370263 shutdown_connections 2024-09-19T04:21:21.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 1 -- 172.21.15.94:0/74370263 wait complete. 2024-09-19T04:21:21.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.310+0000 7f21fb12f740 20 client.25712 trim_cache size 0 max 16384 2024-09-19T04:21:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.328+0000 7f21fb12f740 1 Processor -- start 2024-09-19T04:21:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.328+0000 7f21fb12f740 1 -- start start 2024-09-19T04:21:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.328+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.328+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afbb40 0x557240afd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 0x557240b25510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21fb12f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21d40110b0 con 0x557240b05370 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21fb12f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21d400e980 con 0x557240afbb40 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21fb12f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21d400c7d0 con 0x557240afa5d0 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47238/0 (socket says 172.21.15.94:47238) 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f259c640 1 -- 172.21.15.94:0/2152234796 learned_addr learned my addr 172.21.15.94:0/2152234796 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/2152234796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afbb40 0x557240afd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f2d9d640 1 --2- 172.21.15.94:0/2152234796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 0x557240b25510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f259c640 1 -- 172.21.15.94:0/2152234796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afbb40 msgr2=0x557240afd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f259c640 1 --2- 172.21.15.94:0/2152234796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afbb40 0x557240afd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f259c640 1 -- 172.21.15.94:0/2152234796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 msgr2=0x557240b25510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f259c640 1 --2- 172.21.15.94:0/2152234796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 0x557240b25510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f259c640 1 -- 172.21.15.94:0/2152234796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557240af7dd0 con 0x557240afa5d0 2024-09-19T04:21:21.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.329+0000 7f21f259c640 1 --2- 172.21.15.94:0/2152234796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afd120 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f21e4000c00 tx=0x7f21e4005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42ce05cbbc861a72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.330+0000 7f21dbfff640 1 -- 172.21.15.94:0/2152234796 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e404e040 con 0x557240afa5d0 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.330+0000 7f21dbfff640 1 -- 172.21.15.94:0/2152234796 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21e4032d50 con 0x557240afa5d0 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.330+0000 7f21dbfff640 1 -- 172.21.15.94:0/2152234796 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4035040 con 0x557240afa5d0 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 -- 172.21.15.94:0/2152234796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 msgr2=0x557240afd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 --2- 172.21.15.94:0/2152234796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afd120 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f21e4000c00 tx=0x7f21e4005b70 comp rx=0 tx=0).stop 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 -- 172.21.15.94:0/2152234796 shutdown_connections 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 --2- 172.21.15.94:0/2152234796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240b05370 0x557240b25510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 --2- 172.21.15.94:0/2152234796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240afbb40 0x557240afd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 --2- 172.21.15.94:0/2152234796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afa5d0 0x557240afd120 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 -- 172.21.15.94:0/2152234796 >> 172.21.15.94:0/2152234796 conn(0x557240ac3310 msgr2=0x557240af5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 -- 172.21.15.94:0/2152234796 shutdown_connections 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 -- 172.21.15.94:0/2152234796 wait complete. 2024-09-19T04:21:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 -- start start 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.331+0000 7f21fb12f740 1 Processor -- start 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afa5d0 0x557240b11d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21fb12f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 0x557240b127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21fb12f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21d400ce40 con 0x557240afa5d0 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21fb12f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21d400e2b0 con 0x557240b05370 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21fb12f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21d400d510 con 0x557240afbb40 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21fb12f740 10 client.0 _ll_register_callbacks cb 0x7f21e4040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47240/0 (socket says 172.21.15.94:47240) 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f1d9b640 1 -- 172.21.15.94:0/991555727 learned_addr learned my addr 172.21.15.94:0/991555727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f2d9d640 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 0x557240b127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f259c640 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afa5d0 0x557240b11d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f1d9b640 1 -- 172.21.15.94:0/991555727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 msgr2=0x557240b127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 0x557240b127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f1d9b640 1 -- 172.21.15.94:0/991555727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afa5d0 msgr2=0x557240b11d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afa5d0 0x557240b11d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f1d9b640 1 -- 172.21.15.94:0/991555727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557240b87780 con 0x557240afbb40 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21f1d9b640 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b122a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f21e8099a60 tx=0x7f21e8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=575bd9dcc10a7117 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.332+0000 7f21d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.333+0000 7f21d9ffb640 1 -- 172.21.15.94:0/991555727 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8020040 con 0x557240afbb40 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.333+0000 7f21fb12f740 1 -- 172.21.15.94:0/991555727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e4032d50 con 0x557240afbb40 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.333+0000 7f21d9ffb640 1 -- 172.21.15.94:0/991555727 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f21e8097500 con 0x557240afbb40 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.333+0000 7f21d9ffb640 1 -- 172.21.15.94:0/991555727 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f21e8097e20 con 0x557240afbb40 2024-09-19T04:21:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.333+0000 7f21fb12f740 10 client.25699 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.333+0000 7f21fb12f740 1 -- 172.21.15.94:0/991555727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557240b88a60 con 0x557240afbb40 2024-09-19T04:21:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.334+0000 7f21d9ffb640 1 -- 172.21.15.94:0/991555727 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21e4032d50 con 0x557240afbb40 2024-09-19T04:21:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.334+0000 7f21d9ffb640 1 -- 172.21.15.94:0/991555727 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f21e809a740 con 0x557240afbb40 2024-09-19T04:21:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.334+0000 7f21fb12f740 10 client.25699 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.334+0000 7f21fb12f740 10 client.25699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:21:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.334+0000 7f21fb12f740 20 client.25699 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.334+0000 7f21fb12f740 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557240b1d870 0x557240b8adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.335+0000 7f21fb12f740 4 client.25699 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.335+0000 7f21fb12f740 1 -- 172.21.15.94:0/991555727 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557240a6d860 con 0x557240b1d870 2024-09-19T04:21:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.335+0000 7f21f259c640 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557240b1d870 0x557240b8adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.335+0000 7f21d9ffb640 10 client.25699 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.335+0000 7f21f259c640 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557240b1d870 0x557240b8adb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34e9de4cf85c62d5 server_cookie=785d7ff8a3852cb0 in_seq=0 out_seq=0 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21d9ffb640 1 -- 172.21.15.94:0/991555727 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557240a6d860 con 0x557240b1d870 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21d9ffb640 10 client.25699 handle_command_reply: tid=0 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 client.25699 shutdown 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 -- 172.21.15.94:0/991555727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 msgr2=0x557240b122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b122a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f21e8099a60 tx=0x7f21e8000bc0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 -- 172.21.15.94:0/991555727 shutdown_connections 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557240b1d870 0x557240b8adb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557240b05370 0x557240b127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557240afbb40 0x557240b122a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 --2- 172.21.15.94:0/991555727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557240afa5d0 0x557240b11d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 -- 172.21.15.94:0/991555727 >> 172.21.15.94:0/991555727 conn(0x557240ac3310 msgr2=0x557240b038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21d9ffb640 0 client.25699 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:21:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 -- 172.21.15.94:0/991555727 shutdown_connections 2024-09-19T04:21:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 1 -- 172.21.15.94:0/991555727 wait complete. 2024-09-19T04:21:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.336+0000 7f21fb12f740 20 client.25699 trim_cache size 0 max 16384 2024-09-19T04:21:21.341 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 -- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f21dc0a34b0 msgr2=0x7f21dc0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f21dc0a34b0 0x7f21dc0a5970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005ab0 tx=0x7f21e4005a20 comp rx=0 tx=0).stop 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 -- 172.21.15.94:0/3060796163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4101e20 msgr2=0x7f21f410e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4101e20 0x7f21f410e4b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21e800d6d0 tx=0x7f21e800db90 comp rx=0 tx=0).stop 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 -- 172.21.15.94:0/3060796163 shutdown_connections 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f21dc0a34b0 0x7f21dc0a5970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21f4105de0 0x7f21f410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4101e20 0x7f21f410e4b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 --2- 172.21.15.94:0/3060796163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101470 0x7f21f410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 -- 172.21.15.94:0/3060796163 >> 172.21.15.94:0/3060796163 conn(0x7f21f4077230 msgr2=0x7f21f4104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 -- 172.21.15.94:0/3060796163 shutdown_connections 2024-09-19T04:21:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:21.337+0000 7f21f8d56640 1 -- 172.21.15.94:0/3060796163 wait complete. 2024-09-19T04:21:21.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:21 smithi094 ceph-mon[30972]: pgmap v192: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 21 MiB/s wr, 1.27k op/s 2024-09-19T04:21:21.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:21 smithi112 ceph-mon[35051]: pgmap v192: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 21 MiB/s wr, 1.27k op/s 2024-09-19T04:21:21.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:21 smithi104 ceph-mon[34145]: pgmap v192: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 21 MiB/s wr, 1.27k op/s 2024-09-19T04:21:22.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:21:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.793+0000 7fa09d555640 1 -- 172.21.15.94:0/2041491839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa098102570 msgr2=0x7fa0981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.793+0000 7fa09d555640 1 --2- 172.21.15.94:0/2041491839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa098102570 0x7fa0981029b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009900 tx=0x7fa08c02a480 comp rx=0 tx=0).stop 2024-09-19T04:21:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.793+0000 7fa09d555640 1 -- 172.21.15.94:0/2041491839 shutdown_connections 2024-09-19T04:21:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.793+0000 7fa09d555640 1 --2- 172.21.15.94:0/2041491839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa098102ef0 0x7fa09810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.793+0000 7fa09d555640 1 --2- 172.21.15.94:0/2041491839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa098102570 0x7fa0981029b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.793+0000 7fa09d555640 1 --2- 172.21.15.94:0/2041491839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098100e20 0x7fa098101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.793+0000 7fa09d555640 1 -- 172.21.15.94:0/2041491839 >> 172.21.15.94:0/2041491839 conn(0x7fa0980fc7d0 msgr2=0x7fa0980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.793+0000 7fa09d555640 1 -- 172.21.15.94:0/2041491839 shutdown_connections 2024-09-19T04:21:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.793+0000 7fa09d555640 1 -- 172.21.15.94:0/2041491839 wait complete. 2024-09-19T04:21:22.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa09d555640 1 Processor -- start 2024-09-19T04:21:22.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa09d555640 1 -- start start 2024-09-19T04:21:22.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa09d555640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa098100e20 0x7fa09819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa09d555640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098102570 0x7fa09819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa09d555640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa098102ef0 0x7fa0981a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa09d555640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa098110e50 con 0x7fa098102ef0 2024-09-19T04:21:22.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa09d555640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa098110cd0 con 0x7fa098100e20 2024-09-19T04:21:22.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa09d555640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa098110fd0 con 0x7fa098102570 2024-09-19T04:21:22.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa0967fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098102570 0x7fa09819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa098100e20 0x7fa09819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa0967fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098102570 0x7fa09819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47478/0 (socket says 172.21.15.94:47478) 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa0967fc640 1 -- 172.21.15.94:0/2646760641 learned_addr learned my addr 172.21.15.94:0/2646760641 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.794+0000 7fa0977fe640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa098102ef0 0x7fa0981a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa0967fc640 1 -- 172.21.15.94:0/2646760641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa098100e20 msgr2=0x7fa09819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa0967fc640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa098100e20 0x7fa09819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa0967fc640 1 -- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa098102ef0 msgr2=0x7fa0981a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa0967fc640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa098102ef0 0x7fa0981a3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa0967fc640 1 -- 172.21.15.94:0/2646760641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0981a3ab0 con 0x7fa098102570 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa096ffd640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa098100e20 0x7fa09819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa0967fc640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098102570 0x7fa09819f260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa08c02aa70 tx=0x7fa08c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa07ffff640 1 -- 172.21.15.94:0/2646760641 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c03f070 con 0x7fa098102570 2024-09-19T04:21:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa09d555640 1 -- 172.21.15.94:0/2646760641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0981a3d40 con 0x7fa098102570 2024-09-19T04:21:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa09d555640 1 -- 172.21.15.94:0/2646760641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0981ab630 con 0x7fa098102570 2024-09-19T04:21:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa07ffff640 1 -- 172.21.15.94:0/2646760641 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa08c03bb20 con 0x7fa098102570 2024-09-19T04:21:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.795+0000 7fa07ffff640 1 -- 172.21.15.94:0/2646760641 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c03f070 con 0x7fa098102570 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.799+0000 7fa09f92e740 1 Processor -- start 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.799+0000 7fa09f92e740 1 -- start start 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.799+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fd98090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.799+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fd98a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.799+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 0x55e23fda5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.799+0000 7fa09f92e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e23fcfe2f0 con 0x55e23fd985d0 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.799+0000 7fa09f92e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e23fca5cf0 con 0x55e23fd99b40 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.799+0000 7fa09f92e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e23fcfa2b0 con 0x55e23fda3370 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.799+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fd98a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.800+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fd98a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38894/0 (socket says 172.21.15.94:38894) 2024-09-19T04:21:22.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.800+0000 7fa096ffd640 1 -- 172.21.15.94:0/584129660 learned_addr learned my addr 172.21.15.94:0/584129660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:22.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.800+0000 7fa07ffff640 1 -- 172.21.15.94:0/2646760641 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa08c042070 con 0x7fa098102570 2024-09-19T04:21:22.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.800+0000 7fa096ffd640 1 -- 172.21.15.94:0/584129660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 msgr2=0x55e23fda5760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:21:22.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.800+0000 7fa096ffd640 1 --2- 172.21.15.94:0/584129660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 0x55e23fda5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.800+0000 7fa096ffd640 1 -- 172.21.15.94:0/584129660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 msgr2=0x55e23fd98090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:22.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.800+0000 7fa096ffd640 1 --2- 172.21.15.94:0/584129660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fd98090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.800+0000 7fa096ffd640 1 -- 172.21.15.94:0/584129660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e23fc8a3a0 con 0x55e23fd985d0 2024-09-19T04:21:22.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.800+0000 7fa096ffd640 1 --2- 172.21.15.94:0/584129660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fd98a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa0880114f0 tx=0x7fa0880119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16163a5042446514 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:22.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.801+0000 7fa07d7fa640 1 -- 172.21.15.94:0/584129660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08801e070 con 0x55e23fd985d0 2024-09-19T04:21:22.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.801+0000 7fa07d7fa640 1 -- 172.21.15.94:0/584129660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0880035f0 con 0x55e23fd985d0 2024-09-19T04:21:22.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.801+0000 7fa07d7fa640 1 -- 172.21.15.94:0/584129660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08800f820 con 0x55e23fd985d0 2024-09-19T04:21:22.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.801+0000 7fa07ffff640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa0800a3630 0x7fa0800a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.801+0000 7fa07ffff640 1 -- 172.21.15.94:0/2646760641 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fa08c0eff20 con 0x7fa098102570 2024-09-19T04:21:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.802+0000 7fa09f92e740 1 -- 172.21.15.94:0/584129660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 msgr2=0x55e23fd98a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.802+0000 7fa09f92e740 1 --2- 172.21.15.94:0/584129660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fd98a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa0880114f0 tx=0x7fa0880119b0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.802+0000 7fa0977fe640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa0800a3630 0x7fa0800a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.802+0000 7fa09f92e740 1 -- 172.21.15.94:0/584129660 shutdown_connections 2024-09-19T04:21:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.802+0000 7fa09f92e740 1 --2- 172.21.15.94:0/584129660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 0x55e23fda5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.802+0000 7fa09f92e740 1 --2- 172.21.15.94:0/584129660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fd98a10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.802+0000 7fa09f92e740 1 --2- 172.21.15.94:0/584129660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fd98090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.802+0000 7fa09f92e740 1 -- 172.21.15.94:0/584129660 >> 172.21.15.94:0/584129660 conn(0x55e23fd61310 msgr2=0x55e23fd94c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.802+0000 7fa09f92e740 1 -- 172.21.15.94:0/584129660 shutdown_connections 2024-09-19T04:21:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.803+0000 7fa09f92e740 1 -- 172.21.15.94:0/584129660 wait complete. 2024-09-19T04:21:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.803+0000 7fa0977fe640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa0800a3630 0x7fa0800a5af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa090005fd0 tx=0x7fa090004a40 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.803+0000 7fa09f92e740 1 -- start start 2024-09-19T04:21:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.803+0000 7fa09f92e740 1 Processor -- start 2024-09-19T04:21:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fda2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fda0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 0x55e23fda07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa09f92e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e23fda6070 con 0x55e23fd985d0 2024-09-19T04:21:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa09f92e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e23fcb9060 con 0x55e23fd99b40 2024-09-19T04:21:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa09f92e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e23fda61f0 con 0x55e23fda3370 2024-09-19T04:21:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa09f92e740 10 client.0 _ll_register_callbacks cb 0x55e23fa0c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fda2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa0967fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fda0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fda2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38902/0 (socket says 172.21.15.94:38902) 2024-09-19T04:21:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa096ffd640 1 -- 172.21.15.94:0/3085428422 learned_addr learned my addr 172.21.15.94:0/3085428422 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa0977fe640 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 0x55e23fda07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa0967fc640 1 -- 172.21.15.94:0/3085428422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 msgr2=0x55e23fda07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa0967fc640 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 0x55e23fda07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa0967fc640 1 -- 172.21.15.94:0/3085428422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 msgr2=0x55e23fda2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa0967fc640 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fda2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa0967fc640 1 -- 172.21.15.94:0/3085428422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e23fa6f3c0 con 0x55e23fd99b40 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.804+0000 7fa0977fe640 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 0x55e23fda07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.805+0000 7fa0967fc640 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fda0090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0ed660 tx=0x7fa08c0bc130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8f8323cf9818e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.805+0000 7fa07aff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.805+0000 7fa07aff5640 1 -- 172.21.15.94:0/3085428422 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c049030 con 0x55e23fd99b40 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.805+0000 7fa09f92e740 1 -- 172.21.15.94:0/3085428422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0880035f0 con 0x55e23fd99b40 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.805+0000 7fa07aff5640 1 -- 172.21.15.94:0/3085428422 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fa08c0b8910 con 0x55e23fd99b40 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.805+0000 7fa07aff5640 1 -- 172.21.15.94:0/3085428422 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa08c0c05a0 con 0x55e23fd99b40 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.805+0000 7fa09f92e740 10 client.25727 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.805+0000 7fa09f92e740 1 -- 172.21.15.94:0/3085428422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e23fdb1ba0 con 0x55e23fd99b40 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.806+0000 7fa07aff5640 1 -- 172.21.15.94:0/3085428422 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0880035f0 con 0x55e23fd99b40 2024-09-19T04:21:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.806+0000 7fa07aff5640 1 -- 172.21.15.94:0/3085428422 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fa08c0c0120 con 0x55e23fd99b40 2024-09-19T04:21:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.806+0000 7fa09f92e740 10 client.25727 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.806+0000 7fa09f92e740 10 client.25727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:21:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.806+0000 7fa09f92e740 20 client.25727 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.806+0000 7fa09f92e740 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55e23fdb3230 0x55e23fdb5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.806+0000 7fa09f92e740 4 client.25727 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.806+0000 7fa09f92e740 1 -- 172.21.15.94:0/3085428422 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e23fd0b860 con 0x55e23fdb3230 2024-09-19T04:21:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.806+0000 7fa096ffd640 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55e23fdb3230 0x55e23fdb5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.807+0000 7fa096ffd640 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55e23fdb3230 0x55e23fdb5620 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dc294d14c67f1e4 server_cookie=21ad9d1d939628df in_seq=0 out_seq=0 2024-09-19T04:21:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.807+0000 7fa07aff5640 10 client.25727 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:21:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa07aff5640 1 -- 172.21.15.94:0/3085428422 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e23fd0b860 con 0x55e23fdb3230 2024-09-19T04:21:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa07aff5640 10 client.25727 handle_command_reply: tid=0 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 client.25727 shutdown 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 -- 172.21.15.94:0/3085428422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 msgr2=0x55e23fda0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fda0090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0ed660 tx=0x7fa08c0bc130 comp rx=0 tx=0).stop 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 -- 172.21.15.94:0/3085428422 shutdown_connections 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55e23fdb3230 0x55e23fdb5620 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fda3370 0x55e23fda07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa07aff5640 0 client.25727 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fda0090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 --2- 172.21.15.94:0/3085428422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fd985d0 0x55e23fda2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 -- 172.21.15.94:0/3085428422 >> 172.21.15.94:0/3085428422 conn(0x55e23fd61310 msgr2=0x55e23fd9a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 -- 172.21.15.94:0/3085428422 shutdown_connections 2024-09-19T04:21:22.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 1 -- 172.21.15.94:0/3085428422 wait complete. 2024-09-19T04:21:22.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.808+0000 7fa09f92e740 20 client.25727 trim_cache size 0 max 16384 2024-09-19T04:21:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.831+0000 7fa09f92e740 1 Processor -- start 2024-09-19T04:21:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.831+0000 7fa09f92e740 1 -- start start 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd985d0 0x55e23fd94c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd99b40 0x55e23fdaf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 0x55e23fdbaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd985d0 0x55e23fd94c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd985d0 0x55e23fd94c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:60750/0 (socket says 172.21.15.94:60750) 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa09f92e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981b6680 con 0x55e23fda3370 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa09f92e740 1 -- 172.21.15.94:0/1343864547 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981b3f50 con 0x55e23fd985d0 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa096ffd640 1 -- 172.21.15.94:0/1343864547 learned_addr learned my addr 172.21.15.94:0/1343864547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa09f92e740 1 -- 172.21.15.94:0/1343864547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981b1da0 con 0x55e23fd99b40 2024-09-19T04:21:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa0967fc640 1 --2- 172.21.15.94:0/1343864547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd99b40 0x55e23fdaf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.832+0000 7fa0977fe640 1 --2- 172.21.15.94:0/1343864547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 0x55e23fdbaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.833+0000 7fa0967fc640 1 -- 172.21.15.94:0/1343864547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd985d0 msgr2=0x55e23fd94c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.833+0000 7fa0967fc640 1 --2- 172.21.15.94:0/1343864547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd985d0 0x55e23fd94c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.833+0000 7fa0967fc640 1 -- 172.21.15.94:0/1343864547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 msgr2=0x55e23fdbaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.833+0000 7fa0967fc640 1 --2- 172.21.15.94:0/1343864547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 0x55e23fdbaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.833+0000 7fa0967fc640 1 -- 172.21.15.94:0/1343864547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e23fda5f50 con 0x55e23fd99b40 2024-09-19T04:21:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.833+0000 7fa0967fc640 1 --2- 172.21.15.94:0/1343864547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd99b40 0x55e23fdaf310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0ed710 tx=0x7fa08c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca94c2efaec44804 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.833+0000 7fa07cff9640 1 -- 172.21.15.94:0/1343864547 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c049070 con 0x55e23fd99b40 2024-09-19T04:21:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.833+0000 7fa07cff9640 1 -- 172.21.15.94:0/1343864547 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa08c0c4e60 con 0x55e23fd99b40 2024-09-19T04:21:22.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.833+0000 7fa07cff9640 1 -- 172.21.15.94:0/1343864547 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0ed8e0 con 0x55e23fd99b40 2024-09-19T04:21:22.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.834+0000 7fa09f92e740 1 -- 172.21.15.94:0/1343864547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd99b40 msgr2=0x55e23fdaf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.834+0000 7fa09f92e740 1 --2- 172.21.15.94:0/1343864547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd99b40 0x55e23fdaf310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0ed710 tx=0x7fa08c004610 comp rx=0 tx=0).stop 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.834+0000 7fa09f92e740 1 -- 172.21.15.94:0/1343864547 shutdown_connections 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.834+0000 7fa09f92e740 1 --2- 172.21.15.94:0/1343864547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 0x55e23fdbaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.834+0000 7fa09f92e740 1 --2- 172.21.15.94:0/1343864547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd99b40 0x55e23fdaf310 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.834+0000 7fa09f92e740 1 --2- 172.21.15.94:0/1343864547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd985d0 0x55e23fd94c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.834+0000 7fa09f92e740 1 -- 172.21.15.94:0/1343864547 >> 172.21.15.94:0/1343864547 conn(0x55e23fd61310 msgr2=0x55e23fd9f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.834+0000 7fa09f92e740 1 -- 172.21.15.94:0/1343864547 shutdown_connections 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.834+0000 7fa09f92e740 1 -- 172.21.15.94:0/1343864547 wait complete. 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.835+0000 7fa09f92e740 1 -- start start 2024-09-19T04:21:22.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.835+0000 7fa09f92e740 1 Processor -- start 2024-09-19T04:21:22.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.835+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd985d0 0x55e23fdab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.835+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fdabba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.835+0000 7fa09f92e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 0x55e23fd9c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa09f92e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981b2410 con 0x55e23fda3370 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa09f92e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981b3880 con 0x55e23fd99b40 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa09f92e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981b2ae0 con 0x55e23fd985d0 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa09f92e740 10 client.0 _ll_register_callbacks cb 0x7fa08c031540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd985d0 0x55e23fdab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa0967fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fdabba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd985d0 0x55e23fdab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47502/0 (socket says 172.21.15.94:47502) 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa096ffd640 1 -- 172.21.15.94:0/476675820 learned_addr learned my addr 172.21.15.94:0/476675820 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa0977fe640 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 0x55e23fd9c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa096ffd640 1 -- 172.21.15.94:0/476675820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 msgr2=0x55e23fdabba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:22.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa096ffd640 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fdabba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa096ffd640 1 -- 172.21.15.94:0/476675820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 msgr2=0x55e23fd9c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa096ffd640 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 0x55e23fd9c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa096ffd640 1 -- 172.21.15.94:0/476675820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e23fd9d0d0 con 0x55e23fd985d0 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa0977fe640 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 0x55e23fd9c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa096ffd640 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd985d0 0x55e23fdab660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa088009250 tx=0x7fa0880036a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0b2362c456b0ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa07aff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.836+0000 7fa07aff5640 1 -- 172.21.15.94:0/476675820 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08801e040 con 0x55e23fd985d0 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa07aff5640 1 -- 172.21.15.94:0/476675820 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fa088003a80 con 0x55e23fd985d0 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa09f92e740 1 -- 172.21.15.94:0/476675820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa08c0c4e60 con 0x55e23fd985d0 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa07aff5640 1 -- 172.21.15.94:0/476675820 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa088003f10 con 0x55e23fd985d0 2024-09-19T04:21:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa09f92e740 10 client.25714 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa09f92e740 1 -- 172.21.15.94:0/476675820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e23fe1d920 con 0x55e23fd985d0 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa07aff5640 1 -- 172.21.15.94:0/476675820 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0880054f0 con 0x55e23fd985d0 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa07aff5640 1 -- 172.21.15.94:0/476675820 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fa088004740 con 0x55e23fd985d0 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa09f92e740 10 client.25714 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa09f92e740 10 client.25714 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa09f92e740 20 client.25714 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa09f92e740 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55e23fdb3230 0x55e23fe1fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa09f92e740 4 client.25714 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa09f92e740 1 -- 172.21.15.94:0/476675820 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa08c0c4e60 con 0x55e23fdb3230 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.837+0000 7fa0967fc640 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55e23fdb3230 0x55e23fe1fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa07aff5640 10 client.25714 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa0967fc640 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55e23fdb3230 0x55e23fe1fe10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae52e7df50a1251c server_cookie=4bfab66041cd2c48 in_seq=0 out_seq=0 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa07aff5640 1 -- 172.21.15.94:0/476675820 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa08c0c4e60 con 0x55e23fdb3230 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa07aff5640 10 client.25714 handle_command_reply: tid=0 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 client.25714 shutdown 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 -- 172.21.15.94:0/476675820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd985d0 msgr2=0x55e23fdab660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd985d0 0x55e23fdab660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa088009250 tx=0x7fa0880036a0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 -- 172.21.15.94:0/476675820 shutdown_connections 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55e23fdb3230 0x55e23fe1fe10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e23fda3370 0x55e23fd9c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e23fd99b40 0x55e23fdabba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 --2- 172.21.15.94:0/476675820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e23fd985d0 0x55e23fdab660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 -- 172.21.15.94:0/476675820 >> 172.21.15.94:0/476675820 conn(0x55e23fd61310 msgr2=0x55e23fd934e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:22.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa07aff5640 0 client.25714 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:21:22.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.838+0000 7fa09f92e740 1 -- 172.21.15.94:0/476675820 shutdown_connections 2024-09-19T04:21:22.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.839+0000 7fa09f92e740 1 -- 172.21.15.94:0/476675820 wait complete. 2024-09-19T04:21:22.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.839+0000 7fa09f92e740 20 client.25714 trim_cache size 0 max 16384 2024-09-19T04:21:22.847 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.839+0000 7fa09d555640 1 -- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa0800a3630 msgr2=0x7fa0800a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.839+0000 7fa09d555640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa0800a3630 0x7fa0800a5af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa090005fd0 tx=0x7fa090004a40 comp rx=0 tx=0).stop 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.839+0000 7fa09d555640 1 -- 172.21.15.94:0/2646760641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098102570 msgr2=0x7fa09819f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.839+0000 7fa09d555640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098102570 0x7fa09819f260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa08c02aa70 tx=0x7fa08c003c10 comp rx=0 tx=0).stop 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.840+0000 7fa09d555640 1 -- 172.21.15.94:0/2646760641 shutdown_connections 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.840+0000 7fa09d555640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa0800a3630 0x7fa0800a5af0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.840+0000 7fa09d555640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa098102ef0 0x7fa0981a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.840+0000 7fa09d555640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098102570 0x7fa09819f260 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.840+0000 7fa09d555640 1 --2- 172.21.15.94:0/2646760641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa098100e20 0x7fa09819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.840+0000 7fa09d555640 1 -- 172.21.15.94:0/2646760641 >> 172.21.15.94:0/2646760641 conn(0x7fa0980fc7d0 msgr2=0x7fa09806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:22.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.840+0000 7fa09d555640 1 -- 172.21.15.94:0/2646760641 shutdown_connections 2024-09-19T04:21:22.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:22.840+0000 7fa09d555640 1 -- 172.21.15.94:0/2646760641 wait complete. 2024-09-19T04:21:23.542 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:23 smithi094 ceph-mon[30972]: pgmap v193: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 21 MiB/s wr, 1.27k op/s 2024-09-19T04:21:23.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:21:23.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:23 smithi112 ceph-mon[35051]: pgmap v193: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 21 MiB/s wr, 1.27k op/s 2024-09-19T04:21:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:23 smithi104 ceph-mon[34145]: pgmap v193: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 21 MiB/s wr, 1.27k op/s 2024-09-19T04:21:24.033 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.825+0000 7ff11f7fe640 1 -- 172.21.15.94:0/4017826249 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11802c040 con 0x7ff120114ab0 2024-09-19T04:21:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.826+0000 7ff126268640 1 -- 172.21.15.94:0/4017826249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff120114ab0 msgr2=0x7ff120116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.826+0000 7ff126268640 1 --2- 172.21.15.94:0/4017826249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff120114ab0 0x7ff120116ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff118009ca0 tx=0x7ff11802a950 comp rx=0 tx=0).stop 2024-09-19T04:21:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.826+0000 7ff126268640 1 -- 172.21.15.94:0/4017826249 shutdown_connections 2024-09-19T04:21:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.826+0000 7ff126268640 1 --2- 172.21.15.94:0/4017826249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff120114ab0 0x7ff120116ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.826+0000 7ff126268640 1 --2- 172.21.15.94:0/4017826249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff120109b10 0x7ff120114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.826+0000 7ff126268640 1 --2- 172.21.15.94:0/4017826249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1201091f0 0x7ff1201095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.826+0000 7ff126268640 1 -- 172.21.15.94:0/4017826249 >> 172.21.15.94:0/4017826249 conn(0x7ff12006e140 msgr2=0x7ff12006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:24.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.827+0000 7ff126268640 1 -- 172.21.15.94:0/4017826249 shutdown_connections 2024-09-19T04:21:24.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.827+0000 7ff126268640 1 -- 172.21.15.94:0/4017826249 wait complete. 2024-09-19T04:21:24.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.827+0000 7ff126268640 1 Processor -- start 2024-09-19T04:21:24.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.827+0000 7ff126268640 1 -- start start 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff126268640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1201091f0 0x7ff120114470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff126268640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff120109b10 0x7ff120111520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff126268640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff120114ab0 0x7ff120111ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff126268640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff12011c8b0 con 0x7ff1201091f0 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff126268640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff12011c730 con 0x7ff120114ab0 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff126268640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff12011ca30 con 0x7ff120109b10 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff124805640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff120114ab0 0x7ff120111ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff124805640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff120114ab0 0x7ff120111ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:60768/0 (socket says 172.21.15.94:60768) 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff124805640 1 -- 172.21.15.94:0/809355476 learned_addr learned my addr 172.21.15.94:0/809355476 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.828+0000 7ff117fff640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff120109b10 0x7ff120111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.829+0000 7ff117fff640 1 -- 172.21.15.94:0/809355476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff120114ab0 msgr2=0x7ff120111ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.829+0000 7ff117fff640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff120114ab0 0x7ff120111ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.829+0000 7ff117fff640 1 -- 172.21.15.94:0/809355476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1201091f0 msgr2=0x7ff120114470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.829+0000 7ff117fff640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1201091f0 0x7ff120114470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.829+0000 7ff117fff640 1 -- 172.21.15.94:0/809355476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1201b3590 con 0x7ff120109b10 2024-09-19T04:21:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.829+0000 7ff117fff640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff120109b10 0x7ff120111520 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff10800e9f0 tx=0x7ff10800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:24.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.829+0000 7ff11dffb640 1 -- 172.21.15.94:0/809355476 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10800ccc0 con 0x7ff120109b10 2024-09-19T04:21:24.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.830+0000 7ff11dffb640 1 -- 172.21.15.94:0/809355476 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1080108e0 con 0x7ff120109b10 2024-09-19T04:21:24.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.830+0000 7ff11dffb640 1 -- 172.21.15.94:0/809355476 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108018880 con 0x7ff120109b10 2024-09-19T04:21:24.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.830+0000 7ff126268640 1 -- 172.21.15.94:0/809355476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1201b3760 con 0x7ff120109b10 2024-09-19T04:21:24.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.830+0000 7ff126268640 1 -- 172.21.15.94:0/809355476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1201b3bc0 con 0x7ff120109b10 2024-09-19T04:21:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.834+0000 7ff128641740 1 Processor -- start 2024-09-19T04:21:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.834+0000 7ff11dffb640 1 -- 172.21.15.94:0/809355476 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff108010a90 con 0x7ff120109b10 2024-09-19T04:21:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.834+0000 7ff128641740 1 -- start start 2024-09-19T04:21:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.835+0000 7ff128641740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff124805640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff128641740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff128641740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 0x5614ad1b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff128641740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5614ad10d2f0 con 0x5614ad1a8b40 2024-09-19T04:21:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff128641740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5614ad0b4cf0 con 0x5614ad1b2370 2024-09-19T04:21:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff128641740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5614ad1092b0 con 0x5614ad1a75d0 2024-09-19T04:21:24.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff117fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 0x5614ad1b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff124805640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38954/0 (socket says 172.21.15.94:38954) 2024-09-19T04:21:24.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff124805640 1 -- 172.21.15.94:0/1563081626 learned_addr learned my addr 172.21.15.94:0/1563081626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:24.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff11dffb640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff10c0a3580 0x7ff10c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.836+0000 7ff11ffff640 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.837+0000 7ff124805640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff10c0a3580 0x7ff10c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.837+0000 7ff124805640 1 -- 172.21.15.94:0/1563081626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 msgr2=0x5614ad1a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:24.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.837+0000 7ff124805640 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.837+0000 7ff124805640 1 -- 172.21.15.94:0/1563081626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 msgr2=0x5614ad1b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:24.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.837+0000 7ff124805640 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 0x5614ad1b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.837+0000 7ff124805640 1 -- 172.21.15.94:0/1563081626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ad0993a0 con 0x5614ad1a8b40 2024-09-19T04:21:24.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.838+0000 7ff117fff640 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 0x5614ad1b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:24.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.838+0000 7ff11ffff640 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:24.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.838+0000 7ff11dffb640 1 -- 172.21.15.94:0/809355476 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff1080cfff0 con 0x7ff120109b10 2024-09-19T04:21:24.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.838+0000 7ff124805640 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1a7090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff11802ae70 tx=0x7ff118003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f54e8b0a19e485e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:24.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.839+0000 7ff124805640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff10c0a3580 0x7ff10c0a5a40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff118048000 tx=0x7ff11804a000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:24.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.839+0000 7ff1167fc640 1 -- 172.21.15.94:0/1563081626 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118043050 con 0x5614ad1a8b40 2024-09-19T04:21:24.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.839+0000 7ff1167fc640 1 -- 172.21.15.94:0/1563081626 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff11804d020 con 0x5614ad1a8b40 2024-09-19T04:21:24.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.839+0000 7ff1167fc640 1 -- 172.21.15.94:0/1563081626 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118034040 con 0x5614ad1a8b40 2024-09-19T04:21:24.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.840+0000 7ff128641740 1 -- 172.21.15.94:0/1563081626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 msgr2=0x5614ad1a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:24.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.840+0000 7ff128641740 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1a7090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff11802ae70 tx=0x7ff118003a70 comp rx=0 tx=0).stop 2024-09-19T04:21:24.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.840+0000 7ff128641740 1 -- 172.21.15.94:0/1563081626 shutdown_connections 2024-09-19T04:21:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.840+0000 7ff128641740 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 0x5614ad1b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.840+0000 7ff128641740 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.840+0000 7ff128641740 1 --2- 172.21.15.94:0/1563081626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1a7090 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.840+0000 7ff128641740 1 -- 172.21.15.94:0/1563081626 >> 172.21.15.94:0/1563081626 conn(0x5614ad170310 msgr2=0x5614ad1a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.840+0000 7ff128641740 1 -- 172.21.15.94:0/1563081626 shutdown_connections 2024-09-19T04:21:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.841+0000 7ff128641740 1 -- 172.21.15.94:0/1563081626 wait complete. 2024-09-19T04:21:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.841+0000 7ff128641740 1 -- start start 2024-09-19T04:21:24.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.842+0000 7ff128641740 1 Processor -- start 2024-09-19T04:21:24.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.842+0000 7ff128641740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.842+0000 7ff128641740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a8b40 0x5614ad1af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff128641740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1b2370 0x5614ad1af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff117fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a8b40 0x5614ad1af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff117fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a8b40 0x5614ad1af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47532/0 (socket says 172.21.15.94:47532) 2024-09-19T04:21:24.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff124805640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1b2370 0x5614ad1af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff124805640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1b2370 0x5614ad1af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38962/0 (socket says 172.21.15.94:38962) 2024-09-19T04:21:24.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff117fff640 1 -- 172.21.15.94:0/2929276066 learned_addr learned my addr 172.21.15.94:0/2929276066 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:24.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5614ad1b5010 con 0x5614ad1b2370 2024-09-19T04:21:24.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff11ffff640 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5614ad0c8060 con 0x5614ad1a75d0 2024-09-19T04:21:24.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.843+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5614ad1b5190 con 0x5614ad1a8b40 2024-09-19T04:21:24.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.844+0000 7ff128641740 10 client.0 _ll_register_callbacks cb 0x5614ace1b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:24.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.844+0000 7ff11ffff640 1 -- 172.21.15.94:0/2929276066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a8b40 msgr2=0x5614ad1af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:24.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.844+0000 7ff11ffff640 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a8b40 0x5614ad1af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.844+0000 7ff11ffff640 1 -- 172.21.15.94:0/2929276066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1b2370 msgr2=0x5614ad1af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:24.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.844+0000 7ff11ffff640 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1b2370 0x5614ad1af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.844+0000 7ff11ffff640 1 -- 172.21.15.94:0/2929276066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ace7e3c0 con 0x5614ad1a75d0 2024-09-19T04:21:24.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.844+0000 7ff11ffff640 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b1db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff110011440 tx=0x7ff110011840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b219d2813231a02a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:24.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.844+0000 7ff0ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:21:24.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.845+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2929276066 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11001e060 con 0x5614ad1a75d0 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.845+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2929276066 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff1100035f0 con 0x5614ad1a75d0 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.845+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff11804d020 con 0x5614ad1a75d0 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.845+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2929276066 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff1100037f0 con 0x5614ad1a75d0 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.845+0000 7ff128641740 10 client.25745 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.845+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5614ad1c0bb0 con 0x5614ad1a75d0 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.846+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2929276066 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff11804d020 con 0x5614ad1a75d0 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.846+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2929276066 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7ff110021a70 con 0x5614ad1a75d0 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.846+0000 7ff128641740 10 client.25745 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.846+0000 7ff128641740 10 client.25745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:24.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.846+0000 7ff128641740 20 client.25745 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:24.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.846+0000 7ff128641740 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5614ad1c2430 0x5614ad1c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.846+0000 7ff128641740 4 client.25745 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:24.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.846+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614ad11a860 con 0x5614ad1c2430 2024-09-19T04:21:24.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.846+0000 7ff117fff640 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5614ad1c2430 0x5614ad1c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.847+0000 7ff117fff640 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5614ad1c2430 0x5614ad1c48d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f2a48b60574e11 server_cookie=35d0f1b77a17800b in_seq=0 out_seq=0 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.847+0000 7ff0ff7fe640 10 client.25745 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.847+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/2929276066 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614ad11a860 con 0x5614ad1c2430 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.847+0000 7ff0ff7fe640 10 client.25745 handle_command_reply: tid=0 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff128641740 1 client.25745 shutdown 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 msgr2=0x5614ad1b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff128641740 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b1db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff110011440 tx=0x7ff110011840 comp rx=0 tx=0).stop 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 shutdown_connections 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff128641740 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5614ad1c2430 0x5614ad1c48d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff128641740 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1b2370 0x5614ad1af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff128641740 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a8b40 0x5614ad1af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff0ff7fe640 0 client.25745 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:24.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff128641740 1 --2- 172.21.15.94:0/2929276066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b1db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.848+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 >> 172.21.15.94:0/2929276066 conn(0x5614ad170310 msgr2=0x5614ad1a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:24.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.849+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 shutdown_connections 2024-09-19T04:21:24.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.849+0000 7ff128641740 1 -- 172.21.15.94:0/2929276066 wait complete. 2024-09-19T04:21:24.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.849+0000 7ff128641740 20 client.25745 trim_cache size 0 max 16384 2024-09-19T04:21:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.871+0000 7ff128641740 1 Processor -- start 2024-09-19T04:21:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.871+0000 7ff128641740 1 -- start start 2024-09-19T04:21:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff128641740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff128641740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff128641740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 0x5614ad1ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff11ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff11ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47548/0 (socket says 172.21.15.94:47548) 2024-09-19T04:21:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff128641740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1201bf040 con 0x5614ad1a8b40 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff128641740 1 -- 172.21.15.94:0/2870049345 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1201bc910 con 0x5614ad1b2370 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff11ffff640 1 -- 172.21.15.94:0/2870049345 learned_addr learned my addr 172.21.15.94:0/2870049345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff128641740 1 -- 172.21.15.94:0/2870049345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1201ba760 con 0x5614ad1a75d0 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.872+0000 7ff124805640 1 --2- 172.21.15.94:0/2870049345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 0x5614ad1ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.873+0000 7ff117fff640 1 --2- 172.21.15.94:0/2870049345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.873+0000 7ff11ffff640 1 -- 172.21.15.94:0/2870049345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 msgr2=0x5614ad1ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.873+0000 7ff11ffff640 1 --2- 172.21.15.94:0/2870049345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 0x5614ad1ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.873+0000 7ff11ffff640 1 -- 172.21.15.94:0/2870049345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 msgr2=0x5614ad1ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.873+0000 7ff11ffff640 1 --2- 172.21.15.94:0/2870049345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.873+0000 7ff11ffff640 1 -- 172.21.15.94:0/2870049345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ad1afb60 con 0x5614ad1a75d0 2024-09-19T04:21:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.873+0000 7ff11ffff640 1 --2- 172.21.15.94:0/2870049345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1ca1e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff110022000 tx=0x7ff110009320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=838ee99128d70967 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.873+0000 7ff115ffb640 1 -- 172.21.15.94:0/2870049345 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11002e030 con 0x5614ad1a75d0 2024-09-19T04:21:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.874+0000 7ff115ffb640 1 -- 172.21.15.94:0/2870049345 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1100035f0 con 0x5614ad1a75d0 2024-09-19T04:21:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.874+0000 7ff115ffb640 1 -- 172.21.15.94:0/2870049345 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11000eb60 con 0x5614ad1a75d0 2024-09-19T04:21:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 -- 172.21.15.94:0/2870049345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 msgr2=0x5614ad1ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 --2- 172.21.15.94:0/2870049345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1ca1e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff110022000 tx=0x7ff110009320 comp rx=0 tx=0).stop 2024-09-19T04:21:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 -- 172.21.15.94:0/2870049345 shutdown_connections 2024-09-19T04:21:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 --2- 172.21.15.94:0/2870049345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1b2370 0x5614ad1ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 --2- 172.21.15.94:0/2870049345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 --2- 172.21.15.94:0/2870049345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1a75d0 0x5614ad1ca1e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 -- 172.21.15.94:0/2870049345 >> 172.21.15.94:0/2870049345 conn(0x5614ad170310 msgr2=0x5614ad1ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 -- 172.21.15.94:0/2870049345 shutdown_connections 2024-09-19T04:21:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 -- 172.21.15.94:0/2870049345 wait complete. 2024-09-19T04:21:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.875+0000 7ff128641740 1 -- start start 2024-09-19T04:21:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.876+0000 7ff128641740 1 Processor -- start 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff128641740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff128641740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff128641740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1b2370 0x5614ad1b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff117fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff117fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38968/0 (socket says 172.21.15.94:38968) 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff117fff640 1 -- 172.21.15.94:0/3292432676 learned_addr learned my addr 172.21.15.94:0/3292432676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1201badd0 con 0x5614ad1a8b40 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1201bc240 con 0x5614ad1a75d0 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1201bb4a0 con 0x5614ad1b2370 2024-09-19T04:21:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff128641740 10 client.0 _ll_register_callbacks cb 0x7ff1100096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.877+0000 7ff124805640 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1b2370 0x5614ad1b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.878+0000 7ff11ffff640 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.879+0000 7ff117fff640 1 -- 172.21.15.94:0/3292432676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1b2370 msgr2=0x5614ad1b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.879+0000 7ff117fff640 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1b2370 0x5614ad1b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.879+0000 7ff117fff640 1 -- 172.21.15.94:0/3292432676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 msgr2=0x5614ad1b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.879+0000 7ff117fff640 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.879+0000 7ff117fff640 1 -- 172.21.15.94:0/3292432676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ad22c1b0 con 0x5614ad1a8b40 2024-09-19T04:21:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.879+0000 7ff11ffff640 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.880+0000 7ff117fff640 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1b65e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff10809bf60 tx=0x7ff1080bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6e2f5e0e1928d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.880+0000 7ff0ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:21:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.880+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3292432676 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1080a4ed0 con 0x5614ad1a8b40 2024-09-19T04:21:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.880+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3292432676 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7ff1080988d0 con 0x5614ad1a8b40 2024-09-19T04:21:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.881+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1100035f0 con 0x5614ad1a8b40 2024-09-19T04:21:24.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.881+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3292432676 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff10809c260 con 0x5614ad1a8b40 2024-09-19T04:21:24.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.881+0000 7ff128641740 10 client.16164 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:24.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.881+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5614ad22d3b0 con 0x5614ad1a8b40 2024-09-19T04:21:24.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.881+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3292432676 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1080b5240 con 0x5614ad1a8b40 2024-09-19T04:21:24.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.882+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3292432676 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7ff1080a0170 con 0x5614ad1a8b40 2024-09-19T04:21:24.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.882+0000 7ff128641740 10 client.16164 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:24.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.882+0000 7ff128641740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:24.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.882+0000 7ff128641740 20 client.16164 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:24.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.882+0000 7ff128641740 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5614ad1c2430 0x5614ad22fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:24.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.882+0000 7ff128641740 4 client.16164 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:24.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.882+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff1100035f0 con 0x5614ad1c2430 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.882+0000 7ff11ffff640 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5614ad1c2430 0x5614ad22fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.883+0000 7ff11ffff640 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5614ad1c2430 0x5614ad22fbd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ed83fa7f317bca1 server_cookie=a4124a0fc9320c77 in_seq=0 out_seq=0 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.883+0000 7ff0ff7fe640 10 client.16164 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.883+0000 7ff0ff7fe640 1 -- 172.21.15.94:0/3292432676 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff1100035f0 con 0x5614ad1c2430 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.883+0000 7ff0ff7fe640 10 client.16164 handle_command_reply: tid=0 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.883+0000 7ff128641740 1 client.16164 shutdown 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.883+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 msgr2=0x5614ad1b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.883+0000 7ff128641740 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1b65e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff10809bf60 tx=0x7ff1080bc040 comp rx=0 tx=0).stop 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 shutdown_connections 2024-09-19T04:21:24.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff128641740 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5614ad1c2430 0x5614ad22fbd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff128641740 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614ad1b2370 0x5614ad1b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff128641740 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614ad1a8b40 0x5614ad1b65e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff128641740 1 --2- 172.21.15.94:0/3292432676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614ad1a75d0 0x5614ad1b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 >> 172.21.15.94:0/3292432676 conn(0x5614ad170310 msgr2=0x5614ad1a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:24.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff0ff7fe640 0 client.16164 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:24.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 shutdown_connections 2024-09-19T04:21:24.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff128641740 1 -- 172.21.15.94:0/3292432676 wait complete. 2024-09-19T04:21:24.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.884+0000 7ff128641740 20 client.16164 trim_cache size 0 max 16384 2024-09-19T04:21:24.888 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:21:24.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 -- 172.21.15.94:0/809355476 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff10c0a3580 msgr2=0x7ff10c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:24.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff10c0a3580 0x7ff10c0a5a40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff118048000 tx=0x7ff11804a000 comp rx=0 tx=0).stop 2024-09-19T04:21:24.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 -- 172.21.15.94:0/809355476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff120109b10 msgr2=0x7ff120111520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:24.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff120109b10 0x7ff120111520 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff10800e9f0 tx=0x7ff10800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 -- 172.21.15.94:0/809355476 shutdown_connections 2024-09-19T04:21:24.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff10c0a3580 0x7ff10c0a5a40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff120114ab0 0x7ff120111ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff120109b10 0x7ff120111520 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 --2- 172.21.15.94:0/809355476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1201091f0 0x7ff120114470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:24.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 -- 172.21.15.94:0/809355476 >> 172.21.15.94:0/809355476 conn(0x7ff12006e140 msgr2=0x7ff120071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:24.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 -- 172.21.15.94:0/809355476 shutdown_connections 2024-09-19T04:21:24.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:24.885+0000 7ff126268640 1 -- 172.21.15.94:0/809355476 wait complete. 2024-09-19T04:21:25.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:25 smithi094 ceph-mon[30972]: pgmap v194: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 27 MiB/s wr, 1.75k op/s 2024-09-19T04:21:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 310.0 s, 759.2 tps, lat 133.325 ms stddev 362.165 2024-09-19T04:21:25.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:25 smithi112 ceph-mon[35051]: pgmap v194: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 27 MiB/s wr, 1.75k op/s 2024-09-19T04:21:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:25 smithi104 ceph-mon[34145]: pgmap v194: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 27 MiB/s wr, 1.75k op/s 2024-09-19T04:21:26.613 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:21:26.613 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:21:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.623+0000 7f9275449640 1 -- 172.21.15.94:0/3270249223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270102570 msgr2=0x7f92701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.623+0000 7f9275449640 1 --2- 172.21.15.94:0/3270249223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270102570 0x7f92701029b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9264009890 tx=0x7f926402a400 comp rx=0 tx=0).stop 2024-09-19T04:21:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.623+0000 7f9275449640 1 -- 172.21.15.94:0/3270249223 shutdown_connections 2024-09-19T04:21:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.623+0000 7f9275449640 1 --2- 172.21.15.94:0/3270249223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9270102ef0 0x7f927010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.623+0000 7f9275449640 1 --2- 172.21.15.94:0/3270249223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270102570 0x7f92701029b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.623+0000 7f9275449640 1 --2- 172.21.15.94:0/3270249223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9270100e20 0x7f9270101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.623+0000 7f9275449640 1 -- 172.21.15.94:0/3270249223 >> 172.21.15.94:0/3270249223 conn(0x7f92700fc7d0 msgr2=0x7f92700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.623+0000 7f9275449640 1 -- 172.21.15.94:0/3270249223 shutdown_connections 2024-09-19T04:21:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.623+0000 7f9275449640 1 -- 172.21.15.94:0/3270249223 wait complete. 2024-09-19T04:21:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.624+0000 7f9275449640 1 Processor -- start 2024-09-19T04:21:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.624+0000 7f9275449640 1 -- start start 2024-09-19T04:21:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.624+0000 7f9275449640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9270100e20 0x7f9270197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.624+0000 7f9275449640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9270102570 0x7f9270197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.624+0000 7f9275449640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270102ef0 0x7f927019c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.624+0000 7f9275449640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9270110e80 con 0x7f9270100e20 2024-09-19T04:21:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.624+0000 7f9275449640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9270110d00 con 0x7f9270102570 2024-09-19T04:21:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.624+0000 7f9275449640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9270111000 con 0x7f9270102ef0 2024-09-19T04:21:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9270100e20 0x7f9270197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9270102570 0x7f9270197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9270100e20 0x7f9270197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38978/0 (socket says 172.21.15.94:38978) 2024-09-19T04:21:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926effd640 1 -- 172.21.15.94:0/2774036543 learned_addr learned my addr 172.21.15.94:0/2774036543 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926f7fe640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270102ef0 0x7f927019c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926e7fc640 1 -- 172.21.15.94:0/2774036543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270102ef0 msgr2=0x7f927019c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926e7fc640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270102ef0 0x7f927019c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926e7fc640 1 -- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9270100e20 msgr2=0x7f9270197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926e7fc640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9270100e20 0x7f9270197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926e7fc640 1 -- 172.21.15.94:0/2774036543 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927019c6b0 con 0x7f9270102570 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.625+0000 7f926effd640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9270100e20 0x7f9270197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.626+0000 7f926e7fc640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9270102570 0x7f9270197660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9264009780 tx=0x7f9264004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.626+0000 7f9257fff640 1 -- 172.21.15.94:0/2774036543 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264037050 con 0x7f9270102570 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.626+0000 7f9275449640 1 -- 172.21.15.94:0/2774036543 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927019c940 con 0x7f9270102570 2024-09-19T04:21:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.626+0000 7f9257fff640 1 -- 172.21.15.94:0/2774036543 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9264043070 con 0x7f9270102570 2024-09-19T04:21:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.626+0000 7f9257fff640 1 -- 172.21.15.94:0/2774036543 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264030ac0 con 0x7f9270102570 2024-09-19T04:21:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.626+0000 7f9275449640 1 -- 172.21.15.94:0/2774036543 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927019ce70 con 0x7f9270102570 2024-09-19T04:21:27.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.629+0000 7f9257fff640 1 -- 172.21.15.94:0/2774036543 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f926402bd10 con 0x7f9270102570 2024-09-19T04:21:27.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.630+0000 7f9257fff640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f92580a35f0 0x7f92580a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.630+0000 7f9277822740 1 Processor -- start 2024-09-19T04:21:27.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.630+0000 7f9277822740 1 -- start start 2024-09-19T04:21:27.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.630+0000 7f9257fff640 1 -- 172.21.15.94:0/2774036543 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f9264030860 con 0x7f9270102570 2024-09-19T04:21:27.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926effd640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f92580a35f0 0x7f92580a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f9277822740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357db7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f9277822740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dc2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f9277822740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f9277822740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f357d1d280 con 0x55f357dc2d40 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f9277822740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f357cc4b10 con 0x55f357db7b60 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f9277822740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f357d19240 con 0x55f357db8480 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dc2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dc2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47588/0 (socket says 172.21.15.94:47588) 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926effd640 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357db7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926e7fc640 1 -- 172.21.15.94:0/1781794647 learned_addr learned my addr 172.21.15.94:0/1781794647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926e7fc640 1 -- 172.21.15.94:0/1781794647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 msgr2=0x55f357db7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926e7fc640 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357db7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926f7fe640 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926e7fc640 1 -- 172.21.15.94:0/1781794647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 msgr2=0x55f357dc5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926effd640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f92580a35f0 0x7f92580a5ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f926800f8b0 tx=0x7f926800fcb0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926e7fc640 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926e7fc640 1 -- 172.21.15.94:0/1781794647 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f357cd11f0 con 0x55f357db8480 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.631+0000 7f926f7fe640 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.632+0000 7f926e7fc640 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dc2800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92640bb8f0 tx=0x7f92640b7fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60701ef7c0434535 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.632+0000 7f92557fa640 1 -- 172.21.15.94:0/1781794647 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92640bc2f0 con 0x55f357db8480 2024-09-19T04:21:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.632+0000 7f92557fa640 1 -- 172.21.15.94:0/1781794647 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9264038040 con 0x55f357db8480 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.632+0000 7f92557fa640 1 -- 172.21.15.94:0/1781794647 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92640ed2a0 con 0x55f357db8480 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.633+0000 7f9277822740 1 -- 172.21.15.94:0/1781794647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 msgr2=0x55f357dc2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.633+0000 7f9277822740 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dc2800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92640bb8f0 tx=0x7f92640b7fb0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.633+0000 7f9277822740 1 -- 172.21.15.94:0/1781794647 shutdown_connections 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.633+0000 7f9277822740 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.633+0000 7f9277822740 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dc2800 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.633+0000 7f9277822740 1 --2- 172.21.15.94:0/1781794647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357db7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.633+0000 7f9277822740 1 -- 172.21.15.94:0/1781794647 >> 172.21.15.94:0/1781794647 conn(0x55f357db40f0 msgr2=0x55f357db44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.634+0000 7f9277822740 1 -- 172.21.15.94:0/1781794647 shutdown_connections 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.634+0000 7f9277822740 1 -- 172.21.15.94:0/1781794647 wait complete. 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.634+0000 7f9277822740 1 -- start start 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.634+0000 7f9277822740 1 Processor -- start 2024-09-19T04:21:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.635+0000 7f9277822740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357dbfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.635+0000 7f9277822740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db8480 0x55f357dc0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.635+0000 7f9277822740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357dc2d40 0x55f357dc06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.635+0000 7f9277822740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f357dc56d0 con 0x55f357db7b60 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.635+0000 7f9277822740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f357cd7f10 con 0x55f357db8480 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.635+0000 7f9277822740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f357dc5850 con 0x55f357dc2d40 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.635+0000 7f926f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357dc2d40 0x55f357dc06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.635+0000 7f926f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357dc2d40 0x55f357dc06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47592/0 (socket says 172.21.15.94:47592) 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.635+0000 7f926f7fe640 1 -- 172.21.15.94:0/1560088226 learned_addr learned my addr 172.21.15.94:0/1560088226 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f9277822740 10 client.0 _ll_register_callbacks cb 0x55f357ca0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f926e7fc640 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db8480 0x55f357dc0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f926effd640 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357dbfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f926f7fe640 1 -- 172.21.15.94:0/1560088226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db8480 msgr2=0x55f357dc0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:27.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f926f7fe640 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db8480 0x55f357dc0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f926f7fe640 1 -- 172.21.15.94:0/1560088226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 msgr2=0x55f357dbfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:27.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f926f7fe640 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357dbfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f926f7fe640 1 -- 172.21.15.94:0/1560088226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f357a8e6a0 con 0x55f357dc2d40 2024-09-19T04:21:27.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f926f7fe640 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357dc2d40 0x55f357dc06c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f926000b830 tx=0x7f926000bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a35c5ea57d3fae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:27.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f9252ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:21:27.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.636+0000 7f9252ff5640 1 -- 172.21.15.94:0/1560088226 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926001b470 con 0x55f357dc2d40 2024-09-19T04:21:27.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.637+0000 7f9252ff5640 1 -- 172.21.15.94:0/1560088226 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f926001b610 con 0x55f357dc2d40 2024-09-19T04:21:27.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.637+0000 7f9277822740 1 -- 172.21.15.94:0/1560088226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9264038040 con 0x55f357dc2d40 2024-09-19T04:21:27.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.637+0000 7f9252ff5640 1 -- 172.21.15.94:0/1560088226 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9260005020 con 0x55f357dc2d40 2024-09-19T04:21:27.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.637+0000 7f9277822740 10 client.25744 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:27.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.637+0000 7f9277822740 1 -- 172.21.15.94:0/1560088226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f357dd1220 con 0x55f357dc2d40 2024-09-19T04:21:27.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.637+0000 7f9252ff5640 1 -- 172.21.15.94:0/1560088226 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9260016070 con 0x55f357dc2d40 2024-09-19T04:21:27.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.637+0000 7f9252ff5640 1 -- 172.21.15.94:0/1560088226 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f92600047a0 con 0x55f357dc2d40 2024-09-19T04:21:27.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.637+0000 7f9277822740 10 client.25744 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.637+0000 7f9277822740 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.638+0000 7f9277822740 20 client.25744 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.638+0000 7f9277822740 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f357dd2730 0x55f357dd4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.638+0000 7f9277822740 4 client.25744 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.638+0000 7f9277822740 1 -- 172.21.15.94:0/1560088226 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9264038040 con 0x55f357dd2730 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.638+0000 7f926effd640 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f357dd2730 0x55f357dd4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.638+0000 7f9252ff5640 10 client.25744 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.638+0000 7f926effd640 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f357dd2730 0x55f357dd4b20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab422d7a5e6991c5 server_cookie=5d2f8bc8c447a949 in_seq=0 out_seq=0 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9252ff5640 1 -- 172.21.15.94:0/1560088226 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9264038040 con 0x55f357dd2730 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9252ff5640 10 client.25744 handle_command_reply: tid=0 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9277822740 1 client.25744 shutdown 2024-09-19T04:21:27.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9277822740 1 -- 172.21.15.94:0/1560088226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357dc2d40 msgr2=0x55f357dc06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9277822740 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357dc2d40 0x55f357dc06c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f926000b830 tx=0x7f926000bcf0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9277822740 1 -- 172.21.15.94:0/1560088226 shutdown_connections 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9277822740 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f357dd2730 0x55f357dd4b20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9277822740 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357dc2d40 0x55f357dc06c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9277822740 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db8480 0x55f357dc0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9277822740 1 --2- 172.21.15.94:0/1560088226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357dbfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.639+0000 7f9277822740 1 -- 172.21.15.94:0/1560088226 >> 172.21.15.94:0/1560088226 conn(0x55f357db40f0 msgr2=0x55f357db9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.640+0000 7f9252ff5640 0 client.25744 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.640+0000 7f9277822740 1 -- 172.21.15.94:0/1560088226 shutdown_connections 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.640+0000 7f9277822740 1 -- 172.21.15.94:0/1560088226 wait complete. 2024-09-19T04:21:27.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.640+0000 7f9277822740 20 client.25744 trim_cache size 0 max 16384 2024-09-19T04:21:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.669+0000 7f9277822740 1 Processor -- start 2024-09-19T04:21:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.669+0000 7f9277822740 1 -- start start 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f9277822740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357db9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f9277822740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dda430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f9277822740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357dc2d40 0x55f357dda970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f9277822740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f925c011120 con 0x55f357db7b60 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f9277822740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f925c00e9f0 con 0x55f357dc2d40 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f9277822740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f925c00c840 con 0x55f357db8480 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dda430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357db9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357db9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39000/0 (socket says 172.21.15.94:39000) 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dda430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47606/0 (socket says 172.21.15.94:47606) 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926effd640 1 -- 172.21.15.94:0/2942418878 learned_addr learned my addr 172.21.15.94:0/2942418878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926f7fe640 1 --2- 172.21.15.94:0/2942418878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357dc2d40 0x55f357dda970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926effd640 1 -- 172.21.15.94:0/2942418878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 msgr2=0x55f357dda430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926effd640 1 --2- 172.21.15.94:0/2942418878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dda430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926effd640 1 -- 172.21.15.94:0/2942418878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357dc2d40 msgr2=0x55f357dda970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926effd640 1 --2- 172.21.15.94:0/2942418878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357dc2d40 0x55f357dda970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.670+0000 7f926effd640 1 -- 172.21.15.94:0/2942418878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f357daf830 con 0x55f357db7b60 2024-09-19T04:21:27.674 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:27 smithi094 ceph-mon[30972]: pgmap v195: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-09-19T04:21:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.671+0000 7f926effd640 1 --2- 172.21.15.94:0/2942418878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357db9cc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9268009c30 tx=0x7f926801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64adfe55a372f4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.672+0000 7f9254ff9640 1 -- 172.21.15.94:0/2942418878 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926800ce80 con 0x55f357db7b60 2024-09-19T04:21:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.672+0000 7f9254ff9640 1 -- 172.21.15.94:0/2942418878 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9268019040 con 0x55f357db7b60 2024-09-19T04:21:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.672+0000 7f9254ff9640 1 -- 172.21.15.94:0/2942418878 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926802ad20 con 0x55f357db7b60 2024-09-19T04:21:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.673+0000 7f9277822740 1 -- 172.21.15.94:0/2942418878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 msgr2=0x55f357db9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.673+0000 7f9277822740 1 --2- 172.21.15.94:0/2942418878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357db9cc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9268009c30 tx=0x7f926801f000 comp rx=0 tx=0).stop 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.673+0000 7f9277822740 1 -- 172.21.15.94:0/2942418878 shutdown_connections 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.673+0000 7f9277822740 1 --2- 172.21.15.94:0/2942418878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357dc2d40 0x55f357dda970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.673+0000 7f9277822740 1 --2- 172.21.15.94:0/2942418878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dda430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.673+0000 7f9277822740 1 --2- 172.21.15.94:0/2942418878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357db7b60 0x55f357db9cc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.673+0000 7f9277822740 1 -- 172.21.15.94:0/2942418878 >> 172.21.15.94:0/2942418878 conn(0x55f357db40f0 msgr2=0x55f357db95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.673+0000 7f9277822740 1 -- 172.21.15.94:0/2942418878 shutdown_connections 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.673+0000 7f9277822740 1 -- 172.21.15.94:0/2942418878 wait complete. 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f9277822740 1 -- start start 2024-09-19T04:21:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f9277822740 1 Processor -- start 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f9277822740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357dc6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f9277822740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dc6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f9277822740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f9277822740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f925c00ceb0 con 0x55f357dc2d40 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f9277822740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f925c00e320 con 0x55f357db7b60 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f9277822740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f925c00d580 con 0x55f357db8480 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f9277822740 10 client.0 _ll_register_callbacks cb 0x7f9260004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.674+0000 7f926effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357dc6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357dc6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:60850/0 (socket says 172.21.15.94:60850) 2024-09-19T04:21:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926effd640 1 -- 172.21.15.94:0/69319968 learned_addr learned my addr 172.21.15.94:0/69319968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926f7fe640 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926effd640 1 -- 172.21.15.94:0/69319968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 msgr2=0x55f357dc6c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926effd640 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dc6c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926effd640 1 -- 172.21.15.94:0/69319968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 msgr2=0x55f357dc71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926effd640 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926effd640 1 -- 172.21.15.94:0/69319968 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f357e3c450 con 0x55f357db7b60 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926f7fe640 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f926effd640 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357dc6720 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f926802a8a0 tx=0x7f926802a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea1189f0a2b8026 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.675+0000 7f9252ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.676+0000 7f9252ff5640 1 -- 172.21.15.94:0/69319968 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268041060 con 0x55f357db7b60 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.676+0000 7f9252ff5640 1 -- 172.21.15.94:0/69319968 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f9268004c50 con 0x55f357db7b60 2024-09-19T04:21:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.676+0000 7f9277822740 1 -- 172.21.15.94:0/69319968 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9268019040 con 0x55f357db7b60 2024-09-19T04:21:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.676+0000 7f9252ff5640 1 -- 172.21.15.94:0/69319968 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9268004e50 con 0x55f357db7b60 2024-09-19T04:21:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.676+0000 7f9277822740 10 client.25769 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.676+0000 7f9277822740 1 -- 172.21.15.94:0/69319968 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f357e3d770 con 0x55f357db7b60 2024-09-19T04:21:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.676+0000 7f9252ff5640 1 -- 172.21.15.94:0/69319968 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f926802a9a0 con 0x55f357db7b60 2024-09-19T04:21:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.677+0000 7f9252ff5640 1 -- 172.21.15.94:0/69319968 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f926800b040 con 0x55f357db7b60 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.677+0000 7f9277822740 10 client.25769 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.677+0000 7f9277822740 10 client.25769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.677+0000 7f9277822740 20 client.25769 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.677+0000 7f9277822740 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f357e3de70 0x55f357e40290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.677+0000 7f9277822740 4 client.25769 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.677+0000 7f9277822740 1 -- 172.21.15.94:0/69319968 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f9268019040 con 0x55f357e3de70 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.677+0000 7f926e7fc640 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f357e3de70 0x55f357e40290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.678+0000 7f9252ff5640 10 client.25769 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.678+0000 7f926e7fc640 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f357e3de70 0x55f357e40290 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f2c826329d6020 server_cookie=43c043b8f83c8a92 in_seq=0 out_seq=0 2024-09-19T04:21:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9252ff5640 1 -- 172.21.15.94:0/69319968 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f9268019040 con 0x55f357e3de70 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9252ff5640 10 client.25769 handle_command_reply: tid=0 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9277822740 1 client.25769 shutdown 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9277822740 1 -- 172.21.15.94:0/69319968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 msgr2=0x55f357dc6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9277822740 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357dc6720 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f926802a8a0 tx=0x7f926802a670 comp rx=0 tx=0).stop 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9277822740 1 -- 172.21.15.94:0/69319968 shutdown_connections 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9277822740 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f357e3de70 0x55f357e40290 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9277822740 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f357dc2d40 0x55f357dc71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9277822740 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f357db8480 0x55f357dc6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9277822740 1 --2- 172.21.15.94:0/69319968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f357db7b60 0x55f357dc6720 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f926802a8a0 tx=0x7f926802a670 comp rx=0 tx=0).stop 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9277822740 1 -- 172.21.15.94:0/69319968 >> 172.21.15.94:0/69319968 conn(0x55f357db40f0 msgr2=0x55f357db2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.679+0000 7f9252ff5640 0 client.25769 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.680+0000 7f9277822740 1 -- 172.21.15.94:0/69319968 shutdown_connections 2024-09-19T04:21:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.680+0000 7f9277822740 1 -- 172.21.15.94:0/69319968 wait complete. 2024-09-19T04:21:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.680+0000 7f9277822740 20 client.25769 trim_cache size 0 max 16384 2024-09-19T04:21:27.684 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:21:27.684 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:21:27.684 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "405e4026-07c9-42a8-9941-bac61f889769", 2024-09-19T04:21:27.684 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:21:27.685 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:21:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 -- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f92580a35f0 msgr2=0x7f92580a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f92580a35f0 0x7f92580a5ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f926800f8b0 tx=0x7f926800fcb0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 -- 172.21.15.94:0/2774036543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9270102570 msgr2=0x7f9270197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:27.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9270102570 0x7f9270197660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9264009780 tx=0x7f9264004270 comp rx=0 tx=0).stop 2024-09-19T04:21:27.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 -- 172.21.15.94:0/2774036543 shutdown_connections 2024-09-19T04:21:27.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f92580a35f0 0x7f92580a5ab0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270102ef0 0x7f927019c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9270102570 0x7f9270197660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 --2- 172.21.15.94:0/2774036543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9270100e20 0x7f9270197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 -- 172.21.15.94:0/2774036543 >> 172.21.15.94:0/2774036543 conn(0x7f92700fc7d0 msgr2=0x7f9270104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 -- 172.21.15.94:0/2774036543 shutdown_connections 2024-09-19T04:21:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:27.681+0000 7f9275449640 1 -- 172.21.15.94:0/2774036543 wait complete. 2024-09-19T04:21:27.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:27 smithi112 ceph-mon[35051]: pgmap v195: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-09-19T04:21:27.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:27 smithi104 ceph-mon[34145]: pgmap v195: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-09-19T04:21:28.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:21:28.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:21:28.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:21:28.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:21:28.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:21:28.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:21:28.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:28 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:28 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:21:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:21:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:21:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:21:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:21:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:28.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:28 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:29.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.416+0000 7fef47fff640 1 -- 172.21.15.94:0/1406808703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef4810e7f0 msgr2=0x7fef4810eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:29.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.416+0000 7fef47fff640 1 --2- 172.21.15.94:0/1406808703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef4810e7f0 0x7fef4810eca0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fef40009be0 tx=0x7fef4002a6d0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.416+0000 7fef47fff640 1 -- 172.21.15.94:0/1406808703 shutdown_connections 2024-09-19T04:21:29.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.416+0000 7fef47fff640 1 --2- 172.21.15.94:0/1406808703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef4810e7f0 0x7fef4810eca0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.416+0000 7fef47fff640 1 --2- 172.21.15.94:0/1406808703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef48109d70 0x7fef4810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.416+0000 7fef47fff640 1 --2- 172.21.15.94:0/1406808703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef48109450 0x7fef48109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.416+0000 7fef47fff640 1 -- 172.21.15.94:0/1406808703 >> 172.21.15.94:0/1406808703 conn(0x7fef48073660 msgr2=0x7fef48073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.416+0000 7fef47fff640 1 -- 172.21.15.94:0/1406808703 shutdown_connections 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.417+0000 7fef47fff640 1 -- 172.21.15.94:0/1406808703 wait complete. 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.418+0000 7fef47fff640 1 Processor -- start 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.418+0000 7fef47fff640 1 -- start start 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.418+0000 7fef47fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef48109450 0x7fef4811e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.418+0000 7fef47fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef48109d70 0x7fef48118590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.418+0000 7fef47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef4810e7f0 0x7fef48118ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.418+0000 7fef47fff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef48117030 con 0x7fef4810e7f0 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.418+0000 7fef47fff640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef48116eb0 con 0x7fef48109450 2024-09-19T04:21:29.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.418+0000 7fef47fff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef481171b0 con 0x7fef48109d70 2024-09-19T04:21:29.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.420+0000 7fef46ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef48109450 0x7fef4811e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.420+0000 7fef46ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef48109450 0x7fef4811e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:60876/0 (socket says 172.21.15.94:60876) 2024-09-19T04:21:29.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.420+0000 7fef46ffd640 1 -- 172.21.15.94:0/3835150605 learned_addr learned my addr 172.21.15.94:0/3835150605 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:29.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.420+0000 7fef467fc640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef48109d70 0x7fef48118590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.420+0000 7fef46ffd640 1 -- 172.21.15.94:0/3835150605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef48109d70 msgr2=0x7fef48118590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:29.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.420+0000 7fef46ffd640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef48109d70 0x7fef48118590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.420+0000 7fef46ffd640 1 -- 172.21.15.94:0/3835150605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef4810e7f0 msgr2=0x7fef48118ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:21:29.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.420+0000 7fef46ffd640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef4810e7f0 0x7fef48118ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.420+0000 7fef46ffd640 1 -- 172.21.15.94:0/3835150605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef48119040 con 0x7fef48109450 2024-09-19T04:21:29.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.421+0000 7fef46ffd640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef48109450 0x7fef4811e510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef3800e9c0 tx=0x7fef3800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:29.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.422+0000 7fef4c885640 1 -- 172.21.15.94:0/3835150605 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3800ccc0 con 0x7fef48109450 2024-09-19T04:21:29.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.423+0000 7fef47fff640 1 -- 172.21.15.94:0/3835150605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef48119330 con 0x7fef48109450 2024-09-19T04:21:29.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.423+0000 7fef47fff640 1 -- 172.21.15.94:0/3835150605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef481265c0 con 0x7fef48109450 2024-09-19T04:21:29.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.423+0000 7fef4c885640 1 -- 172.21.15.94:0/3835150605 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef380054d0 con 0x7fef48109450 2024-09-19T04:21:29.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.423+0000 7fef4c885640 1 -- 172.21.15.94:0/3835150605 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38010430 con 0x7fef48109450 2024-09-19T04:21:29.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.426+0000 7fef4fc60740 1 Processor -- start 2024-09-19T04:21:29.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.426+0000 7fef4fc60740 1 -- start start 2024-09-19T04:21:29.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 0x55d18eee1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1f90 0x55d18eeec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eeeec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef4fc60740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d18ee472e0 con 0x55d18eeec850 2024-09-19T04:21:29.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef4fc60740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d18edeece0 con 0x55d18eee1670 2024-09-19T04:21:29.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef4fc60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d18ee432a0 con 0x55d18eee1f90 2024-09-19T04:21:29.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef467fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eeeec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef467fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eeeec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39028/0 (socket says 172.21.15.94:39028) 2024-09-19T04:21:29.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef467fc640 1 -- 172.21.15.94:0/2113575603 learned_addr learned my addr 172.21.15.94:0/2113575603 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:29.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef467fc640 1 -- 172.21.15.94:0/2113575603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1f90 msgr2=0x55d18eeec310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:21:29.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef467fc640 1 --2- 172.21.15.94:0/2113575603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1f90 0x55d18eeec310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef467fc640 1 -- 172.21.15.94:0/2113575603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 msgr2=0x55d18eee1a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:21:29.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.427+0000 7fef467fc640 1 --2- 172.21.15.94:0/2113575603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 0x55d18eee1a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.428+0000 7fef467fc640 1 -- 172.21.15.94:0/2113575603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18edd3440 con 0x55d18eeec850 2024-09-19T04:21:29.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.428+0000 7fef4c885640 1 -- 172.21.15.94:0/3835150605 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fef380105d0 con 0x7fef48109450 2024-09-19T04:21:29.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.428+0000 7fef467fc640 1 --2- 172.21.15.94:0/2113575603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eeeec40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009c20 tx=0x7fef3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02b100515a8860d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:29.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.429+0000 7fef2dffb640 1 -- 172.21.15.94:0/2113575603 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c009dd0 con 0x55d18eeec850 2024-09-19T04:21:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.429+0000 7fef2dffb640 1 -- 172.21.15.94:0/2113575603 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef3c005020 con 0x55d18eeec850 2024-09-19T04:21:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.430+0000 7fef4fc60740 1 -- 172.21.15.94:0/2113575603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 msgr2=0x55d18eeeec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.430+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2113575603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eeeec40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009c20 tx=0x7fef3c011810 comp rx=0 tx=0).stop 2024-09-19T04:21:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.430+0000 7fef4c885640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef300a3430 0x7fef300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.430+0000 7fef4c885640 1 -- 172.21.15.94:0/3835150605 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fef380cfef0 con 0x7fef48109450 2024-09-19T04:21:29.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.431+0000 7fef4fc60740 1 -- 172.21.15.94:0/2113575603 shutdown_connections 2024-09-19T04:21:29.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.431+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2113575603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eeeec40 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.431+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2113575603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1f90 0x55d18eeec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.431+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2113575603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 0x55d18eee1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.431+0000 7fef4fc60740 1 -- 172.21.15.94:0/2113575603 >> 172.21.15.94:0/2113575603 conn(0x55d18eeddc00 msgr2=0x55d18eeddfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:29.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.431+0000 7fef477fe640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef300a3430 0x7fef300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.431+0000 7fef477fe640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef300a3430 0x7fef300a58f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef400099a0 tx=0x7fef40005fd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.431+0000 7fef4fc60740 1 -- 172.21.15.94:0/2113575603 shutdown_connections 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.431+0000 7fef4fc60740 1 -- 172.21.15.94:0/2113575603 wait complete. 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.432+0000 7fef4fc60740 1 -- start start 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.432+0000 7fef4fc60740 1 Processor -- start 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eee1670 0x55d18eeec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 0x55d18eee9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eeec850 0x55d18eee9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef4fc60740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d18eeef470 con 0x55d18eee1670 2024-09-19T04:21:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef4fc60740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d18ee02050 con 0x55d18eee1f90 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef4fc60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d18eeef5f0 con 0x55d18eeec850 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef4fc60740 10 client.0 _ll_register_callbacks cb 0x55d18ee42160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eee1670 0x55d18eeec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eee1670 0x55d18eeec0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39032/0 (socket says 172.21.15.94:39032) 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef46ffd640 1 -- 172.21.15.94:0/2478833512 learned_addr learned my addr 172.21.15.94:0/2478833512 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef477fe640 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eeec850 0x55d18eee9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef467fc640 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 0x55d18eee9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef46ffd640 1 -- 172.21.15.94:0/2478833512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eeec850 msgr2=0x55d18eee9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef46ffd640 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eeec850 0x55d18eee9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef46ffd640 1 -- 172.21.15.94:0/2478833512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 msgr2=0x55d18eee9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:29.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef46ffd640 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 0x55d18eee9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.433+0000 7fef46ffd640 1 -- 172.21.15.94:0/2478833512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18ebb83c0 con 0x55d18eee1670 2024-09-19T04:21:29.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.434+0000 7fef477fe640 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eeec850 0x55d18eee9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:29.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.434+0000 7fef46ffd640 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eee1670 0x55d18eeec0a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fef380cdc30 tx=0x7fef380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce854893e6e3f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:29.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.434+0000 7fef26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.434+0000 7fef26ffd640 1 -- 172.21.15.94:0/2478833512 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef380a4e30 con 0x55d18eee1670 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.434+0000 7fef4fc60740 1 -- 172.21.15.94:0/2478833512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef3c005020 con 0x55d18eee1670 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.435+0000 7fef26ffd640 1 -- 172.21.15.94:0/2478833512 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fef380bc0f0 con 0x55d18eee1670 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.435+0000 7fef26ffd640 1 -- 172.21.15.94:0/2478833512 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fef380046c0 con 0x55d18eee1670 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.435+0000 7fef4fc60740 10 client.16188 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.435+0000 7fef4fc60740 1 -- 172.21.15.94:0/2478833512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d18eefaea0 con 0x55d18eee1670 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.435+0000 7fef26ffd640 1 -- 172.21.15.94:0/2478833512 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef38004860 con 0x55d18eee1670 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.435+0000 7fef26ffd640 1 -- 172.21.15.94:0/2478833512 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fef380a49a0 con 0x55d18eee1670 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.436+0000 7fef4fc60740 10 client.16188 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.436+0000 7fef4fc60740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.436+0000 7fef4fc60740 20 client.16188 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.436+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d18eefc320 0x55d18eefe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.436+0000 7fef4fc60740 4 client.16188 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:29.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.436+0000 7fef4fc60740 1 -- 172.21.15.94:0/2478833512 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fef3c005020 con 0x55d18eefc320 2024-09-19T04:21:29.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.437+0000 7fef467fc640 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d18eefc320 0x55d18eefe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.437+0000 7fef467fc640 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d18eefc320 0x55d18eefe760 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b10eb97fb9e545 server_cookie=d95978a69c9dc4b6 in_seq=0 out_seq=0 2024-09-19T04:21:29.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.437+0000 7fef26ffd640 10 client.16188 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:29.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.438+0000 7fef26ffd640 1 -- 172.21.15.94:0/2478833512 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fef3c005020 con 0x55d18eefc320 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.438+0000 7fef26ffd640 10 client.16188 handle_command_reply: tid=0 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.438+0000 7fef4fc60740 1 client.16188 shutdown 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 -- 172.21.15.94:0/2478833512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eee1670 msgr2=0x55d18eeec0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eee1670 0x55d18eeec0a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fef380cdc30 tx=0x7fef380057c0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 -- 172.21.15.94:0/2478833512 shutdown_connections 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d18eefc320 0x55d18eefe760 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eeec850 0x55d18eee9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 0x55d18eee9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2478833512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eee1670 0x55d18eeec0a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 -- 172.21.15.94:0/2478833512 >> 172.21.15.94:0/2478833512 conn(0x55d18eeddc00 msgr2=0x55d18eee38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef26ffd640 0 client.16188 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:29.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 -- 172.21.15.94:0/2478833512 shutdown_connections 2024-09-19T04:21:29.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 1 -- 172.21.15.94:0/2478833512 wait complete. 2024-09-19T04:21:29.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.439+0000 7fef4fc60740 20 client.16188 trim_cache size 0 max 16384 2024-09-19T04:21:29.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.471+0000 7fef4fc60740 1 Processor -- start 2024-09-19T04:21:29.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:29 smithi094 ceph-mon[30972]: pgmap v196: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 22 MiB/s wr, 1.63k op/s 2024-09-19T04:21:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.472+0000 7fef4fc60740 1 -- start start 2024-09-19T04:21:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.472+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1670 0x55d18eef87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.472+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 0x55d18eef4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.472+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.472+0000 7fef4fc60740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef34011130 con 0x55d18eeec850 2024-09-19T04:21:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.472+0000 7fef4fc60740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef3400ea00 con 0x55d18eee1f90 2024-09-19T04:21:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.472+0000 7fef4fc60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef3400c850 con 0x55d18eee1670 2024-09-19T04:21:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.473+0000 7fef467fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 0x55d18eef4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.473+0000 7fef46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1670 0x55d18eef87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.473+0000 7fef477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.473+0000 7fef477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39042/0 (socket says 172.21.15.94:39042) 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.473+0000 7fef477fe640 1 -- 172.21.15.94:0/4276488456 learned_addr learned my addr 172.21.15.94:0/4276488456 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.474+0000 7fef477fe640 1 -- 172.21.15.94:0/4276488456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1670 msgr2=0x55d18eef87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.474+0000 7fef477fe640 1 --2- 172.21.15.94:0/4276488456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1670 0x55d18eef87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.474+0000 7fef477fe640 1 -- 172.21.15.94:0/4276488456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 msgr2=0x55d18eef4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.474+0000 7fef477fe640 1 --2- 172.21.15.94:0/4276488456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 0x55d18eef4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.474+0000 7fef477fe640 1 -- 172.21.15.94:0/4276488456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18eedda30 con 0x55d18eeec850 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.474+0000 7fef467fc640 1 --2- 172.21.15.94:0/4276488456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 0x55d18eef4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.474+0000 7fef477fe640 1 --2- 172.21.15.94:0/4276488456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef4e80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fef40033000 tx=0x7fef4003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3778df1ae31e184 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.474+0000 7fef2d7fa640 1 -- 172.21.15.94:0/4276488456 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4004d070 con 0x55d18eeec850 2024-09-19T04:21:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.475+0000 7fef2d7fa640 1 -- 172.21.15.94:0/4276488456 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef40044040 con 0x55d18eeec850 2024-09-19T04:21:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.475+0000 7fef2d7fa640 1 -- 172.21.15.94:0/4276488456 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40043040 con 0x55d18eeec850 2024-09-19T04:21:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.476+0000 7fef4fc60740 1 -- 172.21.15.94:0/4276488456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 msgr2=0x55d18eef4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.476+0000 7fef4fc60740 1 --2- 172.21.15.94:0/4276488456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef4e80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fef40033000 tx=0x7fef4003a000 comp rx=0 tx=0).stop 2024-09-19T04:21:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.476+0000 7fef4fc60740 1 -- 172.21.15.94:0/4276488456 shutdown_connections 2024-09-19T04:21:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.476+0000 7fef4fc60740 1 --2- 172.21.15.94:0/4276488456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef4e80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.476+0000 7fef4fc60740 1 --2- 172.21.15.94:0/4276488456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1f90 0x55d18eef4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.476+0000 7fef4fc60740 1 --2- 172.21.15.94:0/4276488456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1670 0x55d18eef87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.476+0000 7fef4fc60740 1 -- 172.21.15.94:0/4276488456 >> 172.21.15.94:0/4276488456 conn(0x55d18eeddc00 msgr2=0x55d18eeeb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.476+0000 7fef4fc60740 1 -- 172.21.15.94:0/4276488456 shutdown_connections 2024-09-19T04:21:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.476+0000 7fef4fc60740 1 -- 172.21.15.94:0/4276488456 wait complete. 2024-09-19T04:21:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.477+0000 7fef4fc60740 1 -- start start 2024-09-19T04:21:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.477+0000 7fef4fc60740 1 Processor -- start 2024-09-19T04:21:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.477+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 0x55d18ef5e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.477+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1f90 0x55d18ef5e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.477+0000 7fef4fc60740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.477+0000 7fef4fc60740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef3400cec0 con 0x55d18eeec850 2024-09-19T04:21:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.477+0000 7fef4fc60740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef3400e330 con 0x55d18eee1670 2024-09-19T04:21:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.477+0000 7fef4fc60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef3400d590 con 0x55d18eee1f90 2024-09-19T04:21:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.477+0000 7fef4fc60740 10 client.0 _ll_register_callbacks cb 0x7fef3800ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef46ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 0x55d18ef5e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef46ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 0x55d18ef5e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:60932/0 (socket says 172.21.15.94:60932) 2024-09-19T04:21:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef46ffd640 1 -- 172.21.15.94:0/2990165274 learned_addr learned my addr 172.21.15.94:0/2990165274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:21:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef467fc640 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1f90 0x55d18ef5e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef477fe640 1 -- 172.21.15.94:0/2990165274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1f90 msgr2=0x55d18ef5e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef477fe640 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1f90 0x55d18ef5e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef477fe640 1 -- 172.21.15.94:0/2990165274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 msgr2=0x55d18ef5e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef477fe640 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 0x55d18ef5e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef477fe640 1 -- 172.21.15.94:0/2990165274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18eef4730 con 0x55d18eeec850 2024-09-19T04:21:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef477fe640 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef3f20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fef40034040 tx=0x7fef40009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b54f9e1c2a443e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.478+0000 7fef26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:21:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.479+0000 7fef26ffd640 1 -- 172.21.15.94:0/2990165274 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40059020 con 0x55d18eeec850 2024-09-19T04:21:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.479+0000 7fef26ffd640 1 -- 172.21.15.94:0/2990165274 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7fef40004000 con 0x55d18eeec850 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.479+0000 7fef4fc60740 1 -- 172.21.15.94:0/2990165274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef40044040 con 0x55d18eeec850 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.479+0000 7fef26ffd640 1 -- 172.21.15.94:0/2990165274 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fef40064050 con 0x55d18eeec850 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.479+0000 7fef4fc60740 10 client.16194 fetch_fsmap learned FSMap version 36 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.479+0000 7fef4fc60740 1 -- 172.21.15.94:0/2990165274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d18ef66bd0 con 0x55d18eeec850 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.479+0000 7fef26ffd640 1 -- 172.21.15.94:0/2990165274 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef40044040 con 0x55d18eeec850 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.480+0000 7fef26ffd640 1 -- 172.21.15.94:0/2990165274 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fef4003c470 con 0x55d18eeec850 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.480+0000 7fef4fc60740 10 client.16194 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.480+0000 7fef4fc60740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.480+0000 7fef4fc60740 20 client.16194 populate_metadata read hostname 'smithi094' 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.480+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d18ef67240 0x55d18ef69660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.480+0000 7fef4fc60740 4 client.16194 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:21:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.480+0000 7fef4fc60740 1 -- 172.21.15.94:0/2990165274 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d18ee54850 con 0x55d18ef67240 2024-09-19T04:21:29.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.483+0000 7fef46ffd640 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d18ef67240 0x55d18ef69660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:29.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.483+0000 7fef26ffd640 10 client.16194 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:29.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.483+0000 7fef46ffd640 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d18ef67240 0x55d18ef69660 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a796365253183e server_cookie=8119d7b50adc494a in_seq=0 out_seq=0 2024-09-19T04:21:29.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.483+0000 7fef26ffd640 1 -- 172.21.15.94:0/2990165274 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d18ee54850 con 0x55d18ef67240 2024-09-19T04:21:29.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.483+0000 7fef26ffd640 10 client.16194 handle_command_reply: tid=0 2024-09-19T04:21:29.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.483+0000 7fef4fc60740 1 client.16194 shutdown 2024-09-19T04:21:29.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef4fc60740 1 -- 172.21.15.94:0/2990165274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 msgr2=0x55d18eef3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef3f20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fef40034040 tx=0x7fef40009500 comp rx=0 tx=0).stop 2024-09-19T04:21:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef4fc60740 1 -- 172.21.15.94:0/2990165274 shutdown_connections 2024-09-19T04:21:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d18ef67240 0x55d18ef69660 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.487 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:21:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:21:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:21:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "405e4026-07c9-42a8-9941-bac61f889769": { 2024-09-19T04:21:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:21:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "405e4026-07c9-42a8-9941-bac61f889769", 2024-09-19T04:21:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:21:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:21:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:21:29.488 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:21:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18eeec850 0x55d18eef3f20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef26ffd640 0 client.16194 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:21:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d18eee1f90 0x55d18ef5e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef4fc60740 1 --2- 172.21.15.94:0/2990165274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d18eee1670 0x55d18ef5e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef4fc60740 1 -- 172.21.15.94:0/2990165274 >> 172.21.15.94:0/2990165274 conn(0x55d18eeddc00 msgr2=0x55d18eef77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.484+0000 7fef4fc60740 1 -- 172.21.15.94:0/2990165274 shutdown_connections 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.485+0000 7fef4fc60740 1 -- 172.21.15.94:0/2990165274 wait complete. 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.485+0000 7fef4fc60740 20 client.16194 trim_cache size 0 max 16384 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.486+0000 7fef47fff640 1 -- 172.21.15.94:0/3835150605 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef300a3430 msgr2=0x7fef300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.486+0000 7fef47fff640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef300a3430 0x7fef300a58f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef400099a0 tx=0x7fef40005fd0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.486+0000 7fef47fff640 1 -- 172.21.15.94:0/3835150605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef48109450 msgr2=0x7fef4811e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.486+0000 7fef47fff640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef48109450 0x7fef4811e510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef3800e9c0 tx=0x7fef3800ee80 comp rx=0 tx=0).stop 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.487+0000 7fef47fff640 1 -- 172.21.15.94:0/3835150605 shutdown_connections 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.487+0000 7fef47fff640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef300a3430 0x7fef300a58f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.487+0000 7fef47fff640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef4810e7f0 0x7fef48118ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.487+0000 7fef47fff640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef48109d70 0x7fef48118590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.487+0000 7fef47fff640 1 --2- 172.21.15.94:0/3835150605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef48109450 0x7fef4811e510 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.487+0000 7fef47fff640 1 -- 172.21.15.94:0/3835150605 >> 172.21.15.94:0/3835150605 conn(0x7fef48073660 msgr2=0x7fef4810c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.487+0000 7fef47fff640 1 -- 172.21.15.94:0/3835150605 shutdown_connections 2024-09-19T04:21:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:21:29.487+0000 7fef47fff640 1 -- 172.21.15.94:0/3835150605 wait complete. 2024-09-19T04:21:29.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:29 smithi112 ceph-mon[35051]: pgmap v196: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 22 MiB/s wr, 1.63k op/s 2024-09-19T04:21:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:29 smithi104 ceph-mon[34145]: pgmap v196: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 22 MiB/s wr, 1.63k op/s 2024-09-19T04:21:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 315.0 s, 1311.0 tps, lat 76.089 ms stddev 26.465 2024-09-19T04:21:30.879 INFO:tasks.cephfs.filesystem:scrub status for tag:405e4026-07c9-42a8-9941-bac61f889769 - {'path': '/', 'tag': '405e4026-07c9-42a8-9941-bac61f889769', 'options': 'recursive,force'} 2024-09-19T04:21:31.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:31 smithi112 ceph-mon[35051]: pgmap v197: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-09-19T04:21:31.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:31 smithi094 ceph-mon[30972]: pgmap v197: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-09-19T04:21:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:31 smithi104 ceph-mon[34145]: pgmap v197: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-09-19T04:21:33.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:33 smithi112 ceph-mon[35051]: pgmap v198: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-09-19T04:21:33.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:33 smithi094 ceph-mon[30972]: pgmap v198: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-09-19T04:21:33.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:33 smithi104 ceph-mon[34145]: pgmap v198: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-09-19T04:21:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 320.0 s, 1519.2 tps, lat 66.062 ms stddev 19.221 2024-09-19T04:21:35.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:35 smithi112 ceph-mon[35051]: pgmap v199: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-09-19T04:21:35.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:35 smithi094 ceph-mon[30972]: pgmap v199: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-09-19T04:21:35.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:35 smithi104 ceph-mon[34145]: pgmap v199: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-09-19T04:21:37.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:37 smithi112 ceph-mon[35051]: pgmap v200: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-09-19T04:21:37.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:37 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:21:37.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:37 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:21:37.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:37 smithi094 ceph-mon[30972]: pgmap v200: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-09-19T04:21:37.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:37 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:21:37.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:37 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:21:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:37 smithi104 ceph-mon[34145]: pgmap v200: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-09-19T04:21:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:37 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:21:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:37 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:21:38.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:38 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:21:38.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:38 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:21:38.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:38 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:21:39.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:39 smithi112 ceph-mon[35051]: pgmap v201: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 21 MiB/s wr, 1.87k op/s 2024-09-19T04:21:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:39 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:21:39.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:39 smithi094 ceph-mon[30972]: pgmap v201: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 21 MiB/s wr, 1.87k op/s 2024-09-19T04:21:39.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:39 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:21:39.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:39 smithi104 ceph-mon[34145]: pgmap v201: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 21 MiB/s wr, 1.87k op/s 2024-09-19T04:21:39.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:39 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:21:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 325.0 s, 1474.4 tps, lat 67.725 ms stddev 22.712 2024-09-19T04:21:41.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:41 smithi112 ceph-mon[35051]: pgmap v202: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.33k op/s 2024-09-19T04:21:41.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:41 smithi094 ceph-mon[30972]: pgmap v202: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.33k op/s 2024-09-19T04:21:41.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:41 smithi104 ceph-mon[34145]: pgmap v202: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.33k op/s 2024-09-19T04:21:43.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:43 smithi112 ceph-mon[35051]: pgmap v203: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.33k op/s 2024-09-19T04:21:43.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:43 smithi094 ceph-mon[30972]: pgmap v203: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.33k op/s 2024-09-19T04:21:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:43 smithi104 ceph-mon[34145]: pgmap v203: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.33k op/s 2024-09-19T04:21:44.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:44 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:21:44.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:44 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:21:44.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:44 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:21:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 330.0 s, 1566.6 tps, lat 63.820 ms stddev 32.543 2024-09-19T04:21:45.755 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:45 smithi104 ceph-mon[34145]: pgmap v204: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 21 MiB/s wr, 2.01k op/s 2024-09-19T04:21:45.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:45 smithi112 ceph-mon[35051]: pgmap v204: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 21 MiB/s wr, 2.01k op/s 2024-09-19T04:21:45.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:45 smithi094 ceph-mon[30972]: pgmap v204: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 21 MiB/s wr, 2.01k op/s 2024-09-19T04:21:47.792 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:47 smithi112 ceph-mon[35051]: pgmap v205: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-09-19T04:21:47.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:47 smithi094 ceph-mon[30972]: pgmap v205: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-09-19T04:21:47.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:47 smithi104 ceph-mon[34145]: pgmap v205: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-09-19T04:21:49.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:49 smithi112 ceph-mon[35051]: pgmap v206: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 21 MiB/s wr, 2.09k op/s 2024-09-19T04:21:49.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:49 smithi094 ceph-mon[30972]: pgmap v206: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 21 MiB/s wr, 2.09k op/s 2024-09-19T04:21:49.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:49 smithi104 ceph-mon[34145]: pgmap v206: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 21 MiB/s wr, 2.09k op/s 2024-09-19T04:21:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 335.0 s, 1560.6 tps, lat 64.133 ms stddev 17.711 2024-09-19T04:21:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:51 smithi112 ceph-mon[35051]: pgmap v207: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-19T04:21:51.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:51 smithi094 ceph-mon[30972]: pgmap v207: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-19T04:21:51.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:51 smithi104 ceph-mon[34145]: pgmap v207: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-19T04:21:53.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:53 smithi112 ceph-mon[35051]: pgmap v208: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-19T04:21:53.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:53 smithi094 ceph-mon[30972]: pgmap v208: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-19T04:21:53.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:53 smithi104 ceph-mon[34145]: pgmap v208: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-19T04:21:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 340.0 s, 1522.6 tps, lat 65.649 ms stddev 20.508 2024-09-19T04:21:55.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:55 smithi112 ceph-mon[35051]: pgmap v209: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 20 MiB/s wr, 2.03k op/s 2024-09-19T04:21:55.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:55 smithi094 ceph-mon[30972]: pgmap v209: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 20 MiB/s wr, 2.03k op/s 2024-09-19T04:21:55.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:55 smithi104 ceph-mon[34145]: pgmap v209: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 20 MiB/s wr, 2.03k op/s 2024-09-19T04:21:57.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:57 smithi112 ceph-mon[35051]: pgmap v210: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 13 MiB/s wr, 1.34k op/s 2024-09-19T04:21:57.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:57 smithi094 ceph-mon[30972]: pgmap v210: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 13 MiB/s wr, 1.34k op/s 2024-09-19T04:21:57.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:57 smithi104 ceph-mon[34145]: pgmap v210: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 13 MiB/s wr, 1.34k op/s 2024-09-19T04:21:59.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:21:59 smithi112 ceph-mon[35051]: pgmap v211: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 20 MiB/s wr, 1.99k op/s 2024-09-19T04:21:59.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:21:59 smithi094 ceph-mon[30972]: pgmap v211: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 20 MiB/s wr, 1.99k op/s 2024-09-19T04:21:59.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:21:59 smithi104 ceph-mon[34145]: pgmap v211: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 20 MiB/s wr, 1.99k op/s 2024-09-19T04:22:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 345.0 s, 1619.4 tps, lat 61.877 ms stddev 19.727 2024-09-19T04:22:00.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:22:01.304 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:01 smithi094 ceph-mon[30972]: pgmap v212: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-19T04:22:01.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:01 smithi112 ceph-mon[35051]: pgmap v212: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-19T04:22:01.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:01 smithi104 ceph-mon[34145]: pgmap v212: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-19T04:22:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.858+0000 7f43b3688640 1 -- 172.21.15.94:0/1098772266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac109b10 msgr2=0x7f43ac114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.858+0000 7f43b3688640 1 --2- 172.21.15.94:0/1098772266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac109b10 0x7f43ac114570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f439c009900 tx=0x7f439c02a480 comp rx=0 tx=0).stop 2024-09-19T04:22:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.858+0000 7f43b3688640 1 -- 172.21.15.94:0/1098772266 shutdown_connections 2024-09-19T04:22:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.858+0000 7f43b3688640 1 --2- 172.21.15.94:0/1098772266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ac114ab0 0x7f43ac116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.858+0000 7f43b3688640 1 --2- 172.21.15.94:0/1098772266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac109b10 0x7f43ac114570 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.858+0000 7f43b3688640 1 --2- 172.21.15.94:0/1098772266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43ac1091f0 0x7f43ac1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.858+0000 7f43b3688640 1 -- 172.21.15.94:0/1098772266 >> 172.21.15.94:0/1098772266 conn(0x7f43ac06e140 msgr2=0x7f43ac06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.858+0000 7f43b3688640 1 -- 172.21.15.94:0/1098772266 shutdown_connections 2024-09-19T04:22:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.858+0000 7f43b3688640 1 -- 172.21.15.94:0/1098772266 wait complete. 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.859+0000 7f43b3688640 1 Processor -- start 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.859+0000 7f43b3688640 1 -- start start 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b3688640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43ac1091f0 0x7f43ac1141e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b3688640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ac109b10 0x7f43ac111290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b3688640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac114ab0 0x7f43ac1117d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b3688640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac10ba90 con 0x7f43ac109b10 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b3688640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac10b910 con 0x7f43ac1091f0 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b3688640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac10bc10 con 0x7f43ac114ab0 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b0c23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ac109b10 0x7f43ac111290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b0c23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ac109b10 0x7f43ac111290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33190/0 (socket says 172.21.15.94:33190) 2024-09-19T04:22:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b0c23640 1 -- 172.21.15.94:0/1055321239 learned_addr learned my addr 172.21.15.94:0/1055321239 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b1c25640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac114ab0 0x7f43ac1117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.860+0000 7f43b1424640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43ac1091f0 0x7f43ac1141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.861+0000 7f43b0c23640 1 -- 172.21.15.94:0/1055321239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac114ab0 msgr2=0x7f43ac1117d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.861+0000 7f43b0c23640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac114ab0 0x7f43ac1117d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.861+0000 7f43b0c23640 1 -- 172.21.15.94:0/1055321239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43ac1091f0 msgr2=0x7f43ac1141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.861+0000 7f43b0c23640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43ac1091f0 0x7f43ac1141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.861+0000 7f43b0c23640 1 -- 172.21.15.94:0/1055321239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac111f00 con 0x7f43ac109b10 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.862+0000 7f43b0c23640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ac109b10 0x7f43ac111290 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f439c005940 tx=0x7f439c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.862+0000 7f43a27fc640 1 -- 172.21.15.94:0/1055321239 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c03f030 con 0x7f43ac109b10 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.862+0000 7f43a27fc640 1 -- 172.21.15.94:0/1055321239 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f439c003aa0 con 0x7f43ac109b10 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.863+0000 7f43b3688640 1 -- 172.21.15.94:0/1055321239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ac1b3840 con 0x7f43ac109b10 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.863+0000 7f43b3688640 1 -- 172.21.15.94:0/1055321239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ac1b3be0 con 0x7f43ac109b10 2024-09-19T04:22:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.863+0000 7f43a27fc640 1 -- 172.21.15.94:0/1055321239 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c035070 con 0x7f43ac109b10 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.869+0000 7f43a27fc640 1 -- 172.21.15.94:0/1055321239 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f439c042030 con 0x7f43ac109b10 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.869+0000 7f43b5a61740 1 Processor -- start 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.870+0000 7f43b5a61740 1 -- start start 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.870+0000 7f43a27fc640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f43980a3360 0x7f43980a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43a27fc640 1 -- 172.21.15.94:0/1055321239 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f439c0f0160 con 0x7f43ac109b10 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce096d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b1c25640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f43980a3360 0x7f43980a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096ddc0 0x55fce0978140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 0x55fce097aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b5a61740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fce08d32e0 con 0x55fce096ddc0 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b5a61740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fce087ace0 con 0x55fce0978680 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b5a61740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fce08cf2a0 con 0x55fce096d4a0 2024-09-19T04:22:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b0c23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096ddc0 0x55fce0978140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b0c23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096ddc0 0x55fce0978140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33204/0 (socket says 172.21.15.94:33204) 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.871+0000 7f43b0c23640 1 -- 172.21.15.94:0/1780393454 learned_addr learned my addr 172.21.15.94:0/1780393454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.872+0000 7f43b1424640 1 --2- 172.21.15.94:0/1780393454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce096d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.872+0000 7f43b1c25640 1 --2- 172.21.15.94:0/1780393454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 0x55fce097aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.872+0000 7f43b1424640 1 -- 172.21.15.94:0/1780393454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 msgr2=0x55fce097aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.872+0000 7f43b1424640 1 --2- 172.21.15.94:0/1780393454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 0x55fce097aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.872+0000 7f43b1424640 1 -- 172.21.15.94:0/1780393454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096ddc0 msgr2=0x55fce0978140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.872+0000 7f43b1424640 1 --2- 172.21.15.94:0/1780393454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096ddc0 0x55fce0978140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.872+0000 7f43b1424640 1 -- 172.21.15.94:0/1780393454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce085f440 con 0x55fce096d4a0 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.872+0000 7f43b1424640 1 --2- 172.21.15.94:0/1780393454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce096d880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f43a800bbf0 tx=0x7f43a80103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25da7ff6c0a4cae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.872+0000 7f43977fe640 1 -- 172.21.15.94:0/1780393454 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8004030 con 0x55fce096d4a0 2024-09-19T04:22:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.873+0000 7f43977fe640 1 -- 172.21.15.94:0/1780393454 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43a80035f0 con 0x55fce096d4a0 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.873+0000 7f43977fe640 1 -- 172.21.15.94:0/1780393454 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80116a0 con 0x55fce096d4a0 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 -- 172.21.15.94:0/1780393454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 msgr2=0x55fce096d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 --2- 172.21.15.94:0/1780393454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce096d880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f43a800bbf0 tx=0x7f43a80103d0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b1c25640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f43980a3360 0x7f43980a5820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009fd0 tx=0x7f43a4009ac0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 -- 172.21.15.94:0/1780393454 shutdown_connections 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 --2- 172.21.15.94:0/1780393454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 0x55fce097aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 --2- 172.21.15.94:0/1780393454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096ddc0 0x55fce0978140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 --2- 172.21.15.94:0/1780393454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce096d880 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 -- 172.21.15.94:0/1780393454 >> 172.21.15.94:0/1780393454 conn(0x55fce0969a30 msgr2=0x55fce0969e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 -- 172.21.15.94:0/1780393454 shutdown_connections 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 -- 172.21.15.94:0/1780393454 wait complete. 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.874+0000 7f43b5a61740 1 -- start start 2024-09-19T04:22:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.875+0000 7f43b5a61740 1 Processor -- start 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.875+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096d4a0 0x55fce0977fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.875+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce0975250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.875+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 0x55fce0975790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.875+0000 7f43b5a61740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fce097b250 con 0x55fce096d4a0 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.875+0000 7f43b5a61740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fce088e050 con 0x55fce0978680 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.875+0000 7f43b5a61740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fce097b3d0 con 0x55fce096ddc0 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.875+0000 7f43b5a61740 10 client.0 _ll_register_callbacks cb 0x55fce08ce160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b0c23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce0975250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b1424640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096d4a0 0x55fce0977fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b0c23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce0975250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55948/0 (socket says 172.21.15.94:55948) 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b0c23640 1 -- 172.21.15.94:0/379636684 learned_addr learned my addr 172.21.15.94:0/379636684 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b1c25640 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 0x55fce0975790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b0c23640 1 -- 172.21.15.94:0/379636684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 msgr2=0x55fce0975790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b0c23640 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 0x55fce0975790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b0c23640 1 -- 172.21.15.94:0/379636684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096d4a0 msgr2=0x55fce0977fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b0c23640 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096d4a0 0x55fce0977fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b0c23640 1 -- 172.21.15.94:0/379636684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce06443c0 con 0x55fce096ddc0 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f4394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b0c23640 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce0975250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f439c005e00 tx=0x7f439c0b87e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdf749c5f43e7548 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f4394ff9640 1 -- 172.21.15.94:0/379636684 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c0d4070 con 0x55fce096ddc0 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f4394ff9640 1 -- 172.21.15.94:0/379636684 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f439c0edac0 con 0x55fce096ddc0 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.876+0000 7f43b5a61740 1 -- 172.21.15.94:0/379636684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a80035f0 con 0x55fce096ddc0 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f4394ff9640 1 -- 172.21.15.94:0/379636684 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f439c049040 con 0x55fce096ddc0 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f43b5a61740 10 client.25774 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f43b5a61740 1 -- 172.21.15.94:0/379636684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fce097b2b0 con 0x55fce096ddc0 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f4394ff9640 1 -- 172.21.15.94:0/379636684 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f439c0edcc0 con 0x55fce096ddc0 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f4394ff9640 1 -- 172.21.15.94:0/379636684 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f439c0bc6e0 con 0x55fce096ddc0 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f43b5a61740 10 client.25774 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f43b5a61740 10 client.25774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f43b5a61740 20 client.25774 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f43b5a61740 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fce09880d0 0x55fce098a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f43b5a61740 4 client.25774 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.877+0000 7f43b5a61740 1 -- 172.21.15.94:0/379636684 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f43a80035f0 con 0x55fce09880d0 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.878+0000 7f43b1424640 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fce09880d0 0x55fce098a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.879+0000 7f43b1424640 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fce09880d0 0x55fce098a530 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f462b4e1a71b7a7f server_cookie=5247f4751ae5e3d1 in_seq=0 out_seq=0 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.879+0000 7f4394ff9640 10 client.25774 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.880+0000 7f4394ff9640 1 -- 172.21.15.94:0/379636684 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f43a80035f0 con 0x55fce09880d0 2024-09-19T04:22:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.880+0000 7f4394ff9640 10 client.25774 handle_command_reply: tid=0 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.880+0000 7f43b5a61740 1 client.25774 shutdown 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.880+0000 7f43b5a61740 1 -- 172.21.15.94:0/379636684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 msgr2=0x55fce0975250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.880+0000 7f43b5a61740 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce0975250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f439c005e00 tx=0x7f439c0b87e0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f43b5a61740 1 -- 172.21.15.94:0/379636684 shutdown_connections 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f43b5a61740 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fce09880d0 0x55fce098a530 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f43b5a61740 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce0978680 0x55fce0975790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f43b5a61740 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce0975250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f43b5a61740 1 --2- 172.21.15.94:0/379636684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce096d4a0 0x55fce0977fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f4394ff9640 0 client.25774 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f43b5a61740 1 -- 172.21.15.94:0/379636684 >> 172.21.15.94:0/379636684 conn(0x55fce0969a30 msgr2=0x55fce0968b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f43b5a61740 1 -- 172.21.15.94:0/379636684 shutdown_connections 2024-09-19T04:22:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f43b5a61740 1 -- 172.21.15.94:0/379636684 wait complete. 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.881+0000 7f43b5a61740 20 client.25774 trim_cache size 0 max 16384 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.911+0000 7f43b5a61740 1 Processor -- start 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.911+0000 7f43b5a61740 1 -- start start 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.912+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce09844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.912+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096d4a0 0x55fce0975210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.912+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce098fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.912+0000 7f43b5a61740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1bec50 con 0x55fce0969a30 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.912+0000 7f43b1424640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce09844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.912+0000 7f43b1424640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce09844b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33220/0 (socket says 172.21.15.94:33220) 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.912+0000 7f43b1424640 1 -- 172.21.15.94:0/2952405183 learned_addr learned my addr 172.21.15.94:0/2952405183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.912+0000 7f43b5a61740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1bc4c0 con 0x55fce096d4a0 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.913+0000 7f43b1c25640 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce098fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.913+0000 7f43b0c23640 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096d4a0 0x55fce0975210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.913+0000 7f43b5a61740 1 -- 172.21.15.94:0/2952405183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1ba310 con 0x55fce096ddc0 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.913+0000 7f43b1424640 1 -- 172.21.15.94:0/2952405183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 msgr2=0x55fce098fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.914+0000 7f43b1424640 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce098fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.914+0000 7f43b1424640 1 -- 172.21.15.94:0/2952405183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096d4a0 msgr2=0x55fce0975210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.914+0000 7f43b1424640 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096d4a0 0x55fce0975210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.914+0000 7f43b1424640 1 -- 172.21.15.94:0/2952405183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce0978310 con 0x55fce0969a30 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.914+0000 7f43b0c23640 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096d4a0 0x55fce0975210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.914+0000 7f43b1424640 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce09844b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f43a8000c00 tx=0x7f43a800b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d63963854ddba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.915+0000 7f4396ffd640 1 -- 172.21.15.94:0/2952405183 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a802c020 con 0x55fce0969a30 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.915+0000 7f4396ffd640 1 -- 172.21.15.94:0/2952405183 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43a8005490 con 0x55fce0969a30 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.915+0000 7f4396ffd640 1 -- 172.21.15.94:0/2952405183 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8011910 con 0x55fce0969a30 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.916+0000 7f43b5a61740 1 -- 172.21.15.94:0/2952405183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 msgr2=0x55fce09844b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.916+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce09844b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f43a8000c00 tx=0x7f43a800b590 comp rx=0 tx=0).stop 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.916+0000 7f43b5a61740 1 -- 172.21.15.94:0/2952405183 shutdown_connections 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.916+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096ddc0 0x55fce098fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.916+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096d4a0 0x55fce0975210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.916+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2952405183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce09844b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.916+0000 7f43b5a61740 1 -- 172.21.15.94:0/2952405183 >> 172.21.15.94:0/2952405183 conn(0x55fce09758f0 msgr2=0x55fce0974340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.916+0000 7f43b5a61740 1 -- 172.21.15.94:0/2952405183 shutdown_connections 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.916+0000 7f43b5a61740 1 -- 172.21.15.94:0/2952405183 wait complete. 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.917+0000 7f43b5a61740 1 -- start start 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.917+0000 7f43b5a61740 1 Processor -- start 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.917+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce0980c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce097df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b5a61740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096ddc0 0x55fce097e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b5a61740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1ba980 con 0x55fce0969a30 2024-09-19T04:22:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b5a61740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1bbdf0 con 0x55fce096ddc0 2024-09-19T04:22:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b5a61740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1bb050 con 0x55fce096d4a0 2024-09-19T04:22:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b5a61740 10 client.0 _ll_register_callbacks cb 0x7f439c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b0c23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce097df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b0c23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce097df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:55960/0 (socket says 172.21.15.94:55960) 2024-09-19T04:22:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b0c23640 1 -- 172.21.15.94:0/2593817301 learned_addr learned my addr 172.21.15.94:0/2593817301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b1c25640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096ddc0 0x55fce097e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.918+0000 7f43b1424640 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce0980c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.919+0000 7f43b1424640 1 -- 172.21.15.94:0/2593817301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 msgr2=0x55fce097df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.919+0000 7f43b1424640 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce097df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.919+0000 7f43b1424640 1 -- 172.21.15.94:0/2593817301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096ddc0 msgr2=0x55fce097e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.919+0000 7f43b1424640 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096ddc0 0x55fce097e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.919+0000 7f43b1424640 1 -- 172.21.15.94:0/2593817301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce097eba0 con 0x55fce0969a30 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.919+0000 7f43b1c25640 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096ddc0 0x55fce097e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.919+0000 7f43b0c23640 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce097df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.920+0000 7f43b1424640 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce0980c20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43a800bbc0 tx=0x7f43a8010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36710f9a97ab4d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.920+0000 7f4394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:22:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.920+0000 7f4394ff9640 1 -- 172.21.15.94:0/2593817301 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8035060 con 0x55fce0969a30 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.920+0000 7f4394ff9640 1 -- 172.21.15.94:0/2593817301 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f43a8003a00 con 0x55fce0969a30 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.920+0000 7f43b5a61740 1 -- 172.21.15.94:0/2593817301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a8005490 con 0x55fce0969a30 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.920+0000 7f4394ff9640 1 -- 172.21.15.94:0/2593817301 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f43a8003c40 con 0x55fce0969a30 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f43b5a61740 10 client.16212 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f43b5a61740 1 -- 172.21.15.94:0/2593817301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fce09f2c90 con 0x55fce0969a30 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f4394ff9640 1 -- 172.21.15.94:0/2593817301 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43a8005490 con 0x55fce0969a30 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f4394ff9640 1 -- 172.21.15.94:0/2593817301 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f43a8027550 con 0x55fce0969a30 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f43b5a61740 10 client.16212 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f43b5a61740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f43b5a61740 20 client.16212 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fce0978680 0x55fce09f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f43b5a61740 4 client.16212 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:22:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.921+0000 7f43b5a61740 1 -- 172.21.15.94:0/2593817301 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fce08e0850 con 0x55fce0978680 2024-09-19T04:22:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.922+0000 7f43b0c23640 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fce0978680 0x55fce09f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.922+0000 7f4394ff9640 10 client.16212 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.922+0000 7f43b0c23640 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fce0978680 0x55fce09f50d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5bee36697c9232 server_cookie=767b3630cc5a7ef1 in_seq=0 out_seq=0 2024-09-19T04:22:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.922+0000 7f4394ff9640 1 -- 172.21.15.94:0/2593817301 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fce08e0850 con 0x55fce0978680 2024-09-19T04:22:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f4394ff9640 10 client.16212 handle_command_reply: tid=0 2024-09-19T04:22:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 client.16212 shutdown 2024-09-19T04:22:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 -- 172.21.15.94:0/2593817301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 msgr2=0x55fce0980c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce0980c20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43a800bbc0 tx=0x7f43a8010f20 comp rx=0 tx=0).stop 2024-09-19T04:22:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 -- 172.21.15.94:0/2593817301 shutdown_connections 2024-09-19T04:22:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fce0978680 0x55fce09f50d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce096ddc0 0x55fce097e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fce096d4a0 0x55fce097df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 --2- 172.21.15.94:0/2593817301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fce0969a30 0x55fce0980c20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f4394ff9640 0 client.16212 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 -- 172.21.15.94:0/2593817301 >> 172.21.15.94:0/2593817301 conn(0x55fce09758f0 msgr2=0x55fce0976ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.147 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:22:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:22:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:22:02.147 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:22:02.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 -- 172.21.15.94:0/2593817301 shutdown_connections 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 1 -- 172.21.15.94:0/2593817301 wait complete. 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.923+0000 7f43b5a61740 20 client.16212 trim_cache size 0 max 16384 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.924+0000 7f43b3688640 1 -- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f43980a3360 msgr2=0x7f43980a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.924+0000 7f43b3688640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f43980a3360 0x7f43980a5820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009fd0 tx=0x7f43a4009ac0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.924+0000 7f43b3688640 1 -- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ac109b10 msgr2=0x7f43ac111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.924+0000 7f43b3688640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ac109b10 0x7f43ac111290 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f439c005940 tx=0x7f439c004270 comp rx=0 tx=0).stop 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.925+0000 7f43b3688640 1 -- 172.21.15.94:0/1055321239 shutdown_connections 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.925+0000 7f43b3688640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f43980a3360 0x7f43980a5820 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.925+0000 7f43b3688640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac114ab0 0x7f43ac1117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.925+0000 7f43b3688640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ac109b10 0x7f43ac111290 secure :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f439c005940 tx=0x7f439c004270 comp rx=0 tx=0).stop 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.925+0000 7f43b3688640 1 --2- 172.21.15.94:0/1055321239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43ac1091f0 0x7f43ac1141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.925+0000 7f43b3688640 1 -- 172.21.15.94:0/1055321239 >> 172.21.15.94:0/1055321239 conn(0x7f43ac06e140 msgr2=0x7f43ac0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.925+0000 7f43b3688640 1 -- 172.21.15.94:0/1055321239 shutdown_connections 2024-09-19T04:22:02.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:01.925+0000 7f43b3688640 1 -- 172.21.15.94:0/1055321239 wait complete. 2024-09-19T04:22:02.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:02 smithi112 ceph-mon[35051]: pgmap v213: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-19T04:22:02.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:02 smithi094 ceph-mon[30972]: pgmap v213: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-19T04:22:02.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:02 smithi104 ceph-mon[34145]: pgmap v213: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-19T04:22:03.071 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:22:03.071 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:22:03.482 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:04.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.079+0000 7f94e7d08640 1 -- 172.21.15.94:0/1508332467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94e01091f0 msgr2=0x7f94e01095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.079+0000 7f94e7d08640 1 --2- 172.21.15.94:0/1508332467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94e01091f0 0x7f94e01095d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009900 tx=0x7f94d002a480 comp rx=0 tx=0).stop 2024-09-19T04:22:04.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.080+0000 7f94e7d08640 1 -- 172.21.15.94:0/1508332467 shutdown_connections 2024-09-19T04:22:04.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.080+0000 7f94e7d08640 1 --2- 172.21.15.94:0/1508332467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e0114ab0 0x7f94e0116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.080+0000 7f94e7d08640 1 --2- 172.21.15.94:0/1508332467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e0109b10 0x7f94e0114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.080+0000 7f94e7d08640 1 --2- 172.21.15.94:0/1508332467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94e01091f0 0x7f94e01095d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.080+0000 7f94e7d08640 1 -- 172.21.15.94:0/1508332467 >> 172.21.15.94:0/1508332467 conn(0x7f94e006e140 msgr2=0x7f94e006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:04.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.080+0000 7f94e7d08640 1 -- 172.21.15.94:0/1508332467 shutdown_connections 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.080+0000 7f94e7d08640 1 -- 172.21.15.94:0/1508332467 wait complete. 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e7d08640 1 Processor -- start 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e7d08640 1 -- start start 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e7d08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94e01091f0 0x7f94e01acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e7d08640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e0109b10 0x7f94e01ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e7d08640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e0114ab0 0x7f94e0113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e7d08640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94e011a6c0 con 0x7f94e01091f0 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e7d08640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94e011a540 con 0x7f94e0114ab0 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e7d08640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94e011a840 con 0x7f94e0109b10 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e52a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e0109b10 0x7f94e01ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e52a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e0109b10 0x7f94e01ad460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58734/0 (socket says 172.21.15.94:58734) 2024-09-19T04:22:04.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e52a3640 1 -- 172.21.15.94:0/2418688969 learned_addr learned my addr 172.21.15.94:0/2418688969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:04.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e62a5640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e0114ab0 0x7f94e0113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.081+0000 7f94e5aa4640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94e01091f0 0x7f94e01acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.082+0000 7f94e62a5640 1 -- 172.21.15.94:0/2418688969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e0109b10 msgr2=0x7f94e01ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.082+0000 7f94e62a5640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e0109b10 0x7f94e01ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.082+0000 7f94e62a5640 1 -- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94e01091f0 msgr2=0x7f94e01acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.082+0000 7f94e62a5640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94e01091f0 0x7f94e01acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.082+0000 7f94e62a5640 1 -- 172.21.15.94:0/2418688969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e0113fc0 con 0x7f94e0114ab0 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.082+0000 7f94e62a5640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e0114ab0 0x7f94e0113790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f94e00743f0 tx=0x7f94dc00dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.082+0000 7f94d6ffd640 1 -- 172.21.15.94:0/2418688969 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc017070 con 0x7f94e0114ab0 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.083+0000 7f94d6ffd640 1 -- 172.21.15.94:0/2418688969 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94dc0047a0 con 0x7f94e0114ab0 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.083+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e01142b0 con 0x7f94e0114ab0 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.083+0000 7f94d6ffd640 1 -- 172.21.15.94:0/2418688969 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc009c40 con 0x7f94e0114ab0 2024-09-19T04:22:04.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.083+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e01b3ee0 con 0x7f94e0114ab0 2024-09-19T04:22:04.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.084+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e0073a40 con 0x7f94e0114ab0 2024-09-19T04:22:04.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.086+0000 7f94d6ffd640 1 -- 172.21.15.94:0/2418688969 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f94dc004950 con 0x7f94e0114ab0 2024-09-19T04:22:04.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.088+0000 7f94d6ffd640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94cc0a3500 0x7f94cc0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.088+0000 7f94e5aa4640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94cc0a3500 0x7f94cc0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.088+0000 7f94d6ffd640 1 -- 172.21.15.94:0/2418688969 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f94dc0cf480 con 0x7f94e0114ab0 2024-09-19T04:22:04.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.088+0000 7f94e5aa4640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94cc0a3500 0x7f94cc0a59c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005ab0 tx=0x7f94d0000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:04.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.092+0000 7f94d6ffd640 1 -- 172.21.15.94:0/2418688969 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f94dc09b7a0 con 0x7f94e0114ab0 2024-09-19T04:22:04.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.457+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94e010da20 con 0x7f94e0114ab0 2024-09-19T04:22:04.459 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:22:04.459 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":38,"btime":"2024-09-19T04:21:43:442341+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:21:42.753568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:22:04.460 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 38 2024-09-19T04:22:04.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.458+0000 7f94d6ffd640 1 -- 172.21.15.94:0/2418688969 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11921 (secure 0 0 0) 0x7f94dc09faa0 con 0x7f94e0114ab0 2024-09-19T04:22:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.461+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94cc0a3500 msgr2=0x7f94cc0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.461+0000 7f94e7d08640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94cc0a3500 0x7f94cc0a59c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005ab0 tx=0x7f94d0000bc0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.461+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e0114ab0 msgr2=0x7f94e0113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.461+0000 7f94e7d08640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e0114ab0 0x7f94e0113790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f94e00743f0 tx=0x7f94dc00dae0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.462+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 shutdown_connections 2024-09-19T04:22:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.462+0000 7f94e7d08640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f94cc0a3500 0x7f94cc0a59c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.462+0000 7f94e7d08640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e0114ab0 0x7f94e0113790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.462+0000 7f94e7d08640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e0109b10 0x7f94e01ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.462+0000 7f94e7d08640 1 --2- 172.21.15.94:0/2418688969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94e01091f0 0x7f94e01acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.462+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 >> 172.21.15.94:0/2418688969 conn(0x7f94e006e140 msgr2=0x7f94e0070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.463+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 shutdown_connections 2024-09-19T04:22:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:04.463+0000 7f94e7d08640 1 -- 172.21.15.94:0/2418688969 wait complete. 2024-09-19T04:22:04.793 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:04 smithi094 ceph-mon[30972]: pgmap v214: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 19 MiB/s wr, 1.95k op/s 2024-09-19T04:22:04.793 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:04 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2418688969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:22:04.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:04 smithi104 ceph-mon[34145]: pgmap v214: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 19 MiB/s wr, 1.95k op/s 2024-09-19T04:22:04.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:04 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2418688969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:22:05.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:04 smithi112 ceph-mon[35051]: pgmap v214: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 19 MiB/s wr, 1.95k op/s 2024-09-19T04:22:05.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:04 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2418688969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:22:05.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:22:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 350.0 s, 1486.6 tps, lat 66.991 ms stddev 19.867 2024-09-19T04:22:06.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.179+0000 7f20d4192640 1 -- 172.21.15.94:0/3023266120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20cc10e500 msgr2=0x7f20cc110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:06.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.179+0000 7f20d4192640 1 --2- 172.21.15.94:0/3023266120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20cc10e500 0x7f20cc110910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009be0 tx=0x7f20c402a6c0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.180+0000 7f20d4192640 1 -- 172.21.15.94:0/3023266120 shutdown_connections 2024-09-19T04:22:06.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.180+0000 7f20d4192640 1 --2- 172.21.15.94:0/3023266120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20cc10e500 0x7f20cc110910 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.180+0000 7f20d4192640 1 --2- 172.21.15.94:0/3023266120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20cc101440 0x7f20cc10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.180+0000 7f20d4192640 1 --2- 172.21.15.94:0/3023266120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20cc100b20 0x7f20cc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.180+0000 7f20d4192640 1 -- 172.21.15.94:0/3023266120 >> 172.21.15.94:0/3023266120 conn(0x7f20cc0fc7d0 msgr2=0x7f20cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:06.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.180+0000 7f20d4192640 1 -- 172.21.15.94:0/3023266120 shutdown_connections 2024-09-19T04:22:06.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.180+0000 7f20d4192640 1 -- 172.21.15.94:0/3023266120 wait complete. 2024-09-19T04:22:06.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.181+0000 7f20d4192640 1 Processor -- start 2024-09-19T04:22:06.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.181+0000 7f20d4192640 1 -- start start 2024-09-19T04:22:06.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.181+0000 7f20d4192640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20cc100b20 0x7f20cc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.181+0000 7f20d4192640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20cc101440 0x7f20cc19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.181+0000 7f20d4192640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20cc10e500 0x7f20cc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.181+0000 7f20d4192640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc114100 con 0x7f20cc100b20 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.181+0000 7f20d4192640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc113f80 con 0x7f20cc10e500 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.181+0000 7f20d4192640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc114280 con 0x7f20cc101440 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.181+0000 7f20d272f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20cc10e500 0x7f20cc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.182+0000 7f20d272f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20cc10e500 0x7f20cc19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:54558/0 (socket says 172.21.15.94:54558) 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.182+0000 7f20d272f640 1 -- 172.21.15.94:0/1340006851 learned_addr learned my addr 172.21.15.94:0/1340006851 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.182+0000 7f20d172d640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20cc101440 0x7f20cc19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.182+0000 7f20d272f640 1 -- 172.21.15.94:0/1340006851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20cc101440 msgr2=0x7f20cc19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.182+0000 7f20d272f640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20cc101440 0x7f20cc19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.182+0000 7f20d272f640 1 -- 172.21.15.94:0/1340006851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20cc100b20 msgr2=0x7f20cc19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.182+0000 7f20d272f640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20cc100b20 0x7f20cc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.182+0000 7f20d272f640 1 -- 172.21.15.94:0/1340006851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc19f670 con 0x7f20cc10e500 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.182+0000 7f20d272f640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20cc10e500 0x7f20cc19ef90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f20c4034040 tx=0x7f20c4003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.183+0000 7f20c2ffd640 1 -- 172.21.15.94:0/1340006851 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4041020 con 0x7f20cc10e500 2024-09-19T04:22:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.183+0000 7f20c2ffd640 1 -- 172.21.15.94:0/1340006851 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20c4037040 con 0x7f20cc10e500 2024-09-19T04:22:06.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.183+0000 7f20c2ffd640 1 -- 172.21.15.94:0/1340006851 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4036070 con 0x7f20cc10e500 2024-09-19T04:22:06.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.183+0000 7f20d4192640 1 -- 172.21.15.94:0/1340006851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc19f900 con 0x7f20cc10e500 2024-09-19T04:22:06.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.183+0000 7f20d4192640 1 -- 172.21.15.94:0/1340006851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc19fd90 con 0x7f20cc10e500 2024-09-19T04:22:06.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.187+0000 7f20d656b740 1 Processor -- start 2024-09-19T04:22:06.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.187+0000 7f20c2ffd640 1 -- 172.21.15.94:0/1340006851 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f20c4044020 con 0x7f20cc10e500 2024-09-19T04:22:06.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.187+0000 7f20d656b740 1 -- start start 2024-09-19T04:22:06.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d656b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 0x5640b04a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d656b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d656b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 0x5640b04b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d656b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5640b04092f0 con 0x5640b04ae370 2024-09-19T04:22:06.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d656b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640b03b0cf0 con 0x5640b04a4b40 2024-09-19T04:22:06.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d656b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5640b04052b0 con 0x5640b04a35d0 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d172d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 0x5640b04b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d1f2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d172d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 0x5640b04b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37380/0 (socket says 172.21.15.94:37380) 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d172d640 1 -- 172.21.15.94:0/4010923892 learned_addr learned my addr 172.21.15.94:0/4010923892 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d172d640 1 --2- 172.21.15.94:0/4010923892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 0x5640b04a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d172d640 1 -- 172.21.15.94:0/4010923892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 msgr2=0x5640b04a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d172d640 1 --2- 172.21.15.94:0/4010923892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d172d640 1 -- 172.21.15.94:0/4010923892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 msgr2=0x5640b04b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.188+0000 7f20d172d640 1 --2- 172.21.15.94:0/4010923892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 0x5640b04b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.189+0000 7f20d172d640 1 -- 172.21.15.94:0/4010923892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b03953a0 con 0x5640b04a4b40 2024-09-19T04:22:06.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.189+0000 7f20c2ffd640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20b80a3540 0x7f20b80a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.189+0000 7f20d172d640 1 --2- 172.21.15.94:0/4010923892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 0x5640b04a3090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0058c0 tx=0x7f20bc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00bb439609f4d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:06.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.189+0000 7f20b7fff640 1 -- 172.21.15.94:0/4010923892 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc018020 con 0x5640b04a4b40 2024-09-19T04:22:06.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.189+0000 7f20c2ffd640 1 -- 172.21.15.94:0/1340006851 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f20c40effb0 con 0x7f20cc10e500 2024-09-19T04:22:06.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.189+0000 7f20b7fff640 1 -- 172.21.15.94:0/4010923892 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20bc00e750 con 0x5640b04a4b40 2024-09-19T04:22:06.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.189+0000 7f20b7fff640 1 -- 172.21.15.94:0/4010923892 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0035f0 con 0x5640b04a4b40 2024-09-19T04:22:06.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.190+0000 7f20d1f2e640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20b80a3540 0x7f20b80a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.190+0000 7f20d656b740 1 -- 172.21.15.94:0/4010923892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 msgr2=0x5640b04a3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.190+0000 7f20d656b740 1 --2- 172.21.15.94:0/4010923892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 0x5640b04a3090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0058c0 tx=0x7f20bc009260 comp rx=0 tx=0).stop 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.190+0000 7f20d1f2e640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20b80a3540 0x7f20b80a5a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005d00 tx=0x7f20c8005f30 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.190+0000 7f20d656b740 1 -- 172.21.15.94:0/4010923892 shutdown_connections 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.190+0000 7f20d656b740 1 --2- 172.21.15.94:0/4010923892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 0x5640b04b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.190+0000 7f20d656b740 1 --2- 172.21.15.94:0/4010923892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.190+0000 7f20d656b740 1 --2- 172.21.15.94:0/4010923892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 0x5640b04a3090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.191+0000 7f20d656b740 1 -- 172.21.15.94:0/4010923892 >> 172.21.15.94:0/4010923892 conn(0x5640b046c310 msgr2=0x5640b049fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.191+0000 7f20d656b740 1 -- 172.21.15.94:0/4010923892 shutdown_connections 2024-09-19T04:22:06.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.191+0000 7f20d656b740 1 -- 172.21.15.94:0/4010923892 wait complete. 2024-09-19T04:22:06.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.192+0000 7f20d656b740 1 -- start start 2024-09-19T04:22:06.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.192+0000 7f20d656b740 1 Processor -- start 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d656b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 0x5640b04add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d656b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 0x5640b04aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d656b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d656b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5640b04b10c0 con 0x5640b04a35d0 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d656b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640b03c4060 con 0x5640b04ae370 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d1f2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 0x5640b04add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d1f2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 0x5640b04add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37394/0 (socket says 172.21.15.94:37394) 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d1f2e640 1 -- 172.21.15.94:0/3060549464 learned_addr learned my addr 172.21.15.94:0/3060549464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d272f640 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d172d640 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 0x5640b04aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d656b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5640b04b1240 con 0x5640b04a4b40 2024-09-19T04:22:06.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.193+0000 7f20d656b740 10 client.0 _ll_register_callbacks cb 0x5640b01178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:06.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20d272f640 1 -- 172.21.15.94:0/3060549464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 msgr2=0x5640b04aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20d272f640 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 0x5640b04aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20d272f640 1 -- 172.21.15.94:0/3060549464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 msgr2=0x5640b04add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20d272f640 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 0x5640b04add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20d272f640 1 -- 172.21.15.94:0/3060549464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b017a3c0 con 0x5640b04ae370 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20d172d640 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 0x5640b04aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20d272f640 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04ab530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f20c40ed690 tx=0x7f20c40bc070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c067fcb8d464ccae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20b57fa640 1 -- 172.21.15.94:0/3060549464 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40d4020 con 0x5640b04ae370 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20b57fa640 1 -- 172.21.15.94:0/3060549464 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f20c40ed910 con 0x5640b04ae370 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.194+0000 7f20d656b740 1 -- 172.21.15.94:0/3060549464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20bc00e750 con 0x5640b04ae370 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.195+0000 7f20b57fa640 1 -- 172.21.15.94:0/3060549464 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f20c40bc9a0 con 0x5640b04ae370 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.195+0000 7f20d656b740 10 client.25820 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:06.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.195+0000 7f20d656b740 1 -- 172.21.15.94:0/3060549464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5640b04bcb20 con 0x5640b04ae370 2024-09-19T04:22:06.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.195+0000 7f20b57fa640 1 -- 172.21.15.94:0/3060549464 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20c40bc2a0 con 0x5640b04ae370 2024-09-19T04:22:06.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.195+0000 7f20b57fa640 1 -- 172.21.15.94:0/3060549464 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f20c4004510 con 0x5640b04ae370 2024-09-19T04:22:06.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.196+0000 7f20d656b740 10 client.25820 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:06.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.196+0000 7f20d656b740 10 client.25820 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:22:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.196+0000 7f20d656b740 20 client.25820 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.196+0000 7f20d656b740 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5640b04be110 0x5640b04c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.196+0000 7f20d656b740 4 client.25820 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.196+0000 7f20d656b740 1 -- 172.21.15.94:0/3060549464 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f20bc00e750 con 0x5640b04be110 2024-09-19T04:22:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.196+0000 7f20d1f2e640 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5640b04be110 0x5640b04c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.197+0000 7f20b57fa640 10 client.25820 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:22:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.197+0000 7f20d1f2e640 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5640b04be110 0x5640b04c0500 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5ccbb1a50f0736d server_cookie=f9ce631cd29445 in_seq=0 out_seq=0 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20b57fa640 1 -- 172.21.15.94:0/3060549464 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f20bc00e750 con 0x5640b04be110 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20b57fa640 10 client.25820 handle_command_reply: tid=0 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20d656b740 1 client.25820 shutdown 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20d656b740 1 -- 172.21.15.94:0/3060549464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 msgr2=0x5640b04ab530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20d656b740 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04ab530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f20c40ed690 tx=0x7f20c40bc070 comp rx=0 tx=0).stop 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20d656b740 1 -- 172.21.15.94:0/3060549464 shutdown_connections 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20d656b740 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5640b04be110 0x5640b04c0500 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20d656b740 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04ab530 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20d656b740 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 0x5640b04aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20d656b740 1 --2- 172.21.15.94:0/3060549464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 0x5640b04add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20b57fa640 0 client.25820 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:22:06.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.198+0000 7f20d656b740 1 -- 172.21.15.94:0/3060549464 >> 172.21.15.94:0/3060549464 conn(0x5640b046c310 msgr2=0x5640b04a5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:06.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.199+0000 7f20d656b740 1 -- 172.21.15.94:0/3060549464 shutdown_connections 2024-09-19T04:22:06.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.199+0000 7f20d656b740 1 -- 172.21.15.94:0/3060549464 wait complete. 2024-09-19T04:22:06.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.199+0000 7f20d656b740 20 client.25820 trim_cache size 0 max 16384 2024-09-19T04:22:06.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.222+0000 7f20d656b740 1 Processor -- start 2024-09-19T04:22:06.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.223+0000 7f20d656b740 1 -- start start 2024-09-19T04:22:06.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.223+0000 7f20d656b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 0x5640b049ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.223+0000 7f20d656b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 0x5640b04a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.223+0000 7f20d656b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04a6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.223+0000 7f20d656b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1b2260 con 0x5640b04a35d0 2024-09-19T04:22:06.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.223+0000 7f20d656b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1afad0 con 0x5640b04ae370 2024-09-19T04:22:06.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d656b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ad920 con 0x5640b04a4b40 2024-09-19T04:22:06.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d172d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 0x5640b04a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d272f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d272f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:54592/0 (socket says 172.21.15.94:54592) 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d272f640 1 -- 172.21.15.94:0/3373488255 learned_addr learned my addr 172.21.15.94:0/3373488255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d1f2e640 1 --2- 172.21.15.94:0/3373488255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 0x5640b049ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d272f640 1 -- 172.21.15.94:0/3373488255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 msgr2=0x5640b04a6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d272f640 1 --2- 172.21.15.94:0/3373488255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 0x5640b04a6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d272f640 1 -- 172.21.15.94:0/3373488255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 msgr2=0x5640b049ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d272f640 1 --2- 172.21.15.94:0/3373488255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 0x5640b049ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d272f640 1 -- 172.21.15.94:0/3373488255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b049fed0 con 0x5640b04ae370 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.224+0000 7f20d272f640 1 --2- 172.21.15.94:0/3373488255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04a6660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20c40c49e0 tx=0x7f20c40bc1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef08e00c233aaecb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.225+0000 7f20b77fe640 1 -- 172.21.15.94:0/3373488255 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40d4070 con 0x5640b04ae370 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.225+0000 7f20b77fe640 1 -- 172.21.15.94:0/3373488255 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20c40c4cf0 con 0x5640b04ae370 2024-09-19T04:22:06.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.225+0000 7f20b77fe640 1 -- 172.21.15.94:0/3373488255 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40c0650 con 0x5640b04ae370 2024-09-19T04:22:06.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.226+0000 7f20d656b740 1 -- 172.21.15.94:0/3373488255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 msgr2=0x5640b04a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:06.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.226+0000 7f20d656b740 1 --2- 172.21.15.94:0/3373488255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04a6660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20c40c49e0 tx=0x7f20c40bc1a0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.226+0000 7f20d656b740 1 -- 172.21.15.94:0/3373488255 shutdown_connections 2024-09-19T04:22:06.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.226+0000 7f20d656b740 1 --2- 172.21.15.94:0/3373488255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04ae370 0x5640b04a6660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.226+0000 7f20d656b740 1 --2- 172.21.15.94:0/3373488255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a4b40 0x5640b04a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.226+0000 7f20d656b740 1 --2- 172.21.15.94:0/3373488255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04a35d0 0x5640b049ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.226+0000 7f20d656b740 1 -- 172.21.15.94:0/3373488255 >> 172.21.15.94:0/3373488255 conn(0x5640b046c310 msgr2=0x5640b049e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:06.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.226+0000 7f20d656b740 1 -- 172.21.15.94:0/3373488255 shutdown_connections 2024-09-19T04:22:06.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.226+0000 7f20d656b740 1 -- 172.21.15.94:0/3373488255 wait complete. 2024-09-19T04:22:06.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.227+0000 7f20d656b740 1 -- start start 2024-09-19T04:22:06.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.227+0000 7f20d656b740 1 Processor -- start 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.227+0000 7f20d656b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d656b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 0x5640b04b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d656b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 0x5640b0526d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d656b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1adf90 con 0x5640b04ae370 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d656b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1af400 con 0x5640b04a4b40 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d656b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ae660 con 0x5640b04a35d0 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d656b740 10 client.0 _ll_register_callbacks cb 0x7f20c40319e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d1f2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d1f2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58790/0 (socket says 172.21.15.94:58790) 2024-09-19T04:22:06.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d1f2e640 1 -- 172.21.15.94:0/3331172684 learned_addr learned my addr 172.21.15.94:0/3331172684 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d172d640 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 0x5640b04b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d1f2e640 1 -- 172.21.15.94:0/3331172684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 msgr2=0x5640b04b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d1f2e640 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 0x5640b04b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d1f2e640 1 -- 172.21.15.94:0/3331172684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 msgr2=0x5640b0526d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d1f2e640 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 0x5640b0526d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.228+0000 7f20d1f2e640 1 -- 172.21.15.94:0/3331172684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b05272d0 con 0x5640b04a35d0 2024-09-19T04:22:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.229+0000 7f20d1f2e640 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04b1880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5640b04c0d40 tx=0x7f20c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de332aeae9c41d57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.229+0000 7f20b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:22:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.229+0000 7f20b57fa640 1 -- 172.21.15.94:0/3331172684 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8028500 con 0x5640b04a35d0 2024-09-19T04:22:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.229+0000 7f20b57fa640 1 -- 172.21.15.94:0/3331172684 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8541+0+0 (secure 0 0 0) 0x7f20c80286a0 con 0x5640b04a35d0 2024-09-19T04:22:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.229+0000 7f20d656b740 1 -- 172.21.15.94:0/3331172684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5640b0416860 con 0x5640b04a35d0 2024-09-19T04:22:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.229+0000 7f20b57fa640 1 -- 172.21.15.94:0/3331172684 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f20c80288a0 con 0x5640b04a35d0 2024-09-19T04:22:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.229+0000 7f20d656b740 10 client.25804 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.229+0000 7f20d656b740 1 -- 172.21.15.94:0/3331172684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5640b0530d20 con 0x5640b04a35d0 2024-09-19T04:22:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.230+0000 7f20b57fa640 1 -- 172.21.15.94:0/3331172684 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5640b0416860 con 0x5640b04a35d0 2024-09-19T04:22:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.230+0000 7f20b57fa640 1 -- 172.21.15.94:0/3331172684 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f20c8003950 con 0x5640b04a35d0 2024-09-19T04:22:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.230+0000 7f20d656b740 10 client.25804 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.230+0000 7f20d656b740 10 client.25804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:22:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.230+0000 7f20d656b740 20 client.25804 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.230+0000 7f20d656b740 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5640b05313d0 0x5640b0533830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.230+0000 7f20d656b740 4 client.25804 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.230+0000 7f20d656b740 1 -- 172.21.15.94:0/3331172684 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640b0410d70 con 0x5640b05313d0 2024-09-19T04:22:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.231+0000 7f20d172d640 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5640b05313d0 0x5640b0533830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.231+0000 7f20d172d640 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5640b05313d0 0x5640b0533830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc67887470c4e437 server_cookie=4480d981bc8fbfb in_seq=0 out_seq=0 2024-09-19T04:22:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.231+0000 7f20b57fa640 10 client.25804 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:22:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.231+0000 7f20b57fa640 1 -- 172.21.15.94:0/3331172684 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640b0410d70 con 0x5640b05313d0 2024-09-19T04:22:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.231+0000 7f20b57fa640 10 client.25804 handle_command_reply: tid=0 2024-09-19T04:22:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20d656b740 1 client.25804 shutdown 2024-09-19T04:22:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20d656b740 1 -- 172.21.15.94:0/3331172684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 msgr2=0x5640b04b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20d656b740 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04b1880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5640b04c0d40 tx=0x7f20c80046c0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20d656b740 1 -- 172.21.15.94:0/3331172684 shutdown_connections 2024-09-19T04:22:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20d656b740 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5640b05313d0 0x5640b0533830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20d656b740 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640b04ae370 0x5640b0526d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20d656b740 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640b04a4b40 0x5640b04b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20d656b740 1 --2- 172.21.15.94:0/3331172684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640b04a35d0 0x5640b04b1880 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20d656b740 1 -- 172.21.15.94:0/3331172684 >> 172.21.15.94:0/3331172684 conn(0x5640b046c310 msgr2=0x5640b04ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.232+0000 7f20b57fa640 0 client.25804 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:22:06.237 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:22:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.233+0000 7f20d656b740 1 -- 172.21.15.94:0/3331172684 shutdown_connections 2024-09-19T04:22:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.233+0000 7f20d656b740 1 -- 172.21.15.94:0/3331172684 wait complete. 2024-09-19T04:22:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.233+0000 7f20d656b740 20 client.25804 trim_cache size 0 max 16384 2024-09-19T04:22:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 -- 172.21.15.94:0/1340006851 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20b80a3540 msgr2=0x7f20b80a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:06.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20b80a3540 0x7f20b80a5a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005d00 tx=0x7f20c8005f30 comp rx=0 tx=0).stop 2024-09-19T04:22:06.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 -- 172.21.15.94:0/1340006851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20cc10e500 msgr2=0x7f20cc19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:06.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20cc10e500 0x7f20cc19ef90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f20c4034040 tx=0x7f20c4003f10 comp rx=0 tx=0).stop 2024-09-19T04:22:06.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 -- 172.21.15.94:0/1340006851 shutdown_connections 2024-09-19T04:22:06.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20b80a3540 0x7f20b80a5a00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20cc10e500 0x7f20cc19ef90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20cc101440 0x7f20cc19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 --2- 172.21.15.94:0/1340006851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20cc100b20 0x7f20cc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 -- 172.21.15.94:0/1340006851 >> 172.21.15.94:0/1340006851 conn(0x7f20cc0fc7d0 msgr2=0x7f20cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 -- 172.21.15.94:0/1340006851 shutdown_connections 2024-09-19T04:22:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:06.234+0000 7f20d4192640 1 -- 172.21.15.94:0/1340006851 wait complete. 2024-09-19T04:22:06.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:06 smithi094 ceph-mon[30972]: pgmap v215: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-09-19T04:22:06.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:06 smithi104 ceph-mon[34145]: pgmap v215: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-09-19T04:22:07.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:06 smithi112 ceph-mon[35051]: pgmap v215: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-09-19T04:22:07.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:22:07.835 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:08.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.399+0000 7fee757cc640 1 -- 172.21.15.94:0/934376147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70102ef0 msgr2=0x7fee7010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:08.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.399+0000 7fee757cc640 1 --2- 172.21.15.94:0/934376147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70102ef0 0x7fee7010da40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fee68009c10 tx=0x7fee6802a770 comp rx=0 tx=0).stop 2024-09-19T04:22:08.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.399+0000 7fee757cc640 1 -- 172.21.15.94:0/934376147 shutdown_connections 2024-09-19T04:22:08.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.399+0000 7fee757cc640 1 --2- 172.21.15.94:0/934376147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70102ef0 0x7fee7010da40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.399+0000 7fee757cc640 1 --2- 172.21.15.94:0/934376147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee70102570 0x7fee701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.399+0000 7fee757cc640 1 --2- 172.21.15.94:0/934376147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee70100e20 0x7fee70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.399+0000 7fee757cc640 1 -- 172.21.15.94:0/934376147 >> 172.21.15.94:0/934376147 conn(0x7fee700fc7d0 msgr2=0x7fee700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.399+0000 7fee757cc640 1 -- 172.21.15.94:0/934376147 shutdown_connections 2024-09-19T04:22:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.399+0000 7fee757cc640 1 -- 172.21.15.94:0/934376147 wait complete. 2024-09-19T04:22:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.400+0000 7fee757cc640 1 Processor -- start 2024-09-19T04:22:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee757cc640 1 -- start start 2024-09-19T04:22:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee757cc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70100e20 0x7fee701a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee757cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee70102570 0x7fee701a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee757cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee70102ef0 0x7fee7010ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee757cc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee70111f40 con 0x7fee70102ef0 2024-09-19T04:22:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee757cc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee70111dc0 con 0x7fee70100e20 2024-09-19T04:22:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee757cc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee701120c0 con 0x7fee70102570 2024-09-19T04:22:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee70102570 0x7fee701a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee6f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee70102ef0 0x7fee7010ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee70102570 0x7fee701a3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58806/0 (socket says 172.21.15.94:58806) 2024-09-19T04:22:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee6e7fc640 1 -- 172.21.15.94:0/202031158 learned_addr learned my addr 172.21.15.94:0/202031158 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee6f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee70102ef0 0x7fee7010ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37422/0 (socket says 172.21.15.94:37422) 2024-09-19T04:22:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.401+0000 7fee6effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70100e20 0x7fee701a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee6effd640 1 -- 172.21.15.94:0/202031158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee70102570 msgr2=0x7fee701a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee6effd640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee70102570 0x7fee701a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee6effd640 1 -- 172.21.15.94:0/202031158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee70102ef0 msgr2=0x7fee7010ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee6effd640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee70102ef0 0x7fee7010ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee6effd640 1 -- 172.21.15.94:0/202031158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7010d1b0 con 0x7fee70100e20 2024-09-19T04:22:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee6effd640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70100e20 0x7fee701a3620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee6000e7c0 tx=0x7fee6000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee57fff640 1 -- 172.21.15.94:0/202031158 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6000ccc0 con 0x7fee70100e20 2024-09-19T04:22:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee57fff640 1 -- 172.21.15.94:0/202031158 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee6000ce60 con 0x7fee70100e20 2024-09-19T04:22:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee57fff640 1 -- 172.21.15.94:0/202031158 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60018680 con 0x7fee70100e20 2024-09-19T04:22:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.402+0000 7fee757cc640 1 -- 172.21.15.94:0/202031158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7010d4a0 con 0x7fee70100e20 2024-09-19T04:22:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.404+0000 7fee757cc640 1 -- 172.21.15.94:0/202031158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee701973b0 con 0x7fee70100e20 2024-09-19T04:22:08.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee77ba5740 1 Processor -- start 2024-09-19T04:22:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee77ba5740 1 -- start start 2024-09-19T04:22:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cab40 0x56315f7c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee77ba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56315f72f2f0 con 0x56315f7cab40 2024-09-19T04:22:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee77ba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56315f6d6cf0 con 0x56315f7c95d0 2024-09-19T04:22:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee77ba5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56315f72b2b0 con 0x56315f7d4370 2024-09-19T04:22:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee6f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cab40 0x56315f7c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58812/0 (socket says 172.21.15.94:58812) 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee6f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cab40 0x56315f7c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37438/0 (socket says 172.21.15.94:37438) 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee6e7fc640 1 -- 172.21.15.94:0/1187168277 learned_addr learned my addr 172.21.15.94:0/1187168277 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.408+0000 7fee57fff640 1 -- 172.21.15.94:0/202031158 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fee60010430 con 0x7fee70100e20 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.409+0000 7fee6effd640 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.409+0000 7fee6e7fc640 1 -- 172.21.15.94:0/1187168277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 msgr2=0x56315f7c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.409+0000 7fee6e7fc640 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.409+0000 7fee6e7fc640 1 -- 172.21.15.94:0/1187168277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cab40 msgr2=0x56315f7c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.409+0000 7fee6e7fc640 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cab40 0x56315f7c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.409+0000 7fee6e7fc640 1 -- 172.21.15.94:0/1187168277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315f6bb3a0 con 0x56315f7d4370 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.409+0000 7fee6f7fe640 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cab40 0x56315f7c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.409+0000 7fee6effd640 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.410+0000 7fee6e7fc640 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d6760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fee64004600 tx=0x7fee6400bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b40c9012797538c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.410+0000 7fee557fa640 1 -- 172.21.15.94:0/1187168277 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64003dd0 con 0x56315f7d4370 2024-09-19T04:22:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.410+0000 7fee557fa640 1 -- 172.21.15.94:0/1187168277 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee64015070 con 0x56315f7d4370 2024-09-19T04:22:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.410+0000 7fee557fa640 1 -- 172.21.15.94:0/1187168277 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64019730 con 0x56315f7d4370 2024-09-19T04:22:08.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.410+0000 7fee57fff640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fee580a3500 0x7fee580a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee57fff640 1 -- 172.21.15.94:0/202031158 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8629+0+0 (secure 0 0 0) 0x7fee600d4060 con 0x7fee70100e20 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee6f7fe640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fee580a3500 0x7fee580a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee77ba5740 1 -- 172.21.15.94:0/1187168277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 msgr2=0x56315f7d6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee77ba5740 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d6760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fee64004600 tx=0x7fee6400bc70 comp rx=0 tx=0).stop 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee77ba5740 1 -- 172.21.15.94:0/1187168277 shutdown_connections 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee77ba5740 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d6760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee77ba5740 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee77ba5740 1 --2- 172.21.15.94:0/1187168277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cab40 0x56315f7c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee77ba5740 1 -- 172.21.15.94:0/1187168277 >> 172.21.15.94:0/1187168277 conn(0x56315f792310 msgr2=0x56315f7c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.411+0000 7fee77ba5740 1 -- 172.21.15.94:0/1187168277 shutdown_connections 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.412+0000 7fee77ba5740 1 -- 172.21.15.94:0/1187168277 wait complete. 2024-09-19T04:22:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.412+0000 7fee77ba5740 1 -- start start 2024-09-19T04:22:08.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.412+0000 7fee77ba5740 1 Processor -- start 2024-09-19T04:22:08.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7c95d0 0x56315f7d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7cab40 0x56315f7d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6f7fe640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fee580a3500 0x7fee580a59c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fee6802ac90 tx=0x7fee68005fd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee77ba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56315f7d7070 con 0x56315f7c95d0 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee77ba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56315f6ea060 con 0x56315f7cab40 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee77ba5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56315f7d71f0 con 0x56315f7d4370 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee77ba5740 10 client.0 _ll_register_callbacks cb 0x56315f43d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7cab40 0x56315f7d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7cab40 0x56315f7d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:54630/0 (socket says 172.21.15.94:54630) 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6e7fc640 1 -- 172.21.15.94:0/3472091695 learned_addr learned my addr 172.21.15.94:0/3472091695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6f7fe640 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6effd640 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7c95d0 0x56315f7d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6e7fc640 1 -- 172.21.15.94:0/3472091695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 msgr2=0x56315f7d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6e7fc640 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6e7fc640 1 -- 172.21.15.94:0/3472091695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7c95d0 msgr2=0x56315f7d3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6e7fc640 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7c95d0 0x56315f7d3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6e7fc640 1 -- 172.21.15.94:0/3472091695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315f4a03c0 con 0x56315f7cab40 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.413+0000 7fee6f7fe640 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.414+0000 7fee6e7fc640 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7cab40 0x56315f7d0f50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee64006010 tx=0x7fee64003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=807111f6fce5b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.414+0000 7fee52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.414+0000 7fee52ff5640 1 -- 172.21.15.94:0/3472091695 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64039060 con 0x56315f7cab40 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.414+0000 7fee77ba5740 1 -- 172.21.15.94:0/3472091695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee64015070 con 0x56315f7cab40 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.414+0000 7fee52ff5640 1 -- 172.21.15.94:0/3472091695 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8629+0+0 (secure 0 0 0) 0x7fee64011850 con 0x56315f7cab40 2024-09-19T04:22:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.414+0000 7fee52ff5640 1 -- 172.21.15.94:0/3472091695 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fee640041f0 con 0x56315f7cab40 2024-09-19T04:22:08.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.414+0000 7fee77ba5740 10 client.25835 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:08.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.414+0000 7fee77ba5740 1 -- 172.21.15.94:0/3472091695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56315f7e29e0 con 0x56315f7cab40 2024-09-19T04:22:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.415+0000 7fee52ff5640 1 -- 172.21.15.94:0/3472091695 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee64015070 con 0x56315f7cab40 2024-09-19T04:22:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.415+0000 7fee52ff5640 1 -- 172.21.15.94:0/3472091695 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fee64034400 con 0x56315f7cab40 2024-09-19T04:22:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.415+0000 7fee77ba5740 10 client.25835 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.415+0000 7fee77ba5740 10 client.25835 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.416+0000 7fee77ba5740 20 client.25835 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.416+0000 7fee77ba5740 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56315f7e3d90 0x56315f7e6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.416+0000 7fee77ba5740 4 client.25835 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.416+0000 7fee77ba5740 1 -- 172.21.15.94:0/3472091695 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56315f73c860 con 0x56315f7e3d90 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.416+0000 7fee6effd640 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56315f7e3d90 0x56315f7e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.416+0000 7fee6effd640 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56315f7e3d90 0x56315f7e6180 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a2ee6b9b1a04cb0 server_cookie=26ed5f8eb88435eb in_seq=0 out_seq=0 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.416+0000 7fee52ff5640 10 client.25835 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.417+0000 7fee52ff5640 1 -- 172.21.15.94:0/3472091695 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56315f73c860 con 0x56315f7e3d90 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.417+0000 7fee52ff5640 10 client.25835 handle_command_reply: tid=0 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.417+0000 7fee77ba5740 1 client.25835 shutdown 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 -- 172.21.15.94:0/3472091695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7cab40 msgr2=0x56315f7d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7cab40 0x56315f7d0f50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee64006010 tx=0x7fee64003e00 comp rx=0 tx=0).stop 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 -- 172.21.15.94:0/3472091695 shutdown_connections 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56315f7e3d90 0x56315f7e6180 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7d4370 0x56315f7d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7cab40 0x56315f7d0f50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 --2- 172.21.15.94:0/3472091695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7c95d0 0x56315f7d3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 -- 172.21.15.94:0/3472091695 >> 172.21.15.94:0/3472091695 conn(0x56315f792310 msgr2=0x56315f7cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee52ff5640 0 client.25835 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 -- 172.21.15.94:0/3472091695 shutdown_connections 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 1 -- 172.21.15.94:0/3472091695 wait complete. 2024-09-19T04:22:08.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.418+0000 7fee77ba5740 20 client.25835 trim_cache size 0 max 16384 2024-09-19T04:22:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.441+0000 7fee77ba5740 1 Processor -- start 2024-09-19T04:22:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.441+0000 7fee77ba5740 1 -- start start 2024-09-19T04:22:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7d4370 0x56315f7cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7cc660 0x56315f7eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee77ba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee7006e930 con 0x56315f7d4370 2024-09-19T04:22:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee77ba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee7006c1a0 con 0x56315f7c95d0 2024-09-19T04:22:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee77ba5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee7019e0d0 con 0x56315f7cc660 2024-09-19T04:22:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7e0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:54632/0 (socket says 172.21.15.94:54632) 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6effd640 1 -- 172.21.15.94:0/1817482975 learned_addr learned my addr 172.21.15.94:0/1817482975 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6f7fe640 1 --2- 172.21.15.94:0/1817482975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7cc660 0x56315f7eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6effd640 1 -- 172.21.15.94:0/1817482975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7cc660 msgr2=0x56315f7eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6effd640 1 --2- 172.21.15.94:0/1817482975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7cc660 0x56315f7eb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6effd640 1 -- 172.21.15.94:0/1817482975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7d4370 msgr2=0x56315f7cc120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6effd640 1 --2- 172.21.15.94:0/1817482975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7d4370 0x56315f7cc120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6effd640 1 -- 172.21.15.94:0/1817482975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315f7c6960 con 0x56315f7c95d0 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.442+0000 7fee6effd640 1 --2- 172.21.15.94:0/1817482975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7e0340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee60098470 tx=0x7fee600bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d41df81626a0626d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.443+0000 7fee54ff9640 1 -- 172.21.15.94:0/1817482975 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6009c010 con 0x56315f7c95d0 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.443+0000 7fee54ff9640 1 -- 172.21.15.94:0/1817482975 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee600d3020 con 0x56315f7c95d0 2024-09-19T04:22:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.443+0000 7fee54ff9640 1 -- 172.21.15.94:0/1817482975 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee600109e0 con 0x56315f7c95d0 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 -- 172.21.15.94:0/1817482975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 msgr2=0x56315f7e0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 --2- 172.21.15.94:0/1817482975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7e0340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee60098470 tx=0x7fee600bc040 comp rx=0 tx=0).stop 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 -- 172.21.15.94:0/1817482975 shutdown_connections 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 --2- 172.21.15.94:0/1817482975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7cc660 0x56315f7eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 --2- 172.21.15.94:0/1817482975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7d4370 0x56315f7cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 --2- 172.21.15.94:0/1817482975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7c95d0 0x56315f7e0340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 -- 172.21.15.94:0/1817482975 >> 172.21.15.94:0/1817482975 conn(0x56315f792310 msgr2=0x56315f7d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 -- 172.21.15.94:0/1817482975 shutdown_connections 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 -- 172.21.15.94:0/1817482975 wait complete. 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.444+0000 7fee77ba5740 1 -- start start 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee77ba5740 1 Processor -- start 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7c95d0 0x56315f7ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cc660 0x56315f7ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee77ba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7d4370 0x56315f7ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee6effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7c95d0 0x56315f7ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cc660 0x56315f7ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cc660 0x56315f7ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37468/0 (socket says 172.21.15.94:37468) 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee6effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7c95d0 0x56315f7ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58836/0 (socket says 172.21.15.94:58836) 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee6e7fc640 1 -- 172.21.15.94:0/2225005467 learned_addr learned my addr 172.21.15.94:0/2225005467 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee7019e720 con 0x56315f7cc660 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.445+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee7006bb00 con 0x56315f7d4370 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.446+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee7019edf0 con 0x56315f7c95d0 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.446+0000 7fee77ba5740 10 client.0 _ll_register_callbacks cb 0x7fee64004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.446+0000 7fee6effd640 1 -- 172.21.15.94:0/2225005467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7d4370 msgr2=0x56315f7ceb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.446+0000 7fee6effd640 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7d4370 0x56315f7ceb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.446+0000 7fee6effd640 1 -- 172.21.15.94:0/2225005467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cc660 msgr2=0x56315f7ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.446+0000 7fee6effd640 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cc660 0x56315f7ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.446+0000 7fee6effd640 1 -- 172.21.15.94:0/2225005467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315f84da90 con 0x56315f7c95d0 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.446+0000 7fee6e7fc640 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cc660 0x56315f7ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.446+0000 7fee6effd640 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7c95d0 0x56315f7ce110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee60000c00 tx=0x7fee60010930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=959bcea579cad158 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee52ff5640 1 -- 172.21.15.94:0/2225005467 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee600a8020 con 0x56315f7c95d0 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee52ff5640 1 -- 172.21.15.94:0/2225005467 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8629+0+0 (secure 0 0 0) 0x7fee600cd700 con 0x56315f7c95d0 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee600d3020 con 0x56315f7c95d0 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee52ff5640 1 -- 172.21.15.94:0/2225005467 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fee600b4030 con 0x56315f7c95d0 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee77ba5740 10 client.25822 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56315f84ed30 con 0x56315f7c95d0 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee52ff5640 1 -- 172.21.15.94:0/2225005467 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee600d3020 con 0x56315f7c95d0 2024-09-19T04:22:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee52ff5640 1 -- 172.21.15.94:0/2225005467 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fee60098680 con 0x56315f7c95d0 2024-09-19T04:22:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.447+0000 7fee77ba5740 10 client.25822 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.448+0000 7fee77ba5740 10 client.25822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:22:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.448+0000 7fee77ba5740 20 client.25822 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.448+0000 7fee77ba5740 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56315f84f350 0x56315f851740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.448+0000 7fee77ba5740 4 client.25822 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.448+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56315f736d70 con 0x56315f84f350 2024-09-19T04:22:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.448+0000 7fee6e7fc640 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56315f84f350 0x56315f851740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee6e7fc640 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56315f84f350 0x56315f851740 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db03a8622e3cc6d server_cookie=3868ce3d6fd5d370 in_seq=0 out_seq=0 2024-09-19T04:22:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee52ff5640 10 client.25822 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:22:08.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee52ff5640 1 -- 172.21.15.94:0/2225005467 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56315f736d70 con 0x56315f84f350 2024-09-19T04:22:08.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee52ff5640 10 client.25822 handle_command_reply: tid=0 2024-09-19T04:22:08.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee77ba5740 1 client.25822 shutdown 2024-09-19T04:22:08.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7c95d0 msgr2=0x56315f7ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:08.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee77ba5740 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7c95d0 0x56315f7ce110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee60000c00 tx=0x7fee60010930 comp rx=0 tx=0).stop 2024-09-19T04:22:08.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 shutdown_connections 2024-09-19T04:22:08.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee77ba5740 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x56315f84f350 0x56315f851740 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee77ba5740 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315f7d4370 0x56315f7ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee77ba5740 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56315f7cc660 0x56315f7ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee77ba5740 1 --2- 172.21.15.94:0/2225005467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f7c95d0 0x56315f7ce110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.449+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 >> 172.21.15.94:0/2225005467 conn(0x56315f792310 msgr2=0x56315f7d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.450+0000 7fee52ff5640 0 client.25822 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:22:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.450+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 shutdown_connections 2024-09-19T04:22:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.450+0000 7fee77ba5740 1 -- 172.21.15.94:0/2225005467 wait complete. 2024-09-19T04:22:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.450+0000 7fee77ba5740 20 client.25822 trim_cache size 0 max 16384 2024-09-19T04:22:08.458 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:22:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 -- 172.21.15.94:0/202031158 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fee580a3500 msgr2=0x7fee580a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fee580a3500 0x7fee580a59c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fee6802ac90 tx=0x7fee68005fd0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 -- 172.21.15.94:0/202031158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70100e20 msgr2=0x7fee701a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70100e20 0x7fee701a3620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee6000e7c0 tx=0x7fee6000ec80 comp rx=0 tx=0).stop 2024-09-19T04:22:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 -- 172.21.15.94:0/202031158 shutdown_connections 2024-09-19T04:22:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fee580a3500 0x7fee580a59c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee70102ef0 0x7fee7010ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee70102570 0x7fee701a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 --2- 172.21.15.94:0/202031158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70100e20 0x7fee701a3620 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee6000e7c0 tx=0x7fee6000ec80 comp rx=0 tx=0).stop 2024-09-19T04:22:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.451+0000 7fee757cc640 1 -- 172.21.15.94:0/202031158 >> 172.21.15.94:0/202031158 conn(0x7fee700fc7d0 msgr2=0x7fee70103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.452+0000 7fee757cc640 1 -- 172.21.15.94:0/202031158 shutdown_connections 2024-09-19T04:22:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:08.452+0000 7fee757cc640 1 -- 172.21.15.94:0/202031158 wait complete. 2024-09-19T04:22:08.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:08 smithi094 ceph-mon[30972]: pgmap v216: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 18 MiB/s wr, 1.96k op/s 2024-09-19T04:22:08.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:08 smithi094 ceph-mon[30972]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:22:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:08 smithi104 ceph-mon[34145]: pgmap v216: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 18 MiB/s wr, 1.96k op/s 2024-09-19T04:22:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:08 smithi104 ceph-mon[34145]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:22:09.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:08 smithi112 ceph-mon[35051]: pgmap v216: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 18 MiB/s wr, 1.96k op/s 2024-09-19T04:22:09.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:08 smithi112 ceph-mon[35051]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:22:09.315 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:22:09.717 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:10.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.269+0000 7f9bdcbad640 1 -- 172.21.15.94:0/2840555777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd8101080 msgr2=0x7f9bd8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.269+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/2840555777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd8101080 0x7f9bd8104e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009860 tx=0x7f9bc402a340 comp rx=0 tx=0).stop 2024-09-19T04:22:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.269+0000 7f9bdcbad640 1 -- 172.21.15.94:0/2840555777 shutdown_connections 2024-09-19T04:22:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.269+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/2840555777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd8105580 0x7f9bd810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.269+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/2840555777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd8101080 0x7f9bd8104e50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.269+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/2840555777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bd81006d0 0x7f9bd8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.269+0000 7f9bdcbad640 1 -- 172.21.15.94:0/2840555777 >> 172.21.15.94:0/2840555777 conn(0x7f9bd80fba90 msgr2=0x7f9bd80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:10.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.269+0000 7f9bdcbad640 1 -- 172.21.15.94:0/2840555777 shutdown_connections 2024-09-19T04:22:10.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.269+0000 7f9bdcbad640 1 -- 172.21.15.94:0/2840555777 wait complete. 2024-09-19T04:22:10.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bdcbad640 1 Processor -- start 2024-09-19T04:22:10.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bdcbad640 1 -- start start 2024-09-19T04:22:10.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bdcbad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bd81006d0 0x7f9bd810d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bdcbad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd8101080 0x7f9bd810a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bdcbad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd8105580 0x7f9bd810a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bdcbad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8113fd0 con 0x7f9bd81006d0 2024-09-19T04:22:10.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bdcbad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8113e50 con 0x7f9bd8101080 2024-09-19T04:22:10.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bdcbad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8114150 con 0x7f9bd8105580 2024-09-19T04:22:10.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bd81006d0 0x7f9bd810d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bd81006d0 0x7f9bd810d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37496/0 (socket says 172.21.15.94:37496) 2024-09-19T04:22:10.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bd659c640 1 -- 172.21.15.94:0/1774319685 learned_addr learned my addr 172.21.15.94:0/1774319685 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:10.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.271+0000 7f9bd6d9d640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd8105580 0x7f9bd810a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.272+0000 7f9bd659c640 1 -- 172.21.15.94:0/1774319685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd8105580 msgr2=0x7f9bd810a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:10.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.272+0000 7f9bd659c640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd8105580 0x7f9bd810a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.272+0000 7f9bd659c640 1 -- 172.21.15.94:0/1774319685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd8101080 msgr2=0x7f9bd810a150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:22:10.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.272+0000 7f9bd659c640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd8101080 0x7f9bd810a150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.272+0000 7f9bd659c640 1 -- 172.21.15.94:0/1774319685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd81a04d0 con 0x7f9bd81006d0 2024-09-19T04:22:10.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.272+0000 7f9bd659c640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bd81006d0 0x7f9bd810d0a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800fb60 tx=0x7f9bc800d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:10.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.273+0000 7f9bcf7fe640 1 -- 172.21.15.94:0/1774319685 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8005220 con 0x7f9bd81006d0 2024-09-19T04:22:10.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.273+0000 7f9bcf7fe640 1 -- 172.21.15.94:0/1774319685 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc8004c50 con 0x7f9bd81006d0 2024-09-19T04:22:10.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.273+0000 7f9bcf7fe640 1 -- 172.21.15.94:0/1774319685 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800bb30 con 0x7f9bd81006d0 2024-09-19T04:22:10.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.273+0000 7f9bdcbad640 1 -- 172.21.15.94:0/1774319685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd81a07c0 con 0x7f9bd81006d0 2024-09-19T04:22:10.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.273+0000 7f9bdcbad640 1 -- 172.21.15.94:0/1774319685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd81a0c20 con 0x7f9bd81006d0 2024-09-19T04:22:10.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.276+0000 7f9bcf7fe640 1 -- 172.21.15.94:0/1774319685 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9bc800bcd0 con 0x7f9bd81006d0 2024-09-19T04:22:10.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.278+0000 7f9bcf7fe640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9bbc0a3580 0x7f9bbc0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.278+0000 7f9bcf7fe640 1 -- 172.21.15.94:0/1774319685 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8629+0+0 (secure 0 0 0) 0x7f9bc80d03a0 con 0x7f9bd81006d0 2024-09-19T04:22:10.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.278+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9bbc0a3580 0x7f9bbc0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.279+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9bbc0a3580 0x7f9bbc0a5a40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9bc402a860 tx=0x7f9bc4000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:10.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.280+0000 7f9bdef86740 1 Processor -- start 2024-09-19T04:22:10.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.280+0000 7f9bdef86740 1 -- start start 2024-09-19T04:22:10.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.280+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d982a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.280+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982a5d0 0x5654d982aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.280+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 0x5654d9837760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.280+0000 7f9bdef86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5654d97902f0 con 0x5654d9835370 2024-09-19T04:22:10.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.280+0000 7f9bdef86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5654d9737cf0 con 0x5654d982bb40 2024-09-19T04:22:10.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.280+0000 7f9bdef86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654d978c2b0 con 0x5654d982a5d0 2024-09-19T04:22:10.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d982a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d982a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:54666/0 (socket says 172.21.15.94:54666) 2024-09-19T04:22:10.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd659c640 1 -- 172.21.15.94:0/1771539770 learned_addr learned my addr 172.21.15.94:0/1771539770 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/1771539770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982a5d0 0x5654d982aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd6d9d640 1 --2- 172.21.15.94:0/1771539770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 0x5654d9837760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd659c640 1 -- 172.21.15.94:0/1771539770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982a5d0 msgr2=0x5654d982aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd659c640 1 --2- 172.21.15.94:0/1771539770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982a5d0 0x5654d982aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd659c640 1 -- 172.21.15.94:0/1771539770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 msgr2=0x5654d9837760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd659c640 1 --2- 172.21.15.94:0/1771539770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 0x5654d9837760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd659c640 1 -- 172.21.15.94:0/1771539770 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d971c3a0 con 0x5654d982bb40 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bd659c640 1 --2- 172.21.15.94:0/1771539770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d982a090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8098f80 tx=0x7f9bc809bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=501443988473ec52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.281+0000 7f9bccff9640 1 -- 172.21.15.94:0/1771539770 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80b4070 con 0x5654d982bb40 2024-09-19T04:22:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.282+0000 7f9bccff9640 1 -- 172.21.15.94:0/1771539770 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc809c5b0 con 0x5654d982bb40 2024-09-19T04:22:10.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.282+0000 7f9bccff9640 1 -- 172.21.15.94:0/1771539770 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80cf070 con 0x5654d982bb40 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.282+0000 7f9bdef86740 1 -- 172.21.15.94:0/1771539770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 msgr2=0x5654d982a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.282+0000 7f9bdef86740 1 --2- 172.21.15.94:0/1771539770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d982a090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8098f80 tx=0x7f9bc809bf10 comp rx=0 tx=0).stop 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.283+0000 7f9bdef86740 1 -- 172.21.15.94:0/1771539770 shutdown_connections 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.283+0000 7f9bdef86740 1 --2- 172.21.15.94:0/1771539770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 0x5654d9837760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.283+0000 7f9bdef86740 1 --2- 172.21.15.94:0/1771539770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982a5d0 0x5654d982aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.283+0000 7f9bdef86740 1 --2- 172.21.15.94:0/1771539770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d982a090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.283+0000 7f9bdef86740 1 -- 172.21.15.94:0/1771539770 >> 172.21.15.94:0/1771539770 conn(0x5654d97f3310 msgr2=0x5654d9826c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.283+0000 7f9bdef86740 1 -- 172.21.15.94:0/1771539770 shutdown_connections 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.283+0000 7f9bdef86740 1 -- 172.21.15.94:0/1771539770 wait complete. 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.283+0000 7f9bdef86740 1 -- start start 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.284+0000 7f9bdef86740 1 Processor -- start 2024-09-19T04:22:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.284+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 0x5654d9834dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.284+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 0x5654d9832080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.284+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d9835370 0x5654d9832810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.284+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 0x5654d9834dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.284+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 0x5654d9834dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37524/0 (socket says 172.21.15.94:37524) 2024-09-19T04:22:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bdef86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5654d9838070 con 0x5654d982a5d0 2024-09-19T04:22:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bdef86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5654d974b060 con 0x5654d9835370 2024-09-19T04:22:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bdef86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654d98381f0 con 0x5654d982bb40 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bdef86740 10 client.0 _ll_register_callbacks cb 0x5654d949e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bd659c640 1 -- 172.21.15.94:0/3060913872 learned_addr learned my addr 172.21.15.94:0/3060913872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 0x5654d9832080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bd6d9d640 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d9835370 0x5654d9832810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bd5d9b640 1 -- 172.21.15.94:0/3060913872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d9835370 msgr2=0x5654d9832810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d9835370 0x5654d9832810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bd5d9b640 1 -- 172.21.15.94:0/3060913872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 msgr2=0x5654d9834dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 0x5654d9834dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.285+0000 7f9bd5d9b640 1 -- 172.21.15.94:0/3060913872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d95013c0 con 0x5654d982bb40 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.286+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 0x5654d9832080 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9bc402a930 tx=0x7f9bc4039000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6001e4feba1129 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.286+0000 7f9bb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.286+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/3060913872 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc404f060 con 0x5654d982bb40 2024-09-19T04:22:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.286+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/3060913872 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8629+0+0 (secure 0 0 0) 0x7f9bc4030650 con 0x5654d982bb40 2024-09-19T04:22:10.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.286+0000 7f9bdef86740 1 -- 172.21.15.94:0/3060913872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc809c5b0 con 0x5654d982bb40 2024-09-19T04:22:10.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/3060913872 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9bc4037040 con 0x5654d982bb40 2024-09-19T04:22:10.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bdef86740 10 client.25834 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:10.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bdef86740 1 -- 172.21.15.94:0/3060913872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5654d9843b90 con 0x5654d982bb40 2024-09-19T04:22:10.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/3060913872 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc809c5b0 con 0x5654d982bb40 2024-09-19T04:22:10.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/3060913872 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9bc403a030 con 0x5654d982bb40 2024-09-19T04:22:10.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bdef86740 10 client.25834 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:10.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bdef86740 10 client.25834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bdef86740 20 client.25834 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bdef86740 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654d9845050 0x5654d9847440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bdef86740 4 client.25834 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bdef86740 1 -- 172.21.15.94:0/3060913872 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d979d860 con 0x5654d9845050 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.287+0000 7f9bd659c640 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654d9845050 0x5654d9847440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.288+0000 7f9bb9ffb640 10 client.25834 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.288+0000 7f9bd659c640 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654d9845050 0x5654d9847440 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8e1a4f8c8aec1f server_cookie=1f327873d83a8f2f in_seq=0 out_seq=0 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/3060913872 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654d979d860 con 0x5654d9845050 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bb9ffb640 10 client.25834 handle_command_reply: tid=0 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bdef86740 1 client.25834 shutdown 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bdef86740 1 -- 172.21.15.94:0/3060913872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 msgr2=0x5654d9832080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bdef86740 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 0x5654d9832080 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9bc402a930 tx=0x7f9bc4039000 comp rx=0 tx=0).stop 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bdef86740 1 -- 172.21.15.94:0/3060913872 shutdown_connections 2024-09-19T04:22:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bdef86740 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654d9845050 0x5654d9847440 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bb9ffb640 0 client.25834 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:10.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bdef86740 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d9835370 0x5654d9832810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bdef86740 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 0x5654d9832080 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bdef86740 1 --2- 172.21.15.94:0/3060913872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 0x5654d9834dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.289+0000 7f9bdef86740 1 -- 172.21.15.94:0/3060913872 >> 172.21.15.94:0/3060913872 conn(0x5654d97f3310 msgr2=0x5654d982c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:10.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.290+0000 7f9bdef86740 1 -- 172.21.15.94:0/3060913872 shutdown_connections 2024-09-19T04:22:10.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.290+0000 7f9bdef86740 1 -- 172.21.15.94:0/3060913872 wait complete. 2024-09-19T04:22:10.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.290+0000 7f9bdef86740 20 client.25834 trim_cache size 0 max 16384 2024-09-19T04:22:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.313+0000 7f9bdef86740 1 Processor -- start 2024-09-19T04:22:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.313+0000 7f9bdef86740 1 -- start start 2024-09-19T04:22:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.313+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982a5d0 0x5654d984cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 0x5654d984d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 0x5654d982f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bdef86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81bf140 con 0x5654d9835370 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bdef86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81bc9b0 con 0x5654d982a5d0 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bdef86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81ba820 con 0x5654d982bb40 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982a5d0 0x5654d984cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982a5d0 0x5654d984cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:54688/0 (socket says 172.21.15.94:54688) 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd659c640 1 -- 172.21.15.94:0/2889273310 learned_addr learned my addr 172.21.15.94:0/2889273310 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd6d9d640 1 --2- 172.21.15.94:0/2889273310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 0x5654d982f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd659c640 1 -- 172.21.15.94:0/2889273310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 msgr2=0x5654d984d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd659c640 1 --2- 172.21.15.94:0/2889273310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 0x5654d984d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd659c640 1 -- 172.21.15.94:0/2889273310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 msgr2=0x5654d982f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:10.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd659c640 1 --2- 172.21.15.94:0/2889273310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 0x5654d982f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd659c640 1 -- 172.21.15.94:0/2889273310 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d9827960 con 0x5654d982a5d0 2024-09-19T04:22:10.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd6d9d640 1 --2- 172.21.15.94:0/2889273310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 0x5654d982f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:10.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.314+0000 7f9bd659c640 1 --2- 172.21.15.94:0/2889273310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982a5d0 0x5654d984cd10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8098f30 tx=0x7f9bc8005560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5400d485b4ff5d80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:10.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.315+0000 7f9bbbfff640 1 -- 172.21.15.94:0/2889273310 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80b9070 con 0x5654d982a5d0 2024-09-19T04:22:10.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.315+0000 7f9bbbfff640 1 -- 172.21.15.94:0/2889273310 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc80a5220 con 0x5654d982a5d0 2024-09-19T04:22:10.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.315+0000 7f9bbbfff640 1 -- 172.21.15.94:0/2889273310 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80cf040 con 0x5654d982a5d0 2024-09-19T04:22:10.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.316+0000 7f9bdef86740 1 -- 172.21.15.94:0/2889273310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982a5d0 msgr2=0x5654d984cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.316+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2889273310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982a5d0 0x5654d984cd10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8098f30 tx=0x7f9bc8005560 comp rx=0 tx=0).stop 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.316+0000 7f9bdef86740 1 -- 172.21.15.94:0/2889273310 shutdown_connections 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.316+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2889273310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d9835370 0x5654d982f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.316+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2889273310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d982bb40 0x5654d984d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.316+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2889273310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982a5d0 0x5654d984cd10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.316+0000 7f9bdef86740 1 -- 172.21.15.94:0/2889273310 >> 172.21.15.94:0/2889273310 conn(0x5654d97f3310 msgr2=0x5654d9830950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.316+0000 7f9bdef86740 1 -- 172.21.15.94:0/2889273310 shutdown_connections 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.316+0000 7f9bdef86740 1 -- 172.21.15.94:0/2889273310 wait complete. 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.317+0000 7f9bdef86740 1 -- start start 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.317+0000 7f9bdef86740 1 Processor -- start 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.317+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 0x5654d983d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.317+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d983dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.317+0000 7f9bdef86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d9835370 0x5654d983a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.317+0000 7f9bd5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d983dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bd5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d983dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:54698/0 (socket says 172.21.15.94:54698) 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bd5d9b640 1 -- 172.21.15.94:0/2148346022 learned_addr learned my addr 172.21.15.94:0/2148346022 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bd6d9d640 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d9835370 0x5654d983a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81bae70 con 0x5654d982a5d0 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81bc2e0 con 0x5654d982bb40 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81bb540 con 0x5654d9835370 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bdef86740 10 client.0 _ll_register_callbacks cb 0x7f9bc8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bd5d9b640 1 -- 172.21.15.94:0/2148346022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d9835370 msgr2=0x5654d983a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d9835370 0x5654d983a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bd5d9b640 1 -- 172.21.15.94:0/2148346022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 msgr2=0x5654d983d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:10.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 0x5654d983d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bd5d9b640 1 -- 172.21.15.94:0/2148346022 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d982d120 con 0x5654d982bb40 2024-09-19T04:22:10.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.318+0000 7f9bd5d9b640 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d983dba0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4030d10 tx=0x7f9bc4030bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fb57a02e7fd3aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:10.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.319+0000 7f9bb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:10.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.319+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/2148346022 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4032e60 con 0x5654d982bb40 2024-09-19T04:22:10.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.319+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/2148346022 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8629+0+0 (secure 0 0 0) 0x7f9bc4030450 con 0x5654d982bb40 2024-09-19T04:22:10.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.319+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc80a5220 con 0x5654d982bb40 2024-09-19T04:22:10.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.319+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/2148346022 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9bc404f070 con 0x5654d982bb40 2024-09-19T04:22:10.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.319+0000 7f9bdef86740 10 client.25853 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.320+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5654d98b0440 con 0x5654d982bb40 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.320+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/2148346022 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc4032440 con 0x5654d982bb40 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.320+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/2148346022 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9bc403a040 con 0x5654d982bb40 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.320+0000 7f9bdef86740 10 client.25853 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.320+0000 7f9bdef86740 10 client.25853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.320+0000 7f9bdef86740 20 client.25853 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.320+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654d98b0c50 0x5654d98b3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.320+0000 7f9bdef86740 4 client.25853 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.320+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9bc80a5220 con 0x5654d98b0c50 2024-09-19T04:22:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.321+0000 7f9bd659c640 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654d98b0c50 0x5654d98b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.322+0000 7f9bd659c640 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654d98b0c50 0x5654d98b3070 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2236b699e435db server_cookie=3d907f5f3cf3728e in_seq=0 out_seq=0 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.322+0000 7f9bb9ffb640 10 client.25853 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.322+0000 7f9bb9ffb640 1 -- 172.21.15.94:0/2148346022 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9bc80a5220 con 0x5654d98b0c50 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.322+0000 7f9bb9ffb640 10 client.25853 handle_command_reply: tid=0 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.322+0000 7f9bdef86740 1 client.25853 shutdown 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.322+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 msgr2=0x5654d983dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.322+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d983dba0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4030d10 tx=0x7f9bc4030bd0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 shutdown_connections 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654d98b0c50 0x5654d98b3070 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d9835370 0x5654d983a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bb9ffb640 0 client.25853 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:10.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654d982bb40 0x5654d983dba0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bdef86740 1 --2- 172.21.15.94:0/2148346022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654d982a5d0 0x5654d983d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 >> 172.21.15.94:0/2148346022 conn(0x5654d97f3310 msgr2=0x5654d98338e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:10.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 shutdown_connections 2024-09-19T04:22:10.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bdef86740 1 -- 172.21.15.94:0/2148346022 wait complete. 2024-09-19T04:22:10.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.323+0000 7f9bdef86740 20 client.25853 trim_cache size 0 max 16384 2024-09-19T04:22:10.326 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:22:10.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.324+0000 7f9bdcbad640 1 -- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9bbc0a3580 msgr2=0x7f9bbc0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:10.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.324+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9bbc0a3580 0x7f9bbc0a5a40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9bc402a860 tx=0x7f9bc4000bc0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.324+0000 7f9bdcbad640 1 -- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bd81006d0 msgr2=0x7f9bd810d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:10.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.324+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bd81006d0 0x7f9bd810d0a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800fb60 tx=0x7f9bc800d7b0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.324+0000 7f9bdcbad640 1 -- 172.21.15.94:0/1774319685 shutdown_connections 2024-09-19T04:22:10.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.324+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9bbc0a3580 0x7f9bbc0a5a40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.325+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd8105580 0x7f9bd810a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.325+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd8101080 0x7f9bd810a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.325+0000 7f9bdcbad640 1 --2- 172.21.15.94:0/1774319685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bd81006d0 0x7f9bd810d0a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:10.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.325+0000 7f9bdcbad640 1 -- 172.21.15.94:0/1774319685 >> 172.21.15.94:0/1774319685 conn(0x7f9bd80fba90 msgr2=0x7f9bd80fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:10.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.325+0000 7f9bdcbad640 1 -- 172.21.15.94:0/1774319685 shutdown_connections 2024-09-19T04:22:10.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:10.325+0000 7f9bdcbad640 1 -- 172.21.15.94:0/1774319685 wait complete. 2024-09-19T04:22:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 355.0 s, 1416.0 tps, lat 70.657 ms stddev 22.424 2024-09-19T04:22:10.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:10 smithi094 ceph-mon[30972]: pgmap v218: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-09-19T04:22:10.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:10 smithi104 ceph-mon[34145]: pgmap v218: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-09-19T04:22:11.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:10 smithi112 ceph-mon[35051]: pgmap v218: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-09-19T04:22:12.146 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:22:12.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:22:12.551 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:12 smithi094 ceph-mon[30972]: pgmap v219: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-09-19T04:22:12.569 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:12.836 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:12 smithi112 ceph-mon[35051]: pgmap v219: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-09-19T04:22:12.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:12 smithi104 ceph-mon[34145]: pgmap v219: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-09-19T04:22:14.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:14 smithi094 ceph-mon[30972]: pgmap v220: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-09-19T04:22:14.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:14 smithi104 ceph-mon[34145]: pgmap v220: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-09-19T04:22:15.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:14 smithi112 ceph-mon[35051]: pgmap v220: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-09-19T04:22:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 360.0 s, 1393.8 tps, lat 71.831 ms stddev 29.855 2024-09-19T04:22:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.291+0000 7f6e5c992640 1 -- 172.21.15.94:0/3022820643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58100e20 msgr2=0x7f6e58101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.291+0000 7f6e5c992640 1 --2- 172.21.15.94:0/3022820643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58100e20 0x7f6e58101200 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009900 tx=0x7f6e4802a480 comp rx=0 tx=0).stop 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.291+0000 7f6e5c992640 1 -- 172.21.15.94:0/3022820643 shutdown_connections 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.291+0000 7f6e5c992640 1 --2- 172.21.15.94:0/3022820643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e58102ef0 0x7f6e5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.291+0000 7f6e5c992640 1 --2- 172.21.15.94:0/3022820643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e58102570 0x7f6e581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.291+0000 7f6e5c992640 1 --2- 172.21.15.94:0/3022820643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58100e20 0x7f6e58101200 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.291+0000 7f6e5c992640 1 -- 172.21.15.94:0/3022820643 >> 172.21.15.94:0/3022820643 conn(0x7f6e580fc7d0 msgr2=0x7f6e580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.291+0000 7f6e5c992640 1 -- 172.21.15.94:0/3022820643 shutdown_connections 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.291+0000 7f6e5c992640 1 -- 172.21.15.94:0/3022820643 wait complete. 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.292+0000 7f6e5c992640 1 Processor -- start 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.292+0000 7f6e5c992640 1 -- start start 2024-09-19T04:22:16.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.292+0000 7f6e5c992640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e58100e20 0x7f6e581973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.292+0000 7f6e5c992640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58102570 0x7f6e58197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.292+0000 7f6e5c992640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e58102ef0 0x7f6e5819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.292+0000 7f6e5c992640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58110ec0 con 0x7f6e58102570 2024-09-19T04:22:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.292+0000 7f6e5c992640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58110d40 con 0x7f6e58102ef0 2024-09-19T04:22:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.292+0000 7f6e5c992640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58111040 con 0x7f6e58100e20 2024-09-19T04:22:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58102570 0x7f6e58197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58102570 0x7f6e58197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49318/0 (socket says 172.21.15.94:49318) 2024-09-19T04:22:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e55d9b640 1 -- 172.21.15.94:0/1245871159 learned_addr learned my addr 172.21.15.94:0/1245871159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e5659c640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e58100e20 0x7f6e581973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e56d9d640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e58102ef0 0x7f6e5819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e55d9b640 1 -- 172.21.15.94:0/1245871159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e58100e20 msgr2=0x7f6e581973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e58100e20 0x7f6e581973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e55d9b640 1 -- 172.21.15.94:0/1245871159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e58102ef0 msgr2=0x7f6e5819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e58102ef0 0x7f6e5819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e55d9b640 1 -- 172.21.15.94:0/1245871159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5819c390 con 0x7f6e58102570 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e56d9d640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e58102ef0 0x7f6e5819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58102570 0x7f6e58197920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c970 tx=0x7f6e4c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e477fe640 1 -- 172.21.15.94:0/1245871159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c007bb0 con 0x7f6e58102570 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e477fe640 1 -- 172.21.15.94:0/1245871159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e4c007d50 con 0x7f6e58102570 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e5c992640 1 -- 172.21.15.94:0/1245871159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5819c680 con 0x7f6e58102570 2024-09-19T04:22:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e477fe640 1 -- 172.21.15.94:0/1245871159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00f450 con 0x7f6e58102570 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.293+0000 7f6e5c992640 1 -- 172.21.15.94:0/1245871159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e58069f70 con 0x7f6e58102570 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.296+0000 7f6e477fe640 1 -- 172.21.15.94:0/1245871159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6e4c004f20 con 0x7f6e58102570 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.296+0000 7f6e477fe640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e400a3430 0x7f6e400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.296+0000 7f6e5ed6b740 1 Processor -- start 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.296+0000 7f6e5659c640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e400a3430 0x7f6e400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.296+0000 7f6e477fe640 1 -- 172.21.15.94:0/1245871159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f6e4c0cfa60 con 0x7f6e58102570 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5ed6b740 1 -- start start 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be73ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 0x5555be74a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be74ad40 0x5555be74d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555be6a5280 con 0x5555be73fb60 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5555be64cb10 con 0x5555be740480 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5555be6a1240 con 0x5555be74ad40 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be73ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e400a3430 0x7f6e400a58f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6e48000c00 tx=0x7f6e48005e80 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be73ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49326/0 (socket says 172.21.15.94:49326) 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 -- 172.21.15.94:0/2738562889 learned_addr learned my addr 172.21.15.94:0/2738562889 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e56d9d640 1 --2- 172.21.15.94:0/2738562889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be74ad40 0x5555be74d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/2738562889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 0x5555be74a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 -- 172.21.15.94:0/2738562889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be74ad40 msgr2=0x5555be74d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 --2- 172.21.15.94:0/2738562889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be74ad40 0x5555be74d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 -- 172.21.15.94:0/2738562889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 msgr2=0x5555be74a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 --2- 172.21.15.94:0/2738562889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 0x5555be74a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 -- 172.21.15.94:0/2738562889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555be6591f0 con 0x5555be73fb60 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.297+0000 7f6e5659c640 1 --2- 172.21.15.94:0/2738562889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be73ff40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009900 tx=0x7f6e4803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ccea05e9008600 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e44ff9640 1 -- 172.21.15.94:0/2738562889 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48035050 con 0x5555be73fb60 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e44ff9640 1 -- 172.21.15.94:0/2738562889 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e4803c8f0 con 0x5555be73fb60 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2738562889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 msgr2=0x5555be73ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2738562889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be73ff40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009900 tx=0x7f6e4803e000 comp rx=0 tx=0).stop 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e44ff9640 1 -- 172.21.15.94:0/2738562889 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 0 0 0) 0x7f6e48031a10 con 0x5555be73fb60 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2738562889 shutdown_connections 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2738562889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be74ad40 0x5555be74d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2738562889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 0x5555be74a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2738562889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be73ff40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2738562889 >> 172.21.15.94:0/2738562889 conn(0x5555be73c0f0 msgr2=0x5555be73c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:16.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2738562889 shutdown_connections 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.298+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2738562889 wait complete. 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5ed6b740 1 -- start start 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5ed6b740 1 Processor -- start 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be747bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be740480 0x5555be748100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 0x5555be748640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555be74d960 con 0x5555be73fb60 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5555be65ff10 con 0x5555be74ad40 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5555be74dae0 con 0x5555be740480 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5ed6b740 10 client.0 _ll_register_callbacks cb 0x5555be628a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be747bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be747bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49334/0 (socket says 172.21.15.94:49334) 2024-09-19T04:22:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5659c640 1 -- 172.21.15.94:0/2301640185 learned_addr learned my addr 172.21.15.94:0/2301640185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be740480 0x5555be748100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e56d9d640 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 0x5555be748640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5659c640 1 -- 172.21.15.94:0/2301640185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be740480 msgr2=0x5555be748100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5659c640 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be740480 0x5555be748100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5659c640 1 -- 172.21.15.94:0/2301640185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 msgr2=0x5555be748640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5659c640 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 0x5555be748640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.299+0000 7f6e5659c640 1 -- 172.21.15.94:0/2301640185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555be4166a0 con 0x5555be73fb60 2024-09-19T04:22:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e5659c640 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be747bc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6e48003ef0 tx=0x7f6e4802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7ba0e37514c05b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2301640185 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48035030 con 0x5555be73fb60 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2301640185 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f6e4803caa0 con 0x5555be73fb60 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2301640185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e4803c8f0 con 0x5555be73fb60 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2301640185 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6e4803cca0 con 0x5555be73fb60 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e5ed6b740 10 client.16269 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2301640185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5555be759560 con 0x5555be73fb60 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2301640185 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e4803c8f0 con 0x5555be73fb60 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.300+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2301640185 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6e4805b760 con 0x5555be73fb60 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.301+0000 7f6e5ed6b740 10 client.16269 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.301+0000 7f6e5ed6b740 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.301+0000 7f6e5ed6b740 20 client.16269 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.301+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5555be75a8b0 0x5555be75cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.301+0000 7f6e5ed6b740 4 client.16269 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.301+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2301640185 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555be6b27f0 con 0x5555be75a8b0 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.301+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5555be75a8b0 0x5555be75cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.301+0000 7f6e3dffb640 10 client.16269 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.301+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5555be75a8b0 0x5555be75cca0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7791a8ce19b51ae server_cookie=f2b3d7b92f63d190 in_seq=0 out_seq=0 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2301640185 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555be6b27f0 con 0x5555be75a8b0 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e3dffb640 10 client.16269 handle_command_reply: tid=0 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 client.16269 shutdown 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2301640185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 msgr2=0x5555be747bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be747bc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6e48003ef0 tx=0x7f6e4802a480 comp rx=0 tx=0).stop 2024-09-19T04:22:16.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2301640185 shutdown_connections 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5555be75a8b0 0x5555be75cca0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 0x5555be748640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e3dffb640 0 client.16269 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be740480 0x5555be748100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2301640185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be73fb60 0x5555be747bc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2301640185 >> 172.21.15.94:0/2301640185 conn(0x5555be73c0f0 msgr2=0x5555be73b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2301640185 shutdown_connections 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2301640185 wait complete. 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.302+0000 7f6e5ed6b740 20 client.16269 trim_cache size 0 max 16384 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.333+0000 7f6e5ed6b740 1 Processor -- start 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5ed6b740 1 -- start start 2024-09-19T04:22:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be73b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 0x5555be7622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be74ad40 0x5555be762810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581b2460 con 0x5555be74ad40 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581afcd0 con 0x5555be740480 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581adb40 con 0x5555be73fb60 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be73b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be73b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:46172/0 (socket says 172.21.15.94:46172) 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e5659c640 1 -- 172.21.15.94:0/16981574 learned_addr learned my addr 172.21.15.94:0/16981574 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e56d9d640 1 --2- 172.21.15.94:0/16981574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be74ad40 0x5555be762810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.334+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/16981574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 0x5555be7622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.335+0000 7f6e56d9d640 1 -- 172.21.15.94:0/16981574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 msgr2=0x5555be73b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.335+0000 7f6e56d9d640 1 --2- 172.21.15.94:0/16981574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be73b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.335+0000 7f6e56d9d640 1 -- 172.21.15.94:0/16981574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 msgr2=0x5555be7622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.335+0000 7f6e56d9d640 1 --2- 172.21.15.94:0/16981574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 0x5555be7622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.335+0000 7f6e56d9d640 1 -- 172.21.15.94:0/16981574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555be7402b0 con 0x5555be74ad40 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.335+0000 7f6e56d9d640 1 --2- 172.21.15.94:0/16981574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be74ad40 0x5555be762810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009ec0 tx=0x7f6e5000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cbc66f1f72d11a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.335+0000 7f6e3ffff640 1 -- 172.21.15.94:0/16981574 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5001c070 con 0x5555be74ad40 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.335+0000 7f6e3ffff640 1 -- 172.21.15.94:0/16981574 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e50016c30 con 0x5555be74ad40 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.335+0000 7f6e3ffff640 1 -- 172.21.15.94:0/16981574 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50017d00 con 0x5555be74ad40 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.336+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/16981574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be74ad40 msgr2=0x5555be762810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.336+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/16981574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be74ad40 0x5555be762810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009ec0 tx=0x7f6e5000a7d0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.337+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/16981574 shutdown_connections 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.337+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/16981574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be74ad40 0x5555be762810 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.337+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/16981574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be740480 0x5555be7622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.337+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/16981574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be73b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.337+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/16981574 >> 172.21.15.94:0/16981574 conn(0x5555be73c0f0 msgr2=0x5555be73aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.337+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/16981574 shutdown_connections 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.337+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/16981574 wait complete. 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.337+0000 7f6e5ed6b740 1 -- start start 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.337+0000 7f6e5ed6b740 1 Processor -- start 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.338+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be752d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.338+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be740480 0x5555be7532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.338+0000 7f6e5ed6b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 0x5555be74fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.338+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ae190 con 0x5555be740480 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581af600 con 0x5555be74ad40 2024-09-19T04:22:16.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e5ed6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ae860 con 0x5555be73fb60 2024-09-19T04:22:16.493 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:22:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:22:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "acfab277-a7a1-44f0-bed2-5e99c8f01145", 2024-09-19T04:22:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:22:16.493 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:22:16.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e5ed6b740 10 client.0 _ll_register_callbacks cb 0x7f6e48003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:16.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be740480 0x5555be7532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be752d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be740480 0x5555be7532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49354/0 (socket says 172.21.15.94:49354) 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be752d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:46180/0 (socket says 172.21.15.94:46180) 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e55d9b640 1 -- 172.21.15.94:0/2014433033 learned_addr learned my addr 172.21.15.94:0/2014433033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e56d9d640 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 0x5555be74fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e55d9b640 1 -- 172.21.15.94:0/2014433033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 msgr2=0x5555be752d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be752d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e55d9b640 1 -- 172.21.15.94:0/2014433033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 msgr2=0x5555be74fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 0x5555be74fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.339+0000 7f6e55d9b640 1 -- 172.21.15.94:0/2014433033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555be7539d0 con 0x5555be740480 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.340+0000 7f6e55d9b640 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be740480 0x5555be7532c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0058c0 tx=0x7f6e4c09be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec76437a5ebaf599 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.340+0000 7f6e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:22:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.340+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2014433033 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0b8030 con 0x5555be740480 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.340+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2014433033 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f6e4c09a000 con 0x5555be740480 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.341+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2014433033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e50016c30 con 0x5555be740480 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.341+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2014433033 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6e4c01d040 con 0x5555be740480 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.341+0000 7f6e5ed6b740 10 client.16275 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.341+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2014433033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5555be7c5900 con 0x5555be740480 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.341+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2014433033 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e50016c30 con 0x5555be740480 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.341+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2014433033 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6e4c0a44e0 con 0x5555be740480 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.342+0000 7f6e5ed6b740 10 client.16275 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.342+0000 7f6e5ed6b740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.342+0000 7f6e5ed6b740 20 client.16275 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.342+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5555be75a8b0 0x5555be7c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:16.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.342+0000 7f6e5ed6b740 4 client.16275 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.342+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2014433033 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555be6acd00 con 0x5555be75a8b0 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.342+0000 7f6e5659c640 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5555be75a8b0 0x5555be7c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.343+0000 7f6e5659c640 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5555be75a8b0 0x5555be7c7f90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86699afae4d75153 server_cookie=aff6852103652039 in_seq=0 out_seq=0 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.344+0000 7f6e3dffb640 10 client.16275 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.344+0000 7f6e3dffb640 1 -- 172.21.15.94:0/2014433033 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555be6acd00 con 0x5555be75a8b0 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.344+0000 7f6e3dffb640 10 client.16275 handle_command_reply: tid=0 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.344+0000 7f6e5ed6b740 1 client.16275 shutdown 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.345+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2014433033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be740480 msgr2=0x5555be7532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.345+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be740480 0x5555be7532c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0058c0 tx=0x7f6e4c09be60 comp rx=0 tx=0).stop 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.346+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2014433033 shutdown_connections 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.346+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5555be75a8b0 0x5555be7c7f90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.346+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555be74ad40 0x5555be74fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.346+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555be740480 0x5555be7532c0 secure :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0058c0 tx=0x7f6e4c09be60 comp rx=0 tx=0).stop 2024-09-19T04:22:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.346+0000 7f6e5ed6b740 1 --2- 172.21.15.94:0/2014433033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555be73fb60 0x5555be752d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.346+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2014433033 >> 172.21.15.94:0/2014433033 conn(0x5555be73c0f0 msgr2=0x5555be741dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.346+0000 7f6e3dffb640 0 client.16275 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.347+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2014433033 shutdown_connections 2024-09-19T04:22:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.347+0000 7f6e5ed6b740 1 -- 172.21.15.94:0/2014433033 wait complete. 2024-09-19T04:22:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.347+0000 7f6e5ed6b740 20 client.16275 trim_cache size 0 max 16384 2024-09-19T04:22:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 -- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e400a3430 msgr2=0x7f6e400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e400a3430 0x7f6e400a58f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6e48000c00 tx=0x7f6e48005e80 comp rx=0 tx=0).stop 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 -- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58102570 msgr2=0x7f6e58197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58102570 0x7f6e58197920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c970 tx=0x7f6e4c00ce30 comp rx=0 tx=0).stop 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 -- 172.21.15.94:0/1245871159 shutdown_connections 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6e400a3430 0x7f6e400a58f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e58102ef0 0x7f6e5819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e58102570 0x7f6e58197920 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 --2- 172.21.15.94:0/1245871159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e58100e20 0x7f6e581973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 -- 172.21.15.94:0/1245871159 >> 172.21.15.94:0/1245871159 conn(0x7f6e580fc7d0 msgr2=0x7f6e580fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.349+0000 7f6e5c992640 1 -- 172.21.15.94:0/1245871159 shutdown_connections 2024-09-19T04:22:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:16.350+0000 7f6e5c992640 1 -- 172.21.15.94:0/1245871159 wait complete. 2024-09-19T04:22:16.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:16 smithi094 ceph-mon[30972]: pgmap v221: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-09-19T04:22:16.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:16 smithi094 ceph-mon[30972]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:22:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:16 smithi104 ceph-mon[34145]: pgmap v221: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-09-19T04:22:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:16 smithi104 ceph-mon[34145]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:22:17.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:16 smithi112 ceph-mon[35051]: pgmap v221: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-09-19T04:22:17.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:16 smithi112 ceph-mon[35051]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:22:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:22:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:22:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:22:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:22:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:22:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:17 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:22:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:22:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:22:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:22:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:22:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:22:17.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:17.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:17 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:18.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:18.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:17 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:22:18.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:22:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:18.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:18 smithi094 ceph-mon[30972]: pgmap v223: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-09-19T04:22:18.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:18 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:22:18.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:18 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:22:18.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:18 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:22:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:18 smithi104 ceph-mon[34145]: pgmap v223: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-09-19T04:22:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:18 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:22:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:18 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:22:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:18 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:22:19.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:18 smithi112 ceph-mon[35051]: pgmap v223: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-09-19T04:22:19.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:18 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:22:19.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:18 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:22:19.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:18 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:22:19.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.203+0000 7f281ec5e640 1 -- 172.21.15.94:0/2590380749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281810e580 msgr2=0x7f281810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.203+0000 7f281ec5e640 1 --2- 172.21.15.94:0/2590380749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281810e580 0x7f281810e960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2808009cd0 tx=0x7f280802a8f0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.203+0000 7f281ec5e640 1 -- 172.21.15.94:0/2590380749 shutdown_connections 2024-09-19T04:22:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.203+0000 7f281ec5e640 1 --2- 172.21.15.94:0/2590380749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f281810d9b0 0x7f2818115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.203+0000 7f281ec5e640 1 --2- 172.21.15.94:0/2590380749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f281810ef30 0x7f281810d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.203+0000 7f281ec5e640 1 --2- 172.21.15.94:0/2590380749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281810e580 0x7f281810e960 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2808009cd0 tx=0x7f280802a8f0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.203+0000 7f281ec5e640 1 -- 172.21.15.94:0/2590380749 >> 172.21.15.94:0/2590380749 conn(0x7f281806e140 msgr2=0x7f281806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:19.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.204+0000 7f281ec5e640 1 -- 172.21.15.94:0/2590380749 shutdown_connections 2024-09-19T04:22:19.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.204+0000 7f281ec5e640 1 -- 172.21.15.94:0/2590380749 wait complete. 2024-09-19T04:22:19.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.205+0000 7f281ec5e640 1 Processor -- start 2024-09-19T04:22:19.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.205+0000 7f281ec5e640 1 -- start start 2024-09-19T04:22:19.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281ec5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f281810d9b0 0x7f281819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281ec5e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281810ef30 0x7f28181a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281ec5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28181a45f0 0x7f28181a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281ec5e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2818118cf0 con 0x7f28181a45f0 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281ec5e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2818118b70 con 0x7f281810ef30 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281ec5e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2818118e70 con 0x7f281810d9b0 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281c9fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f281810d9b0 0x7f281819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281c9fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f281810d9b0 0x7f281819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:46188/0 (socket says 172.21.15.94:46188) 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281c9fa640 1 -- 172.21.15.94:0/882860421 learned_addr learned my addr 172.21.15.94:0/882860421 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f2817fff640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281810ef30 0x7f28181a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281d1fb640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28181a45f0 0x7f28181a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281c9fa640 1 -- 172.21.15.94:0/882860421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281810ef30 msgr2=0x7f28181a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281c9fa640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281810ef30 0x7f28181a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281c9fa640 1 -- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28181a45f0 msgr2=0x7f28181a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281c9fa640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28181a45f0 0x7f28181a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f281c9fa640 1 -- 172.21.15.94:0/882860421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28181a50b0 con 0x7f281810d9b0 2024-09-19T04:22:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.206+0000 7f2817fff640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281810ef30 0x7f28181a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.207+0000 7f281c9fa640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f281810d9b0 0x7f281819fd20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2808009ca0 tx=0x7f2808003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.207+0000 7f2815ffb640 1 -- 172.21.15.94:0/882860421 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808004240 con 0x7f281810d9b0 2024-09-19T04:22:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.207+0000 7f281ec5e640 1 -- 172.21.15.94:0/882860421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28181a5340 con 0x7f281810d9b0 2024-09-19T04:22:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.207+0000 7f2815ffb640 1 -- 172.21.15.94:0/882860421 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2808040030 con 0x7f281810d9b0 2024-09-19T04:22:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.207+0000 7f2815ffb640 1 -- 172.21.15.94:0/882860421 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808031b70 con 0x7f281810d9b0 2024-09-19T04:22:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.208+0000 7f281ec5e640 1 -- 172.21.15.94:0/882860421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2818119730 con 0x7f281810d9b0 2024-09-19T04:22:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.211+0000 7f2821037740 1 Processor -- start 2024-09-19T04:22:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2821037740 1 -- start start 2024-09-19T04:22:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2821037740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168583090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2821037740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1685835d0 0x55b16858d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2821037740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858de90 0x55b168590280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2821037740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b1684e02e0 con 0x55b1685835d0 2024-09-19T04:22:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2821037740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b168487ce0 con 0x55b16858de90 2024-09-19T04:22:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2821037740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b1684dc2a0 con 0x55b168582cb0 2024-09-19T04:22:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f281d1fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168583090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2817fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858de90 0x55b168590280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2817fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858de90 0x55b168590280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33884/0 (socket says 172.21.15.94:33884) 2024-09-19T04:22:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f281d1fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168583090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:46192/0 (socket says 172.21.15.94:46192) 2024-09-19T04:22:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2817fff640 1 -- 172.21.15.94:0/3780645097 learned_addr learned my addr 172.21.15.94:0/3780645097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2817fff640 1 -- 172.21.15.94:0/3780645097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 msgr2=0x55b168583090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2817fff640 1 --2- 172.21.15.94:0/3780645097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168583090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2817fff640 1 -- 172.21.15.94:0/3780645097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1685835d0 msgr2=0x55b16858d950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:22:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2817fff640 1 --2- 172.21.15.94:0/3780645097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1685835d0 0x55b16858d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.212+0000 7f2817fff640 1 -- 172.21.15.94:0/3780645097 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b16846c440 con 0x55b16858de90 2024-09-19T04:22:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.213+0000 7f2817fff640 1 --2- 172.21.15.94:0/3780645097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858de90 0x55b168590280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f280000cbc0 tx=0x7f280000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c87278ec8868a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.213+0000 7f280effd640 1 -- 172.21.15.94:0/3780645097 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800005220 con 0x55b16858de90 2024-09-19T04:22:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.213+0000 7f280effd640 1 -- 172.21.15.94:0/3780645097 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2800004c50 con 0x55b16858de90 2024-09-19T04:22:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.213+0000 7f280effd640 1 -- 172.21.15.94:0/3780645097 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28000107b0 con 0x55b16858de90 2024-09-19T04:22:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.214+0000 7f2815ffb640 1 -- 172.21.15.94:0/882860421 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2808031d10 con 0x7f281810d9b0 2024-09-19T04:22:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.214+0000 7f2821037740 1 -- 172.21.15.94:0/3780645097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858de90 msgr2=0x55b168590280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.214+0000 7f2821037740 1 --2- 172.21.15.94:0/3780645097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858de90 0x55b168590280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f280000cbc0 tx=0x7f280000e720 comp rx=0 tx=0).stop 2024-09-19T04:22:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.214+0000 7f2821037740 1 -- 172.21.15.94:0/3780645097 shutdown_connections 2024-09-19T04:22:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.214+0000 7f2821037740 1 --2- 172.21.15.94:0/3780645097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858de90 0x55b168590280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.214+0000 7f2821037740 1 --2- 172.21.15.94:0/3780645097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1685835d0 0x55b16858d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.214+0000 7f2821037740 1 --2- 172.21.15.94:0/3780645097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168583090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.214+0000 7f2821037740 1 -- 172.21.15.94:0/3780645097 >> 172.21.15.94:0/3780645097 conn(0x55b16857f240 msgr2=0x55b16857f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.215+0000 7f2821037740 1 -- 172.21.15.94:0/3780645097 shutdown_connections 2024-09-19T04:22:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.215+0000 7f2821037740 1 -- 172.21.15.94:0/3780645097 wait complete. 2024-09-19T04:22:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.215+0000 7f2815ffb640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f28040a3360 0x7f28040a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.215+0000 7f2821037740 1 -- start start 2024-09-19T04:22:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.215+0000 7f2815ffb640 1 -- 172.21.15.94:0/882860421 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f28080f0b80 con 0x7f281810d9b0 2024-09-19T04:22:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.215+0000 7f2821037740 1 Processor -- start 2024-09-19T04:22:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.216+0000 7f281d1fb640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f28040a3360 0x7f28040a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.216+0000 7f2821037740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b16858ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.216+0000 7f2821037740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1685835d0 0x55b16858b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.217+0000 7f2821037740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858b7f0 0x55b168599ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.217+0000 7f281c9fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b16858ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.217+0000 7f281c9fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b16858ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:46202/0 (socket says 172.21.15.94:46202) 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.217+0000 7f2821037740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b168590a60 con 0x55b1685835d0 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.217+0000 7f2821037740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b16849b050 con 0x55b16858b7f0 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.217+0000 7f2821037740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b168590be0 con 0x55b168582cb0 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.217+0000 7f2821037740 10 client.0 _ll_register_callbacks cb 0x55b1684db160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.217+0000 7f281c9fa640 1 -- 172.21.15.94:0/3294395596 learned_addr learned my addr 172.21.15.94:0/3294395596 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.217+0000 7f2817fff640 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1685835d0 0x55b16858b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.218+0000 7f281c9fa640 1 -- 172.21.15.94:0/3294395596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858b7f0 msgr2=0x55b168599ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.218+0000 7f281c9fa640 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858b7f0 0x55b168599ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.218+0000 7f281c9fa640 1 -- 172.21.15.94:0/3294395596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1685835d0 msgr2=0x55b16858b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.218+0000 7f281c9fa640 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1685835d0 0x55b16858b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.218+0000 7f281c9fa640 1 -- 172.21.15.94:0/3294395596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1682513c0 con 0x55b168582cb0 2024-09-19T04:22:19.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.218+0000 7f281d1fb640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f28040a3360 0x7f28040a5820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f281000bc60 tx=0x7f281000b450 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:19.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.218+0000 7f281c9fa640 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b16858ad70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f28080ee400 tx=0x7f28080bc890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f2127177697bb78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:19.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.218+0000 7f27f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.219+0000 7f27f7fff640 1 -- 172.21.15.94:0/3294395596 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280804a040 con 0x55b168582cb0 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.219+0000 7f27f7fff640 1 -- 172.21.15.94:0/3294395596 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f28080b95a0 con 0x55b168582cb0 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.219+0000 7f2821037740 1 -- 172.21.15.94:0/3294395596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2800004c50 con 0x55b168582cb0 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.219+0000 7f27f7fff640 1 -- 172.21.15.94:0/3294395596 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f28080bcf30 con 0x55b168582cb0 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.219+0000 7f2821037740 10 client.25861 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.219+0000 7f2821037740 1 -- 172.21.15.94:0/3294395596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b168590ab0 con 0x55b168582cb0 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.219+0000 7f27f7fff640 1 -- 172.21.15.94:0/3294395596 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28080d8080 con 0x55b168582cb0 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.220+0000 7f27f7fff640 1 -- 172.21.15.94:0/3294395596 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f28080c0030 con 0x55b168582cb0 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.220+0000 7f2821037740 10 client.25861 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.220+0000 7f2821037740 10 client.25861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.220+0000 7f2821037740 20 client.25861 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:19.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.220+0000 7f2821037740 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b16859de70 0x55b1685a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.220+0000 7f2821037740 4 client.25861 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:19.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.220+0000 7f2821037740 1 -- 172.21.15.94:0/3294395596 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1684ed850 con 0x55b16859de70 2024-09-19T04:22:19.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.220+0000 7f2817fff640 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b16859de70 0x55b1685a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.221+0000 7f27f7fff640 10 client.25861 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.221+0000 7f2817fff640 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b16859de70 0x55b1685a02c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca9a329978b1afa server_cookie=2315f2a65f8440a in_seq=0 out_seq=0 2024-09-19T04:22:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.222+0000 7f27f7fff640 1 -- 172.21.15.94:0/3294395596 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1684ed850 con 0x55b16859de70 2024-09-19T04:22:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.222+0000 7f27f7fff640 10 client.25861 handle_command_reply: tid=0 2024-09-19T04:22:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.223+0000 7f2821037740 1 client.25861 shutdown 2024-09-19T04:22:19.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.223+0000 7f2821037740 1 -- 172.21.15.94:0/3294395596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 msgr2=0x55b16858ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:19.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.223+0000 7f2821037740 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b16858ad70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f28080ee400 tx=0x7f28080bc890 comp rx=0 tx=0).stop 2024-09-19T04:22:19.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.225+0000 7f2821037740 1 -- 172.21.15.94:0/3294395596 shutdown_connections 2024-09-19T04:22:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.225+0000 7f2821037740 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b16859de70 0x55b1685a02c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.225+0000 7f27f7fff640 0 client.25861 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.225+0000 7f2821037740 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b16858b7f0 0x55b168599ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.225+0000 7f2821037740 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1685835d0 0x55b16858b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.225+0000 7f2821037740 1 --2- 172.21.15.94:0/3294395596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b16858ad70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.225+0000 7f2821037740 1 -- 172.21.15.94:0/3294395596 >> 172.21.15.94:0/3294395596 conn(0x55b16857f240 msgr2=0x55b168575b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.225+0000 7f2821037740 1 -- 172.21.15.94:0/3294395596 shutdown_connections 2024-09-19T04:22:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.225+0000 7f2821037740 1 -- 172.21.15.94:0/3294395596 wait complete. 2024-09-19T04:22:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.226+0000 7f2821037740 20 client.25861 trim_cache size 0 max 16384 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.257+0000 7f2821037740 1 Processor -- start 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.257+0000 7f2821037740 1 -- start start 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.257+0000 7f2821037740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b168582cb0 0x55b1685a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.257+0000 7f2821037740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.257+0000 7f2821037740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b16858b7f0 0x55b168587fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f2821037740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b6d00 con 0x55b168582cb0 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f2821037740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b4570 con 0x55b1685835d0 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f2821037740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b23a0 con 0x55b16858b7f0 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281d1fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b16858b7f0 0x55b168587fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281d1fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b16858b7f0 0x55b168587fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:46210/0 (socket says 172.21.15.94:46210) 2024-09-19T04:22:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281d1fb640 1 -- 172.21.15.94:0/3717417092 learned_addr learned my addr 172.21.15.94:0/3717417092 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f2817fff640 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281d1fb640 1 -- 172.21.15.94:0/3717417092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 msgr2=0x55b1685a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281d1fb640 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281c9fa640 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b168582cb0 0x55b1685a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281d1fb640 1 -- 172.21.15.94:0/3717417092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b168582cb0 msgr2=0x55b1685a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281d1fb640 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b168582cb0 0x55b1685a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281d1fb640 1 -- 172.21.15.94:0/3717417092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1685892c0 con 0x55b16858b7f0 2024-09-19T04:22:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f2817fff640 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.258+0000 7f281c9fa640 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b168582cb0 0x55b1685a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:19.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.259+0000 7f281d1fb640 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b16858b7f0 0x55b168587fd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f281000b760 tx=0x7f2810014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd32210357662b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:19.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.259+0000 7f280e7fc640 1 -- 172.21.15.94:0/3717417092 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281001f420 con 0x55b16858b7f0 2024-09-19T04:22:19.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.259+0000 7f280e7fc640 1 -- 172.21.15.94:0/3717417092 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f281001fa70 con 0x55b16858b7f0 2024-09-19T04:22:19.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.259+0000 7f280e7fc640 1 -- 172.21.15.94:0/3717417092 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810027b70 con 0x55b16858b7f0 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.260+0000 7f2821037740 1 -- 172.21.15.94:0/3717417092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b16858b7f0 msgr2=0x55b168587fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.260+0000 7f2821037740 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b16858b7f0 0x55b168587fd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f281000b760 tx=0x7f2810014040 comp rx=0 tx=0).stop 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.260+0000 7f2821037740 1 -- 172.21.15.94:0/3717417092 shutdown_connections 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.260+0000 7f2821037740 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b16858b7f0 0x55b168587fd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.260+0000 7f2821037740 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.260+0000 7f2821037740 1 --2- 172.21.15.94:0/3717417092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b168582cb0 0x55b1685a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.260+0000 7f2821037740 1 -- 172.21.15.94:0/3717417092 >> 172.21.15.94:0/3717417092 conn(0x55b16857f240 msgr2=0x55b168575b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.260+0000 7f2821037740 1 -- 172.21.15.94:0/3717417092 shutdown_connections 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.260+0000 7f2821037740 1 -- 172.21.15.94:0/3717417092 wait complete. 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.261+0000 7f2821037740 1 -- start start 2024-09-19T04:22:19.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.261+0000 7f2821037740 1 Processor -- start 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f2821037740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168593ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f2821037740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f2821037740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b16858b7f0 0x55b168594620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f281c9fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168593ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f281c9fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168593ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:46218/0 (socket says 172.21.15.94:46218) 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f2817fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b2a30 con 0x55b16858b7f0 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b3ea0 con 0x55b1685835d0 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b3100 con 0x55b168582cb0 2024-09-19T04:22:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f281c9fa640 1 -- 172.21.15.94:0/573737863 learned_addr learned my addr 172.21.15.94:0/573737863 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.262+0000 7f2821037740 10 client.0 _ll_register_callbacks cb 0x7f2808005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f281d1fb640 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b16858b7f0 0x55b168594620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f2817fff640 1 -- 172.21.15.94:0/573737863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 msgr2=0x55b168593ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f2817fff640 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168593ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f2817fff640 1 -- 172.21.15.94:0/573737863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b16858b7f0 msgr2=0x55b168594620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f2817fff640 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b16858b7f0 0x55b168594620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f2817fff640 1 -- 172.21.15.94:0/573737863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b168607b70 con 0x55b1685835d0 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f281d1fb640 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b16858b7f0 0x55b168594620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f2817fff640 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685940e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55b1685802f0 tx=0x7f280000c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe461f8d6298f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f27f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.263+0000 7f27f7fff640 1 -- 172.21.15.94:0/573737863 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800004a70 con 0x55b1685835d0 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f27f7fff640 1 -- 172.21.15.94:0/573737863 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f2800004e00 con 0x55b1685835d0 2024-09-19T04:22:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281001fa70 con 0x55b1685835d0 2024-09-19T04:22:19.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f27f7fff640 1 -- 172.21.15.94:0/573737863 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2800037050 con 0x55b1685835d0 2024-09-19T04:22:19.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f2821037740 10 client.25880 fetch_fsmap learned FSMap version 38 2024-09-19T04:22:19.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b168608e00 con 0x55b1685835d0 2024-09-19T04:22:19.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f27f7fff640 1 -- 172.21.15.94:0/573737863 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f281001fa70 con 0x55b1685835d0 2024-09-19T04:22:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f27f7fff640 1 -- 172.21.15.94:0/573737863 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f2800010830 con 0x55b1685835d0 2024-09-19T04:22:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f2821037740 10 client.25880 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T04:22:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f2821037740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f2821037740 20 client.25880 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.264+0000 7f2821037740 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b16859de70 0x55b16860b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.265+0000 7f2821037740 4 client.25880 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:22:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.265+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1684e7d60 con 0x55b16859de70 2024-09-19T04:22:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.265+0000 7f281c9fa640 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b16859de70 0x55b16860b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:19.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.268+0000 7f27f7fff640 10 client.25880 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:19.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.268+0000 7f281c9fa640 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b16859de70 0x55b16860b480 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47b25c72ff0c87d server_cookie=6a6749ab9c8fc376 in_seq=0 out_seq=0 2024-09-19T04:22:19.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.268+0000 7f27f7fff640 1 -- 172.21.15.94:0/573737863 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b1684e7d60 con 0x55b16859de70 2024-09-19T04:22:19.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.268+0000 7f27f7fff640 10 client.25880 handle_command_reply: tid=0 2024-09-19T04:22:19.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.268+0000 7f2821037740 1 client.25880 shutdown 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.268+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 msgr2=0x55b1685940e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.268+0000 7f2821037740 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685940e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55b1685802f0 tx=0x7f280000c7b0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 shutdown_connections 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f2821037740 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b16859de70 0x55b16860b480 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f2821037740 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b16858b7f0 0x55b168594620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f2821037740 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1685835d0 0x55b1685940e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f2821037740 1 --2- 172.21.15.94:0/573737863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b168582cb0 0x55b168593ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 >> 172.21.15.94:0/573737863 conn(0x55b16857f240 msgr2=0x55b16858c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f27f7fff640 0 client.25880 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 shutdown_connections 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f2821037740 1 -- 172.21.15.94:0/573737863 wait complete. 2024-09-19T04:22:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.269+0000 7f2821037740 20 client.25880 trim_cache size 0 max 16384 2024-09-19T04:22:19.272 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:22:19.272 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:22:19.272 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:22:19.272 INFO:teuthology.orchestra.run.smithi094.stdout: "acfab277-a7a1-44f0-bed2-5e99c8f01145": { 2024-09-19T04:22:19.272 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:22:19.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "acfab277-a7a1-44f0-bed2-5e99c8f01145", 2024-09-19T04:22:19.272 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:22:19.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:22:19.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:22:19.273 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 -- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f28040a3360 msgr2=0x7f28040a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f28040a3360 0x7f28040a5820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f281000bc60 tx=0x7f281000b450 comp rx=0 tx=0).stop 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 -- 172.21.15.94:0/882860421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f281810d9b0 msgr2=0x7f281819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f281810d9b0 0x7f281819fd20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2808009ca0 tx=0x7f2808003ba0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 -- 172.21.15.94:0/882860421 shutdown_connections 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f28040a3360 0x7f28040a5820 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28181a45f0 0x7f28181a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281810ef30 0x7f28181a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 --2- 172.21.15.94:0/882860421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f281810d9b0 0x7f281819fd20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:19.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.270+0000 7f281ec5e640 1 -- 172.21.15.94:0/882860421 >> 172.21.15.94:0/882860421 conn(0x7f281806e140 msgr2=0x7f28180721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:19.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.271+0000 7f281ec5e640 1 -- 172.21.15.94:0/882860421 shutdown_connections 2024-09-19T04:22:19.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:19.271+0000 7f281ec5e640 1 -- 172.21.15.94:0/882860421 wait complete. 2024-09-19T04:22:20.170 INFO:tasks.cephfs.filesystem:scrub status for tag:acfab277-a7a1-44f0-bed2-5e99c8f01145 - {'path': '/', 'tag': 'acfab277-a7a1-44f0-bed2-5e99c8f01145', 'options': 'recursive,force'} 2024-09-19T04:22:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 365.0 s, 1269.0 tps, lat 78.586 ms stddev 26.062 2024-09-19T04:22:20.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:20 smithi094 ceph-mon[30972]: pgmap v224: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:22:20.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:20 smithi104 ceph-mon[34145]: pgmap v224: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:22:21.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:20 smithi112 ceph-mon[35051]: pgmap v224: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:22:22.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:22 smithi094 ceph-mon[30972]: pgmap v225: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:22:22.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:22 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:22:22.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:22 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:22:22.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:22 smithi104 ceph-mon[34145]: pgmap v225: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:22:22.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:22 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:22:22.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:22 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:22:23.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:22 smithi112 ceph-mon[35051]: pgmap v225: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:22:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:22 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:22:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:22 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:22:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:23 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:22:24.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:23 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:22:24.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:23 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:22:24.857 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:24 smithi104 ceph-mon[34145]: pgmap v226: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-09-19T04:22:25.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:24 smithi112 ceph-mon[35051]: pgmap v226: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-09-19T04:22:25.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:24 smithi094 ceph-mon[30972]: pgmap v226: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-09-19T04:22:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 370.0 s, 1533.4 tps, lat 65.446 ms stddev 22.929 2024-09-19T04:22:26.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:26 smithi104 ceph-mon[34145]: pgmap v227: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-09-19T04:22:27.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:26 smithi112 ceph-mon[35051]: pgmap v227: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-09-19T04:22:27.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:26 smithi094 ceph-mon[30972]: pgmap v227: 97 pgs: 97 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-09-19T04:22:29.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:28 smithi112 ceph-mon[35051]: pgmap v228: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.78k op/s 2024-09-19T04:22:29.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:28 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:29.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:28 smithi094 ceph-mon[30972]: pgmap v228: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.78k op/s 2024-09-19T04:22:29.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:28 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:28 smithi104 ceph-mon[34145]: pgmap v228: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.78k op/s 2024-09-19T04:22:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:28 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 375.0 s, 1597.2 tps, lat 62.463 ms stddev 37.333 2024-09-19T04:22:30.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:30 smithi094 ceph-mon[30972]: pgmap v229: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:22:31.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:30 smithi112 ceph-mon[35051]: pgmap v229: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:22:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:30 smithi104 ceph-mon[34145]: pgmap v229: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:22:33.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:32 smithi112 ceph-mon[35051]: pgmap v230: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:22:33.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:32 smithi094 ceph-mon[30972]: pgmap v230: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:22:33.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:32 smithi104 ceph-mon[34145]: pgmap v230: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-19T04:22:35.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:34 smithi112 ceph-mon[35051]: pgmap v231: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 17 MiB/s wr, 1.84k op/s 2024-09-19T04:22:35.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:34 smithi094 ceph-mon[30972]: pgmap v231: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 17 MiB/s wr, 1.84k op/s 2024-09-19T04:22:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:34 smithi104 ceph-mon[34145]: pgmap v231: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 17 MiB/s wr, 1.84k op/s 2024-09-19T04:22:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 380.0 s, 1514.6 tps, lat 65.973 ms stddev 27.319 2024-09-19T04:22:37.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:36 smithi112 ceph-mon[35051]: pgmap v232: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-19T04:22:37.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:36 smithi094 ceph-mon[30972]: pgmap v232: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-19T04:22:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:36 smithi104 ceph-mon[34145]: pgmap v232: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-19T04:22:39.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:38 smithi112 ceph-mon[35051]: pgmap v233: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 18 MiB/s wr, 1.94k op/s 2024-09-19T04:22:39.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:38 smithi094 ceph-mon[30972]: pgmap v233: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 18 MiB/s wr, 1.94k op/s 2024-09-19T04:22:39.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:38 smithi104 ceph-mon[34145]: pgmap v233: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 18 MiB/s wr, 1.94k op/s 2024-09-19T04:22:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 385.0 s, 1510.8 tps, lat 66.193 ms stddev 24.155 2024-09-19T04:22:41.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:40 smithi112 ceph-mon[35051]: pgmap v234: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:22:41.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:40 smithi094 ceph-mon[30972]: pgmap v234: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:22:41.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:40 smithi104 ceph-mon[34145]: pgmap v234: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:22:43.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:42 smithi112 ceph-mon[35051]: pgmap v235: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:22:43.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:42 smithi094 ceph-mon[30972]: pgmap v235: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:22:43.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:42 smithi104 ceph-mon[34145]: pgmap v235: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-19T04:22:44.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:43 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:44.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:43 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:44.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:43 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:45.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:44 smithi112 ceph-mon[35051]: pgmap v236: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 17 MiB/s wr, 1.89k op/s 2024-09-19T04:22:45.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:44 smithi094 ceph-mon[30972]: pgmap v236: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 17 MiB/s wr, 1.89k op/s 2024-09-19T04:22:45.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:44 smithi104 ceph-mon[34145]: pgmap v236: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 17 MiB/s wr, 1.89k op/s 2024-09-19T04:22:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 390.0 s, 1568.0 tps, lat 63.907 ms stddev 26.307 2024-09-19T04:22:46.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:46 smithi104 ceph-mon[34145]: pgmap v237: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-19T04:22:47.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:46 smithi112 ceph-mon[35051]: pgmap v237: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-19T04:22:47.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:46 smithi094 ceph-mon[30972]: pgmap v237: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-19T04:22:49.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:48 smithi112 ceph-mon[35051]: pgmap v238: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 17 MiB/s wr, 1.87k op/s 2024-09-19T04:22:49.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:48 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:49.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:48 smithi094 ceph-mon[30972]: pgmap v238: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 17 MiB/s wr, 1.87k op/s 2024-09-19T04:22:49.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:48 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:49.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:48 smithi104 ceph-mon[34145]: pgmap v238: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 17 MiB/s wr, 1.87k op/s 2024-09-19T04:22:49.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:48 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:50.171 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:22:50.593 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:50.601 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 395.0 s, 1538.0 tps, lat 64.924 ms stddev 25.462 2024-09-19T04:22:50.903 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:50 smithi094 ceph-mon[30972]: pgmap v239: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:22:51.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:50 smithi112 ceph-mon[35051]: pgmap v239: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:22:51.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:50 smithi104 ceph-mon[34145]: pgmap v239: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:22:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.190+0000 7efcfb4ce640 1 -- 172.21.15.94:0/107924641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcf4102c10 msgr2=0x7efcf4102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.190+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/107924641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcf4102c10 0x7efcf4102ff0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009860 tx=0x7efcf002a3d0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.190+0000 7efcfb4ce640 1 -- 172.21.15.94:0/107924641 shutdown_connections 2024-09-19T04:22:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.190+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/107924641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf410bff0 0x7efcf410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.190+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/107924641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf4103530 0x7efcf410ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.190+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/107924641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcf4102c10 0x7efcf4102ff0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.190+0000 7efcfb4ce640 1 -- 172.21.15.94:0/107924641 >> 172.21.15.94:0/107924641 conn(0x7efcf40fc610 msgr2=0x7efcf40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.191+0000 7efcfb4ce640 1 -- 172.21.15.94:0/107924641 shutdown_connections 2024-09-19T04:22:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.191+0000 7efcfb4ce640 1 -- 172.21.15.94:0/107924641 wait complete. 2024-09-19T04:22:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.191+0000 7efcfb4ce640 1 Processor -- start 2024-09-19T04:22:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfb4ce640 1 -- start start 2024-09-19T04:22:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfb4ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf4102c10 0x7efcf4100870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfb4ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf4103530 0x7efcf4100db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfb4ce640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcf410bff0 0x7efcf41012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfb4ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4111cc0 con 0x7efcf4102c10 2024-09-19T04:22:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfb4ce640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4111b40 con 0x7efcf410bff0 2024-09-19T04:22:51.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfb4ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4111e40 con 0x7efcf4103530 2024-09-19T04:22:51.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfa4cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf4102c10 0x7efcf4100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfa4cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf4102c10 0x7efcf4100870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43642/0 (socket says 172.21.15.94:43642) 2024-09-19T04:22:51.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfa4cc640 1 -- 172.21.15.94:0/1244542090 learned_addr learned my addr 172.21.15.94:0/1244542090 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:51.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcfaccd640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcf410bff0 0x7efcf41012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.192+0000 7efcf9ccb640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf4103530 0x7efcf4100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.193+0000 7efcfaccd640 1 -- 172.21.15.94:0/1244542090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf4103530 msgr2=0x7efcf4100db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.193+0000 7efcfaccd640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf4103530 0x7efcf4100db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.193+0000 7efcfaccd640 1 -- 172.21.15.94:0/1244542090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf4102c10 msgr2=0x7efcf4100870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.193+0000 7efcfaccd640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf4102c10 0x7efcf4100870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.193+0000 7efcfaccd640 1 -- 172.21.15.94:0/1244542090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf41a6660 con 0x7efcf410bff0 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.193+0000 7efcfaccd640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcf410bff0 0x7efcf41012f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efcec00ef00 tx=0x7efcec00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.193+0000 7efceb7fe640 1 -- 172.21.15.94:0/1244542090 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec004b20 con 0x7efcf410bff0 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.194+0000 7efceb7fe640 1 -- 172.21.15.94:0/1244542090 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcec003e20 con 0x7efcf410bff0 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.194+0000 7efceb7fe640 1 -- 172.21.15.94:0/1244542090 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec00b570 con 0x7efcf410bff0 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.194+0000 7efcfb4ce640 1 -- 172.21.15.94:0/1244542090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf4197f60 con 0x7efcf410bff0 2024-09-19T04:22:51.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.194+0000 7efcfb4ce640 1 -- 172.21.15.94:0/1244542090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf4198390 con 0x7efcf410bff0 2024-09-19T04:22:51.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.198+0000 7efceb7fe640 1 -- 172.21.15.94:0/1244542090 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7efcec01d020 con 0x7efcf410bff0 2024-09-19T04:22:51.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.198+0000 7efcfeb09740 1 Processor -- start 2024-09-19T04:22:51.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.198+0000 7efcfeb09740 1 -- start start 2024-09-19T04:22:51.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.198+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516066a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516066f90 0x55a516071310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516071850 0x55a516073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfeb09740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a515fcc2e0 con 0x55a516071850 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfeb09740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a515f73ce0 con 0x55a516066670 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfeb09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a515fc82a0 con 0x55a516066f90 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcf9ccb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfa4cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516066f90 0x55a516071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcf9ccb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516066a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50724/0 (socket says 172.21.15.94:50724) 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcf9ccb640 1 -- 172.21.15.94:0/525554789 learned_addr learned my addr 172.21.15.94:0/525554789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfa4cc640 1 -- 172.21.15.94:0/525554789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 msgr2=0x55a516066a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfa4cc640 1 --2- 172.21.15.94:0/525554789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516066a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfa4cc640 1 -- 172.21.15.94:0/525554789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516071850 msgr2=0x55a516073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:51.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfa4cc640 1 --2- 172.21.15.94:0/525554789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516071850 0x55a516073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efceb7fe640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efce00a3500 0x7efce00a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.199+0000 7efcfa4cc640 1 -- 172.21.15.94:0/525554789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a515f58440 con 0x55a516066f90 2024-09-19T04:22:51.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.200+0000 7efceb7fe640 1 -- 172.21.15.94:0/1244542090 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7efcec0cfa10 con 0x7efcf410bff0 2024-09-19T04:22:51.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.200+0000 7efcfa4cc640 1 --2- 172.21.15.94:0/525554789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516066f90 0x55a516071310 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009830 tx=0x7efcf0032f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e03d67923de9940 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:51.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.200+0000 7efcf9ccb640 1 --2- 172.21.15.94:0/525554789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:51.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.200+0000 7efce8ff9640 1 -- 172.21.15.94:0/525554789 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00303a0 con 0x55a516066f90 2024-09-19T04:22:51.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.200+0000 7efce8ff9640 1 -- 172.21.15.94:0/525554789 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcf0030540 con 0x55a516066f90 2024-09-19T04:22:51.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.200+0000 7efce8ff9640 1 -- 172.21.15.94:0/525554789 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0005040 con 0x55a516066f90 2024-09-19T04:22:51.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.201+0000 7efcf9ccb640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efce00a3500 0x7efce00a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.201+0000 7efcfeb09740 1 -- 172.21.15.94:0/525554789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516066f90 msgr2=0x55a516071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.201+0000 7efcfeb09740 1 --2- 172.21.15.94:0/525554789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516066f90 0x55a516071310 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009830 tx=0x7efcf0032f10 comp rx=0 tx=0).stop 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.201+0000 7efcf9ccb640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efce00a3500 0x7efce00a59c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efce40058c0 tx=0x7efce4005850 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.202+0000 7efcfeb09740 1 -- 172.21.15.94:0/525554789 shutdown_connections 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.202+0000 7efcfeb09740 1 --2- 172.21.15.94:0/525554789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516071850 0x55a516073c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.202+0000 7efcfeb09740 1 --2- 172.21.15.94:0/525554789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516066f90 0x55a516071310 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.202+0000 7efcfeb09740 1 --2- 172.21.15.94:0/525554789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.202+0000 7efcfeb09740 1 -- 172.21.15.94:0/525554789 >> 172.21.15.94:0/525554789 conn(0x55a516062c00 msgr2=0x55a516062fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.202+0000 7efcfeb09740 1 -- 172.21.15.94:0/525554789 shutdown_connections 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.202+0000 7efcfeb09740 1 -- 172.21.15.94:0/525554789 wait complete. 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.203+0000 7efcfeb09740 1 -- start start 2024-09-19T04:22:51.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.203+0000 7efcfeb09740 1 Processor -- start 2024-09-19T04:22:51.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.204+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.204+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066f90 0x55a51606e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.204+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a51606e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.204+0000 7efcfa4cc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.204+0000 7efcfa4cc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516071140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50734/0 (socket says 172.21.15.94:50734) 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.204+0000 7efcfa4cc640 1 -- 172.21.15.94:0/252188637 learned_addr learned my addr 172.21.15.94:0/252188637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.204+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a5160743c0 con 0x55a516066f90 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.204+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a515f87050 con 0x55a516066670 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.205+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a516074540 con 0x55a516071850 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.205+0000 7efcfeb09740 10 client.0 _ll_register_callbacks cb 0x55a515fc7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.205+0000 7efcfa4cc640 1 -- 172.21.15.94:0/252188637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 msgr2=0x55a51606e930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.205+0000 7efcfa4cc640 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a51606e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.205+0000 7efcfa4cc640 1 -- 172.21.15.94:0/252188637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066f90 msgr2=0x55a51606e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.205+0000 7efcfa4cc640 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066f90 0x55a51606e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.205+0000 7efcfa4cc640 1 -- 172.21.15.94:0/252188637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a515d3d3c0 con 0x55a516066670 2024-09-19T04:22:51.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.209+0000 7efcfa4cc640 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516071140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efcf004c040 tx=0x7efcf00058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d2d50d47f7770b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:51.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.209+0000 7efcddffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:51.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.209+0000 7efcddffb640 1 -- 172.21.15.94:0/252188637 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0057070 con 0x55a516066670 2024-09-19T04:22:51.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.209+0000 7efcddffb640 1 -- 172.21.15.94:0/252188637 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7efcf0004020 con 0x55a516066670 2024-09-19T04:22:51.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.210+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf0030540 con 0x55a516066670 2024-09-19T04:22:51.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.210+0000 7efcddffb640 1 -- 172.21.15.94:0/252188637 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efcf0063050 con 0x55a516066670 2024-09-19T04:22:51.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.210+0000 7efcfeb09740 10 client.25889 fetch_fsmap learned FSMap version 41 2024-09-19T04:22:51.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.210+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a51607fc80 con 0x55a516066670 2024-09-19T04:22:51.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.211+0000 7efcddffb640 1 -- 172.21.15.94:0/252188637 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcf0030540 con 0x55a516066670 2024-09-19T04:22:51.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.211+0000 7efcddffb640 1 -- 172.21.15.94:0/252188637 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7efcf00046c0 con 0x55a516066670 2024-09-19T04:22:51.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.211+0000 7efcfeb09740 10 client.25889 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:22:51.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.211+0000 7efcfeb09740 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:51.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.211+0000 7efcfeb09740 20 client.25889 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:51.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.211+0000 7efcfeb09740 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a516081250 0x55a516083710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.211+0000 7efcfeb09740 4 client.25889 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:51.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.211+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a515fd9850 con 0x55a516081250 2024-09-19T04:22:51.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.211+0000 7efcf9ccb640 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a516081250 0x55a516083710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.212+0000 7efcf9ccb640 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a516081250 0x55a516083710 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9412a5ff58581dc9 server_cookie=52d68318b7190bcc in_seq=0 out_seq=0 2024-09-19T04:22:51.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.212+0000 7efcddffb640 10 client.25889 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:51.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.213+0000 7efcddffb640 1 -- 172.21.15.94:0/252188637 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a515fd9850 con 0x55a516081250 2024-09-19T04:22:51.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.213+0000 7efcddffb640 10 client.25889 handle_command_reply: tid=0 2024-09-19T04:22:51.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.213+0000 7efcfeb09740 1 client.25889 shutdown 2024-09-19T04:22:51.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.213+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 msgr2=0x55a516071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:51.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.213+0000 7efcfeb09740 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516071140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efcf004c040 tx=0x7efcf00058f0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 shutdown_connections 2024-09-19T04:22:51.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcfeb09740 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a516081250 0x55a516083710 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcfeb09740 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a51606e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcfeb09740 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066f90 0x55a51606e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcfeb09740 1 --2- 172.21.15.94:0/252188637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066670 0x55a516071140 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 >> 172.21.15.94:0/252188637 conn(0x55a516062c00 msgr2=0x55a5160688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:51.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcddffb640 0 client.25889 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:51.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 shutdown_connections 2024-09-19T04:22:51.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcfeb09740 1 -- 172.21.15.94:0/252188637 wait complete. 2024-09-19T04:22:51.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.214+0000 7efcfeb09740 20 client.25889 trim_cache size 0 max 16384 2024-09-19T04:22:51.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.246+0000 7efcfeb09740 1 Processor -- start 2024-09-19T04:22:51.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.246+0000 7efcfeb09740 1 -- start start 2024-09-19T04:22:51.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.246+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 0x55a51607d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.246+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a5160699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.246+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a516089020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.246+0000 7efcfeb09740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41addf0 con 0x55a516066670 2024-09-19T04:22:51.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.246+0000 7efcfeb09740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41ab660 con 0x55a516066f90 2024-09-19T04:22:51.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.246+0000 7efcfeb09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41a94f0 con 0x55a516071850 2024-09-19T04:22:51.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcfaccd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a516089020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcfa4cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 0x55a51607d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcfaccd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a516089020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58924/0 (socket says 172.21.15.94:58924) 2024-09-19T04:22:51.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcfaccd640 1 -- 172.21.15.94:0/3823979785 learned_addr learned my addr 172.21.15.94:0/3823979785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:51.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcf9ccb640 1 --2- 172.21.15.94:0/3823979785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a5160699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcfaccd640 1 -- 172.21.15.94:0/3823979785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 msgr2=0x55a5160699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:51.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcfaccd640 1 --2- 172.21.15.94:0/3823979785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a5160699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcfaccd640 1 -- 172.21.15.94:0/3823979785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 msgr2=0x55a51607d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:51.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcfaccd640 1 --2- 172.21.15.94:0/3823979785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 0x55a51607d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcfaccd640 1 -- 172.21.15.94:0/3823979785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a516081480 con 0x55a516071850 2024-09-19T04:22:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.247+0000 7efcf9ccb640 1 --2- 172.21.15.94:0/3823979785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a5160699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:51.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.248+0000 7efcfaccd640 1 --2- 172.21.15.94:0/3823979785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a516089020 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efcec0cd720 tx=0x7efcec09b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3933cc680a930c09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:51.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.248+0000 7efcdffff640 1 -- 172.21.15.94:0/3823979785 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec01a040 con 0x55a516071850 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.248+0000 7efcdffff640 1 -- 172.21.15.94:0/3823979785 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcec0a0060 con 0x55a516071850 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.248+0000 7efcdffff640 1 -- 172.21.15.94:0/3823979785 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec098320 con 0x55a516071850 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.249+0000 7efcfeb09740 1 -- 172.21.15.94:0/3823979785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 msgr2=0x55a516089020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.249+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3823979785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a516089020 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efcec0cd720 tx=0x7efcec09b6d0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.249+0000 7efcfeb09740 1 -- 172.21.15.94:0/3823979785 shutdown_connections 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.249+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3823979785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a516089020 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.249+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3823979785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a5160699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.249+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3823979785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 0x55a51607d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.249+0000 7efcfeb09740 1 -- 172.21.15.94:0/3823979785 >> 172.21.15.94:0/3823979785 conn(0x55a516062c00 msgr2=0x55a5160692b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.249+0000 7efcfeb09740 1 -- 172.21.15.94:0/3823979785 shutdown_connections 2024-09-19T04:22:51.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.249+0000 7efcfeb09740 1 -- 172.21.15.94:0/3823979785 wait complete. 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcfeb09740 1 -- start start 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcfeb09740 1 Processor -- start 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 0x55a516074d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a516075280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcfeb09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a5160e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcfeb09740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41a9b40 con 0x55a516066670 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcfeb09740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41aaf90 con 0x55a516066f90 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcfeb09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41aa1f0 con 0x55a516071850 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcfeb09740 10 client.0 _ll_register_callbacks cb 0x7efcf00096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.250+0000 7efcf9ccb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a516075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcfa4cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 0x55a516074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcf9ccb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a516075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50746/0 (socket says 172.21.15.94:50746) 2024-09-19T04:22:51.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcf9ccb640 1 -- 172.21.15.94:0/3103017295 learned_addr learned my addr 172.21.15.94:0/3103017295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcfaccd640 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a5160e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcfaccd640 1 -- 172.21.15.94:0/3103017295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 msgr2=0x55a516075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcfaccd640 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a516075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcfaccd640 1 -- 172.21.15.94:0/3103017295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 msgr2=0x55a516074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcfaccd640 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 0x55a516074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcfaccd640 1 -- 172.21.15.94:0/3103017295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5160ea550 con 0x55a516071850 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcf9ccb640 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a516075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcfaccd640 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a5160e9fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55a515fc83d0 tx=0x7efcec0a49f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8193e18224f7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcddffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcddffb640 1 -- 172.21.15.94:0/3103017295 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec01a040 con 0x55a516071850 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcddffb640 1 -- 172.21.15.94:0/3103017295 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7efcec0a4490 con 0x55a516071850 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.251+0000 7efcfeb09740 1 -- 172.21.15.94:0/3103017295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcec0a0060 con 0x55a516071850 2024-09-19T04:22:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcddffb640 1 -- 172.21.15.94:0/3103017295 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efcec0a46b0 con 0x55a516071850 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcfeb09740 10 client.25882 fetch_fsmap learned FSMap version 41 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcfeb09740 1 -- 172.21.15.94:0/3103017295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a5160eb840 con 0x55a516071850 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcddffb640 1 -- 172.21.15.94:0/3103017295 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcec0a0060 con 0x55a516071850 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcddffb640 1 -- 172.21.15.94:0/3103017295 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7efcec00b040 con 0x55a516071850 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcfeb09740 10 client.25882 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcfeb09740 10 client.25882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcfeb09740 20 client.25882 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a5160ebe10 0x55a5160ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcfeb09740 4 client.25882 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.252+0000 7efcfeb09740 1 -- 172.21.15.94:0/3103017295 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a51606ecc0 con 0x55a5160ebe10 2024-09-19T04:22:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.253+0000 7efcfa4cc640 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a5160ebe10 0x55a5160ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:51.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.253+0000 7efcddffb640 10 client.25882 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:51.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.253+0000 7efcfa4cc640 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a5160ebe10 0x55a5160ee230 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2adc46071ee233 server_cookie=eed006bc7b982f16 in_seq=0 out_seq=0 2024-09-19T04:22:51.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcddffb640 1 -- 172.21.15.94:0/3103017295 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a51606ecc0 con 0x55a5160ebe10 2024-09-19T04:22:51.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcddffb640 10 client.25882 handle_command_reply: tid=0 2024-09-19T04:22:51.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcfeb09740 1 client.25882 shutdown 2024-09-19T04:22:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcfeb09740 1 -- 172.21.15.94:0/3103017295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 msgr2=0x55a5160e9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a5160e9fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55a515fc83d0 tx=0x7efcec0a49f0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcfeb09740 1 -- 172.21.15.94:0/3103017295 shutdown_connections 2024-09-19T04:22:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a5160ebe10 0x55a5160ee230 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a516071850 0x55a5160e9fe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a516066f90 0x55a516075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcfeb09740 1 --2- 172.21.15.94:0/3103017295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a516066670 0x55a516074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcfeb09740 1 -- 172.21.15.94:0/3103017295 >> 172.21.15.94:0/3103017295 conn(0x55a516062c00 msgr2=0x55a51606c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.254+0000 7efcddffb640 0 client.25882 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.255+0000 7efcfeb09740 1 -- 172.21.15.94:0/3103017295 shutdown_connections 2024-09-19T04:22:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.255+0000 7efcfeb09740 1 -- 172.21.15.94:0/3103017295 wait complete. 2024-09-19T04:22:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.255+0000 7efcfeb09740 20 client.25882 trim_cache size 0 max 16384 2024-09-19T04:22:51.263 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:22:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:22:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:22:51.264 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:22:51.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.256+0000 7efcfb4ce640 1 -- 172.21.15.94:0/1244542090 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efce00a3500 msgr2=0x7efce00a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:51.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.256+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efce00a3500 0x7efce00a59c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efce40058c0 tx=0x7efce4005850 comp rx=0 tx=0).stop 2024-09-19T04:22:51.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.256+0000 7efcfb4ce640 1 -- 172.21.15.94:0/1244542090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcf410bff0 msgr2=0x7efcf41012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:51.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.256+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcf410bff0 0x7efcf41012f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efcec00ef00 tx=0x7efcec00c740 comp rx=0 tx=0).stop 2024-09-19T04:22:51.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.256+0000 7efcfb4ce640 1 -- 172.21.15.94:0/1244542090 shutdown_connections 2024-09-19T04:22:51.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.257+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efce00a3500 0x7efce00a59c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.257+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcf410bff0 0x7efcf41012f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.257+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf4103530 0x7efcf4100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.257+0000 7efcfb4ce640 1 --2- 172.21.15.94:0/1244542090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf4102c10 0x7efcf4100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:51.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.257+0000 7efcfb4ce640 1 -- 172.21.15.94:0/1244542090 >> 172.21.15.94:0/1244542090 conn(0x7efcf40fc610 msgr2=0x7efcf40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:51.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.257+0000 7efcfb4ce640 1 -- 172.21.15.94:0/1244542090 shutdown_connections 2024-09-19T04:22:51.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:51.258+0000 7efcfb4ce640 1 -- 172.21.15.94:0/1244542090 wait complete. 2024-09-19T04:22:52.140 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:22:52.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:22:52.535 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:53.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:52 smithi094 ceph-mon[30972]: pgmap v240: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:22:53.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:52 smithi112 ceph-mon[35051]: pgmap v240: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:22:53.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:52 smithi104 ceph-mon[34145]: pgmap v240: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:22:53.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.145+0000 7f2bc03c5640 1 -- 172.21.15.94:0/716441452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 msgr2=0x7f2bb810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:53.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.145+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/716441452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 0x7f2bb810ec40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009c10 tx=0x7f2bb002a6f0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.145+0000 7f2bc03c5640 1 -- 172.21.15.94:0/716441452 shutdown_connections 2024-09-19T04:22:53.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.145+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/716441452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 0x7f2bb810ec40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.145+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/716441452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb81036e0 0x7f2bb8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.145+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/716441452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bb8102d30 0x7f2bb8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.145+0000 7f2bc03c5640 1 -- 172.21.15.94:0/716441452 >> 172.21.15.94:0/716441452 conn(0x7f2bb80fc7d0 msgr2=0x7f2bb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.145+0000 7f2bc03c5640 1 -- 172.21.15.94:0/716441452 shutdown_connections 2024-09-19T04:22:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.146+0000 7f2bc03c5640 1 -- 172.21.15.94:0/716441452 wait complete. 2024-09-19T04:22:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.146+0000 7f2bc03c5640 1 Processor -- start 2024-09-19T04:22:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bc03c5640 1 -- start start 2024-09-19T04:22:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bc03c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bb8102d30 0x7f2bb8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bc03c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb81036e0 0x7f2bb8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bc03c5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 0x7f2bb819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bc03c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8111f50 con 0x7f2bb8102d30 2024-09-19T04:22:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bc03c5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8111dd0 con 0x7f2bb81082a0 2024-09-19T04:22:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bc03c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb81120d0 con 0x7f2bb81036e0 2024-09-19T04:22:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bbe962640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 0x7f2bb819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bbe962640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 0x7f2bb819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44910/0 (socket says 172.21.15.94:44910) 2024-09-19T04:22:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bbe962640 1 -- 172.21.15.94:0/4043926848 learned_addr learned my addr 172.21.15.94:0/4043926848 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bbe161640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bb8102d30 0x7f2bb8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.147+0000 7f2bbd960640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb81036e0 0x7f2bb8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.148+0000 7f2bbe962640 1 -- 172.21.15.94:0/4043926848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb81036e0 msgr2=0x7f2bb8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.148+0000 7f2bbe962640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb81036e0 0x7f2bb8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.148+0000 7f2bbe962640 1 -- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bb8102d30 msgr2=0x7f2bb8198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.148+0000 7f2bbe962640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bb8102d30 0x7f2bb8198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.148+0000 7f2bbe962640 1 -- 172.21.15.94:0/4043926848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb819d4c0 con 0x7f2bb81082a0 2024-09-19T04:22:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.148+0000 7f2bbe962640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 0x7f2bb819cde0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0004af0 tx=0x7f2bb0003c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.149+0000 7f2baf7fe640 1 -- 172.21.15.94:0/4043926848 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0041020 con 0x7f2bb81082a0 2024-09-19T04:22:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.149+0000 7f2baf7fe640 1 -- 172.21.15.94:0/4043926848 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bb0037040 con 0x7f2bb81082a0 2024-09-19T04:22:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.149+0000 7f2baf7fe640 1 -- 172.21.15.94:0/4043926848 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0041020 con 0x7f2bb81082a0 2024-09-19T04:22:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.150+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb819d750 con 0x7f2bb81082a0 2024-09-19T04:22:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.150+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb819dbe0 con 0x7f2bb81082a0 2024-09-19T04:22:53.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.152+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb8103110 con 0x7f2bb81082a0 2024-09-19T04:22:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.153+0000 7f2baf7fe640 1 -- 172.21.15.94:0/4043926848 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2bb0044020 con 0x7f2bb81082a0 2024-09-19T04:22:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.154+0000 7f2baf7fe640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2ba40a35b0 0x7f2ba40a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.154+0000 7f2baf7fe640 1 -- 172.21.15.94:0/4043926848 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f2bb00f02d0 con 0x7f2bb81082a0 2024-09-19T04:22:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.154+0000 7f2bbe161640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2ba40a35b0 0x7f2ba40a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.155+0000 7f2bbe161640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2ba40a35b0 0x7f2ba40a5a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40096c0 tx=0x7f2bb4009260 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:53.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.160+0000 7f2baf7fe640 1 -- 172.21.15.94:0/4043926848 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2bb00bcc10 con 0x7f2bb81082a0 2024-09-19T04:22:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.532+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bb810a230 con 0x7f2bb81082a0 2024-09-19T04:22:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.533+0000 7f2baf7fe640 1 -- 172.21.15.94:0/4043926848 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11921 (secure 0 0 0) 0x7f2bb00bc510 con 0x7f2bb81082a0 2024-09-19T04:22:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:22:53.535 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":42,"btime":"2024-09-19T04:22:52:651391+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:22:51.816040+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:22:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 42 2024-09-19T04:22:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2ba40a35b0 msgr2=0x7f2ba40a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2ba40a35b0 0x7f2ba40a5a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40096c0 tx=0x7f2bb4009260 comp rx=0 tx=0).stop 2024-09-19T04:22:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 msgr2=0x7f2bb819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 0x7f2bb819cde0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0004af0 tx=0x7f2bb0003c70 comp rx=0 tx=0).stop 2024-09-19T04:22:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 shutdown_connections 2024-09-19T04:22:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2ba40a35b0 0x7f2ba40a5a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb81082a0 0x7f2bb819cde0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb81036e0 0x7f2bb8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 --2- 172.21.15.94:0/4043926848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bb8102d30 0x7f2bb8198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.537+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 >> 172.21.15.94:0/4043926848 conn(0x7f2bb80fc7d0 msgr2=0x7f2bb8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.538+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 shutdown_connections 2024-09-19T04:22:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:53.538+0000 7f2bc03c5640 1 -- 172.21.15.94:0/4043926848 wait complete. 2024-09-19T04:22:53.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:53 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:53.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:53 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4043926848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:22:54.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:53 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:54.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:53 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4043926848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:22:54.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:53 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:22:54.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:53 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4043926848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:22:54.415 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:22:54.669 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:54 smithi094 ceph-mon[30972]: pgmap v241: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 16 MiB/s wr, 1.82k op/s 2024-09-19T04:22:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:55.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:54 smithi112 ceph-mon[35051]: pgmap v241: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 16 MiB/s wr, 1.82k op/s 2024-09-19T04:22:55.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:54 smithi104 ceph-mon[34145]: pgmap v241: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 16 MiB/s wr, 1.82k op/s 2024-09-19T04:22:55.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.424+0000 7fa8e777b640 1 -- 172.21.15.94:0/3423419435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102ef0 msgr2=0x7fa8e0069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:55.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.424+0000 7fa8e777b640 1 --2- 172.21.15.94:0/3423419435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102ef0 0x7fa8e0069f70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009860 tx=0x7fa8d802a3d0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.425+0000 7fa8e777b640 1 -- 172.21.15.94:0/3423419435 shutdown_connections 2024-09-19T04:22:55.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.425+0000 7fa8e777b640 1 --2- 172.21.15.94:0/3423419435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102ef0 0x7fa8e0069f70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.425+0000 7fa8e777b640 1 --2- 172.21.15.94:0/3423419435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0102570 0x7fa8e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.425+0000 7fa8e777b640 1 --2- 172.21.15.94:0/3423419435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e0100e20 0x7fa8e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.425+0000 7fa8e777b640 1 -- 172.21.15.94:0/3423419435 >> 172.21.15.94:0/3423419435 conn(0x7fa8e00fc7d0 msgr2=0x7fa8e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:55.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.425+0000 7fa8e777b640 1 -- 172.21.15.94:0/3423419435 shutdown_connections 2024-09-19T04:22:55.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.426+0000 7fa8e777b640 1 -- 172.21.15.94:0/3423419435 wait complete. 2024-09-19T04:22:55.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.427+0000 7fa8e777b640 1 Processor -- start 2024-09-19T04:22:55.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.428+0000 7fa8e777b640 1 -- start start 2024-09-19T04:22:55.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.428+0000 7fa8e777b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e0100e20 0x7fa8e01a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.428+0000 7fa8e777b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102570 0x7fa8e01a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.428+0000 7fa8e777b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0102ef0 0x7fa8e01aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.428+0000 7fa8e777b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e010fdd0 con 0x7fa8e0100e20 2024-09-19T04:22:55.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.428+0000 7fa8e777b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e010fc50 con 0x7fa8e0102ef0 2024-09-19T04:22:55.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.428+0000 7fa8e777b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e010ff50 con 0x7fa8e0102570 2024-09-19T04:22:55.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e5d18640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0102ef0 0x7fa8e01aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e4d16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102570 0x7fa8e01a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e5d18640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0102ef0 0x7fa8e01aba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44926/0 (socket says 172.21.15.94:44926) 2024-09-19T04:22:55.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e4d16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102570 0x7fa8e01a76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48868/0 (socket says 172.21.15.94:48868) 2024-09-19T04:22:55.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e5d18640 1 -- 172.21.15.94:0/219533950 learned_addr learned my addr 172.21.15.94:0/219533950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e5517640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e0100e20 0x7fa8e01a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e4d16640 1 -- 172.21.15.94:0/219533950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0102ef0 msgr2=0x7fa8e01aba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e4d16640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0102ef0 0x7fa8e01aba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e4d16640 1 -- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e0100e20 msgr2=0x7fa8e01a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e4d16640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e0100e20 0x7fa8e01a71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.429+0000 7fa8e4d16640 1 -- 172.21.15.94:0/219533950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e01ac160 con 0x7fa8e0102570 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.430+0000 7fa8e4d16640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102570 0x7fa8e01a76f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40097d0 tx=0x7fa8d4009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.430+0000 7fa8de7fc640 1 -- 172.21.15.94:0/219533950 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d401c070 con 0x7fa8e0102570 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.430+0000 7fa8e777b640 1 -- 172.21.15.94:0/219533950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e01ac450 con 0x7fa8e0102570 2024-09-19T04:22:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.430+0000 7fa8de7fc640 1 -- 172.21.15.94:0/219533950 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8d4017410 con 0x7fa8e0102570 2024-09-19T04:22:55.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.430+0000 7fa8de7fc640 1 -- 172.21.15.94:0/219533950 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400c5b0 con 0x7fa8e0102570 2024-09-19T04:22:55.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.430+0000 7fa8e777b640 1 -- 172.21.15.94:0/219533950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e01b3d30 con 0x7fa8e0102570 2024-09-19T04:22:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.434+0000 7fa8de7fc640 1 -- 172.21.15.94:0/219533950 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa8d401f070 con 0x7fa8e0102570 2024-09-19T04:22:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.434+0000 7fa8e9b54740 1 Processor -- start 2024-09-19T04:22:55.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.434+0000 7fa8e9b54740 1 -- start start 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e80b40 0x561584e7f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e7fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e8a370 0x561584e8c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e9b54740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561584de52f0 con 0x561584e8a370 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e9b54740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561584d8ccf0 con 0x561584e7f5d0 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e9b54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561584de12b0 con 0x561584e80b40 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e5517640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e7fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e5d18640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e80b40 0x561584e7f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e5d18640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e80b40 0x561584e7f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48874/0 (socket says 172.21.15.94:48874) 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e5d18640 1 -- 172.21.15.94:0/3593379845 learned_addr learned my addr 172.21.15.94:0/3593379845 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e4d16640 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e8a370 0x561584e8c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e5d18640 1 -- 172.21.15.94:0/3593379845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 msgr2=0x561584e7fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e5d18640 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e7fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.435+0000 7fa8e5d18640 1 -- 172.21.15.94:0/3593379845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e8a370 msgr2=0x561584e8c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.436+0000 7fa8e5d18640 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e8a370 0x561584e8c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.436+0000 7fa8e5d18640 1 -- 172.21.15.94:0/3593379845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561584d713a0 con 0x561584e80b40 2024-09-19T04:22:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.436+0000 7fa8e5517640 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.436+0000 7fa8e4d16640 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e8a370 0x561584e8c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.436+0000 7fa8e5d18640 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e80b40 0x561584e7f090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8030b50 tx=0x7fa8d8030b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39914018f7ca1093 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.436+0000 7fa8de7fc640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8c80a3500 0x7fa8c80a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.436+0000 7fa8c77fe640 1 -- 172.21.15.94:0/3593379845 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8031600 con 0x561584e80b40 2024-09-19T04:22:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.437+0000 7fa8de7fc640 1 -- 172.21.15.94:0/219533950 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fa8d40cf430 con 0x7fa8e0102570 2024-09-19T04:22:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.437+0000 7fa8c77fe640 1 -- 172.21.15.94:0/3593379845 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8d8030e10 con 0x561584e80b40 2024-09-19T04:22:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.437+0000 7fa8c77fe640 1 -- 172.21.15.94:0/3593379845 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8031a00 con 0x561584e80b40 2024-09-19T04:22:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.437+0000 7fa8e5517640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8c80a3500 0x7fa8c80a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.438+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3593379845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e80b40 msgr2=0x561584e7f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.438+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e80b40 0x561584e7f090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8030b50 tx=0x7fa8d8030b80 comp rx=0 tx=0).stop 2024-09-19T04:22:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.438+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3593379845 shutdown_connections 2024-09-19T04:22:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.438+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e8a370 0x561584e8c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.438+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.438+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3593379845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e80b40 0x561584e7f090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.438+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3593379845 >> 172.21.15.94:0/3593379845 conn(0x561584e48310 msgr2=0x561584e7bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.438+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3593379845 shutdown_connections 2024-09-19T04:22:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.439+0000 7fa8e5517640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8c80a3500 0x7fa8c80a59c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x561584e7cac0 tx=0x7fa8d0011d10 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.439+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3593379845 wait complete. 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.439+0000 7fa8e9b54740 1 -- start start 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.439+0000 7fa8e9b54740 1 Processor -- start 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.440+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e89db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.440+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e87060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.440+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 0x561584e877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e9b54740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561584e8d010 con 0x561584e80b40 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e9b54740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561584da0060 con 0x561584e7f5d0 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e9b54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561584e8d190 con 0x561584e8a370 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e9b54740 10 client.0 _ll_register_callbacks cb 0x561584af38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e5517640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e89db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e4d16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e87060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e5d18640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 0x561584e877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e5517640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e89db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44954/0 (socket says 172.21.15.94:44954) 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e4d16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e87060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52320/0 (socket says 172.21.15.94:52320) 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e5517640 1 -- 172.21.15.94:0/3998936438 learned_addr learned my addr 172.21.15.94:0/3998936438 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e5517640 1 -- 172.21.15.94:0/3998936438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 msgr2=0x561584e877f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e5517640 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 0x561584e877f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e5517640 1 -- 172.21.15.94:0/3998936438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 msgr2=0x561584e87060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e5517640 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e87060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.441+0000 7fa8e5517640 1 -- 172.21.15.94:0/3998936438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561584b563c0 con 0x561584e7f5d0 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.442+0000 7fa8e5517640 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e89db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000fc30 tx=0x7fa8d0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0270e29c78c05f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.442+0000 7fa8c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.442+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/3998936438 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d001e420 con 0x561584e7f5d0 2024-09-19T04:22:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.442+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/3998936438 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fa8d001e5c0 con 0x561584e7f5d0 2024-09-19T04:22:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.442+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3998936438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d8030e10 con 0x561584e7f5d0 2024-09-19T04:22:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.442+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/3998936438 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa8d001e7c0 con 0x561584e7f5d0 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.442+0000 7fa8e9b54740 10 client.25910 fetch_fsmap learned FSMap version 42 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.442+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3998936438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561584e98ad0 con 0x561584e7f5d0 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.442+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/3998936438 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8d8030e10 con 0x561584e7f5d0 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.443+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/3998936438 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fa8d00269e0 con 0x561584e7f5d0 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.443+0000 7fa8e9b54740 10 client.25910 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.443+0000 7fa8e9b54740 10 client.25910 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.443+0000 7fa8e9b54740 20 client.25910 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.443+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561584e99f90 0x561584e9c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.443+0000 7fa8e9b54740 4 client.25910 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.443+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3998936438 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561584df2860 con 0x561584e99f90 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.443+0000 7fa8e4d16640 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561584e99f90 0x561584e9c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.444+0000 7fa8c4ff9640 10 client.25910 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.444+0000 7fa8e4d16640 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561584e99f90 0x561584e9c380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfb96b08ed61f12a server_cookie=3575ab40aaa67ded in_seq=0 out_seq=0 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/3998936438 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561584df2860 con 0x561584e99f90 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8c4ff9640 10 client.25910 handle_command_reply: tid=0 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8e9b54740 1 client.25910 shutdown 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3998936438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 msgr2=0x561584e89db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e89db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000fc30 tx=0x7fa8d0004a40 comp rx=0 tx=0).stop 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3998936438 shutdown_connections 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561584e99f90 0x561584e9c380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 0x561584e877f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8c4ff9640 0 client.25910 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e87060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/3998936438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e89db0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.445+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3998936438 >> 172.21.15.94:0/3998936438 conn(0x561584e48310 msgr2=0x561584e81960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.446+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3998936438 shutdown_connections 2024-09-19T04:22:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.446+0000 7fa8e9b54740 1 -- 172.21.15.94:0/3998936438 wait complete. 2024-09-19T04:22:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.446+0000 7fa8e9b54740 20 client.25910 trim_cache size 0 max 16384 2024-09-19T04:22:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.469+0000 7fa8e9b54740 1 Processor -- start 2024-09-19T04:22:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.469+0000 7fa8e9b54740 1 -- start start 2024-09-19T04:22:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.469+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e9c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.470+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e7ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.470+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 0x561584ea1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.470+0000 7fa8e5517640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e9c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.470+0000 7fa8e5517640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e9c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44962/0 (socket says 172.21.15.94:44962) 2024-09-19T04:22:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.470+0000 7fa8e4d16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e7ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.470+0000 7fa8e4d16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e7ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52322/0 (socket says 172.21.15.94:52322) 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.470+0000 7fa8e5517640 1 -- 172.21.15.94:0/774790565 learned_addr learned my addr 172.21.15.94:0/774790565 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.470+0000 7fa8e9b54740 1 -- 172.21.15.94:0/774790565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e01bef70 con 0x561584e80b40 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.471+0000 7fa8e5d18640 1 --2- 172.21.15.94:0/774790565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 0x561584ea1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.471+0000 7fa8e9b54740 1 -- 172.21.15.94:0/774790565 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e01bc7e0 con 0x561584e7f5d0 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.471+0000 7fa8e9b54740 1 -- 172.21.15.94:0/774790565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e01ba650 con 0x561584e8a370 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.471+0000 7fa8e5517640 1 -- 172.21.15.94:0/774790565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 msgr2=0x561584ea1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.471+0000 7fa8e5517640 1 --2- 172.21.15.94:0/774790565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 0x561584ea1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.471+0000 7fa8e5517640 1 -- 172.21.15.94:0/774790565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 msgr2=0x561584e7ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.471+0000 7fa8e5517640 1 --2- 172.21.15.94:0/774790565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e7ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.471+0000 7fa8e5517640 1 -- 172.21.15.94:0/774790565 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561584e82120 con 0x561584e7f5d0 2024-09-19T04:22:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.472+0000 7fa8e5517640 1 --2- 172.21.15.94:0/774790565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e9c380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0011b00 tx=0x7fa8d000f780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92b7f133099b33c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.472+0000 7fa8c6ffd640 1 -- 172.21.15.94:0/774790565 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d001e420 con 0x561584e7f5d0 2024-09-19T04:22:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.472+0000 7fa8c6ffd640 1 -- 172.21.15.94:0/774790565 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8d003c070 con 0x561584e7f5d0 2024-09-19T04:22:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.472+0000 7fa8c6ffd640 1 -- 172.21.15.94:0/774790565 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d001edc0 con 0x561584e7f5d0 2024-09-19T04:22:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 -- 172.21.15.94:0/774790565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 msgr2=0x561584e9c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/774790565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e9c380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0011b00 tx=0x7fa8d000f780 comp rx=0 tx=0).stop 2024-09-19T04:22:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 -- 172.21.15.94:0/774790565 shutdown_connections 2024-09-19T04:22:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/774790565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e8a370 0x561584ea1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/774790565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e7ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/774790565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e7f5d0 0x561584e9c380 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 -- 172.21.15.94:0/774790565 >> 172.21.15.94:0/774790565 conn(0x561584e48310 msgr2=0x561584e7a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 -- 172.21.15.94:0/774790565 shutdown_connections 2024-09-19T04:22:55.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 -- 172.21.15.94:0/774790565 wait complete. 2024-09-19T04:22:55.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 -- start start 2024-09-19T04:22:55.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.473+0000 7fa8e9b54740 1 Processor -- start 2024-09-19T04:22:55.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e7f5d0 0x561584e92600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e92b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e9b54740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e8a370 0x561584e8f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e9b54740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e01baca0 con 0x561584e80b40 2024-09-19T04:22:55.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e9b54740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e01bc110 con 0x561584e8a370 2024-09-19T04:22:55.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e9b54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e01bb370 con 0x561584e7f5d0 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e9b54740 10 client.0 _ll_register_callbacks cb 0x7fa8d00118b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e5517640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e7f5d0 0x561584e92600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e5517640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e7f5d0 0x561584e92600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48896/0 (socket says 172.21.15.94:48896) 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e5517640 1 -- 172.21.15.94:0/501825843 learned_addr learned my addr 172.21.15.94:0/501825843 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e5d18640 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e8a370 0x561584e8f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.474+0000 7fa8e4d16640 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e92b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.475+0000 7fa8e5d18640 1 -- 172.21.15.94:0/501825843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e7f5d0 msgr2=0x561584e92600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.475+0000 7fa8e5d18640 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e7f5d0 0x561584e92600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.475+0000 7fa8e5d18640 1 -- 172.21.15.94:0/501825843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 msgr2=0x561584e92b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.475+0000 7fa8e5d18640 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e92b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.475+0000 7fa8e5d18640 1 -- 172.21.15.94:0/501825843 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561584e8fc20 con 0x561584e8a370 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.475+0000 7fa8e5d18640 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e8a370 0x561584e8f3f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005940 tx=0x7fa8d8005570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9065460d9052415 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.475+0000 7fa8c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/501825843 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8056060 con 0x561584e8a370 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/501825843 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fa8d80046c0 con 0x561584e8a370 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8e9b54740 1 -- 172.21.15.94:0/501825843 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d003c070 con 0x561584e8a370 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/501825843 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa8d80048c0 con 0x561584e8a370 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8e9b54740 10 client.25916 fetch_fsmap learned FSMap version 42 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8e9b54740 1 -- 172.21.15.94:0/501825843 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561584f05680 con 0x561584e8a370 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/501825843 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8d8051460 con 0x561584e8a370 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/501825843 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fa8d8046430 con 0x561584e8a370 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8e9b54740 10 client.25916 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:22:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.476+0000 7fa8e9b54740 10 client.25916 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:22:55.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.477+0000 7fa8e9b54740 20 client.25916 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:55.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.477+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561584e99f90 0x561584f078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:55.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.477+0000 7fa8e9b54740 4 client.25916 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:55.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.477+0000 7fa8e9b54740 1 -- 172.21.15.94:0/501825843 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa8d003c070 con 0x561584e99f90 2024-09-19T04:22:55.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.477+0000 7fa8e5517640 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561584e99f90 0x561584f078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:55.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.477+0000 7fa8c4ff9640 10 client.25916 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:22:55.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.478+0000 7fa8e5517640 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561584e99f90 0x561584f078c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c083f3b7abdeaf81 server_cookie=346179ce05825893 in_seq=0 out_seq=0 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.478+0000 7fa8c4ff9640 1 -- 172.21.15.94:0/501825843 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa8d003c070 con 0x561584e99f90 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.478+0000 7fa8c4ff9640 10 client.25916 handle_command_reply: tid=0 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.478+0000 7fa8e9b54740 1 client.25916 shutdown 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.479+0000 7fa8e9b54740 1 -- 172.21.15.94:0/501825843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e8a370 msgr2=0x561584e8f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.479+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e8a370 0x561584e8f3f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005940 tx=0x7fa8d8005570 comp rx=0 tx=0).stop 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.479+0000 7fa8e9b54740 1 -- 172.21.15.94:0/501825843 shutdown_connections 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.479+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x561584e99f90 0x561584f078c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.479+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561584e8a370 0x561584e8f3f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.479+0000 7fa8c4ff9640 0 client.25916 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.479+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561584e80b40 0x561584e92b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.479+0000 7fa8e9b54740 1 --2- 172.21.15.94:0/501825843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561584e7f5d0 0x561584e92600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.479+0000 7fa8e9b54740 1 -- 172.21.15.94:0/501825843 >> 172.21.15.94:0/501825843 conn(0x561584e48310 msgr2=0x561584e7ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.480+0000 7fa8e9b54740 1 -- 172.21.15.94:0/501825843 shutdown_connections 2024-09-19T04:22:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.480+0000 7fa8e9b54740 1 -- 172.21.15.94:0/501825843 wait complete. 2024-09-19T04:22:55.485 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:22:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.480+0000 7fa8e9b54740 20 client.25916 trim_cache size 0 max 16384 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 -- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8c80a3500 msgr2=0x7fa8c80a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8c80a3500 0x7fa8c80a59c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x561584e7cac0 tx=0x7fa8d0011d10 comp rx=0 tx=0).stop 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 -- 172.21.15.94:0/219533950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102570 msgr2=0x7fa8e01a76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102570 0x7fa8e01a76f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40097d0 tx=0x7fa8d4009c90 comp rx=0 tx=0).stop 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 -- 172.21.15.94:0/219533950 shutdown_connections 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8c80a3500 0x7fa8c80a59c0 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x561584e7cac0 tx=0x7fa8d0011d10 comp rx=0 tx=0).stop 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0102ef0 0x7fa8e01aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8e0102570 0x7fa8e01a76f0 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40097d0 tx=0x7fa8d4009c90 comp rx=0 tx=0).stop 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 --2- 172.21.15.94:0/219533950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e0100e20 0x7fa8e01a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.481+0000 7fa8e777b640 1 -- 172.21.15.94:0/219533950 >> 172.21.15.94:0/219533950 conn(0x7fa8e00fc7d0 msgr2=0x7fa8e0106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.482+0000 7fa8e777b640 1 -- 172.21.15.94:0/219533950 shutdown_connections 2024-09-19T04:22:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:55.482+0000 7fa8e777b640 1 -- 172.21.15.94:0/219533950 wait complete. 2024-09-19T04:22:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 400.0 s, 1429.6 tps, lat 69.740 ms stddev 21.399 2024-09-19T04:22:56.350 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:22:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:56.925 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:56 smithi094 ceph-mon[30972]: pgmap v242: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-19T04:22:57.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:56 smithi112 ceph-mon[35051]: pgmap v242: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-19T04:22:57.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:56 smithi104 ceph-mon[34145]: pgmap v242: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-19T04:22:57.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.349+0000 7f4cc3fff640 1 -- 172.21.15.94:0/1302715358 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc031880 con 0x7f4ccc1083f0 2024-09-19T04:22:57.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.350+0000 7f4cd3a60640 1 -- 172.21.15.94:0/1302715358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ccc1083f0 msgr2=0x7f4ccc1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.350+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/1302715358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ccc1083f0 0x7f4ccc1087d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009900 tx=0x7f4cbc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.350+0000 7f4cd3a60640 1 -- 172.21.15.94:0/1302715358 shutdown_connections 2024-09-19T04:22:57.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.350+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/1302715358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ccc10d520 0x7f4ccc1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.350+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/1302715358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ccc108da0 0x7f4ccc10cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.350+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/1302715358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ccc1083f0 0x7f4ccc1087d0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009900 tx=0x7f4cbc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.350+0000 7f4cd3a60640 1 -- 172.21.15.94:0/1302715358 >> 172.21.15.94:0/1302715358 conn(0x7f4ccc077af0 msgr2=0x7f4ccc079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.351+0000 7f4cd3a60640 1 -- 172.21.15.94:0/1302715358 shutdown_connections 2024-09-19T04:22:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.351+0000 7f4cd3a60640 1 -- 172.21.15.94:0/1302715358 wait complete. 2024-09-19T04:22:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.351+0000 7f4cd3a60640 1 Processor -- start 2024-09-19T04:22:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd3a60640 1 -- start start 2024-09-19T04:22:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd3a60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ccc108da0 0x7f4ccc082ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd3a60640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ccc10d520 0x7f4ccc07bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd3a60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ccc07c170 0x7f4ccc07c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd3a60640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc118440 con 0x7f4ccc07c170 2024-09-19T04:22:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd3a60640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc1182c0 con 0x7f4ccc10d520 2024-09-19T04:22:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd3a60640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc1185c0 con 0x7f4ccc108da0 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd1ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ccc07c170 0x7f4ccc07c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd17fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ccc108da0 0x7f4ccc082ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd1ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ccc07c170 0x7f4ccc07c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52346/0 (socket says 172.21.15.94:52346) 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd0ffb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ccc10d520 0x7f4ccc07bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd1ffd640 1 -- 172.21.15.94:0/2644698161 learned_addr learned my addr 172.21.15.94:0/2644698161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.352+0000 7f4cd17fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ccc108da0 0x7f4ccc082ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48914/0 (socket says 172.21.15.94:48914) 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.353+0000 7f4cd1ffd640 1 -- 172.21.15.94:0/2644698161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ccc108da0 msgr2=0x7f4ccc082ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.353+0000 7f4cd1ffd640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ccc108da0 0x7f4ccc082ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.353+0000 7f4cd1ffd640 1 -- 172.21.15.94:0/2644698161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ccc10d520 msgr2=0x7f4ccc07bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.353+0000 7f4cd1ffd640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ccc10d520 0x7f4ccc07bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.353+0000 7f4cd1ffd640 1 -- 172.21.15.94:0/2644698161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc080890 con 0x7f4ccc07c170 2024-09-19T04:22:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.353+0000 7f4cd0ffb640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ccc10d520 0x7f4ccc07bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.354+0000 7f4cd1ffd640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ccc07c170 0x7f4ccc07c620 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400eb10 tx=0x7f4cc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.355+0000 7f4cc27fc640 1 -- 172.21.15.94:0/2644698161 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4004b20 con 0x7f4ccc07c170 2024-09-19T04:22:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.355+0000 7f4cd3a60640 1 -- 172.21.15.94:0/2644698161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ccc080b80 con 0x7f4ccc07c170 2024-09-19T04:22:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.355+0000 7f4cd3a60640 1 -- 172.21.15.94:0/2644698161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ccc0810c0 con 0x7f4ccc07c170 2024-09-19T04:22:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.355+0000 7f4cc27fc640 1 -- 172.21.15.94:0/2644698161 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc4004cc0 con 0x7f4ccc07c170 2024-09-19T04:22:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.355+0000 7f4cc27fc640 1 -- 172.21.15.94:0/2644698161 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4010690 con 0x7f4ccc07c170 2024-09-19T04:22:57.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.359+0000 7f4cd5e39740 1 Processor -- start 2024-09-19T04:22:57.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.360+0000 7f4cc27fc640 1 -- 172.21.15.94:0/2644698161 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4cc4005020 con 0x7f4ccc07c170 2024-09-19T04:22:57.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.361+0000 7f4cc27fc640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4cb80a3500 0x7f4cb80a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.362+0000 7f4cc27fc640 1 -- 172.21.15.94:0/2644698161 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f4cc40d0140 con 0x7f4ccc07c170 2024-09-19T04:22:57.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.362+0000 7f4cd5e39740 1 -- start start 2024-09-19T04:22:57.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.362+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acb4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.362+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb45d0 0x55fd9acb4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.362+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acbf370 0x55fd9acc1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.362+0000 7f4cd5e39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fd9ac1a2f0 con 0x55fd9acb5b40 2024-09-19T04:22:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.362+0000 7f4cd5e39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fd9abc1cf0 con 0x55fd9acbf370 2024-09-19T04:22:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd5e39740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd9ac162b0 con 0x55fd9acb45d0 2024-09-19T04:22:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd0ffb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb45d0 0x55fd9acb4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd0ffb640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4cb80a3500 0x7f4cb80a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd0ffb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb45d0 0x55fd9acb4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48920/0 (socket says 172.21.15.94:48920) 2024-09-19T04:22:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd0ffb640 1 -- 172.21.15.94:0/721708800 learned_addr learned my addr 172.21.15.94:0/721708800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd17fc640 1 --2- 172.21.15.94:0/721708800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acb4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd0ffb640 1 -- 172.21.15.94:0/721708800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acbf370 msgr2=0x55fd9acc1760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:22:57.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd0ffb640 1 --2- 172.21.15.94:0/721708800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acbf370 0x55fd9acc1760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd0ffb640 1 -- 172.21.15.94:0/721708800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 msgr2=0x55fd9acb4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd0ffb640 1 --2- 172.21.15.94:0/721708800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acb4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.363+0000 7f4cd0ffb640 1 -- 172.21.15.94:0/721708800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd9aba63a0 con 0x55fd9acb45d0 2024-09-19T04:22:57.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.364+0000 7f4cd0ffb640 1 --2- 172.21.15.94:0/721708800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb45d0 0x55fd9acb4a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8013b10 tx=0x7f4cc8010370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a6bf71ee2aaa8b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.364+0000 7f4cd0ffb640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4cb80a3500 0x7f4cb80a59c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80058c0 tx=0x7f4cc8009000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.364+0000 7f4cb77fe640 1 -- 172.21.15.94:0/721708800 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8005220 con 0x55fd9acb45d0 2024-09-19T04:22:57.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.364+0000 7f4cb77fe640 1 -- 172.21.15.94:0/721708800 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc8004c50 con 0x55fd9acb45d0 2024-09-19T04:22:57.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.364+0000 7f4cb77fe640 1 -- 172.21.15.94:0/721708800 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8009760 con 0x55fd9acb45d0 2024-09-19T04:22:57.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.365+0000 7f4cd5e39740 1 -- 172.21.15.94:0/721708800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb45d0 msgr2=0x55fd9acb4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.365+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/721708800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb45d0 0x55fd9acb4a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8013b10 tx=0x7f4cc8010370 comp rx=0 tx=0).stop 2024-09-19T04:22:57.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.365+0000 7f4cd5e39740 1 -- 172.21.15.94:0/721708800 shutdown_connections 2024-09-19T04:22:57.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.365+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/721708800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acbf370 0x55fd9acc1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.365+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/721708800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb45d0 0x55fd9acb4a10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.365+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/721708800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acb4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.365+0000 7f4cd5e39740 1 -- 172.21.15.94:0/721708800 >> 172.21.15.94:0/721708800 conn(0x55fd9ac7d310 msgr2=0x55fd9acb0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.365+0000 7f4cd5e39740 1 -- 172.21.15.94:0/721708800 shutdown_connections 2024-09-19T04:22:57.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.365+0000 7f4cd5e39740 1 -- 172.21.15.94:0/721708800 wait complete. 2024-09-19T04:22:57.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.366+0000 7f4cd5e39740 1 -- start start 2024-09-19T04:22:57.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.366+0000 7f4cd5e39740 1 Processor -- start 2024-09-19T04:22:57.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.367+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acbede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.367+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acbc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.367+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acbc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.367+0000 7f4cd5e39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fd9acc2010 con 0x55fd9acb5b40 2024-09-19T04:22:57.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.367+0000 7f4cd5e39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fd9abd5060 con 0x55fd9acb45d0 2024-09-19T04:22:57.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.367+0000 7f4cd5e39740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd9acc2190 con 0x55fd9acbf370 2024-09-19T04:22:57.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.367+0000 7f4cd5e39740 10 client.0 _ll_register_callbacks cb 0x55fd9a9288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:57.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.368+0000 7f4cd0ffb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acbc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.368+0000 7f4cd17fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acbede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.368+0000 7f4cd0ffb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acbc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52356/0 (socket says 172.21.15.94:52356) 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.368+0000 7f4cd1ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acbc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.368+0000 7f4cd0ffb640 1 -- 172.21.15.94:0/3290206109 learned_addr learned my addr 172.21.15.94:0/3290206109 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cd1ffd640 1 -- 172.21.15.94:0/3290206109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 msgr2=0x55fd9acbede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cd1ffd640 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acbede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cd1ffd640 1 -- 172.21.15.94:0/3290206109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 msgr2=0x55fd9acbc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cd1ffd640 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acbc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cd1ffd640 1 -- 172.21.15.94:0/3290206109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd9a98b3c0 con 0x55fd9acbf370 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cd1ffd640 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acbc7d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40cd850 tx=0x7f4cc409bfe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d55df9bcc3f952fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/3290206109 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc409c3e0 con 0x55fd9acbf370 2024-09-19T04:22:57.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/3290206109 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f4cc40a4bc0 con 0x55fd9acbf370 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3290206109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc8004c50 con 0x55fd9acbf370 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/3290206109 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4cc40a0790 con 0x55fd9acbf370 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cd5e39740 10 client.25918 fetch_fsmap learned FSMap version 42 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.369+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3290206109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fd9accdbd0 con 0x55fd9acbf370 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.370+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/3290206109 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc40a5050 con 0x55fd9acbf370 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.370+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/3290206109 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f4cc4005310 con 0x55fd9acbf370 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.370+0000 7f4cd5e39740 10 client.25918 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.370+0000 7f4cd5e39740 10 client.25918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.370+0000 7f4cd5e39740 20 client.25918 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:57.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.370+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55fd9accf130 0x55fd9acd1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.370+0000 7f4cd5e39740 4 client.25918 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:57.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.370+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3290206109 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4cc8004c50 con 0x55fd9accf130 2024-09-19T04:22:57.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.370+0000 7f4cd17fc640 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55fd9accf130 0x55fd9acd1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.371+0000 7f4cb4ff9640 10 client.25918 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:22:57.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.371+0000 7f4cd17fc640 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55fd9accf130 0x55fd9acd1520 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17d33d3ffb67ce97 server_cookie=c4b5dda6ed019fe4 in_seq=0 out_seq=0 2024-09-19T04:22:57.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/3290206109 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4cc8004c50 con 0x55fd9accf130 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cb4ff9640 10 client.25918 handle_command_reply: tid=0 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cd5e39740 1 client.25918 shutdown 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3290206109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 msgr2=0x55fd9acbc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acbc7d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40cd850 tx=0x7f4cc409bfe0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3290206109 shutdown_connections 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55fd9accf130 0x55fd9acd1520 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acbc7d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acbc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3290206109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acbede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3290206109 >> 172.21.15.94:0/3290206109 conn(0x55fd9ac7d310 msgr2=0x55fd9acb6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.373+0000 7f4cb4ff9640 0 client.25918 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.374+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3290206109 shutdown_connections 2024-09-19T04:22:57.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.374+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3290206109 wait complete. 2024-09-19T04:22:57.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.374+0000 7f4cd5e39740 20 client.25918 trim_cache size 0 max 16384 2024-09-19T04:22:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.398+0000 7f4cd5e39740 1 Processor -- start 2024-09-19T04:22:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.398+0000 7f4cd5e39740 1 -- start start 2024-09-19T04:22:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.398+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9accb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.398+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acb7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.398+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acb7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.398+0000 7f4cd5e39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb0011060 con 0x55fd9acb5b40 2024-09-19T04:22:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.398+0000 7f4cd5e39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb000e8d0 con 0x55fd9acb45d0 2024-09-19T04:22:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.398+0000 7f4cd5e39740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb000c740 con 0x55fd9acbf370 2024-09-19T04:22:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd1ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acb7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd17fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9accb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd1ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acb7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48938/0 (socket says 172.21.15.94:48938) 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd1ffd640 1 -- 172.21.15.94:0/3795361819 learned_addr learned my addr 172.21.15.94:0/3795361819 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd17fc640 1 -- 172.21.15.94:0/3795361819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 msgr2=0x55fd9acb7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd17fc640 1 --2- 172.21.15.94:0/3795361819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acb7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd17fc640 1 -- 172.21.15.94:0/3795361819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 msgr2=0x55fd9acb7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd17fc640 1 --2- 172.21.15.94:0/3795361819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acb7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd17fc640 1 -- 172.21.15.94:0/3795361819 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd9accb880 con 0x55fd9acb45d0 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.399+0000 7f4cd17fc640 1 --2- 172.21.15.94:0/3795361819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9accb340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc109620 tx=0x7f4cbc005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb4369f180bc9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.400+0000 7f4cb6ffd640 1 -- 172.21.15.94:0/3795361819 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc04a070 con 0x55fd9acb45d0 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.400+0000 7f4cb6ffd640 1 -- 172.21.15.94:0/3795361819 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cbc0044a0 con 0x55fd9acb45d0 2024-09-19T04:22:57.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.400+0000 7f4cb6ffd640 1 -- 172.21.15.94:0/3795361819 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc035040 con 0x55fd9acb45d0 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.401+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3795361819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 msgr2=0x55fd9accb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.401+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3795361819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9accb340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc109620 tx=0x7f4cbc005a20 comp rx=0 tx=0).stop 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.401+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3795361819 shutdown_connections 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.401+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3795361819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acbf370 0x55fd9acb7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.401+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3795361819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acb5b40 0x55fd9acb7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.401+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/3795361819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9accb340 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.401+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3795361819 >> 172.21.15.94:0/3795361819 conn(0x55fd9ac7d310 msgr2=0x55fd9acbe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.402+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3795361819 shutdown_connections 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.402+0000 7f4cd5e39740 1 -- 172.21.15.94:0/3795361819 wait complete. 2024-09-19T04:22:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.402+0000 7f4cd5e39740 1 -- start start 2024-09-19T04:22:57.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.403+0000 7f4cd5e39740 1 Processor -- start 2024-09-19T04:22:57.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.403+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acb90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd5e39740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb5b40 0x55fd9acb9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd17fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acb90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd17fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acb90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:45036/0 (socket says 172.21.15.94:45036) 2024-09-19T04:22:57.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd0ffb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb5b40 0x55fd9acb9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd0ffb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb5b40 0x55fd9acb9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48940/0 (socket says 172.21.15.94:48940) 2024-09-19T04:22:57.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd17fc640 1 -- 172.21.15.94:0/2297380356 learned_addr learned my addr 172.21.15.94:0/2297380356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:57.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acbf370 0x55fd9acb9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb000cd90 con 0x55fd9acbf370 2024-09-19T04:22:57.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd1ffd640 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acbf370 0x55fd9acb9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb000e200 con 0x55fd9acb45d0 2024-09-19T04:22:57.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb000d460 con 0x55fd9acb5b40 2024-09-19T04:22:57.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.404+0000 7f4cd5e39740 10 client.0 _ll_register_callbacks cb 0x7f4cc4096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:57.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.405+0000 7f4cd17fc640 1 -- 172.21.15.94:0/2297380356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb5b40 msgr2=0x55fd9acb9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.405+0000 7f4cd17fc640 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb5b40 0x55fd9acb9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.405+0000 7f4cd17fc640 1 -- 172.21.15.94:0/2297380356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acbf370 msgr2=0x55fd9acb9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.405+0000 7f4cd17fc640 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acbf370 0x55fd9acb9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.405+0000 7f4cd17fc640 1 -- 172.21.15.94:0/2297380356 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd9ad38dd0 con 0x55fd9acb45d0 2024-09-19T04:22:57.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.405+0000 7f4cd17fc640 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acb90f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc005070 tx=0x7f4cbc0050a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7eb1d7d3a112cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.405+0000 7f4cd0ffb640 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb5b40 0x55fd9acb9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:57.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.405+0000 7f4cb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:57.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.406+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/2297380356 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc053060 con 0x55fd9acb45d0 2024-09-19T04:22:57.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.406+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/2297380356 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7f4cbc005440 con 0x55fd9acb45d0 2024-09-19T04:22:57.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.406+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cbc0044a0 con 0x55fd9acb45d0 2024-09-19T04:22:57.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.407+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/2297380356 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4cbc035070 con 0x55fd9acb45d0 2024-09-19T04:22:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.407+0000 7f4cd5e39740 10 client.25928 fetch_fsmap learned FSMap version 42 2024-09-19T04:22:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.407+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fd9ad3a0b0 con 0x55fd9acb45d0 2024-09-19T04:22:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.407+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/2297380356 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cbc0044a0 con 0x55fd9acb45d0 2024-09-19T04:22:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.407+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/2297380356 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f4cbc04d030 con 0x55fd9acb45d0 2024-09-19T04:22:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.407+0000 7f4cd5e39740 10 client.25928 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:22:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.407+0000 7f4cd5e39740 10 client.25928 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:22:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.407+0000 7f4cd5e39740 20 client.25928 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.407+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55fd9ad3a6e0 0x55fd9ad3cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.408+0000 7f4cd5e39740 4 client.25928 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.408+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd9ac27860 con 0x55fd9ad3a6e0 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.408+0000 7f4cd0ffb640 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55fd9ad3a6e0 0x55fd9ad3cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.408+0000 7f4cb4ff9640 10 client.25928 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.409+0000 7f4cd0ffb640 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55fd9ad3a6e0 0x55fd9ad3cb00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=574febe0a2cfb233 server_cookie=9534c45d21be7ce in_seq=0 out_seq=0 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.409+0000 7f4cb4ff9640 1 -- 172.21.15.94:0/2297380356 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd9ac27860 con 0x55fd9ad3a6e0 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.409+0000 7f4cb4ff9640 10 client.25928 handle_command_reply: tid=0 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.409+0000 7f4cd5e39740 1 client.25928 shutdown 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 msgr2=0x55fd9acb90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acb90f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc005070 tx=0x7f4cbc0050a0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 shutdown_connections 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55fd9ad3a6e0 0x55fd9ad3cb00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd9acbf370 0x55fd9acb9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd9acb5b40 0x55fd9acb9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 --2- 172.21.15.94:0/2297380356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd9acb45d0 0x55fd9acb90f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 >> 172.21.15.94:0/2297380356 conn(0x55fd9ac7d310 msgr2=0x55fd9acba850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cb4ff9640 0 client.25928 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:22:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 shutdown_connections 2024-09-19T04:22:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 1 -- 172.21.15.94:0/2297380356 wait complete. 2024-09-19T04:22:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.410+0000 7f4cd5e39740 20 client.25928 trim_cache size 0 max 16384 2024-09-19T04:22:57.415 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:22:57.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 -- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4cb80a3500 msgr2=0x7f4cb80a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4cb80a3500 0x7f4cb80a59c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80058c0 tx=0x7f4cc8009000 comp rx=0 tx=0).stop 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 -- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ccc07c170 msgr2=0x7f4ccc07c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ccc07c170 0x7f4ccc07c620 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400eb10 tx=0x7f4cc400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 -- 172.21.15.94:0/2644698161 shutdown_connections 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4cb80a3500 0x7f4cb80a59c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ccc07c170 0x7f4ccc07c620 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ccc10d520 0x7f4ccc07bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 --2- 172.21.15.94:0/2644698161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ccc108da0 0x7f4ccc082ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.411+0000 7f4cd3a60640 1 -- 172.21.15.94:0/2644698161 >> 172.21.15.94:0/2644698161 conn(0x7f4ccc077af0 msgr2=0x7f4ccc078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.412+0000 7f4cd3a60640 1 -- 172.21.15.94:0/2644698161 shutdown_connections 2024-09-19T04:22:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:57.412+0000 7f4cd3a60640 1 -- 172.21.15.94:0/2644698161 wait complete. 2024-09-19T04:22:58.132 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:22:58.546 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:58.992 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:22:58 smithi094 ceph-mon[30972]: pgmap v243: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 16 MiB/s wr, 1.77k op/s 2024-09-19T04:22:59.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:22:58 smithi112 ceph-mon[35051]: pgmap v243: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 16 MiB/s wr, 1.77k op/s 2024-09-19T04:22:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:22:58 smithi104 ceph-mon[34145]: pgmap v243: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 16 MiB/s wr, 1.77k op/s 2024-09-19T04:22:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.124+0000 7fd0aedfa640 1 -- 172.21.15.94:0/1230879688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8108c30 msgr2=0x7fd0a810cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.124+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/1230879688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8108c30 0x7fd0a810cc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd098009900 tx=0x7fd09802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.124+0000 7fd0aedfa640 1 -- 172.21.15.94:0/1230879688 shutdown_connections 2024-09-19T04:22:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.124+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/1230879688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a810d360 0x7fd0a8114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.124+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/1230879688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8108c30 0x7fd0a810cc30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.124+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/1230879688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8108280 0x7fd0a8108660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.124+0000 7fd0aedfa640 1 -- 172.21.15.94:0/1230879688 >> 172.21.15.94:0/1230879688 conn(0x7fd0a80779f0 msgr2=0x7fd0a8079e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.124+0000 7fd0aedfa640 1 -- 172.21.15.94:0/1230879688 shutdown_connections 2024-09-19T04:22:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.124+0000 7fd0aedfa640 1 -- 172.21.15.94:0/1230879688 wait complete. 2024-09-19T04:22:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.125+0000 7fd0aedfa640 1 Processor -- start 2024-09-19T04:22:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0aedfa640 1 -- start start 2024-09-19T04:22:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0aedfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8108280 0x7fd0a81a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0aedfa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8108c30 0x7fd0a81a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0aedfa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a810d360 0x7fd0a819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0aedfa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8118260 con 0x7fd0a8108280 2024-09-19T04:22:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0aedfa640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81180e0 con 0x7fd0a8108c30 2024-09-19T04:22:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0aedfa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81183e0 con 0x7fd0a810d360 2024-09-19T04:22:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0ad5f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8108c30 0x7fd0a81a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0ad5f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8108c30 0x7fd0a81a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:45046/0 (socket says 172.21.15.94:45046) 2024-09-19T04:22:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.126+0000 7fd0ad5f7640 1 -- 172.21.15.94:0/2030069377 learned_addr learned my addr 172.21.15.94:0/2030069377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.127+0000 7fd0addf8640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8108280 0x7fd0a81a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.128+0000 7fd0addf8640 1 -- 172.21.15.94:0/2030069377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a810d360 msgr2=0x7fd0a819e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:22:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.128+0000 7fd0addf8640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a810d360 0x7fd0a819e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.128+0000 7fd0addf8640 1 -- 172.21.15.94:0/2030069377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8108c30 msgr2=0x7fd0a81a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.128+0000 7fd0addf8640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8108c30 0x7fd0a81a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.128+0000 7fd0addf8640 1 -- 172.21.15.94:0/2030069377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a819ece0 con 0x7fd0a8108280 2024-09-19T04:22:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.128+0000 7fd0addf8640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8108280 0x7fd0a81a43f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400e9f0 tx=0x7fd0a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.129+0000 7fd09effd640 1 -- 172.21.15.94:0/2030069377 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400ccc0 con 0x7fd0a8108280 2024-09-19T04:22:59.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.129+0000 7fd09effd640 1 -- 172.21.15.94:0/2030069377 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a40108e0 con 0x7fd0a8108280 2024-09-19T04:22:59.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.129+0000 7fd0aedfa640 1 -- 172.21.15.94:0/2030069377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a819efd0 con 0x7fd0a8108280 2024-09-19T04:22:59.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.130+0000 7fd0aedfa640 1 -- 172.21.15.94:0/2030069377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81ab2c0 con 0x7fd0a8108280 2024-09-19T04:22:59.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.130+0000 7fd09effd640 1 -- 172.21.15.94:0/2030069377 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40188e0 con 0x7fd0a8108280 2024-09-19T04:22:59.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.133+0000 7fd0b2435740 1 Processor -- start 2024-09-19T04:22:59.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.133+0000 7fd0b2435740 1 -- start start 2024-09-19T04:22:59.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.133+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa2cf0 0x558992fa1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.133+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fa1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.133+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fac520 0x558992fae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.133+0000 7fd0b2435740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558992f072f0 con 0x558992fac520 2024-09-19T04:22:59.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.133+0000 7fd0b2435740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558992eaecf0 con 0x558992fa1780 2024-09-19T04:22:59.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.133+0000 7fd0b2435740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558992f032b0 con 0x558992fa2cf0 2024-09-19T04:22:59.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.134+0000 7fd0ae5f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa2cf0 0x558992fa1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.134+0000 7fd0ae5f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa2cf0 0x558992fa1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48952/0 (socket says 172.21.15.94:48952) 2024-09-19T04:22:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.134+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/2879424457 learned_addr learned my addr 172.21.15.94:0/2879424457 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.134+0000 7fd09effd640 1 -- 172.21.15.94:0/2030069377 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd0a4010a90 con 0x7fd0a8108280 2024-09-19T04:22:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.134+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/2879424457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 msgr2=0x558992fa1bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:22:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.134+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/2879424457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fa1bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.134+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/2879424457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fac520 msgr2=0x558992fae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.134+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/2879424457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fac520 0x558992fae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.134+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/2879424457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558992e933a0 con 0x558992fa2cf0 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.135+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/2879424457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa2cf0 0x558992fa1240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009db0 tx=0x7fd0a000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5688d7ff0a534dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.135+0000 7fd09effd640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a3540 0x7fd0940a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.135+0000 7fd09effd640 1 -- 172.21.15.94:0/2030069377 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fd0a40d0060 con 0x7fd0a8108280 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.135+0000 7fd093fff640 1 -- 172.21.15.94:0/2879424457 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0004b20 con 0x558992fa2cf0 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.135+0000 7fd093fff640 1 -- 172.21.15.94:0/2879424457 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0004cc0 con 0x558992fa2cf0 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.136+0000 7fd0addf8640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a3540 0x7fd0940a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.136+0000 7fd093fff640 1 -- 172.21.15.94:0/2879424457 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0010720 con 0x558992fa2cf0 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.137+0000 7fd0b2435740 1 -- 172.21.15.94:0/2879424457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa2cf0 msgr2=0x558992fa1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.137+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2879424457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa2cf0 0x558992fa1240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009db0 tx=0x7fd0a000e720 comp rx=0 tx=0).stop 2024-09-19T04:22:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.137+0000 7fd0addf8640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a3540 0x7fd0940a5a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x558992f9ec70 tx=0x7fd0a409c160 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.137+0000 7fd0b2435740 1 -- 172.21.15.94:0/2879424457 shutdown_connections 2024-09-19T04:22:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.137+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2879424457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fac520 0x558992fae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.137+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2879424457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fa1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.137+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2879424457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa2cf0 0x558992fa1240 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.137+0000 7fd0b2435740 1 -- 172.21.15.94:0/2879424457 >> 172.21.15.94:0/2879424457 conn(0x558992f6a310 msgr2=0x558992f9de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.137+0000 7fd0b2435740 1 -- 172.21.15.94:0/2879424457 shutdown_connections 2024-09-19T04:22:59.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.138+0000 7fd0b2435740 1 -- 172.21.15.94:0/2879424457 wait complete. 2024-09-19T04:22:59.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.138+0000 7fd0b2435740 1 -- start start 2024-09-19T04:22:59.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.139+0000 7fd0b2435740 1 Processor -- start 2024-09-19T04:22:59.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.139+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fabec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fa2cf0 0x558992fa9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 0x558992fa9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0b2435740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558992faf2c0 con 0x558992fa2cf0 2024-09-19T04:22:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0b2435740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558992ec2060 con 0x558992fa1780 2024-09-19T04:22:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0b2435740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558992faf440 con 0x558992fac520 2024-09-19T04:22:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0b2435740 10 client.0 _ll_register_callbacks cb 0x558992c158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0ae5f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fabec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0ae5f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fabec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:45060/0 (socket says 172.21.15.94:45060) 2024-09-19T04:22:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/2745571467 learned_addr learned my addr 172.21.15.94:0/2745571467 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.140+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 0x558992fa9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.141+0000 7fd0ad5f7640 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fa2cf0 0x558992fa9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.141+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/2745571467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 msgr2=0x558992fa9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.141+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 0x558992fa9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.141+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/2745571467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fa2cf0 msgr2=0x558992fa9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.141+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fa2cf0 0x558992fa9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.141+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/2745571467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558992c783c0 con 0x558992fa1780 2024-09-19T04:22:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.141+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fabec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00059a0 tx=0x7fd0a00055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f68ab65d3f6e5e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.141+0000 7fd0917fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.142+0000 7fd0917fa640 1 -- 172.21.15.94:0/2745571467 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a001c070 con 0x558992fa1780 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.142+0000 7fd0b2435740 1 -- 172.21.15.94:0/2745571467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a0004cc0 con 0x558992fa1780 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.142+0000 7fd0917fa640 1 -- 172.21.15.94:0/2745571467 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fd0a0003d80 con 0x558992fa1780 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.142+0000 7fd0917fa640 1 -- 172.21.15.94:0/2745571467 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd0a0030050 con 0x558992fa1780 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.142+0000 7fd0b2435740 10 client.25937 fetch_fsmap learned FSMap version 42 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.142+0000 7fd0b2435740 1 -- 172.21.15.94:0/2745571467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558992fbad10 con 0x558992fa1780 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.142+0000 7fd0917fa640 1 -- 172.21.15.94:0/2745571467 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0004cc0 con 0x558992fa1780 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.143+0000 7fd0917fa640 1 -- 172.21.15.94:0/2745571467 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd0a00104a0 con 0x558992fa1780 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.143+0000 7fd0b2435740 10 client.25937 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:22:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.143+0000 7fd0b2435740 10 client.25937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.143+0000 7fd0b2435740 20 client.25937 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.143+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558992fbc0c0 0x558992fbe4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.143+0000 7fd0b2435740 4 client.25937 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.143+0000 7fd0b2435740 1 -- 172.21.15.94:0/2745571467 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558992f14860 con 0x558992fbc0c0 2024-09-19T04:22:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.143+0000 7fd0ad5f7640 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558992fbc0c0 0x558992fbe4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.144+0000 7fd0917fa640 10 client.25937 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.144+0000 7fd0ad5f7640 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558992fbc0c0 0x558992fbe4b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a8eab3a0f001e7 server_cookie=7c00c7673e4561dc in_seq=0 out_seq=0 2024-09-19T04:22:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.145+0000 7fd0917fa640 1 -- 172.21.15.94:0/2745571467 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558992f14860 con 0x558992fbc0c0 2024-09-19T04:22:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.145+0000 7fd0917fa640 10 client.25937 handle_command_reply: tid=0 2024-09-19T04:22:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.145+0000 7fd0b2435740 1 client.25937 shutdown 2024-09-19T04:22:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.145+0000 7fd0b2435740 1 -- 172.21.15.94:0/2745571467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 msgr2=0x558992fabec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.145+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fabec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00059a0 tx=0x7fd0a00055f0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.146+0000 7fd0b2435740 1 -- 172.21.15.94:0/2745571467 shutdown_connections 2024-09-19T04:22:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.146+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558992fbc0c0 0x558992fbe4b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.146+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 0x558992fa9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.146+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fa2cf0 0x558992fa9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.146+0000 7fd0b2435740 1 --2- 172.21.15.94:0/2745571467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fabec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.146+0000 7fd0b2435740 1 -- 172.21.15.94:0/2745571467 >> 172.21.15.94:0/2745571467 conn(0x558992f6a310 msgr2=0x558992fa3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.146+0000 7fd0917fa640 0 client.25937 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:59.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.147+0000 7fd0b2435740 1 -- 172.21.15.94:0/2745571467 shutdown_connections 2024-09-19T04:22:59.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.147+0000 7fd0b2435740 1 -- 172.21.15.94:0/2745571467 wait complete. 2024-09-19T04:22:59.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.147+0000 7fd0b2435740 20 client.25937 trim_cache size 0 max 16384 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0b2435740 1 Processor -- start 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0b2435740 1 -- start start 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa1780 0x558992fb8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa2cf0 0x558992f9c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fac520 0x558992fa42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0ae5f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa1780 0x558992fb8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0ae5f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa1780 0x558992fb8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48966/0 (socket says 172.21.15.94:48966) 2024-09-19T04:22:59.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0ad5f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa2cf0 0x558992f9c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0ad5f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa2cf0 0x558992f9c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:45066/0 (socket says 172.21.15.94:45066) 2024-09-19T04:22:59.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/3576009255 learned_addr learned my addr 172.21.15.94:0/3576009255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0b2435740 1 -- 172.21.15.94:0/3576009255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81b6710 con 0x558992fac520 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0b2435740 1 -- 172.21.15.94:0/3576009255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81b3f80 con 0x558992fa2cf0 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.169+0000 7fd0b2435740 1 -- 172.21.15.94:0/3576009255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81b1df0 con 0x558992fa1780 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.170+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/3576009255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fac520 0x558992fa42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.170+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/3576009255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa2cf0 msgr2=0x558992f9c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.170+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/3576009255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa2cf0 0x558992f9c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.170+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/3576009255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fac520 msgr2=0x558992fa42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.170+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/3576009255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fac520 0x558992fa42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.170+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/3576009255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558992f9ef80 con 0x558992fa1780 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.170+0000 7fd0ad5f7640 1 --2- 172.21.15.94:0/3576009255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa2cf0 0x558992f9c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.170+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/3576009255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa1780 0x558992fb8560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0006010 tx=0x7fd0a00021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=473c81d08dec8770 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.171+0000 7fd0937fe640 1 -- 172.21.15.94:0/3576009255 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a001c070 con 0x558992fa1780 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.171+0000 7fd0937fe640 1 -- 172.21.15.94:0/3576009255 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0021070 con 0x558992fa1780 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.171+0000 7fd0937fe640 1 -- 172.21.15.94:0/3576009255 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0023030 con 0x558992fa1780 2024-09-19T04:22:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.172+0000 7fd0b2435740 1 -- 172.21.15.94:0/3576009255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa1780 msgr2=0x558992fb8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.172+0000 7fd0b2435740 1 --2- 172.21.15.94:0/3576009255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa1780 0x558992fb8560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0006010 tx=0x7fd0a00021d0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.172+0000 7fd0b2435740 1 -- 172.21.15.94:0/3576009255 shutdown_connections 2024-09-19T04:22:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.172+0000 7fd0b2435740 1 --2- 172.21.15.94:0/3576009255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fac520 0x558992fa42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.172+0000 7fd0b2435740 1 --2- 172.21.15.94:0/3576009255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa2cf0 0x558992f9c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.172+0000 7fd0b2435740 1 --2- 172.21.15.94:0/3576009255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fa1780 0x558992fb8560 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.172+0000 7fd0b2435740 1 -- 172.21.15.94:0/3576009255 >> 172.21.15.94:0/3576009255 conn(0x558992f6a310 msgr2=0x558992fab1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.173+0000 7fd0b2435740 1 -- 172.21.15.94:0/3576009255 shutdown_connections 2024-09-19T04:22:59.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.173+0000 7fd0b2435740 1 -- 172.21.15.94:0/3576009255 wait complete. 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.174+0000 7fd0b2435740 1 -- start start 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.174+0000 7fd0b2435740 1 Processor -- start 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fa6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fa2cf0 0x558992fa6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0b2435740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 0x558992fa6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0b2435740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81b2440 con 0x558992fa2cf0 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0b2435740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81b38b0 con 0x558992fa1780 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0b2435740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81b2b10 con 0x558992fac520 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0ae5f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fa6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0b2435740 10 client.0 _ll_register_callbacks cb 0x7fd0a0009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0ae5f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fa6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:45068/0 (socket says 172.21.15.94:45068) 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.175+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/1181322080 learned_addr learned my addr 172.21.15.94:0/1181322080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.176+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 0x558992fa6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.176+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/1181322080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 msgr2=0x558992fa6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.176+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 0x558992fa6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.176+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/1181322080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fa2cf0 msgr2=0x558992fa6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.176+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fa2cf0 0x558992fa6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.176+0000 7fd0ae5f9640 1 -- 172.21.15.94:0/1181322080 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558993025bc0 con 0x558992fa1780 2024-09-19T04:22:59.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.176+0000 7fd0ae5f9640 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fa6340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00095c0 tx=0x7fd0a000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d741d45de8b6ea76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.177+0000 7fd0917fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:22:59.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.177+0000 7fd0917fa640 1 -- 172.21.15.94:0/1181322080 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0047060 con 0x558992fa1780 2024-09-19T04:22:59.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.177+0000 7fd0917fa640 1 -- 172.21.15.94:0/1181322080 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fd0a0018470 con 0x558992fa1780 2024-09-19T04:22:59.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.177+0000 7fd0b2435740 1 -- 172.21.15.94:0/1181322080 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a0021070 con 0x558992fa1780 2024-09-19T04:22:59.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.178+0000 7fd0917fa640 1 -- 172.21.15.94:0/1181322080 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd0a001c030 con 0x558992fa1780 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.178+0000 7fd0b2435740 10 client.25943 fetch_fsmap learned FSMap version 42 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.178+0000 7fd0b2435740 1 -- 172.21.15.94:0/1181322080 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558993026ea0 con 0x558992fa1780 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.178+0000 7fd0917fa640 1 -- 172.21.15.94:0/1181322080 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0024040 con 0x558992fa1780 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.178+0000 7fd0917fa640 1 -- 172.21.15.94:0/1181322080 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd0a0040420 con 0x558992fa1780 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.178+0000 7fd0b2435740 10 client.25943 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.179+0000 7fd0b2435740 10 client.25943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.179+0000 7fd0b2435740 20 client.25943 populate_metadata read hostname 'smithi094' 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.179+0000 7fd0b2435740 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5589930275c0 0x5589930299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.179+0000 7fd0b2435740 4 client.25943 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.179+0000 7fd0b2435740 1 -- 172.21.15.94:0/1181322080 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd0a0021070 con 0x5589930275c0 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.180+0000 7fd0ad5f7640 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5589930275c0 0x5589930299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.180+0000 7fd0917fa640 10 client.25943 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.180+0000 7fd0ad5f7640 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5589930275c0 0x5589930299e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ea3860fa905562 server_cookie=4d1cc6f80423c162 in_seq=0 out_seq=0 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0917fa640 1 -- 172.21.15.94:0/1181322080 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd0a0021070 con 0x5589930275c0 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0917fa640 10 client.25943 handle_command_reply: tid=0 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 client.25943 shutdown 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 -- 172.21.15.94:0/1181322080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 msgr2=0x558992fa6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fa6340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00095c0 tx=0x7fd0a000ec20 comp rx=0 tx=0).stop 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 -- 172.21.15.94:0/1181322080 shutdown_connections 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5589930275c0 0x5589930299e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558992fac520 0x558992fa6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558992fa2cf0 0x558992fa6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 --2- 172.21.15.94:0/1181322080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558992fa1780 0x558992fa6340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 -- 172.21.15.94:0/1181322080 >> 172.21.15.94:0/1181322080 conn(0x558992f6a310 msgr2=0x558992fb7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0917fa640 0 client.25943 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:22:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 -- 172.21.15.94:0/1181322080 shutdown_connections 2024-09-19T04:22:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 1 -- 172.21.15.94:0/1181322080 wait complete. 2024-09-19T04:22:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.181+0000 7fd0b2435740 20 client.25943 trim_cache size 0 max 16384 2024-09-19T04:22:59.184 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.182+0000 7fd0aedfa640 1 -- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a3540 msgr2=0x7fd0940a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.182+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a3540 0x7fd0940a5a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x558992f9ec70 tx=0x7fd0a409c160 comp rx=0 tx=0).stop 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.182+0000 7fd0aedfa640 1 -- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8108280 msgr2=0x7fd0a81a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.182+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8108280 0x7fd0a81a43f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400e9f0 tx=0x7fd0a400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.183+0000 7fd0aedfa640 1 -- 172.21.15.94:0/2030069377 shutdown_connections 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.183+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd0940a3540 0x7fd0940a5a00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.183+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a810d360 0x7fd0a819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.183+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a8108c30 0x7fd0a81a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.183+0000 7fd0aedfa640 1 --2- 172.21.15.94:0/2030069377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a8108280 0x7fd0a81a43f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.183+0000 7fd0aedfa640 1 -- 172.21.15.94:0/2030069377 >> 172.21.15.94:0/2030069377 conn(0x7fd0a80779f0 msgr2=0x7fd0a8078790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.183+0000 7fd0aedfa640 1 -- 172.21.15.94:0/2030069377 shutdown_connections 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:22:59.183+0000 7fd0aedfa640 1 -- 172.21.15.94:0/2030069377 wait complete. 2024-09-19T04:23:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 405.0 s, 1285.4 tps, lat 78.082 ms stddev 28.796 2024-09-19T04:23:01.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:00 smithi112 ceph-mon[35051]: pgmap v244: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-09-19T04:23:01.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:00 smithi094 ceph-mon[30972]: pgmap v244: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-09-19T04:23:01.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:00 smithi104 ceph-mon[34145]: pgmap v244: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-09-19T04:23:01.248 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:23:01.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:23:01.684 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.288+0000 7ff1342ae640 1 -- 172.21.15.94:0/3691932445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12c101450 msgr2=0x7ff12c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.288+0000 7ff1342ae640 1 --2- 172.21.15.94:0/3691932445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12c101450 0x7ff12c105920 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009900 tx=0x7ff11c02a470 comp rx=0 tx=0).stop 2024-09-19T04:23:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.289+0000 7ff1342ae640 1 -- 172.21.15.94:0/3691932445 shutdown_connections 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.289+0000 7ff1342ae640 1 --2- 172.21.15.94:0/3691932445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff12c105f40 0x7ff12c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.289+0000 7ff1342ae640 1 --2- 172.21.15.94:0/3691932445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12c101450 0x7ff12c105920 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.289+0000 7ff1342ae640 1 --2- 172.21.15.94:0/3691932445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c100b30 0x7ff12c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.289+0000 7ff1342ae640 1 -- 172.21.15.94:0/3691932445 >> 172.21.15.94:0/3691932445 conn(0x7ff12c0fc7a0 msgr2=0x7ff12c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.289+0000 7ff1342ae640 1 -- 172.21.15.94:0/3691932445 shutdown_connections 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.289+0000 7ff1342ae640 1 -- 172.21.15.94:0/3691932445 wait complete. 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.290+0000 7ff1342ae640 1 Processor -- start 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.290+0000 7ff1342ae640 1 -- start start 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.290+0000 7ff1342ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12c100b30 0x7ff12c1a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.290+0000 7ff1342ae640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff12c101450 0x7ff12c1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.290+0000 7ff1342ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c105f40 0x7ff12c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.290+0000 7ff1342ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c118440 con 0x7ff12c100b30 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff1342ae640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1182c0 con 0x7ff12c101450 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff1342ae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1185c0 con 0x7ff12c105f40 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff13284b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c105f40 0x7ff12c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff13204a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12c100b30 0x7ff12c1a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff13284b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c105f40 0x7ff12c19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:57302/0 (socket says 172.21.15.94:57302) 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff13284b640 1 -- 172.21.15.94:0/1200997258 learned_addr learned my addr 172.21.15.94:0/1200997258 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff131849640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff12c101450 0x7ff12c1a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff13204a640 1 -- 172.21.15.94:0/1200997258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c105f40 msgr2=0x7ff12c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff13204a640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c105f40 0x7ff12c19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff13204a640 1 -- 172.21.15.94:0/1200997258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff12c101450 msgr2=0x7ff12c1a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff13204a640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff12c101450 0x7ff12c1a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff13204a640 1 -- 172.21.15.94:0/1200997258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c19ef70 con 0x7ff12c100b30 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.291+0000 7ff131849640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff12c101450 0x7ff12c1a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.292+0000 7ff13204a640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12c100b30 0x7ff12c1a4650 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff12800d6d0 tx=0x7ff12800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.292+0000 7ff1237fe640 1 -- 172.21.15.94:0/1200997258 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128017070 con 0x7ff12c100b30 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.292+0000 7ff1237fe640 1 -- 172.21.15.94:0/1200997258 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff128009820 con 0x7ff12c100b30 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.292+0000 7ff1237fe640 1 -- 172.21.15.94:0/1200997258 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128004c40 con 0x7ff12c100b30 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.292+0000 7ff1342ae640 1 -- 172.21.15.94:0/1200997258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c19f260 con 0x7ff12c100b30 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.292+0000 7ff1342ae640 1 -- 172.21.15.94:0/1200997258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c1ab4b0 con 0x7ff12c100b30 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.296+0000 7ff136687740 1 Processor -- start 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.296+0000 7ff136687740 1 -- start start 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff136687740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3521f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff136687740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 0x55a6f352c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff136687740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 0x55a6f352f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff136687740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a6f3487280 con 0x55a6f3521b60 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff136687740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a6f342eb10 con 0x55a6f3522480 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff136687740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a6f3483240 con 0x55a6f352cd40 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff13284b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3521f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff13284b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3521f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54994/0 (socket says 172.21.15.94:54994) 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff13284b640 1 -- 172.21.15.94:0/2052921497 learned_addr learned my addr 172.21.15.94:0/2052921497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff131849640 1 --2- 172.21.15.94:0/2052921497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 0x55a6f352f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.297+0000 7ff1237fe640 1 -- 172.21.15.94:0/1200997258 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff128005220 con 0x7ff12c100b30 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.298+0000 7ff13204a640 1 --2- 172.21.15.94:0/2052921497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 0x55a6f352c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.298+0000 7ff13284b640 1 -- 172.21.15.94:0/2052921497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 msgr2=0x55a6f352f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.298+0000 7ff13284b640 1 --2- 172.21.15.94:0/2052921497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 0x55a6f352f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.298+0000 7ff13284b640 1 -- 172.21.15.94:0/2052921497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 msgr2=0x55a6f352c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.298+0000 7ff13284b640 1 --2- 172.21.15.94:0/2052921497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 0x55a6f352c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.298+0000 7ff13284b640 1 -- 172.21.15.94:0/2052921497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f343b1f0 con 0x55a6f3521b60 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.299+0000 7ff13284b640 1 --2- 172.21.15.94:0/2052921497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3521f40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff124009f30 tx=0x7ff124010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f6c17e745a6078 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.299+0000 7ff1237fe640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1180a3540 0x7ff1180a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.300+0000 7ff120dea640 1 -- 172.21.15.94:0/2052921497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124016070 con 0x55a6f3521b60 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.300+0000 7ff120dea640 1 -- 172.21.15.94:0/2052921497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff124012900 con 0x55a6f3521b60 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.300+0000 7ff120dea640 1 -- 172.21.15.94:0/2052921497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124011930 con 0x55a6f3521b60 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.300+0000 7ff1237fe640 1 -- 172.21.15.94:0/1200997258 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7ff1280cf760 con 0x7ff12c100b30 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.301+0000 7ff13204a640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1180a3540 0x7ff1180a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.301+0000 7ff136687740 1 -- 172.21.15.94:0/2052921497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 msgr2=0x55a6f3521f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.301+0000 7ff136687740 1 --2- 172.21.15.94:0/2052921497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3521f40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff124009f30 tx=0x7ff124010800 comp rx=0 tx=0).stop 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.301+0000 7ff136687740 1 -- 172.21.15.94:0/2052921497 shutdown_connections 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.301+0000 7ff136687740 1 --2- 172.21.15.94:0/2052921497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 0x55a6f352f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.301+0000 7ff136687740 1 --2- 172.21.15.94:0/2052921497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 0x55a6f352c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.301+0000 7ff136687740 1 --2- 172.21.15.94:0/2052921497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3521f40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.301+0000 7ff136687740 1 -- 172.21.15.94:0/2052921497 >> 172.21.15.94:0/2052921497 conn(0x55a6f351e0f0 msgr2=0x55a6f351e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.302+0000 7ff136687740 1 -- 172.21.15.94:0/2052921497 shutdown_connections 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.302+0000 7ff136687740 1 -- 172.21.15.94:0/2052921497 wait complete. 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.303+0000 7ff136687740 1 -- start start 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.303+0000 7ff13204a640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1180a3540 0x7ff1180a5a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55a6f351ef50 tx=0x7ff128000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.303+0000 7ff136687740 1 Processor -- start 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.303+0000 7ff136687740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3521b60 0x55a6f3529ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.303+0000 7ff136687740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3522480 0x55a6f352a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff13284b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3521b60 0x55a6f3529ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff13284b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3521b60 0x55a6f3529ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44374/0 (socket says 172.21.15.94:44374) 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff13284b640 1 -- 172.21.15.94:0/1295620759 learned_addr learned my addr 172.21.15.94:0/1295620759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff136687740 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f352cd40 0x55a6f352a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a6f352f910 con 0x55a6f352cd40 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff131849640 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3522480 0x55a6f352a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a6f3441f10 con 0x55a6f3521b60 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a6f352fa90 con 0x55a6f3522480 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff13284b640 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f352cd40 0x55a6f352a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff13284b640 1 -- 172.21.15.94:0/1295620759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3522480 msgr2=0x55a6f352a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff13284b640 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3522480 0x55a6f352a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff13284b640 1 -- 172.21.15.94:0/1295620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f352cd40 msgr2=0x55a6f352a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff13284b640 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f352cd40 0x55a6f352a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.304+0000 7ff13284b640 1 -- 172.21.15.94:0/1295620759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f31f86a0 con 0x55a6f3521b60 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.305+0000 7ff13284b640 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3521b60 0x55a6f3529ca0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff124005b40 tx=0x7ff124010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa5b0e4ef6a862c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.305+0000 7ff115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:23:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.305+0000 7ff136687740 10 client.0 _ll_register_callbacks cb 0x55a6f340aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.305+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff124012900 con 0x55a6f3521b60 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.305+0000 7ff115ffb640 1 -- 172.21.15.94:0/1295620759 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124016070 con 0x55a6f3521b60 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.305+0000 7ff115ffb640 1 -- 172.21.15.94:0/1295620759 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7ff124004c40 con 0x55a6f3521b60 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.305+0000 7ff115ffb640 1 -- 172.21.15.94:0/1295620759 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff124031050 con 0x55a6f3521b60 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.305+0000 7ff136687740 10 client.25955 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.305+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55a6f353b2b0 con 0x55a6f3521b60 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.306+0000 7ff115ffb640 1 -- 172.21.15.94:0/1295620759 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7ff12400f040 con 0x55a6f3521b60 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.306+0000 7ff136687740 10 client.25955 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.306+0000 7ff136687740 10 client.25955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.306+0000 7ff136687740 20 client.25955 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.306+0000 7ff136687740 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a6f353c680 0x55a6f353ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.306+0000 7ff136687740 4 client.25955 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.306+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f34947f0 con 0x55a6f353c680 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.308+0000 7ff131849640 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a6f353c680 0x55a6f353ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.309+0000 7ff131849640 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a6f353c680 0x55a6f353ea70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d333da38963416 server_cookie=aa0ede66c41d78c1 in_seq=0 out_seq=0 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.309+0000 7ff115ffb640 10 client.25955 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.309+0000 7ff115ffb640 1 -- 172.21.15.94:0/1295620759 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6f34947f0 con 0x55a6f353c680 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.309+0000 7ff115ffb640 10 client.25955 handle_command_reply: tid=0 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 client.25955 shutdown 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3521b60 msgr2=0x55a6f3529ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3521b60 0x55a6f3529ca0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff124005b40 tx=0x7ff124010fd0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 shutdown_connections 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a6f353c680 0x55a6f353ea70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f352cd40 0x55a6f352a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3522480 0x55a6f352a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 --2- 172.21.15.94:0/1295620759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3521b60 0x55a6f3529ca0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 >> 172.21.15.94:0/1295620759 conn(0x55a6f351e0f0 msgr2=0x55a6f3523e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff115ffb640 0 client.25955 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.310+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 shutdown_connections 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.311+0000 7ff136687740 1 -- 172.21.15.94:0/1295620759 wait complete. 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.311+0000 7ff136687740 20 client.25955 trim_cache size 0 max 16384 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.341+0000 7ff136687740 1 Processor -- start 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.341+0000 7ff136687740 1 -- start start 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.341+0000 7ff136687740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3521b60 0x55a6f3523d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff136687740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3522480 0x55a6f352a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff136687740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f352cd40 0x55a6f352a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff136687740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100110c0 con 0x55a6f3522480 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff136687740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff11000e930 con 0x55a6f352cd40 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff136687740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff11000c7a0 con 0x55a6f3521b60 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff131849640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3522480 0x55a6f352a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff131849640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3522480 0x55a6f352a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55010/0 (socket says 172.21.15.94:55010) 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff131849640 1 -- 172.21.15.94:0/1101952384 learned_addr learned my addr 172.21.15.94:0/1101952384 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff13284b640 1 --2- 172.21.15.94:0/1101952384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3521b60 0x55a6f3523d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.342+0000 7ff13284b640 1 --2- 172.21.15.94:0/1101952384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f352cd40 0x55a6f352a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.343+0000 7ff13284b640 1 -- 172.21.15.94:0/1101952384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f352cd40 msgr2=0x55a6f352a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.343+0000 7ff13284b640 1 --2- 172.21.15.94:0/1101952384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f352cd40 0x55a6f352a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.343+0000 7ff13284b640 1 -- 172.21.15.94:0/1101952384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3522480 msgr2=0x55a6f352a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.343+0000 7ff13284b640 1 --2- 172.21.15.94:0/1101952384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3522480 0x55a6f352a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.343+0000 7ff13284b640 1 -- 172.21.15.94:0/1101952384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f352faa0 con 0x55a6f3521b60 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.343+0000 7ff13284b640 1 --2- 172.21.15.94:0/1101952384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3521b60 0x55a6f3523d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1240114e0 tx=0x7ff12401d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7864d86867e5cb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.343+0000 7ff117fff640 1 -- 172.21.15.94:0/1101952384 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124004c00 con 0x55a6f3521b60 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.343+0000 7ff117fff640 1 -- 172.21.15.94:0/1101952384 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff124012900 con 0x55a6f3521b60 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.343+0000 7ff117fff640 1 -- 172.21.15.94:0/1101952384 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124005000 con 0x55a6f3521b60 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.344+0000 7ff136687740 1 -- 172.21.15.94:0/1101952384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3521b60 msgr2=0x55a6f3523d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.344+0000 7ff136687740 1 --2- 172.21.15.94:0/1101952384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3521b60 0x55a6f3523d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1240114e0 tx=0x7ff12401d070 comp rx=0 tx=0).stop 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.344+0000 7ff136687740 1 -- 172.21.15.94:0/1101952384 shutdown_connections 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.344+0000 7ff136687740 1 --2- 172.21.15.94:0/1101952384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f352cd40 0x55a6f352a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.344+0000 7ff136687740 1 --2- 172.21.15.94:0/1101952384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3522480 0x55a6f352a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.344+0000 7ff136687740 1 --2- 172.21.15.94:0/1101952384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f3521b60 0x55a6f3523d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.344+0000 7ff136687740 1 -- 172.21.15.94:0/1101952384 >> 172.21.15.94:0/1101952384 conn(0x55a6f351e0f0 msgr2=0x55a6f3523600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.345+0000 7ff136687740 1 -- 172.21.15.94:0/1101952384 shutdown_connections 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.345+0000 7ff136687740 1 -- 172.21.15.94:0/1101952384 wait complete. 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff136687740 1 -- start start 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff136687740 1 Processor -- start 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff136687740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3530270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff136687740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 0x55a6f35307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff136687740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 0x55a6f35a5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff13284b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3530270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.517 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:23:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:23:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "bfcd8b2a-098f-4eec-97de-a0e3356c3569", 2024-09-19T04:23:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:23:02.517 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:23:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff13284b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3530270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55012/0 (socket says 172.21.15.94:55012) 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff13284b640 1 -- 172.21.15.94:0/3552073679 learned_addr learned my addr 172.21.15.94:0/3552073679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff11000cdf0 con 0x55a6f3521b60 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff11000e260 con 0x55a6f3522480 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.346+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff11000d4c0 con 0x55a6f352cd40 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.347+0000 7ff13284b640 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 0x55a6f35a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.347+0000 7ff131849640 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 0x55a6f35307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.347+0000 7ff136687740 10 client.0 _ll_register_callbacks cb 0x7ff11c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.347+0000 7ff13284b640 1 -- 172.21.15.94:0/3552073679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 msgr2=0x55a6f35a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.347+0000 7ff13284b640 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 0x55a6f35a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.347+0000 7ff13284b640 1 -- 172.21.15.94:0/3552073679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 msgr2=0x55a6f35307b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.347+0000 7ff13284b640 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 0x55a6f35307b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.347+0000 7ff13284b640 1 -- 172.21.15.94:0/3552073679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f35a5690 con 0x55a6f3521b60 2024-09-19T04:23:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.348+0000 7ff13284b640 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3530270 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff124011ea0 tx=0x7ff124010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b779c64ad9ae7c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.348+0000 7ff115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.348+0000 7ff115ffb640 1 -- 172.21.15.94:0/3552073679 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124016070 con 0x55a6f3521b60 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.348+0000 7ff115ffb640 1 -- 172.21.15.94:0/3552073679 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7ff124011510 con 0x55a6f3521b60 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.348+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff124012900 con 0x55a6f3521b60 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.348+0000 7ff115ffb640 1 -- 172.21.15.94:0/3552073679 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff124011710 con 0x55a6f3521b60 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.348+0000 7ff136687740 10 client.16362 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.348+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a6f35a6910 con 0x55a6f3521b60 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.349+0000 7ff115ffb640 1 -- 172.21.15.94:0/3552073679 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff124012900 con 0x55a6f3521b60 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.349+0000 7ff115ffb640 1 -- 172.21.15.94:0/3552073679 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7ff12404c020 con 0x55a6f3521b60 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.349+0000 7ff136687740 10 client.16362 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.349+0000 7ff136687740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.349+0000 7ff136687740 20 client.16362 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.349+0000 7ff136687740 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a6f35a6f90 0x55a6f35a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.349+0000 7ff136687740 4 client.16362 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.349+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a6f348ed00 con 0x55a6f35a6f90 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.349+0000 7ff131849640 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a6f35a6f90 0x55a6f35a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.351+0000 7ff131849640 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a6f35a6f90 0x55a6f35a93b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0a4b4d9588b04 server_cookie=92aac3f8c7a772f7 in_seq=0 out_seq=0 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.351+0000 7ff115ffb640 10 client.16362 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff115ffb640 1 -- 172.21.15.94:0/3552073679 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a6f348ed00 con 0x55a6f35a6f90 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff115ffb640 10 client.16362 handle_command_reply: tid=0 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff136687740 1 client.16362 shutdown 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 msgr2=0x55a6f3530270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff136687740 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3530270 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff124011ea0 tx=0x7ff124010800 comp rx=0 tx=0).stop 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 shutdown_connections 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff136687740 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55a6f35a6f90 0x55a6f35a93b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff136687740 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6f352cd40 0x55a6f35a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff136687740 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f3522480 0x55a6f35307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff136687740 1 --2- 172.21.15.94:0/3552073679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f3521b60 0x55a6f3530270 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 >> 172.21.15.94:0/3552073679 conn(0x55a6f351e0f0 msgr2=0x55a6f3523dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.352+0000 7ff115ffb640 0 client.16362 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.353+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 shutdown_connections 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.353+0000 7ff136687740 1 -- 172.21.15.94:0/3552073679 wait complete. 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.354+0000 7ff136687740 20 client.16362 trim_cache size 0 max 16384 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.356+0000 7ff1342ae640 1 -- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1180a3540 msgr2=0x7ff1180a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.356+0000 7ff1342ae640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1180a3540 0x7ff1180a5a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55a6f351ef50 tx=0x7ff128000bc0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.356+0000 7ff1342ae640 1 -- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12c100b30 msgr2=0x7ff12c1a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.356+0000 7ff1342ae640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12c100b30 0x7ff12c1a4650 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff12800d6d0 tx=0x7ff12800db90 comp rx=0 tx=0).stop 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.357+0000 7ff1342ae640 1 -- 172.21.15.94:0/1200997258 shutdown_connections 2024-09-19T04:23:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.357+0000 7ff1342ae640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1180a3540 0x7ff1180a5a00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.357+0000 7ff1342ae640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c105f40 0x7ff12c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.357+0000 7ff1342ae640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff12c101450 0x7ff12c1a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.357+0000 7ff1342ae640 1 --2- 172.21.15.94:0/1200997258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12c100b30 0x7ff12c1a4650 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.357+0000 7ff1342ae640 1 -- 172.21.15.94:0/1200997258 >> 172.21.15.94:0/1200997258 conn(0x7ff12c0fc7a0 msgr2=0x7ff12c10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.357+0000 7ff1342ae640 1 -- 172.21.15.94:0/1200997258 shutdown_connections 2024-09-19T04:23:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:02.357+0000 7ff1342ae640 1 -- 172.21.15.94:0/1200997258 wait complete. 2024-09-19T04:23:02.953 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:02 smithi094 ceph-mon[30972]: pgmap v245: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-09-19T04:23:03.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:02 smithi112 ceph-mon[35051]: pgmap v245: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-09-19T04:23:03.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:02 smithi104 ceph-mon[34145]: pgmap v245: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-09-19T04:23:03.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:23:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:03.949 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:23:03.949 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:03.949 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:23:03.949 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:23:03.949 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:23:03.949 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:03.949 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:23:03.949 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:03.949 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:03.950 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:03 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:03 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:23:04.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:23:04.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:23:04.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:23:04.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:23:04.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:03 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.350+0000 7fefb986a640 1 -- 172.21.15.94:0/4020784803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 msgr2=0x7fefb410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.350+0000 7fefb986a640 1 --2- 172.21.15.94:0/4020784803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 0x7fefb410bc00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fefac009ca0 tx=0x7fefac02e9a0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.350+0000 7fefb986a640 1 -- 172.21.15.94:0/4020784803 shutdown_connections 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.350+0000 7fefb986a640 1 --2- 172.21.15.94:0/4020784803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefb410c2f0 0x7fefb410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.350+0000 7fefb986a640 1 --2- 172.21.15.94:0/4020784803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 0x7fefb410bc00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.350+0000 7fefb986a640 1 --2- 172.21.15.94:0/4020784803 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102df0 0x7fefb41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.350+0000 7fefb986a640 1 -- 172.21.15.94:0/4020784803 >> 172.21.15.94:0/4020784803 conn(0x7fefb40fc7d0 msgr2=0x7fefb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.350+0000 7fefb986a640 1 -- 172.21.15.94:0/4020784803 shutdown_connections 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.350+0000 7fefb986a640 1 -- 172.21.15.94:0/4020784803 wait complete. 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.352+0000 7fefb986a640 1 Processor -- start 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.352+0000 7fefb986a640 1 -- start start 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.352+0000 7fefb986a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102df0 0x7fefb419d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.352+0000 7fefb986a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 0x7fefb4196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.352+0000 7fefb986a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefb410c2f0 0x7fefb4196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.352+0000 7fefb986a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410fdb0 con 0x7fefb410c2f0 2024-09-19T04:23:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.352+0000 7fefb986a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410fc30 con 0x7fefb4102df0 2024-09-19T04:23:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.352+0000 7fefb986a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410ff30 con 0x7fefb41037a0 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 0x7fefb4196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102df0 0x7fefb419d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 0x7fefb4196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:57356/0 (socket says 172.21.15.94:57356) 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102df0 0x7fefb419d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44408/0 (socket says 172.21.15.94:44408) 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb27fc640 1 -- 172.21.15.94:0/1031746368 learned_addr learned my addr 172.21.15.94:0/1031746368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb37fe640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefb410c2f0 0x7fefb4196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb27fc640 1 -- 172.21.15.94:0/1031746368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102df0 msgr2=0x7fefb419d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb27fc640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102df0 0x7fefb419d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb27fc640 1 -- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefb410c2f0 msgr2=0x7fefb4196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb27fc640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefb410c2f0 0x7fefb4196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb27fc640 1 -- 172.21.15.94:0/1031746368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb4196d00 con 0x7fefb41037a0 2024-09-19T04:23:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.353+0000 7fefb27fc640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 0x7fefb4196150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefac02eec0 tx=0x7fefac0093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.354+0000 7fefb8868640 1 -- 172.21.15.94:0/1031746368 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac037070 con 0x7fefb41037a0 2024-09-19T04:23:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.354+0000 7fefb986a640 1 -- 172.21.15.94:0/1031746368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb4196f90 con 0x7fefb41037a0 2024-09-19T04:23:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.354+0000 7fefb986a640 1 -- 172.21.15.94:0/1031746368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb419af00 con 0x7fefb41037a0 2024-09-19T04:23:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.354+0000 7fefb8868640 1 -- 172.21.15.94:0/1031746368 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefac051030 con 0x7fefb41037a0 2024-09-19T04:23:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.354+0000 7fefb8868640 1 -- 172.21.15.94:0/1031746368 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac004280 con 0x7fefb41037a0 2024-09-19T04:23:04.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.357+0000 7fefb8868640 1 -- 172.21.15.94:0/1031746368 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fefac03a070 con 0x7fefb41037a0 2024-09-19T04:23:04.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.357+0000 7fefbbc43740 1 Processor -- start 2024-09-19T04:23:04.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.357+0000 7fefbbc43740 1 -- start start 2024-09-19T04:23:04.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 0x564985269880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x564985274140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985276a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefbbc43740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5649851cf2e0 con 0x5649852694a0 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefbbc43740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564985176ce0 con 0x564985269dc0 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefbbc43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5649851cb2a0 con 0x564985274680 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x564985274140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x564985274140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44416/0 (socket says 172.21.15.94:44416) 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb2ffd640 1 -- 172.21.15.94:0/4224324603 learned_addr learned my addr 172.21.15.94:0/4224324603 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb27fc640 1 --2- 172.21.15.94:0/4224324603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985276a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb2ffd640 1 -- 172.21.15.94:0/4224324603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 msgr2=0x564985276a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/4224324603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985276a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb2ffd640 1 -- 172.21.15.94:0/4224324603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 msgr2=0x564985269880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/4224324603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 0x564985269880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb2ffd640 1 -- 172.21.15.94:0/4224324603 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498515b440 con 0x564985269dc0 2024-09-19T04:23:04.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.358+0000 7fefb27fc640 1 --2- 172.21.15.94:0/4224324603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985276a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:04.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.359+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/4224324603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x564985274140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009cf0 tx=0x7fefa4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f82d7cc98404ff73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:04.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.359+0000 7fefb8868640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef9c0a3500 0x7fef9c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.359+0000 7fef99ffb640 1 -- 172.21.15.94:0/4224324603 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4004230 con 0x564985269dc0 2024-09-19T04:23:04.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.359+0000 7fefb8868640 1 -- 172.21.15.94:0/1031746368 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fefac101ed0 con 0x7fefb41037a0 2024-09-19T04:23:04.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.360+0000 7fef99ffb640 1 -- 172.21.15.94:0/4224324603 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa40043d0 con 0x564985269dc0 2024-09-19T04:23:04.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.360+0000 7fef99ffb640 1 -- 172.21.15.94:0/4224324603 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4019820 con 0x564985269dc0 2024-09-19T04:23:04.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.361+0000 7fefb37fe640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef9c0a3500 0x7fef9c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.361+0000 7fefbbc43740 1 -- 172.21.15.94:0/4224324603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 msgr2=0x564985274140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:04.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.361+0000 7fefbbc43740 1 --2- 172.21.15.94:0/4224324603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x564985274140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009cf0 tx=0x7fefa4011810 comp rx=0 tx=0).stop 2024-09-19T04:23:04.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.361+0000 7fefbbc43740 1 -- 172.21.15.94:0/4224324603 shutdown_connections 2024-09-19T04:23:04.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.361+0000 7fefbbc43740 1 --2- 172.21.15.94:0/4224324603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985276a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.361+0000 7fefbbc43740 1 --2- 172.21.15.94:0/4224324603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x564985274140 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.361+0000 7fefbbc43740 1 --2- 172.21.15.94:0/4224324603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 0x564985269880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.361+0000 7fefbbc43740 1 -- 172.21.15.94:0/4224324603 >> 172.21.15.94:0/4224324603 conn(0x564985265a30 msgr2=0x564985265e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:04.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.361+0000 7fefbbc43740 1 -- 172.21.15.94:0/4224324603 shutdown_connections 2024-09-19T04:23:04.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.362+0000 7fefb37fe640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef9c0a3500 0x7fef9c0a59c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009c50 tx=0x7fefa80094d0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:04.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.362+0000 7fefbbc43740 1 -- 172.21.15.94:0/4224324603 wait complete. 2024-09-19T04:23:04.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.362+0000 7fefbbc43740 1 -- start start 2024-09-19T04:23:04.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.362+0000 7fefbbc43740 1 Processor -- start 2024-09-19T04:23:04.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.363+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x564985273f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.363+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564985269dc0 0x5649852711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.363+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985271720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.363+0000 7fefbbc43740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564985277250 con 0x564985269dc0 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.363+0000 7fefbbc43740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56498518a050 con 0x5649852694a0 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefbbc43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5649852773d0 con 0x564985274680 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefb37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985271720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefb37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985271720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:57372/0 (socket says 172.21.15.94:57372) 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefb37fe640 1 -- 172.21.15.94:0/2946650358 learned_addr learned my addr 172.21.15.94:0/2946650358 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x564985273f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefbbc43740 10 client.0 _ll_register_callbacks cb 0x5649851ca160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:04.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefb37fe640 1 -- 172.21.15.94:0/2946650358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 msgr2=0x564985273f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:04.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefb37fe640 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x564985273f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefb37fe640 1 -- 172.21.15.94:0/2946650358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564985269dc0 msgr2=0x5649852711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:04.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefb37fe640 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564985269dc0 0x5649852711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.364+0000 7fefb37fe640 1 -- 172.21.15.94:0/2946650358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564984f403c0 con 0x564985274680 2024-09-19T04:23:04.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.365+0000 7fefb37fe640 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985271720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fefa8004600 tx=0x7fefa8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3274a741474f947d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:04.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.365+0000 7fef92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.365+0000 7fef92ffd640 1 -- 172.21.15.94:0/2946650358 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa801e8f0 con 0x564985274680 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.365+0000 7fef92ffd640 1 -- 172.21.15.94:0/2946650358 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fefa801ea90 con 0x564985274680 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.365+0000 7fefbbc43740 1 -- 172.21.15.94:0/2946650358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa40043d0 con 0x564985274680 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.365+0000 7fef92ffd640 1 -- 172.21.15.94:0/2946650358 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fefa801ec90 con 0x564985274680 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.365+0000 7fefbbc43740 10 client.25969 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.365+0000 7fefbbc43740 1 -- 172.21.15.94:0/2946650358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564985282c90 con 0x564985274680 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.366+0000 7fef92ffd640 1 -- 172.21.15.94:0/2946650358 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa40043d0 con 0x564985274680 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.366+0000 7fef92ffd640 1 -- 172.21.15.94:0/2946650358 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fefa801d430 con 0x564985274680 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.366+0000 7fefbbc43740 10 client.25969 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.366+0000 7fefbbc43740 10 client.25969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.366+0000 7fefbbc43740 20 client.25969 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:04.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.366+0000 7fefbbc43740 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564985284120 0x5649852865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.366+0000 7fefbbc43740 4 client.25969 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:04.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.366+0000 7fefbbc43740 1 -- 172.21.15.94:0/2946650358 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649851dc850 con 0x564985284120 2024-09-19T04:23:04.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.366+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564985284120 0x5649852865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.367+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564985284120 0x5649852865c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7030551d29bfd27 server_cookie=b04935766b87d9c5 in_seq=0 out_seq=0 2024-09-19T04:23:04.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.367+0000 7fef92ffd640 10 client.25969 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:04.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.368+0000 7fef92ffd640 1 -- 172.21.15.94:0/2946650358 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649851dc850 con 0x564985284120 2024-09-19T04:23:04.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.368+0000 7fef92ffd640 10 client.25969 handle_command_reply: tid=0 2024-09-19T04:23:04.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.368+0000 7fefbbc43740 1 client.25969 shutdown 2024-09-19T04:23:04.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.368+0000 7fefbbc43740 1 -- 172.21.15.94:0/2946650358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 msgr2=0x564985271720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.368+0000 7fefbbc43740 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985271720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fefa8004600 tx=0x7fefa8005f90 comp rx=0 tx=0).stop 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fefbbc43740 1 -- 172.21.15.94:0/2946650358 shutdown_connections 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fefbbc43740 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564985284120 0x5649852865c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fefbbc43740 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x564985271720 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fef92ffd640 0 client.25969 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fefbbc43740 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564985269dc0 0x5649852711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fefbbc43740 1 --2- 172.21.15.94:0/2946650358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x564985273f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fefbbc43740 1 -- 172.21.15.94:0/2946650358 >> 172.21.15.94:0/2946650358 conn(0x564985265a30 msgr2=0x56498526b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fefbbc43740 1 -- 172.21.15.94:0/2946650358 shutdown_connections 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fefbbc43740 1 -- 172.21.15.94:0/2946650358 wait complete. 2024-09-19T04:23:04.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.369+0000 7fefbbc43740 20 client.25969 trim_cache size 0 max 16384 2024-09-19T04:23:04.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.398+0000 7fefbbc43740 1 Processor -- start 2024-09-19T04:23:04.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.398+0000 7fefbbc43740 1 -- start start 2024-09-19T04:23:04.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 0x564985280540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x56498528be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x56498528c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefbbc43740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0011210 con 0x5649852694a0 2024-09-19T04:23:04.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefbbc43740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fefa000ea80 con 0x564985269dc0 2024-09-19T04:23:04.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefbbc43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefa000c8f0 con 0x564985274680 2024-09-19T04:23:04.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 0x564985280540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb27fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x56498528be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 0x564985280540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55074/0 (socket says 172.21.15.94:55074) 2024-09-19T04:23:04.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb2ffd640 1 -- 172.21.15.94:0/1233139869 learned_addr learned my addr 172.21.15.94:0/1233139869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb37fe640 1 --2- 172.21.15.94:0/1233139869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x56498528c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb2ffd640 1 -- 172.21.15.94:0/1233139869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 msgr2=0x56498528c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/1233139869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x56498528c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb2ffd640 1 -- 172.21.15.94:0/1233139869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 msgr2=0x56498528be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/1233139869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x56498528be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.399+0000 7fefb2ffd640 1 -- 172.21.15.94:0/1233139869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649852742a0 con 0x5649852694a0 2024-09-19T04:23:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.400+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/1233139869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 0x564985280540 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fefa401b040 tx=0x7fefa4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63874a1e800ab6a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.400+0000 7fef997fa640 1 -- 172.21.15.94:0/1233139869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4030070 con 0x5649852694a0 2024-09-19T04:23:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.400+0000 7fef997fa640 1 -- 172.21.15.94:0/1233139869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa4003a00 con 0x5649852694a0 2024-09-19T04:23:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.400+0000 7fef997fa640 1 -- 172.21.15.94:0/1233139869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4019910 con 0x5649852694a0 2024-09-19T04:23:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.401+0000 7fefbbc43740 1 -- 172.21.15.94:0/1233139869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 msgr2=0x564985280540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.401+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1233139869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 0x564985280540 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fefa401b040 tx=0x7fefa4003d40 comp rx=0 tx=0).stop 2024-09-19T04:23:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.401+0000 7fefbbc43740 1 -- 172.21.15.94:0/1233139869 shutdown_connections 2024-09-19T04:23:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.401+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1233139869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985274680 0x56498528c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.401+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1233139869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564985269dc0 0x56498528be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.401+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1233139869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649852694a0 0x564985280540 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.402+0000 7fefbbc43740 1 -- 172.21.15.94:0/1233139869 >> 172.21.15.94:0/1233139869 conn(0x564985265a30 msgr2=0x56498526fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.402+0000 7fefbbc43740 1 -- 172.21.15.94:0/1233139869 shutdown_connections 2024-09-19T04:23:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.402+0000 7fefbbc43740 1 -- 172.21.15.94:0/1233139869 wait complete. 2024-09-19T04:23:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.402+0000 7fefbbc43740 1 -- start start 2024-09-19T04:23:04.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.403+0000 7fefbbc43740 1 Processor -- start 2024-09-19T04:23:04.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.403+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x5649852783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985269dc0 0x5649852788f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefbbc43740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564985274680 0x564985278e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefbbc43740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefa000cf40 con 0x564985274680 2024-09-19T04:23:04.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefbbc43740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fefa000e3b0 con 0x5649852694a0 2024-09-19T04:23:04.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x5649852783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefbbc43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefa000d610 con 0x564985269dc0 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x5649852783b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44430/0 (socket says 172.21.15.94:44430) 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb2ffd640 1 -- 172.21.15.94:0/1536798411 learned_addr learned my addr 172.21.15.94:0/1536798411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb27fc640 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985269dc0 0x5649852788f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefbbc43740 10 client.0 _ll_register_callbacks cb 0x7fefa8009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb2ffd640 1 -- 172.21.15.94:0/1536798411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985269dc0 msgr2=0x5649852788f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985269dc0 0x5649852788f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb2ffd640 1 -- 172.21.15.94:0/1536798411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564985274680 msgr2=0x564985278e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564985274680 0x564985278e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb2ffd640 1 -- 172.21.15.94:0/1536798411 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649852ede60 con 0x5649852694a0 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fefb2ffd640 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x5649852783b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fefa4011850 tx=0x7fefa4009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e562f06d6fa54595 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.404+0000 7fef92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:23:04.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.405+0000 7fef92ffd640 1 -- 172.21.15.94:0/1536798411 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4039020 con 0x5649852694a0 2024-09-19T04:23:04.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.405+0000 7fef92ffd640 1 -- 172.21.15.94:0/1536798411 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fefa4019420 con 0x5649852694a0 2024-09-19T04:23:04.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.405+0000 7fefbbc43740 1 -- 172.21.15.94:0/1536798411 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa4003a00 con 0x5649852694a0 2024-09-19T04:23:04.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.405+0000 7fef92ffd640 1 -- 172.21.15.94:0/1536798411 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fefa4030070 con 0x5649852694a0 2024-09-19T04:23:04.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.405+0000 7fefbbc43740 10 client.25979 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:04.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.405+0000 7fefbbc43740 1 -- 172.21.15.94:0/1536798411 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5649852ef0d0 con 0x5649852694a0 2024-09-19T04:23:04.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.405+0000 7fef92ffd640 1 -- 172.21.15.94:0/1536798411 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa4003a00 con 0x5649852694a0 2024-09-19T04:23:04.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.406+0000 7fef92ffd640 1 -- 172.21.15.94:0/1536798411 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fefa4019b70 con 0x5649852694a0 2024-09-19T04:23:04.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.406+0000 7fefbbc43740 10 client.25979 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:04.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.406+0000 7fefbbc43740 10 client.25979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:04.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.406+0000 7fefbbc43740 20 client.25979 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:04.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.406+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564985284120 0x5649852f1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:04.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.406+0000 7fefbbc43740 4 client.25979 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:23:04.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.406+0000 7fefbbc43740 1 -- 172.21.15.94:0/1536798411 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564985271ab0 con 0x564985284120 2024-09-19T04:23:04.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.406+0000 7fefb27fc640 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564985284120 0x5649852f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.407+0000 7fefb27fc640 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564985284120 0x5649852f1620 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18a5a39be2d9d2 server_cookie=8c1ea87031105ba5 in_seq=0 out_seq=0 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.407+0000 7fef92ffd640 10 client.25979 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fef92ffd640 1 -- 172.21.15.94:0/1536798411 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564985271ab0 con 0x564985284120 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fef92ffd640 10 client.25979 handle_command_reply: tid=0 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fefbbc43740 1 client.25979 shutdown 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fefbbc43740 1 -- 172.21.15.94:0/1536798411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 msgr2=0x5649852783b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x5649852783b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fefa4011850 tx=0x7fefa4009680 comp rx=0 tx=0).stop 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fefbbc43740 1 -- 172.21.15.94:0/1536798411 shutdown_connections 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564985284120 0x5649852f1620 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564985274680 0x564985278e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fef92ffd640 0 client.25979 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564985269dc0 0x5649852788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.408+0000 7fefbbc43740 1 --2- 172.21.15.94:0/1536798411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649852694a0 0x5649852783b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.409+0000 7fefbbc43740 1 -- 172.21.15.94:0/1536798411 >> 172.21.15.94:0/1536798411 conn(0x564985265a30 msgr2=0x564985264b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.409+0000 7fefbbc43740 1 -- 172.21.15.94:0/1536798411 shutdown_connections 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.409+0000 7fefbbc43740 1 -- 172.21.15.94:0/1536798411 wait complete. 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.409+0000 7fefbbc43740 20 client.25979 trim_cache size 0 max 16384 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:23:04.412 INFO:teuthology.orchestra.run.smithi094.stdout: "bfcd8b2a-098f-4eec-97de-a0e3356c3569": { 2024-09-19T04:23:04.413 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:23:04.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "bfcd8b2a-098f-4eec-97de-a0e3356c3569", 2024-09-19T04:23:04.413 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:23:04.413 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:23:04.413 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:23:04.413 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:23:04.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.410+0000 7fefb986a640 1 -- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef9c0a3500 msgr2=0x7fef9c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:04.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.410+0000 7fefb986a640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef9c0a3500 0x7fef9c0a59c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009c50 tx=0x7fefa80094d0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.410+0000 7fefb986a640 1 -- 172.21.15.94:0/1031746368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 msgr2=0x7fefb4196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:04.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.410+0000 7fefb986a640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 0x7fefb4196150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefac02eec0 tx=0x7fefac0093d0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.412+0000 7fefb986a640 1 -- 172.21.15.94:0/1031746368 shutdown_connections 2024-09-19T04:23:04.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.412+0000 7fefb986a640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fef9c0a3500 0x7fef9c0a59c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.412+0000 7fefb986a640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefb410c2f0 0x7fefb4196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.412+0000 7fefb986a640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb41037a0 0x7fefb4196150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.412+0000 7fefb986a640 1 --2- 172.21.15.94:0/1031746368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102df0 0x7fefb419d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:04.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.412+0000 7fefb986a640 1 -- 172.21.15.94:0/1031746368 >> 172.21.15.94:0/1031746368 conn(0x7fefb40fc7d0 msgr2=0x7fefb40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:04.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.412+0000 7fefb986a640 1 -- 172.21.15.94:0/1031746368 shutdown_connections 2024-09-19T04:23:04.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:04.412+0000 7fefb986a640 1 -- 172.21.15.94:0/1031746368 wait complete. 2024-09-19T04:23:04.995 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:04 smithi094 ceph-mon[30972]: pgmap v246: 97 pgs: 97 active+clean; 4.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 1.72k op/s 2024-09-19T04:23:05.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:04 smithi112 ceph-mon[35051]: pgmap v246: 97 pgs: 97 active+clean; 4.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 1.72k op/s 2024-09-19T04:23:05.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:04 smithi104 ceph-mon[34145]: pgmap v246: 97 pgs: 97 active+clean; 4.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 1.72k op/s 2024-09-19T04:23:05.174 INFO:tasks.cephfs.filesystem:scrub status for tag:bfcd8b2a-098f-4eec-97de-a0e3356c3569 - {'path': '/', 'tag': 'bfcd8b2a-098f-4eec-97de-a0e3356c3569', 'options': 'recursive,force'} 2024-09-19T04:23:05.603 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 410.0 s, 1394.3 tps, lat 71.718 ms stddev 22.158 2024-09-19T04:23:07.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:06 smithi112 ceph-mon[35051]: pgmap v247: 97 pgs: 97 active+clean; 4.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-09-19T04:23:07.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:06 smithi094 ceph-mon[30972]: pgmap v247: 97 pgs: 97 active+clean; 4.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-09-19T04:23:07.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:06 smithi104 ceph-mon[34145]: pgmap v247: 97 pgs: 97 active+clean; 4.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-09-19T04:23:09.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:08 smithi112 ceph-mon[35051]: pgmap v248: 97 pgs: 97 active+clean; 4.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 1.78k op/s 2024-09-19T04:23:09.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:08 smithi112 ceph-mon[35051]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:23:09.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:08 smithi094 ceph-mon[30972]: pgmap v248: 97 pgs: 97 active+clean; 4.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 1.78k op/s 2024-09-19T04:23:09.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:08 smithi094 ceph-mon[30972]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:23:09.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:08 smithi104 ceph-mon[34145]: pgmap v248: 97 pgs: 97 active+clean; 4.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 1.78k op/s 2024-09-19T04:23:09.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:08 smithi104 ceph-mon[34145]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:23:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 415.0 s, 1519.1 tps, lat 64.212 ms stddev 24.079 2024-09-19T04:23:11.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:10 smithi112 ceph-mon[35051]: pgmap v250: 97 pgs: 97 active+clean; 4.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-09-19T04:23:11.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:10 smithi094 ceph-mon[30972]: pgmap v250: 97 pgs: 97 active+clean; 4.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-09-19T04:23:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:10 smithi104 ceph-mon[34145]: pgmap v250: 97 pgs: 97 active+clean; 4.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-09-19T04:23:13.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:12 smithi112 ceph-mon[35051]: pgmap v251: 97 pgs: 97 active+clean; 4.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-09-19T04:23:13.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:12 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:23:13.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:12 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:23:13.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:12 smithi094 ceph-mon[30972]: pgmap v251: 97 pgs: 97 active+clean; 4.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-09-19T04:23:13.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:12 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:23:13.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:12 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:23:13.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:12 smithi104 ceph-mon[34145]: pgmap v251: 97 pgs: 97 active+clean; 4.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-09-19T04:23:13.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:12 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:23:13.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:12 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:23:14.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:13 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:23:14.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:13 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:23:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:13 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:23:15.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:14 smithi112 ceph-mon[35051]: pgmap v252: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-09-19T04:23:15.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:14 smithi094 ceph-mon[30972]: pgmap v252: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-09-19T04:23:15.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:14 smithi104 ceph-mon[34145]: pgmap v252: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-09-19T04:23:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 420.0 s, 1464.8 tps, lat 69.780 ms stddev 47.618 2024-09-19T04:23:16.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:16 smithi094 ceph-mon[30972]: pgmap v253: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-09-19T04:23:16.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:16 smithi094 ceph-mon[30972]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:23:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:16 smithi104 ceph-mon[34145]: pgmap v253: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-09-19T04:23:16.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:16 smithi104 ceph-mon[34145]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:23:17.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:16 smithi112 ceph-mon[35051]: pgmap v253: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-09-19T04:23:17.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:16 smithi112 ceph-mon[35051]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:23:18.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:18 smithi094 ceph-mon[30972]: pgmap v255: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-09-19T04:23:18.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:18 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:23:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:18 smithi104 ceph-mon[34145]: pgmap v255: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-09-19T04:23:18.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:18 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:23:19.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:18 smithi112 ceph-mon[35051]: pgmap v255: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-09-19T04:23:19.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:18 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:23:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:19 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:23:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:19 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:23:19.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:19 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:23:20.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:19 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:23:20.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:19 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:23:20.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:19 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:23:20.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:19 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:23:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:19 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:23:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:19 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:23:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 425.0 s, 1500.8 tps, lat 66.739 ms stddev 34.812 2024-09-19T04:23:21.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:21 smithi094 ceph-mon[30972]: pgmap v256: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:23:21.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:21 smithi104 ceph-mon[34145]: pgmap v256: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:23:21.537 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:21 smithi112 ceph-mon[35051]: pgmap v256: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:23:23.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:23 smithi094 ceph-mon[30972]: pgmap v257: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:23:23.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:23 smithi104 ceph-mon[34145]: pgmap v257: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:23:23.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:23 smithi112 ceph-mon[35051]: pgmap v257: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-09-19T04:23:25.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:25 smithi094 ceph-mon[30972]: pgmap v258: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:23:25.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:25 smithi104 ceph-mon[34145]: pgmap v258: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:23:25.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:25 smithi112 ceph-mon[35051]: pgmap v258: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:23:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 430.0 s, 1481.6 tps, lat 67.438 ms stddev 24.403 2024-09-19T04:23:27.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:27 smithi094 ceph-mon[30972]: pgmap v259: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:23:27.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:27 smithi104 ceph-mon[34145]: pgmap v259: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:23:27.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:27 smithi112 ceph-mon[35051]: pgmap v259: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-09-19T04:23:29.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:29 smithi094 ceph-mon[30972]: pgmap v260: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 1.74k op/s 2024-09-19T04:23:29.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:29 smithi104 ceph-mon[34145]: pgmap v260: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 1.74k op/s 2024-09-19T04:23:29.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:29 smithi112 ceph-mon[35051]: pgmap v260: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 1.74k op/s 2024-09-19T04:23:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 435.0 s, 1415.8 tps, lat 70.704 ms stddev 21.628 2024-09-19T04:23:31.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:31 smithi104 ceph-mon[34145]: pgmap v261: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-09-19T04:23:31.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:31 smithi112 ceph-mon[35051]: pgmap v261: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-09-19T04:23:31.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:31 smithi094 ceph-mon[30972]: pgmap v261: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-09-19T04:23:33.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:33 smithi104 ceph-mon[34145]: pgmap v262: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-09-19T04:23:33.506 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:33 smithi094 ceph-mon[30972]: pgmap v262: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-09-19T04:23:33.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:33 smithi112 ceph-mon[35051]: pgmap v262: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-09-19T04:23:35.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:23:35.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:35 smithi104 ceph-mon[34145]: pgmap v263: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-09-19T04:23:35.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:35 smithi112 ceph-mon[35051]: pgmap v263: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-09-19T04:23:35.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:35 smithi094 ceph-mon[30972]: pgmap v263: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-09-19T04:23:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 440.0 s, 1446.0 tps, lat 68.896 ms stddev 23.056 2024-09-19T04:23:36.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.159+0000 7f11ef3a8640 1 -- 172.21.15.94:0/341001502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e81083f0 msgr2=0x7f11e81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:36.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.159+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/341001502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e81083f0 0x7f11e81087d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009ca0 tx=0x7f11e002a8b0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.161+0000 7f11ef3a8640 1 -- 172.21.15.94:0/341001502 shutdown_connections 2024-09-19T04:23:36.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.161+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/341001502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e810d520 0x7f11e81150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.161+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/341001502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e8108da0 0x7f11e810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.161+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/341001502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e81083f0 0x7f11e81087d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.161+0000 7f11ef3a8640 1 -- 172.21.15.94:0/341001502 >> 172.21.15.94:0/341001502 conn(0x7f11e8077af0 msgr2=0x7f11e8079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:36.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.163+0000 7f11ef3a8640 1 -- 172.21.15.94:0/341001502 shutdown_connections 2024-09-19T04:23:36.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.163+0000 7f11ef3a8640 1 -- 172.21.15.94:0/341001502 wait complete. 2024-09-19T04:23:36.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.164+0000 7f11ef3a8640 1 Processor -- start 2024-09-19T04:23:36.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.164+0000 7f11ef3a8640 1 -- start start 2024-09-19T04:23:36.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.164+0000 7f11ef3a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e81083f0 0x7f11e81a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.164+0000 7f11ef3a8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e8108da0 0x7f11e81a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.164+0000 7f11ef3a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e810d520 0x7f11e819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ef3a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8118440 con 0x7f11e81083f0 2024-09-19T04:23:36.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ef3a8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81182c0 con 0x7f11e8108da0 2024-09-19T04:23:36.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ef3a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81185c0 con 0x7f11e810d520 2024-09-19T04:23:36.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ed144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e81083f0 0x7f11e81a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ed144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e81083f0 0x7f11e81a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48604/0 (socket says 172.21.15.94:48604) 2024-09-19T04:23:36.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ec943640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e8108da0 0x7f11e81a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ed144640 1 -- 172.21.15.94:0/2120108325 learned_addr learned my addr 172.21.15.94:0/2120108325 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:36.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ed945640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e810d520 0x7f11e819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ed144640 1 -- 172.21.15.94:0/2120108325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e810d520 msgr2=0x7f11e819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ed144640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e810d520 0x7f11e819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ed144640 1 -- 172.21.15.94:0/2120108325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e8108da0 msgr2=0x7f11e81a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ed144640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e8108da0 0x7f11e81a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ed144640 1 -- 172.21.15.94:0/2120108325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e819ef70 con 0x7f11e81083f0 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.165+0000 7f11ec943640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e8108da0 0x7f11e81a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.166+0000 7f11ed144640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e81083f0 0x7f11e81a4650 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f11e0005e30 tx=0x7f11e0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.166+0000 7f11de7fc640 1 -- 172.21.15.94:0/2120108325 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0004040 con 0x7f11e81083f0 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.166+0000 7f11de7fc640 1 -- 172.21.15.94:0/2120108325 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11e0040070 con 0x7f11e81083f0 2024-09-19T04:23:36.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.166+0000 7f11ef3a8640 1 -- 172.21.15.94:0/2120108325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e819f200 con 0x7f11e81083f0 2024-09-19T04:23:36.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.166+0000 7f11ef3a8640 1 -- 172.21.15.94:0/2120108325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e81ab4b0 con 0x7f11e81083f0 2024-09-19T04:23:36.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.166+0000 7f11de7fc640 1 -- 172.21.15.94:0/2120108325 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0031ba0 con 0x7f11e81083f0 2024-09-19T04:23:36.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.169+0000 7f11f1781740 1 Processor -- start 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11f1781740 1 -- start start 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11f1781740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddc360 0x563fbdddc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11f1781740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddcc80 0x563fbddf8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11de7fc640 1 -- 172.21.15.94:0/2120108325 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f11e003b430 con 0x7f11e81083f0 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11f1781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 0x563fbddfaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11f1781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563fbdd422e0 con 0x563fbddf8690 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11f1781740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563fbdce9ce0 con 0x563fbdddc360 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11f1781740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563fbdd3e2a0 con 0x563fbdddcc80 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11ed945640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddc360 0x563fbdddc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11ed945640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddc360 0x563fbdddc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33382/0 (socket says 172.21.15.94:33382) 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11ed945640 1 -- 172.21.15.94:0/3915823540 learned_addr learned my addr 172.21.15.94:0/3915823540 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.170+0000 7f11ec943640 1 --2- 172.21.15.94:0/3915823540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 0x563fbddfaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.172+0000 7f11ed945640 1 -- 172.21.15.94:0/3915823540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddcc80 msgr2=0x563fbddf8150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:23:36.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.172+0000 7f11ed945640 1 --2- 172.21.15.94:0/3915823540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddcc80 0x563fbddf8150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.172+0000 7f11ed945640 1 -- 172.21.15.94:0/3915823540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 msgr2=0x563fbddfaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:36.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.172+0000 7f11ed945640 1 --2- 172.21.15.94:0/3915823540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 0x563fbddfaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.172+0000 7f11ed945640 1 -- 172.21.15.94:0/3915823540 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fbdcce440 con 0x563fbdddc360 2024-09-19T04:23:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.172+0000 7f11ed945640 1 --2- 172.21.15.94:0/3915823540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddc360 0x563fbdddc740 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009c20 tx=0x7f11d8033820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48a2273d78f52161 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.172+0000 7f11de7fc640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11d40a34b0 0x7f11d40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.172+0000 7f11d37fe640 1 -- 172.21.15.94:0/3915823540 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8040070 con 0x563fbdddc360 2024-09-19T04:23:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11d37fe640 1 -- 172.21.15.94:0/3915823540 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d803b8c0 con 0x563fbdddc360 2024-09-19T04:23:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11d37fe640 1 -- 172.21.15.94:0/3915823540 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800e770 con 0x563fbdddc360 2024-09-19T04:23:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11de7fc640 1 -- 172.21.15.94:0/2120108325 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f11e00f0cd0 con 0x7f11e81083f0 2024-09-19T04:23:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11ed144640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11d40a34b0 0x7f11d40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11f1781740 1 -- 172.21.15.94:0/3915823540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddc360 msgr2=0x563fbdddc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11f1781740 1 --2- 172.21.15.94:0/3915823540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddc360 0x563fbdddc740 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009c20 tx=0x7f11d8033820 comp rx=0 tx=0).stop 2024-09-19T04:23:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11f1781740 1 -- 172.21.15.94:0/3915823540 shutdown_connections 2024-09-19T04:23:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11f1781740 1 --2- 172.21.15.94:0/3915823540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 0x563fbddfaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11f1781740 1 --2- 172.21.15.94:0/3915823540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddcc80 0x563fbddf8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11f1781740 1 --2- 172.21.15.94:0/3915823540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddc360 0x563fbdddc740 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.173+0000 7f11f1781740 1 -- 172.21.15.94:0/3915823540 >> 172.21.15.94:0/3915823540 conn(0x563fbddd8a30 msgr2=0x563fbddd8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.174+0000 7f11f1781740 1 -- 172.21.15.94:0/3915823540 shutdown_connections 2024-09-19T04:23:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.174+0000 7f11f1781740 1 -- 172.21.15.94:0/3915823540 wait complete. 2024-09-19T04:23:36.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.174+0000 7f11f1781740 1 -- start start 2024-09-19T04:23:36.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.174+0000 7f11ed144640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11d40a34b0 0x7f11d40a5970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f11e00097b0 tx=0x7f11e0000f40 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:36.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.175+0000 7f11f1781740 1 Processor -- start 2024-09-19T04:23:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.175+0000 7f11f1781740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbdde96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.175+0000 7f11f1781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddcc80 0x563fbdde9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11f1781740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbddf8690 0x563fbdde4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11f1781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563fbdddf320 con 0x563fbdddcc80 2024-09-19T04:23:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11f1781740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563fbdcfd050 con 0x563fbddf8690 2024-09-19T04:23:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11f1781740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563fbdddf4a0 con 0x563fbdddc360 2024-09-19T04:23:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ed945640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbdde96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ed945640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbdde96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58044/0 (socket says 172.21.15.94:58044) 2024-09-19T04:23:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11f1781740 10 client.0 _ll_register_callbacks cb 0x563fbdd3d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ed945640 1 -- 172.21.15.94:0/161201123 learned_addr learned my addr 172.21.15.94:0/161201123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ec943640 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddcc80 0x563fbdde9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ed945640 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbddf8690 0x563fbdde4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ed945640 1 -- 172.21.15.94:0/161201123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbddf8690 msgr2=0x563fbdde4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ed945640 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbddf8690 0x563fbdde4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ed945640 1 -- 172.21.15.94:0/161201123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddcc80 msgr2=0x563fbdde9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ed945640 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddcc80 0x563fbdde9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ed945640 1 -- 172.21.15.94:0/161201123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fbdab33c0 con 0x563fbdddc360 2024-09-19T04:23:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.176+0000 7f11ec943640 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddcc80 0x563fbdde9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:23:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.177+0000 7f11ed945640 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbdde96e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009ed0 tx=0x7f11d80099b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=417f18ef7880cbdd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.177+0000 7f11d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:23:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.177+0000 7f11d0ff9640 1 -- 172.21.15.94:0/161201123 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8040070 con 0x563fbdddc360 2024-09-19T04:23:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.177+0000 7f11d0ff9640 1 -- 172.21.15.94:0/161201123 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f11d8005250 con 0x563fbdddc360 2024-09-19T04:23:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.177+0000 7f11f1781740 1 -- 172.21.15.94:0/161201123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d803b8c0 con 0x563fbdddc360 2024-09-19T04:23:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.177+0000 7f11d0ff9640 1 -- 172.21.15.94:0/161201123 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f11d8005450 con 0x563fbdddc360 2024-09-19T04:23:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.177+0000 7f11f1781740 10 client.25981 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.177+0000 7f11f1781740 1 -- 172.21.15.94:0/161201123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563fbddf33b0 con 0x563fbdddc360 2024-09-19T04:23:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.178+0000 7f11d0ff9640 1 -- 172.21.15.94:0/161201123 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d803b8c0 con 0x563fbdddc360 2024-09-19T04:23:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.178+0000 7f11d0ff9640 1 -- 172.21.15.94:0/161201123 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f11d800e4f0 con 0x563fbdddc360 2024-09-19T04:23:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.178+0000 7f11f1781740 10 client.25981 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.178+0000 7f11f1781740 10 client.25981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.178+0000 7f11f1781740 20 client.25981 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.178+0000 7f11f1781740 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563fbddf4b80 0x563fbddf6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.178+0000 7f11f1781740 4 client.25981 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.178+0000 7f11f1781740 1 -- 172.21.15.94:0/161201123 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fbdd4f850 con 0x563fbddf4b80 2024-09-19T04:23:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.178+0000 7f11ec943640 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563fbddf4b80 0x563fbddf6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.179+0000 7f11ec943640 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563fbddf4b80 0x563fbddf6fa0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a569729846219ecc server_cookie=c8be513cea991109 in_seq=0 out_seq=0 2024-09-19T04:23:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.179+0000 7f11d0ff9640 10 client.25981 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:36.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.180+0000 7f11d0ff9640 1 -- 172.21.15.94:0/161201123 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fbdd4f850 con 0x563fbddf4b80 2024-09-19T04:23:36.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.180+0000 7f11d0ff9640 10 client.25981 handle_command_reply: tid=0 2024-09-19T04:23:36.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.180+0000 7f11f1781740 1 client.25981 shutdown 2024-09-19T04:23:36.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.181+0000 7f11f1781740 1 -- 172.21.15.94:0/161201123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 msgr2=0x563fbdde96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:36.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.181+0000 7f11f1781740 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbdde96e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009ed0 tx=0x7f11d80099b0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.181+0000 7f11f1781740 1 -- 172.21.15.94:0/161201123 shutdown_connections 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.181+0000 7f11f1781740 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563fbddf4b80 0x563fbddf6fa0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.181+0000 7f11f1781740 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbddf8690 0x563fbdde4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.181+0000 7f11f1781740 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddcc80 0x563fbdde9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.181+0000 7f11f1781740 1 --2- 172.21.15.94:0/161201123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbdde96e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.181+0000 7f11f1781740 1 -- 172.21.15.94:0/161201123 >> 172.21.15.94:0/161201123 conn(0x563fbddd8a30 msgr2=0x563fbddde790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.181+0000 7f11d0ff9640 0 client.25981 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.182+0000 7f11f1781740 1 -- 172.21.15.94:0/161201123 shutdown_connections 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.182+0000 7f11f1781740 1 -- 172.21.15.94:0/161201123 wait complete. 2024-09-19T04:23:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.182+0000 7f11f1781740 20 client.25981 trim_cache size 0 max 16384 2024-09-19T04:23:36.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.211+0000 7f11f1781740 1 Processor -- start 2024-09-19T04:23:36.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.211+0000 7f11f1781740 1 -- start start 2024-09-19T04:23:36.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.212+0000 7f11f1781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddc360 0x563fbddf7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.212+0000 7f11f1781740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbdde1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.212+0000 7f11f1781740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbddf8690 0x563fbdde21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.212+0000 7f11f1781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc011150 con 0x563fbdddc360 2024-09-19T04:23:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.212+0000 7f11f1781740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc00e9c0 con 0x563fbdddcc80 2024-09-19T04:23:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.212+0000 7f11f1781740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc00c830 con 0x563fbddf8690 2024-09-19T04:23:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.212+0000 7f11ed945640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbddf8690 0x563fbdde21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.212+0000 7f11ed945640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbddf8690 0x563fbdde21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58046/0 (socket says 172.21.15.94:58046) 2024-09-19T04:23:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.212+0000 7f11ed945640 1 -- 172.21.15.94:0/1989778817 learned_addr learned my addr 172.21.15.94:0/1989778817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.213+0000 7f11ec943640 1 --2- 172.21.15.94:0/1989778817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbdde1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.213+0000 7f11ec943640 1 -- 172.21.15.94:0/1989778817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbddf8690 msgr2=0x563fbdde21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.213+0000 7f11ec943640 1 --2- 172.21.15.94:0/1989778817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbddf8690 0x563fbdde21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.213+0000 7f11ec943640 1 -- 172.21.15.94:0/1989778817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddc360 msgr2=0x563fbddf7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.213+0000 7f11ec943640 1 --2- 172.21.15.94:0/1989778817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddc360 0x563fbddf7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.213+0000 7f11ec943640 1 -- 172.21.15.94:0/1989778817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fbddd8860 con 0x563fbdddcc80 2024-09-19T04:23:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.215+0000 7f11ec943640 1 --2- 172.21.15.94:0/1989778817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbdde1c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x563fbdde4290 tx=0x7f11e400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69da4e9acf557cfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.216+0000 7f11d2ffd640 1 -- 172.21.15.94:0/1989778817 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4004220 con 0x563fbdddcc80 2024-09-19T04:23:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.216+0000 7f11d2ffd640 1 -- 172.21.15.94:0/1989778817 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11e40043c0 con 0x563fbdddcc80 2024-09-19T04:23:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.216+0000 7f11d2ffd640 1 -- 172.21.15.94:0/1989778817 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e40047a0 con 0x563fbdddcc80 2024-09-19T04:23:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.217+0000 7f11f1781740 1 -- 172.21.15.94:0/1989778817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 msgr2=0x563fbdde1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.217+0000 7f11f1781740 1 --2- 172.21.15.94:0/1989778817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbdde1c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x563fbdde4290 tx=0x7f11e400b390 comp rx=0 tx=0).stop 2024-09-19T04:23:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.217+0000 7f11f1781740 1 -- 172.21.15.94:0/1989778817 shutdown_connections 2024-09-19T04:23:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.217+0000 7f11f1781740 1 --2- 172.21.15.94:0/1989778817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbddf8690 0x563fbdde21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.217+0000 7f11f1781740 1 --2- 172.21.15.94:0/1989778817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbdde1c60 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x563fbdde4290 tx=0x7f11e400b390 comp rx=0 tx=0).stop 2024-09-19T04:23:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.217+0000 7f11f1781740 1 --2- 172.21.15.94:0/1989778817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbdddc360 0x563fbddf7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.217+0000 7f11f1781740 1 -- 172.21.15.94:0/1989778817 >> 172.21.15.94:0/1989778817 conn(0x563fbddd8a30 msgr2=0x563fbddf7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.217+0000 7f11f1781740 1 -- 172.21.15.94:0/1989778817 shutdown_connections 2024-09-19T04:23:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.218+0000 7f11f1781740 1 -- 172.21.15.94:0/1989778817 wait complete. 2024-09-19T04:23:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.218+0000 7f11f1781740 1 -- start start 2024-09-19T04:23:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.218+0000 7f11f1781740 1 Processor -- start 2024-09-19T04:23:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11f1781740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbddfc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11f1781740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbddfc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11f1781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 0x563fbddfcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11f1781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc00ce80 con 0x563fbddf8690 2024-09-19T04:23:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11f1781740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc00e2f0 con 0x563fbdddcc80 2024-09-19T04:23:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11f1781740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc00d550 con 0x563fbdddc360 2024-09-19T04:23:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11f1781740 10 client.0 _ll_register_callbacks cb 0x7f11d803e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11ed945640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbddfc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11ec943640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbddfc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11ed945640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbddfc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58060/0 (socket says 172.21.15.94:58060) 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11ec943640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbddfc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33416/0 (socket says 172.21.15.94:33416) 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.219+0000 7f11ed945640 1 -- 172.21.15.94:0/4156675563 learned_addr learned my addr 172.21.15.94:0/4156675563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11ed945640 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 0x563fbddfcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11ec943640 1 -- 172.21.15.94:0/4156675563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 msgr2=0x563fbddfc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11ec943640 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbddfc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11ec943640 1 -- 172.21.15.94:0/4156675563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 msgr2=0x563fbddfcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11ec943640 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 0x563fbddfcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11ec943640 1 -- 172.21.15.94:0/4156675563 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fbdde59a0 con 0x563fbdddcc80 2024-09-19T04:23:36.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11ed945640 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbddfc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:36.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11ec943640 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbddfc750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f11e4000c00 tx=0x7f11e4005580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=345a8901f6485994 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:36.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.220+0000 7f11d0ff9640 1 -- 172.21.15.94:0/4156675563 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4010b00 con 0x563fbdddcc80 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.221+0000 7f11d0ff9640 1 -- 172.21.15.94:0/4156675563 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f11e4010ca0 con 0x563fbdddcc80 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.221+0000 7f11f1781740 1 -- 172.21.15.94:0/4156675563 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e40043c0 con 0x563fbdddcc80 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.221+0000 7f11d0ff9640 1 -- 172.21.15.94:0/4156675563 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f11e40035f0 con 0x563fbdddcc80 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.221+0000 7f11f1781740 10 client.25997 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.221+0000 7f11f1781740 1 -- 172.21.15.94:0/4156675563 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563fbde613f0 con 0x563fbdddcc80 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.221+0000 7f11d0ff9640 1 -- 172.21.15.94:0/4156675563 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11e40043c0 con 0x563fbdddcc80 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.222+0000 7f11d0ff9640 1 -- 172.21.15.94:0/4156675563 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f11e4024070 con 0x563fbdddcc80 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.222+0000 7f11f1781740 10 client.25997 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.222+0000 7f11f1781740 10 client.25997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:36.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.222+0000 7f11f1781740 20 client.25997 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:36.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.222+0000 7f11f1781740 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563fbde61af0 0x563fbde63f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:36.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.222+0000 7f11f1781740 4 client.25997 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:23:36.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.222+0000 7f11f1781740 1 -- 172.21.15.94:0/4156675563 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fbdd49d60 con 0x563fbde61af0 2024-09-19T04:23:36.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.222+0000 7f11ed945640 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563fbde61af0 0x563fbde63f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:36.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.223+0000 7f11ed945640 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563fbde61af0 0x563fbde63f10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89189a4938124c45 server_cookie=35c8050f66691f72 in_seq=0 out_seq=0 2024-09-19T04:23:36.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.223+0000 7f11d0ff9640 10 client.25997 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:36.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.223+0000 7f11d0ff9640 1 -- 172.21.15.94:0/4156675563 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563fbdd49d60 con 0x563fbde61af0 2024-09-19T04:23:36.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.223+0000 7f11d0ff9640 10 client.25997 handle_command_reply: tid=0 2024-09-19T04:23:36.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.223+0000 7f11f1781740 1 client.25997 shutdown 2024-09-19T04:23:36.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11f1781740 1 -- 172.21.15.94:0/4156675563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 msgr2=0x563fbddfc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:36.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11f1781740 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbddfc750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f11e4000c00 tx=0x7f11e4005580 comp rx=0 tx=0).stop 2024-09-19T04:23:36.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11f1781740 1 -- 172.21.15.94:0/4156675563 shutdown_connections 2024-09-19T04:23:36.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11f1781740 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563fbde61af0 0x563fbde63f10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11f1781740 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fbddf8690 0x563fbddfcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11f1781740 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fbdddcc80 0x563fbddfc750 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11d0ff9640 0 client.25997 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:36.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11f1781740 1 --2- 172.21.15.94:0/4156675563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbdddc360 0x563fbddfc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11f1781740 1 -- 172.21.15.94:0/4156675563 >> 172.21.15.94:0/4156675563 conn(0x563fbddd8a30 msgr2=0x563fbdddf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:36.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.225+0000 7f11f1781740 1 -- 172.21.15.94:0/4156675563 shutdown_connections 2024-09-19T04:23:36.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.226+0000 7f11f1781740 1 -- 172.21.15.94:0/4156675563 wait complete. 2024-09-19T04:23:36.228 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:23:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:23:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:23:36.229 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:23:36.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.226+0000 7f11f1781740 20 client.25997 trim_cache size 0 max 16384 2024-09-19T04:23:36.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 -- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11d40a34b0 msgr2=0x7f11d40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:36.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11d40a34b0 0x7f11d40a5970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f11e00097b0 tx=0x7f11e0000f40 comp rx=0 tx=0).stop 2024-09-19T04:23:36.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 -- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e81083f0 msgr2=0x7f11e81a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:36.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e81083f0 0x7f11e81a4650 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f11e0005e30 tx=0x7f11e0003970 comp rx=0 tx=0).stop 2024-09-19T04:23:36.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 -- 172.21.15.94:0/2120108325 shutdown_connections 2024-09-19T04:23:36.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11d40a34b0 0x7f11d40a5970 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11e810d520 0x7f11e819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e8108da0 0x7f11e81a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 --2- 172.21.15.94:0/2120108325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e81083f0 0x7f11e81a4650 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:36.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.227+0000 7f11ef3a8640 1 -- 172.21.15.94:0/2120108325 >> 172.21.15.94:0/2120108325 conn(0x7f11e8077af0 msgr2=0x7f11e8078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:36.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.228+0000 7f11ef3a8640 1 -- 172.21.15.94:0/2120108325 shutdown_connections 2024-09-19T04:23:36.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:36.228+0000 7f11ef3a8640 1 -- 172.21.15.94:0/2120108325 wait complete. 2024-09-19T04:23:37.110 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:23:37.111 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:23:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:37 smithi094 ceph-mon[30972]: pgmap v264: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.8 MiB/s wr, 1.14k op/s 2024-09-19T04:23:37.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:37 smithi104 ceph-mon[34145]: pgmap v264: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.8 MiB/s wr, 1.14k op/s 2024-09-19T04:23:37.528 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:37.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:37 smithi112 ceph-mon[35051]: pgmap v264: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.8 MiB/s wr, 1.14k op/s 2024-09-19T04:23:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.329+0000 7fd3acb5b640 1 -- 172.21.15.94:0/3824290354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8077040 msgr2=0x7fd3a81124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.329+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/3824290354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8077040 0x7fd3a81124f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd398009860 tx=0x7fd39802a340 comp rx=0 tx=0).stop 2024-09-19T04:23:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.329+0000 7fd3acb5b640 1 -- 172.21.15.94:0/3824290354 shutdown_connections 2024-09-19T04:23:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.329+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/3824290354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a8112a30 0x7fd3a8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.329+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/3824290354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8077040 0x7fd3a81124f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.329+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/3824290354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a8076720 0x7fd3a8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.329+0000 7fd3acb5b640 1 -- 172.21.15.94:0/3824290354 >> 172.21.15.94:0/3824290354 conn(0x7fd3a80ffab0 msgr2=0x7fd3a8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.330+0000 7fd3acb5b640 1 -- 172.21.15.94:0/3824290354 shutdown_connections 2024-09-19T04:23:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.330+0000 7fd3acb5b640 1 -- 172.21.15.94:0/3824290354 wait complete. 2024-09-19T04:23:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.331+0000 7fd3acb5b640 1 Processor -- start 2024-09-19T04:23:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.331+0000 7fd3acb5b640 1 -- start start 2024-09-19T04:23:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.332+0000 7fd3acb5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8076720 0x7fd3a8112040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.332+0000 7fd3acb5b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a8077040 0x7fd3a810b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.332+0000 7fd3acb5b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a8112a30 0x7fd3a810b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.332+0000 7fd3acb5b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8118450 con 0x7fd3a8076720 2024-09-19T04:23:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.332+0000 7fd3acb5b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81182d0 con 0x7fd3a8077040 2024-09-19T04:23:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.332+0000 7fd3acb5b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81185d0 con 0x7fd3a8112a30 2024-09-19T04:23:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.332+0000 7fd3a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8076720 0x7fd3a8112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.333+0000 7fd3a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8076720 0x7fd3a8112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48648/0 (socket says 172.21.15.94:48648) 2024-09-19T04:23:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.333+0000 7fd3a659c640 1 -- 172.21.15.94:0/25431629 learned_addr learned my addr 172.21.15.94:0/25431629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.333+0000 7fd3a6d9d640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a8112a30 0x7fd3a810b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.333+0000 7fd3a659c640 1 -- 172.21.15.94:0/25431629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a8112a30 msgr2=0x7fd3a810b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.333+0000 7fd3a659c640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a8112a30 0x7fd3a810b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.333+0000 7fd3a659c640 1 -- 172.21.15.94:0/25431629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a8077040 msgr2=0x7fd3a810b0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:23:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.333+0000 7fd3a659c640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a8077040 0x7fd3a810b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.333+0000 7fd3a659c640 1 -- 172.21.15.94:0/25431629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a810bc00 con 0x7fd3a8076720 2024-09-19T04:23:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.334+0000 7fd3a659c640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8076720 0x7fd3a8112040 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd39000e9c0 tx=0x7fd39000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.334+0000 7fd39ffff640 1 -- 172.21.15.94:0/25431629 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39000ccc0 con 0x7fd3a8076720 2024-09-19T04:23:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.334+0000 7fd39ffff640 1 -- 172.21.15.94:0/25431629 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3900054d0 con 0x7fd3a8076720 2024-09-19T04:23:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.334+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a810bef0 con 0x7fd3a8076720 2024-09-19T04:23:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.334+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a810fee0 con 0x7fd3a8076720 2024-09-19T04:23:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.335+0000 7fd39ffff640 1 -- 172.21.15.94:0/25431629 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39000ccc0 con 0x7fd3a8076720 2024-09-19T04:23:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.336+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a8077e60 con 0x7fd3a8076720 2024-09-19T04:23:38.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.338+0000 7fd39ffff640 1 -- 172.21.15.94:0/25431629 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd390004030 con 0x7fd3a8076720 2024-09-19T04:23:38.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.339+0000 7fd39ffff640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd3940a3830 0x7fd3940a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.339+0000 7fd39ffff640 1 -- 172.21.15.94:0/25431629 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fd39000cac0 con 0x7fd3a8076720 2024-09-19T04:23:38.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.340+0000 7fd39dd9b640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd3940a3830 0x7fd3940a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.341+0000 7fd39dd9b640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd3940a3830 0x7fd3940a5cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd39802a860 tx=0x7fd398000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:38.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.345+0000 7fd39ffff640 1 -- 172.21.15.94:0/25431629 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd39009c580 con 0x7fd3a8076720 2024-09-19T04:23:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.727+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3a810c600 con 0x7fd3a8076720 2024-09-19T04:23:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.728+0000 7fd39ffff640 1 -- 172.21.15.94:0/25431629 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11921 (secure 0 0 0) 0x7fd39009be80 con 0x7fd3a8076720 2024-09-19T04:23:38.730 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:23:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":42,"btime":"2024-09-19T04:22:52:651391+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:22:51.816040+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:23:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 42 2024-09-19T04:23:38.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd3940a3830 msgr2=0x7fd3940a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:38.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd3940a3830 0x7fd3940a5cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd39802a860 tx=0x7fd398000bc0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8076720 msgr2=0x7fd3a8112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:38.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8076720 0x7fd3a8112040 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd39000e9c0 tx=0x7fd39000ee80 comp rx=0 tx=0).stop 2024-09-19T04:23:38.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 shutdown_connections 2024-09-19T04:23:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd3940a3830 0x7fd3940a5cf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a8112a30 0x7fd3a810b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a8077040 0x7fd3a810b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 --2- 172.21.15.94:0/25431629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3a8076720 0x7fd3a8112040 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.731+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 >> 172.21.15.94:0/25431629 conn(0x7fd3a80ffab0 msgr2=0x7fd3a8101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.732+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 shutdown_connections 2024-09-19T04:23:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:38.732+0000 7fd3acb5b640 1 -- 172.21.15.94:0/25431629 wait complete. 2024-09-19T04:23:39.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:39 smithi094 ceph-mon[30972]: pgmap v265: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 15 MiB/s wr, 1.72k op/s 2024-09-19T04:23:39.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:39 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/25431629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:23:39.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:39 smithi112 ceph-mon[35051]: pgmap v265: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 15 MiB/s wr, 1.72k op/s 2024-09-19T04:23:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:39 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/25431629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:23:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:39 smithi104 ceph-mon[34145]: pgmap v265: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 15 MiB/s wr, 1.72k op/s 2024-09-19T04:23:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:39 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/25431629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:23:39.665 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:23:40.094 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 445.0 s, 1323.6 tps, lat 75.706 ms stddev 25.289 2024-09-19T04:23:40.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.656+0000 7f7f580c7640 1 -- 172.21.15.94:0/2584295068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f50100e20 msgr2=0x7f7f50101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.656+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2584295068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f50100e20 0x7f7f50101200 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009930 tx=0x7f7f4002a4d0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.656+0000 7f7f580c7640 1 -- 172.21.15.94:0/2584295068 shutdown_connections 2024-09-19T04:23:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.656+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2584295068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f50102ef0 0x7f7f5010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.656+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2584295068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50102570 0x7f7f501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.656+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2584295068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f50100e20 0x7f7f50101200 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.656+0000 7f7f580c7640 1 -- 172.21.15.94:0/2584295068 >> 172.21.15.94:0/2584295068 conn(0x7f7f500fc7d0 msgr2=0x7f7f500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.656+0000 7f7f580c7640 1 -- 172.21.15.94:0/2584295068 shutdown_connections 2024-09-19T04:23:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.656+0000 7f7f580c7640 1 -- 172.21.15.94:0/2584295068 wait complete. 2024-09-19T04:23:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.657+0000 7f7f580c7640 1 Processor -- start 2024-09-19T04:23:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.657+0000 7f7f580c7640 1 -- start start 2024-09-19T04:23:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f580c7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f50100e20 0x7f7f5019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f580c7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f50102570 0x7f7f5019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f580c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50102ef0 0x7f7f5019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f580c7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5010f400 con 0x7f7f50102ef0 2024-09-19T04:23:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f580c7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5010f280 con 0x7f7f50102570 2024-09-19T04:23:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f580c7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5010f580 con 0x7f7f50100e20 2024-09-19T04:23:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f55e63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f50100e20 0x7f7f5019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f56664640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50102ef0 0x7f7f5019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f56664640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50102ef0 0x7f7f5019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48658/0 (socket says 172.21.15.94:48658) 2024-09-19T04:23:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f56664640 1 -- 172.21.15.94:0/2655061459 learned_addr learned my addr 172.21.15.94:0/2655061459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f55662640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f50102570 0x7f7f5019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.658+0000 7f7f55e63640 1 -- 172.21.15.94:0/2655061459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f50102570 msgr2=0x7f7f5019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:40.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.659+0000 7f7f55e63640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f50102570 0x7f7f5019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.659+0000 7f7f55e63640 1 -- 172.21.15.94:0/2655061459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50102ef0 msgr2=0x7f7f5019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:40.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.659+0000 7f7f55e63640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50102ef0 0x7f7f5019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.659+0000 7f7f55e63640 1 -- 172.21.15.94:0/2655061459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5019f410 con 0x7f7f50100e20 2024-09-19T04:23:40.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.659+0000 7f7f55e63640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f50100e20 0x7f7f5019a460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009850 tx=0x7f7f40030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:40.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.659+0000 7f7f4effd640 1 -- 172.21.15.94:0/2655061459 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4003f070 con 0x7f7f50100e20 2024-09-19T04:23:40.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.659+0000 7f7f4effd640 1 -- 172.21.15.94:0/2655061459 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f40004460 con 0x7f7f50100e20 2024-09-19T04:23:40.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.659+0000 7f7f4effd640 1 -- 172.21.15.94:0/2655061459 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40035030 con 0x7f7f50100e20 2024-09-19T04:23:40.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.659+0000 7f7f580c7640 1 -- 172.21.15.94:0/2655061459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5019f6a0 con 0x7f7f50100e20 2024-09-19T04:23:40.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.660+0000 7f7f580c7640 1 -- 172.21.15.94:0/2655061459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5019fb00 con 0x7f7f50100e20 2024-09-19T04:23:40.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.663+0000 7f7f4effd640 1 -- 172.21.15.94:0/2655061459 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7f40042070 con 0x7f7f50100e20 2024-09-19T04:23:40.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.663+0000 7f7f5a4a0740 1 Processor -- start 2024-09-19T04:23:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.663+0000 7f7f5a4a0740 1 -- start start 2024-09-19T04:23:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 0x560c7df10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df105d0 0x560c7df10a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df23ba0 0x560c7df25f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560c7de762f0 con 0x560c7df105d0 2024-09-19T04:23:40.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560c7de1dcf0 con 0x560c7df23ba0 2024-09-19T04:23:40.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560c7de722b0 con 0x560c7df11b50 2024-09-19T04:23:40.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f56664640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 0x560c7df10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f55662640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df23ba0 0x560c7df25f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f55662640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df23ba0 0x560c7df25f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33460/0 (socket says 172.21.15.94:33460) 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f55662640 1 -- 172.21.15.94:0/7791620 learned_addr learned my addr 172.21.15.94:0/7791620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f55e63640 1 --2- 172.21.15.94:0/7791620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df105d0 0x560c7df10a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f55662640 1 -- 172.21.15.94:0/7791620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 msgr2=0x560c7df10090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.664+0000 7f7f55662640 1 --2- 172.21.15.94:0/7791620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 0x560c7df10090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.665+0000 7f7f55662640 1 -- 172.21.15.94:0/7791620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df105d0 msgr2=0x560c7df10a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.665+0000 7f7f55662640 1 --2- 172.21.15.94:0/7791620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df105d0 0x560c7df10a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.665+0000 7f7f55662640 1 -- 172.21.15.94:0/7791620 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7de023a0 con 0x560c7df23ba0 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.665+0000 7f7f4effd640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7f380a3580 0x7f7f380a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.665+0000 7f7f4effd640 1 -- 172.21.15.94:0/2655061459 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7f400f0490 con 0x7f7f50100e20 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.665+0000 7f7f56664640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7f380a3580 0x7f7f380a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.665+0000 7f7f55662640 1 --2- 172.21.15.94:0/7791620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df23ba0 0x560c7df25f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400bd10 tx=0x7f7f440107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277dd1c64cb7a10a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.666+0000 7f7f37fff640 1 -- 172.21.15.94:0/7791620 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44005220 con 0x560c7df23ba0 2024-09-19T04:23:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.666+0000 7f7f37fff640 1 -- 172.21.15.94:0/7791620 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f44004c50 con 0x560c7df23ba0 2024-09-19T04:23:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.666+0000 7f7f37fff640 1 -- 172.21.15.94:0/7791620 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44011820 con 0x560c7df23ba0 2024-09-19T04:23:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.667+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/7791620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df23ba0 msgr2=0x560c7df25f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.667+0000 7f7f56664640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7f380a3580 0x7f7f380a5a40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7f480049d0 tx=0x7f7f48009500 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:40.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.667+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/7791620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df23ba0 0x560c7df25f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400bd10 tx=0x7f7f440107a0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.667+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/7791620 shutdown_connections 2024-09-19T04:23:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.667+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/7791620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df23ba0 0x560c7df25f90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.667+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/7791620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df105d0 0x560c7df10a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.667+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/7791620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 0x560c7df10090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.667+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/7791620 >> 172.21.15.94:0/7791620 conn(0x560c7ded9310 msgr2=0x560c7df0cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.667+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/7791620 shutdown_connections 2024-09-19T04:23:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.668+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/7791620 wait complete. 2024-09-19T04:23:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.668+0000 7f7f5a4a0740 1 -- start start 2024-09-19T04:23:40.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.668+0000 7f7f5a4a0740 1 Processor -- start 2024-09-19T04:23:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 0x560c7df20780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560c7df26920 con 0x560c7df23ba0 2024-09-19T04:23:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560c7de31060 con 0x560c7df105d0 2024-09-19T04:23:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560c7df26aa0 con 0x560c7df11b50 2024-09-19T04:23:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f5a4a0740 10 client.0 _ll_register_callbacks cb 0x560c7db848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f55e63640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f55662640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 0x560c7df20780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f56664640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f55e63640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df234d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33462/0 (socket says 172.21.15.94:33462) 2024-09-19T04:23:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f55e63640 1 -- 172.21.15.94:0/4261612686 learned_addr learned my addr 172.21.15.94:0/4261612686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f55662640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 0x560c7df20780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:58132/0 (socket says 172.21.15.94:58132) 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f55e63640 1 -- 172.21.15.94:0/4261612686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 msgr2=0x560c7df20780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f55e63640 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 0x560c7df20780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f55e63640 1 -- 172.21.15.94:0/4261612686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 msgr2=0x560c7df20cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.669+0000 7f7f55e63640 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df20cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.670+0000 7f7f55e63640 1 -- 172.21.15.94:0/4261612686 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7dbe73c0 con 0x560c7df105d0 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.670+0000 7f7f56664640 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df20cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.670+0000 7f7f55e63640 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df234d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7f400b92a0 tx=0x7f7f40005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44aa1832e38e2320 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.670+0000 7f7f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.670+0000 7f7f357fa640 1 -- 172.21.15.94:0/4261612686 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40049030 con 0x560c7df105d0 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.670+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/4261612686 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f44004c50 con 0x560c7df105d0 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.670+0000 7f7f357fa640 1 -- 172.21.15.94:0/4261612686 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7f400b8f40 con 0x560c7df105d0 2024-09-19T04:23:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.671+0000 7f7f357fa640 1 -- 172.21.15.94:0/4261612686 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7f400bc410 con 0x560c7df105d0 2024-09-19T04:23:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.671+0000 7f7f5a4a0740 10 client.26012 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.671+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/4261612686 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560c7df32360 con 0x560c7df105d0 2024-09-19T04:23:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.671+0000 7f7f357fa640 1 -- 172.21.15.94:0/4261612686 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f44004c50 con 0x560c7df105d0 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.672+0000 7f7f357fa640 1 -- 172.21.15.94:0/4261612686 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f7f400f0070 con 0x560c7df105d0 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.672+0000 7f7f5a4a0740 10 client.26012 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.672+0000 7f7f5a4a0740 10 client.26012 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.672+0000 7f7f5a4a0740 20 client.26012 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.672+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560c7df337c0 0x560c7df35c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.672+0000 7f7f5a4a0740 4 client.26012 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.672+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/4261612686 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c7de83860 con 0x560c7df337c0 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.673+0000 7f7f55662640 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560c7df337c0 0x560c7df35c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.673+0000 7f7f55662640 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560c7df337c0 0x560c7df35c50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=763efcbde98c0a75 server_cookie=ff0499e292177c57 in_seq=0 out_seq=0 2024-09-19T04:23:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.673+0000 7f7f357fa640 10 client.26012 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:23:40.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.674+0000 7f7f357fa640 1 -- 172.21.15.94:0/4261612686 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c7de83860 con 0x560c7df337c0 2024-09-19T04:23:40.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.674+0000 7f7f357fa640 10 client.26012 handle_command_reply: tid=0 2024-09-19T04:23:40.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.674+0000 7f7f5a4a0740 1 client.26012 shutdown 2024-09-19T04:23:40.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.674+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/4261612686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 msgr2=0x560c7df234d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:40.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.674+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df234d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7f400b92a0 tx=0x7f7f40005d50 comp rx=0 tx=0).stop 2024-09-19T04:23:40.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/4261612686 shutdown_connections 2024-09-19T04:23:40.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560c7df337c0 0x560c7df35c50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df20cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df11b50 0x560c7df20780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/4261612686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df234d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/4261612686 >> 172.21.15.94:0/4261612686 conn(0x560c7ded9310 msgr2=0x560c7df12970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:40.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f357fa640 0 client.26012 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:23:40.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/4261612686 shutdown_connections 2024-09-19T04:23:40.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/4261612686 wait complete. 2024-09-19T04:23:40.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.675+0000 7f7f5a4a0740 20 client.26012 trim_cache size 0 max 16384 2024-09-19T04:23:40.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.699+0000 7f7f5a4a0740 1 Processor -- start 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.699+0000 7f7f5a4a0740 1 -- start start 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.699+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df105d0 0x560c7df2fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.699+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df11b50 0x560c7df0c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.699+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df3b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.699+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501ae030 con 0x560c7df23ba0 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.699+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501ab8a0 con 0x560c7df11b50 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.700+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501a9750 con 0x560c7df105d0 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.700+0000 7f7f56664640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df3b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.700+0000 7f7f56664640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df3b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48684/0 (socket says 172.21.15.94:48684) 2024-09-19T04:23:40.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.700+0000 7f7f56664640 1 -- 172.21.15.94:0/2397876731 learned_addr learned my addr 172.21.15.94:0/2397876731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:40.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.700+0000 7f7f55e63640 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df105d0 0x560c7df2fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.700+0000 7f7f55662640 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df11b50 0x560c7df0c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.701+0000 7f7f56664640 1 -- 172.21.15.94:0/2397876731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df105d0 msgr2=0x560c7df2fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.701+0000 7f7f56664640 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df105d0 0x560c7df2fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.701+0000 7f7f56664640 1 -- 172.21.15.94:0/2397876731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df11b50 msgr2=0x560c7df0c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.701+0000 7f7f56664640 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df11b50 0x560c7df0c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.701+0000 7f7f56664640 1 -- 172.21.15.94:0/2397876731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7df26300 con 0x560c7df23ba0 2024-09-19T04:23:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.701+0000 7f7f55662640 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df11b50 0x560c7df0c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:23:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.701+0000 7f7f56664640 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df3b4f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800fc50 tx=0x7f7f48016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300d7d184aea672c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.702+0000 7f7f377fe640 1 -- 172.21.15.94:0/2397876731 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4801c910 con 0x560c7df23ba0 2024-09-19T04:23:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.702+0000 7f7f377fe640 1 -- 172.21.15.94:0/2397876731 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f4802d070 con 0x560c7df23ba0 2024-09-19T04:23:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.702+0000 7f7f377fe640 1 -- 172.21.15.94:0/2397876731 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48028dc0 con 0x560c7df23ba0 2024-09-19T04:23:40.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.703+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/2397876731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 msgr2=0x560c7df3b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:40.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.703+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df3b4f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800fc50 tx=0x7f7f48016000 comp rx=0 tx=0).stop 2024-09-19T04:23:40.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.703+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/2397876731 shutdown_connections 2024-09-19T04:23:40.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.703+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df23ba0 0x560c7df3b4f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.703+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df11b50 0x560c7df0c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.703+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/2397876731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df105d0 0x560c7df2fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.703+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/2397876731 >> 172.21.15.94:0/2397876731 conn(0x560c7ded9310 msgr2=0x560c7df1f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:40.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.704+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/2397876731 shutdown_connections 2024-09-19T04:23:40.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.704+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/2397876731 wait complete. 2024-09-19T04:23:40.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.704+0000 7f7f5a4a0740 1 -- start start 2024-09-19T04:23:40.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.704+0000 7f7f5a4a0740 1 Processor -- start 2024-09-19T04:23:40.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df99860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df11b50 0x560c7df99da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f5a4a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df23ba0 0x560c7df2bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f55e63640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df99860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501a9da0 con 0x560c7df11b50 2024-09-19T04:23:40.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501ab1d0 con 0x560c7df105d0 2024-09-19T04:23:40.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f5a4a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501aa430 con 0x560c7df23ba0 2024-09-19T04:23:40.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f55662640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df11b50 0x560c7df99da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f55e63640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df99860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33484/0 (socket says 172.21.15.94:33484) 2024-09-19T04:23:40.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f55e63640 1 -- 172.21.15.94:0/1933642747 learned_addr learned my addr 172.21.15.94:0/1933642747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:40.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.705+0000 7f7f5a4a0740 10 client.0 _ll_register_callbacks cb 0x7f7f40009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.706+0000 7f7f55662640 1 -- 172.21.15.94:0/1933642747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df23ba0 msgr2=0x560c7df2bd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.706+0000 7f7f55662640 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df23ba0 0x560c7df2bd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.706+0000 7f7f55662640 1 -- 172.21.15.94:0/1933642747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 msgr2=0x560c7df99860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.706+0000 7f7f55662640 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df99860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.706+0000 7f7f55662640 1 -- 172.21.15.94:0/1933642747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7df2c5b0 con 0x560c7df11b50 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.706+0000 7f7f55662640 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df11b50 0x560c7df99da0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005d00 tx=0x7f7f4400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd9f3355e7820fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.706+0000 7f7f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.707+0000 7f7f357fa640 1 -- 172.21.15.94:0/1933642747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4402c020 con 0x560c7df11b50 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.707+0000 7f7f357fa640 1 -- 172.21.15.94:0/1933642747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7f44005450 con 0x560c7df11b50 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.707+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/1933642747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4802d070 con 0x560c7df11b50 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.707+0000 7f7f357fa640 1 -- 172.21.15.94:0/1933642747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7f44037050 con 0x560c7df11b50 2024-09-19T04:23:40.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.707+0000 7f7f5a4a0740 10 client.16410 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:40.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.707+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/1933642747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560c7df1dd60 con 0x560c7df11b50 2024-09-19T04:23:40.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.708+0000 7f7f357fa640 1 -- 172.21.15.94:0/1933642747 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f4802d070 con 0x560c7df11b50 2024-09-19T04:23:40.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.708+0000 7f7f357fa640 1 -- 172.21.15.94:0/1933642747 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f7f44029430 con 0x560c7df11b50 2024-09-19T04:23:40.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.708+0000 7f7f5a4a0740 10 client.16410 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:40.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.708+0000 7f7f5a4a0740 10 client.16410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:23:40.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.708+0000 7f7f5a4a0740 20 client.16410 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:40.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.708+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560c7df17420 0x560c7df19840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:40.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.708+0000 7f7f5a4a0740 4 client.16410 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:23:40.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.708+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/1933642747 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c7de7dd70 con 0x560c7df17420 2024-09-19T04:23:40.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.709+0000 7f7f55e63640 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560c7df17420 0x560c7df19840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:40.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.709+0000 7f7f55e63640 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560c7df17420 0x560c7df19840 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae4f5a8dfe499709 server_cookie=f833906f21d49f76 in_seq=0 out_seq=0 2024-09-19T04:23:40.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.709+0000 7f7f357fa640 10 client.16410 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:23:40.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.709+0000 7f7f357fa640 1 -- 172.21.15.94:0/1933642747 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c7de7dd70 con 0x560c7df17420 2024-09-19T04:23:40.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.709+0000 7f7f357fa640 10 client.16410 handle_command_reply: tid=0 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.709+0000 7f7f5a4a0740 1 client.16410 shutdown 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/1933642747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df11b50 msgr2=0x560c7df99da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df11b50 0x560c7df99da0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005d00 tx=0x7f7f4400be00 comp rx=0 tx=0).stop 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/1933642747 shutdown_connections 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x560c7df17420 0x560c7df19840 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c7df23ba0 0x560c7df2bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c7df11b50 0x560c7df99da0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 --2- 172.21.15.94:0/1933642747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c7df105d0 0x560c7df99860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/1933642747 >> 172.21.15.94:0/1933642747 conn(0x560c7ded9310 msgr2=0x560c7df1f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f357fa640 0 client.16410 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/1933642747 shutdown_connections 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 1 -- 172.21.15.94:0/1933642747 wait complete. 2024-09-19T04:23:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.710+0000 7f7f5a4a0740 20 client.16410 trim_cache size 0 max 16384 2024-09-19T04:23:40.714 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:23:40.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 -- 172.21.15.94:0/2655061459 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7f380a3580 msgr2=0x7f7f380a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7f380a3580 0x7f7f380a5a40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7f480049d0 tx=0x7f7f48009500 comp rx=0 tx=0).stop 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 -- 172.21.15.94:0/2655061459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f50100e20 msgr2=0x7f7f5019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f50100e20 0x7f7f5019a460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009850 tx=0x7f7f40030650 comp rx=0 tx=0).stop 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 -- 172.21.15.94:0/2655061459 shutdown_connections 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f7f380a3580 0x7f7f380a5a40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50102ef0 0x7f7f5019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f50102570 0x7f7f5019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 --2- 172.21.15.94:0/2655061459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f50100e20 0x7f7f5019a460 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 -- 172.21.15.94:0/2655061459 >> 172.21.15.94:0/2655061459 conn(0x7f7f500fc7d0 msgr2=0x7f7f500fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 -- 172.21.15.94:0/2655061459 shutdown_connections 2024-09-19T04:23:40.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:40.711+0000 7f7f580c7640 1 -- 172.21.15.94:0/2655061459 wait complete. 2024-09-19T04:23:41.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:41 smithi094 ceph-mon[30972]: pgmap v266: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-09-19T04:23:41.433 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:23:41.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:41 smithi112 ceph-mon[35051]: pgmap v266: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-09-19T04:23:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:41 smithi104 ceph-mon[34145]: pgmap v266: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-09-19T04:23:41.885 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.446+0000 7f1ac375a640 1 -- 172.21.15.94:0/1904790118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc100e20 msgr2=0x7f1abc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.446+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1904790118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc100e20 0x7f1abc101200 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009900 tx=0x7f1ab802a480 comp rx=0 tx=0).stop 2024-09-19T04:23:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.447+0000 7f1ac375a640 1 -- 172.21.15.94:0/1904790118 shutdown_connections 2024-09-19T04:23:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.447+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1904790118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1abc102ef0 0x7f1abc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.447+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1904790118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1abc102570 0x7f1abc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.447+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1904790118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc100e20 0x7f1abc101200 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.447+0000 7f1ac375a640 1 -- 172.21.15.94:0/1904790118 >> 172.21.15.94:0/1904790118 conn(0x7f1abc0fc7d0 msgr2=0x7f1abc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.447+0000 7f1ac375a640 1 -- 172.21.15.94:0/1904790118 shutdown_connections 2024-09-19T04:23:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.447+0000 7f1ac375a640 1 -- 172.21.15.94:0/1904790118 wait complete. 2024-09-19T04:23:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac375a640 1 Processor -- start 2024-09-19T04:23:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac375a640 1 -- start start 2024-09-19T04:23:42.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac375a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1abc100e20 0x7f1abc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac375a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc102570 0x7f1abc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac375a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1abc102ef0 0x7f1abc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac375a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc10fd30 con 0x7f1abc102570 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac375a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc10fbb0 con 0x7f1abc102ef0 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac375a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc10feb0 con 0x7f1abc100e20 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac0cf5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc102570 0x7f1abc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac0cf5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc102570 0x7f1abc19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47240/0 (socket says 172.21.15.94:47240) 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.448+0000 7f1ac0cf5640 1 -- 172.21.15.94:0/1151197502 learned_addr learned my addr 172.21.15.94:0/1151197502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac1cf7640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1abc102ef0 0x7f1abc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac1cf7640 1 -- 172.21.15.94:0/1151197502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1abc100e20 msgr2=0x7f1abc19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac1cf7640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1abc100e20 0x7f1abc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac1cf7640 1 -- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc102570 msgr2=0x7f1abc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:42.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac1cf7640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc102570 0x7f1abc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac1cf7640 1 -- 172.21.15.94:0/1151197502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc19f660 con 0x7f1abc102ef0 2024-09-19T04:23:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc102570 0x7f1abc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac1cf7640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1abc102ef0 0x7f1abc19ef80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009fd0 tx=0x7f1ab400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1aae7fc640 1 -- 172.21.15.94:0/1151197502 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400ce20 con 0x7f1abc102ef0 2024-09-19T04:23:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac375a640 1 -- 172.21.15.94:0/1151197502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc19f950 con 0x7f1abc102ef0 2024-09-19T04:23:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1aae7fc640 1 -- 172.21.15.94:0/1151197502 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ab4005020 con 0x7f1abc102ef0 2024-09-19T04:23:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1aae7fc640 1 -- 172.21.15.94:0/1151197502 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4010670 con 0x7f1abc102ef0 2024-09-19T04:23:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.449+0000 7f1ac375a640 1 -- 172.21.15.94:0/1151197502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc1a7230 con 0x7f1abc102ef0 2024-09-19T04:23:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.453+0000 7f1aae7fc640 1 -- 172.21.15.94:0/1151197502 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1ab40051d0 con 0x7f1abc102ef0 2024-09-19T04:23:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.453+0000 7f1ac5b33740 1 Processor -- start 2024-09-19T04:23:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.453+0000 7f1ac5b33740 1 -- start start 2024-09-19T04:23:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d47b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d47ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d490f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac5b33740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a69d3e12f0 con 0x55a69d48eb80 2024-09-19T04:23:42.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac5b33740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a69d388cf0 con 0x55a69d47b5d0 2024-09-19T04:23:42.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac5b33740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a69d3dd2b0 con 0x55a69d47cb40 2024-09-19T04:23:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac14f6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d47ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac14f6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d47ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56066/0 (socket says 172.21.15.94:56066) 2024-09-19T04:23:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac0cf5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d490f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac14f6640 1 -- 172.21.15.94:0/1005422813 learned_addr learned my addr 172.21.15.94:0/1005422813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.454+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/1005422813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d47b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.455+0000 7f1aae7fc640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1aa80a33b0 0x7f1aa80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.455+0000 7f1ac1cf7640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1aa80a33b0 0x7f1aa80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.455+0000 7f1ac14f6640 1 -- 172.21.15.94:0/1005422813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 msgr2=0x55a69d47b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.455+0000 7f1ac14f6640 1 --2- 172.21.15.94:0/1005422813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d47b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.455+0000 7f1ac14f6640 1 -- 172.21.15.94:0/1005422813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 msgr2=0x55a69d490f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.455+0000 7f1aae7fc640 1 -- 172.21.15.94:0/1151197502 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f1ab40d0240 con 0x7f1abc102ef0 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.455+0000 7f1ac14f6640 1 --2- 172.21.15.94:0/1005422813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d490f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.455+0000 7f1ac14f6640 1 -- 172.21.15.94:0/1005422813 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69d36d3a0 con 0x55a69d47b5d0 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.455+0000 7f1ac14f6640 1 --2- 172.21.15.94:0/1005422813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d47ba10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8000c00 tx=0x7f1ab8005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae728e2c0be05cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.456+0000 7f1aa77fe640 1 -- 172.21.15.94:0/1005422813 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab803e070 con 0x55a69d47b5d0 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.456+0000 7f1aa77fe640 1 -- 172.21.15.94:0/1005422813 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ab8005430 con 0x55a69d47b5d0 2024-09-19T04:23:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.456+0000 7f1aa77fe640 1 -- 172.21.15.94:0/1005422813 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8035040 con 0x55a69d47b5d0 2024-09-19T04:23:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 -- 172.21.15.94:0/1005422813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 msgr2=0x55a69d47ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/1005422813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d47ba10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8000c00 tx=0x7f1ab8005040 comp rx=0 tx=0).stop 2024-09-19T04:23:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac1cf7640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1aa80a33b0 0x7f1aa80a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40cda00 tx=0x7f1ab4021000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 -- 172.21.15.94:0/1005422813 shutdown_connections 2024-09-19T04:23:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/1005422813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d490f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/1005422813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d47ba10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/1005422813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d47b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 -- 172.21.15.94:0/1005422813 >> 172.21.15.94:0/1005422813 conn(0x55a69d444310 msgr2=0x55a69d477c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 -- 172.21.15.94:0/1005422813 shutdown_connections 2024-09-19T04:23:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 -- 172.21.15.94:0/1005422813 wait complete. 2024-09-19T04:23:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.457+0000 7f1ac5b33740 1 -- start start 2024-09-19T04:23:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.458+0000 7f1ac5b33740 1 Processor -- start 2024-09-19T04:23:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.458+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d48e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.458+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d48b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d48bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac5b33740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a69d4918c0 con 0x55a69d48eb80 2024-09-19T04:23:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac0cf5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d48e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac5b33740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a69d39c060 con 0x55a69d47b5d0 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac5b33740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a69d491a40 con 0x55a69d47cb40 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac14f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d48b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac14f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d48b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47266/0 (socket says 172.21.15.94:47266) 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac14f6640 1 -- 172.21.15.94:0/2775162369 learned_addr learned my addr 172.21.15.94:0/2775162369 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac0cf5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d48e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56072/0 (socket says 172.21.15.94:56072) 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d48bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac5b33740 10 client.0 _ll_register_callbacks cb 0x55a69d0ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac0cf5640 1 -- 172.21.15.94:0/2775162369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 msgr2=0x55a69d48b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d48b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac0cf5640 1 -- 172.21.15.94:0/2775162369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 msgr2=0x55a69d48bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d48bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.459+0000 7f1ac0cf5640 1 -- 172.21.15.94:0/2775162369 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69d1523c0 con 0x55a69d47b5d0 2024-09-19T04:23:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.460+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d48e560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000b3c0 tx=0x7f1ab0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af09227c456cffa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.460+0000 7f1aa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:23:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.460+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2775162369 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0003c90 con 0x55a69d47b5d0 2024-09-19T04:23:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.460+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2775162369 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f1ab0009040 con 0x55a69d47b5d0 2024-09-19T04:23:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.460+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2775162369 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab8005430 con 0x55a69d47b5d0 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.461+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2775162369 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1ab0003e30 con 0x55a69d47b5d0 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.461+0000 7f1ac5b33740 10 client.26027 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.461+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2775162369 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a69d49d380 con 0x55a69d47b5d0 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.461+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2775162369 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ab8005430 con 0x55a69d47b5d0 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.462+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2775162369 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f1ab001a3c0 con 0x55a69d47b5d0 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.462+0000 7f1ac5b33740 10 client.26027 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.462+0000 7f1ac5b33740 10 client.26027 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.462+0000 7f1ac5b33740 20 client.26027 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.462+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55a69d49e8e0 0x55a69d4a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.462+0000 7f1ac5b33740 4 client.26027 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.462+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2775162369 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a69d3ee860 con 0x55a69d49e8e0 2024-09-19T04:23:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.462+0000 7f1ac14f6640 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55a69d49e8e0 0x55a69d4a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.463+0000 7f1ac14f6640 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55a69d49e8e0 0x55a69d4a0d50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bec304a86fc46312 server_cookie=2bf08a384a434c8e in_seq=0 out_seq=0 2024-09-19T04:23:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.463+0000 7f1aa4ff9640 10 client.26027 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:23:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2775162369 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a69d3ee860 con 0x55a69d49e8e0 2024-09-19T04:23:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1aa4ff9640 10 client.26027 handle_command_reply: tid=0 2024-09-19T04:23:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1ac5b33740 1 client.26027 shutdown 2024-09-19T04:23:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2775162369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 msgr2=0x55a69d48e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d48e560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000b3c0 tx=0x7f1ab0004730 comp rx=0 tx=0).stop 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2775162369 shutdown_connections 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55a69d49e8e0 0x55a69d4a0d50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1aa4ff9640 0 client.26027 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d48bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d48b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2775162369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47b5d0 0x55a69d48e560 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.464+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2775162369 >> 172.21.15.94:0/2775162369 conn(0x55a69d444310 msgr2=0x55a69d47d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.465+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2775162369 shutdown_connections 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.465+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2775162369 wait complete. 2024-09-19T04:23:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.465+0000 7f1ac5b33740 20 client.26027 trim_cache size 0 max 16384 2024-09-19T04:23:42.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.487+0000 7f1ac5b33740 1 Processor -- start 2024-09-19T04:23:42.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.487+0000 7f1ac5b33740 1 -- start start 2024-09-19T04:23:42.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.487+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d47b5d0 0x55a69d477e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d49ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac0cf5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d47b5d0 0x55a69d477e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac0cf5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d47b5d0 0x55a69d477e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47250/0 (socket says 172.21.15.94:47250) 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac0cf5640 1 -- 172.21.15.94:0/844610001 learned_addr learned my addr 172.21.15.94:0/844610001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/844610001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d48eb80 0x55a69d48b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac5b33740 1 -- 172.21.15.94:0/844610001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1adf80 con 0x55a69d47b5d0 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac5b33740 1 -- 172.21.15.94:0/844610001 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1ab7f0 con 0x55a69d48eb80 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac5b33740 1 -- 172.21.15.94:0/844610001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1a9660 con 0x55a69d47cb40 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac0cf5640 1 -- 172.21.15.94:0/844610001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 msgr2=0x55a69d49ab60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/844610001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d49ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac0cf5640 1 -- 172.21.15.94:0/844610001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d48eb80 msgr2=0x55a69d48b0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/844610001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d48eb80 0x55a69d48b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.488+0000 7f1ac0cf5640 1 -- 172.21.15.94:0/844610001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69d478960 con 0x55a69d47b5d0 2024-09-19T04:23:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.489+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/844610001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d47b5d0 0x55a69d477e10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1ab001a980 tx=0x7f1ab00169f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609dc91aab865c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:42.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.489+0000 7f1aa6ffd640 1 -- 172.21.15.94:0/844610001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab001ac10 con 0x55a69d47b5d0 2024-09-19T04:23:42.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.489+0000 7f1aa6ffd640 1 -- 172.21.15.94:0/844610001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ab0005d30 con 0x55a69d47b5d0 2024-09-19T04:23:42.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.489+0000 7f1aa6ffd640 1 -- 172.21.15.94:0/844610001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0029dc0 con 0x55a69d47b5d0 2024-09-19T04:23:42.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.490+0000 7f1ac5b33740 1 -- 172.21.15.94:0/844610001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d47b5d0 msgr2=0x55a69d477e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:42.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.490+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/844610001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d47b5d0 0x55a69d477e10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1ab001a980 tx=0x7f1ab00169f0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.491+0000 7f1ac5b33740 1 -- 172.21.15.94:0/844610001 shutdown_connections 2024-09-19T04:23:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.491+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/844610001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d48eb80 0x55a69d48b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.491+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/844610001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47cb40 0x55a69d49ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.491+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/844610001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d47b5d0 0x55a69d477e10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.491+0000 7f1ac5b33740 1 -- 172.21.15.94:0/844610001 >> 172.21.15.94:0/844610001 conn(0x55a69d444310 msgr2=0x55a69d477700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.491+0000 7f1ac5b33740 1 -- 172.21.15.94:0/844610001 shutdown_connections 2024-09-19T04:23:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.491+0000 7f1ac5b33740 1 -- 172.21.15.94:0/844610001 wait complete. 2024-09-19T04:23:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.491+0000 7f1ac5b33740 1 -- start start 2024-09-19T04:23:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.492+0000 7f1ac5b33740 1 Processor -- start 2024-09-19T04:23:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.492+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47b5d0 0x55a69d497020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.492+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47cb40 0x55a69d497560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.492+0000 7f1ac5b33740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d492420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac5b33740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1a9cb0 con 0x55a69d48eb80 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac0cf5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d492420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac0cf5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d492420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47264/0 (socket says 172.21.15.94:47264) 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac0cf5640 1 -- 172.21.15.94:0/2532198933 learned_addr learned my addr 172.21.15.94:0/2532198933 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1ab120 con 0x55a69d47cb40 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1aa380 con 0x55a69d47b5d0 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47b5d0 0x55a69d497020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac5b33740 10 client.0 _ll_register_callbacks cb 0x7f1ab0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac14f6640 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47cb40 0x55a69d497560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac14f6640 1 -- 172.21.15.94:0/2532198933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47b5d0 msgr2=0x55a69d497020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac14f6640 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47b5d0 0x55a69d497020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac14f6640 1 -- 172.21.15.94:0/2532198933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 msgr2=0x55a69d492420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac14f6640 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d492420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.493+0000 7f1ac14f6640 1 -- 172.21.15.94:0/2532198933 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69d492c50 con 0x55a69d47cb40 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.494+0000 7f1ac14f6640 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47cb40 0x55a69d497560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80092a0 tx=0x7f1ab8009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be266bf6a05d6aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.494+0000 7f1aa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.494+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2532198933 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8035040 con 0x55a69d47cb40 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.494+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2532198933 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f1ab80046c0 con 0x55a69d47cb40 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.494+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab0005d30 con 0x55a69d47cb40 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.495+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2532198933 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1ab803e040 con 0x55a69d47cb40 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.495+0000 7f1ac5b33740 10 client.26030 fetch_fsmap learned FSMap version 42 2024-09-19T04:23:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.495+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a69d509250 con 0x55a69d47cb40 2024-09-19T04:23:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.495+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2532198933 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ab0005d30 con 0x55a69d47cb40 2024-09-19T04:23:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.495+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2532198933 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f1ab804c070 con 0x55a69d47cb40 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.496+0000 7f1ac5b33740 10 client.26030 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.496+0000 7f1ac5b33740 10 client.26030 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.496+0000 7f1ac5b33740 20 client.26030 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.496+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55a69d49e8e0 0x55a69d50b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.496+0000 7f1ac5b33740 4 client.26030 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.496+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a69d3e8d70 con 0x55a69d49e8e0 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.497+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55a69d49e8e0 0x55a69d50b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.497+0000 7f1ac0cf5640 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55a69d49e8e0 0x55a69d50b5a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=676087ec4776c080 server_cookie=cace6cfac173106d in_seq=0 out_seq=0 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.497+0000 7f1aa4ff9640 10 client.26030 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.497+0000 7f1aa4ff9640 1 -- 172.21.15.94:0/2532198933 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a69d3e8d70 con 0x55a69d49e8e0 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.497+0000 7f1aa4ff9640 10 client.26030 handle_command_reply: tid=0 2024-09-19T04:23:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.497+0000 7f1ac5b33740 1 client.26030 shutdown 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47cb40 msgr2=0x55a69d497560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47cb40 0x55a69d497560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80092a0 tx=0x7f1ab8009850 comp rx=0 tx=0).stop 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 shutdown_connections 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55a69d49e8e0 0x55a69d50b5a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69d48eb80 0x55a69d492420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1aa4ff9640 0 client.26030 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a69d47cb40 0x55a69d497560 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 --2- 172.21.15.94:0/2532198933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a69d47b5d0 0x55a69d497020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 >> 172.21.15.94:0/2532198933 conn(0x55a69d444310 msgr2=0x55a69d4764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 shutdown_connections 2024-09-19T04:23:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 1 -- 172.21.15.94:0/2532198933 wait complete. 2024-09-19T04:23:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.498+0000 7f1ac5b33740 20 client.26030 trim_cache size 0 max 16384 2024-09-19T04:23:42.502 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 -- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1aa80a33b0 msgr2=0x7f1aa80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1aa80a33b0 0x7f1aa80a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40cda00 tx=0x7f1ab4021000 comp rx=0 tx=0).stop 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 -- 172.21.15.94:0/1151197502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1abc102ef0 msgr2=0x7f1abc19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1abc102ef0 0x7f1abc19ef80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009fd0 tx=0x7f1ab400ef10 comp rx=0 tx=0).stop 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 -- 172.21.15.94:0/1151197502 shutdown_connections 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1aa80a33b0 0x7f1aa80a5870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1abc102ef0 0x7f1abc19ef80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1abc102570 0x7f1abc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 --2- 172.21.15.94:0/1151197502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1abc100e20 0x7f1abc19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.499+0000 7f1ac375a640 1 -- 172.21.15.94:0/1151197502 >> 172.21.15.94:0/1151197502 conn(0x7f1abc0fc7d0 msgr2=0x7f1abc105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.500+0000 7f1ac375a640 1 -- 172.21.15.94:0/1151197502 shutdown_connections 2024-09-19T04:23:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:42.500+0000 7f1ac375a640 1 -- 172.21.15.94:0/1151197502 wait complete. 2024-09-19T04:23:43.199 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:43 smithi094 ceph-mon[30972]: pgmap v267: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-09-19T04:23:43.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:23:43.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:43 smithi112 ceph-mon[35051]: pgmap v267: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-09-19T04:23:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:43 smithi104 ceph-mon[34145]: pgmap v267: 97 pgs: 97 active+clean; 3.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-09-19T04:23:43.677 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.268+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2853699035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f209c102570 msgr2=0x7f209c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.268+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2853699035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f209c102570 0x7f209c1029b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f208c00d990 tx=0x7f208c02c380 comp rx=0 tx=0).stop 2024-09-19T04:23:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.268+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2853699035 shutdown_connections 2024-09-19T04:23:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.268+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2853699035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c102ef0 0x7f209c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.268+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2853699035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f209c102570 0x7f209c1029b0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f208c00d990 tx=0x7f208c02c380 comp rx=0 tx=0).stop 2024-09-19T04:23:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.268+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2853699035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f209c100e20 0x7f209c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.268+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2853699035 >> 172.21.15.94:0/2853699035 conn(0x7f209c0fc7d0 msgr2=0x7f209c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.269+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2853699035 shutdown_connections 2024-09-19T04:23:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.269+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2853699035 wait complete. 2024-09-19T04:23:44.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.270+0000 7f20a3cbc640 1 Processor -- start 2024-09-19T04:23:44.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.270+0000 7f20a3cbc640 1 -- start start 2024-09-19T04:23:44.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.270+0000 7f20a3cbc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c100e20 0x7f209c196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.270+0000 7f20a3cbc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f209c102ef0 0x7f209c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.270+0000 7f20a3cbc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f209c19ad10 0x7f209c19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.270+0000 7f20a3cbc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10fd30 con 0x7f209c19ad10 2024-09-19T04:23:44.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.270+0000 7f20a3cbc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10fbb0 con 0x7f209c102ef0 2024-09-19T04:23:44.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1257640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f209c102ef0 0x7f209c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1a58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c100e20 0x7f209c196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1a58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c100e20 0x7f209c196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47294/0 (socket says 172.21.15.94:47294) 2024-09-19T04:23:44.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1a58640 1 -- 172.21.15.94:0/2599219499 learned_addr learned my addr 172.21.15.94:0/2599219499 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:44.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2599219499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10feb0 con 0x7f209c100e20 2024-09-19T04:23:44.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a2259640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f209c19ad10 0x7f209c19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1257640 1 -- 172.21.15.94:0/2599219499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c100e20 msgr2=0x7f209c196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:44.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1257640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c100e20 0x7f209c196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1257640 1 -- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f209c19ad10 msgr2=0x7f209c19b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:44.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1257640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f209c19ad10 0x7f209c19b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1257640 1 -- 172.21.15.94:0/2599219499 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c19b7d0 con 0x7f209c102ef0 2024-09-19T04:23:44.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.271+0000 7f20a1a58640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c100e20 0x7f209c196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:44.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.272+0000 7f20a1257640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f209c102ef0 0x7f209c196980 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f208c008000 tx=0x7f208c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:44.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.272+0000 7f2092ffd640 1 -- 172.21.15.94:0/2599219499 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c039070 con 0x7f209c102ef0 2024-09-19T04:23:44.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.272+0000 7f2092ffd640 1 -- 172.21.15.94:0/2599219499 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f208c03a070 con 0x7f209c102ef0 2024-09-19T04:23:44.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.272+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2599219499 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c19ba60 con 0x7f209c102ef0 2024-09-19T04:23:44.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.272+0000 7f2092ffd640 1 -- 172.21.15.94:0/2599219499 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c03e810 con 0x7f209c102ef0 2024-09-19T04:23:44.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.272+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2599219499 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c1a3190 con 0x7f209c102ef0 2024-09-19T04:23:44.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.275+0000 7f2092ffd640 1 -- 172.21.15.94:0/2599219499 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f208c0037f0 con 0x7f209c102ef0 2024-09-19T04:23:44.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.276+0000 7f2092ffd640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20880a3580 0x7f20880a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.276+0000 7f20a6095740 1 Processor -- start 2024-09-19T04:23:44.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.277+0000 7f2092ffd640 1 -- 172.21.15.94:0/2599219499 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f208c0efc90 con 0x7f209c102ef0 2024-09-19T04:23:44.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.277+0000 7f20a1a58640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20880a3580 0x7f20880a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.277+0000 7f20a6095740 1 -- start start 2024-09-19T04:23:44.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.278+0000 7f20a1a58640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20880a3580 0x7f20880a5a40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20980096c0 tx=0x7f2098014040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:44.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.278+0000 7f20a6095740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfc3b40 0x555ecbfc2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.278+0000 7f20a6095740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.278+0000 7f20a6095740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfcd370 0x555ecbfcf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.279+0000 7f20a1a58640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfc3b40 0x555ecbfc2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.279+0000 7f20a6095740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555ecbf282f0 con 0x555ecbfcd370 2024-09-19T04:23:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.279+0000 7f20a6095740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555ecbecfcf0 con 0x555ecbfc3b40 2024-09-19T04:23:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.279+0000 7f20a6095740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ecbf242b0 con 0x555ecbfc25d0 2024-09-19T04:23:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.279+0000 7f20a1257640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.279+0000 7f20a1257640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfc2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47300/0 (socket says 172.21.15.94:47300) 2024-09-19T04:23:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.279+0000 7f20a1257640 1 -- 172.21.15.94:0/3350986768 learned_addr learned my addr 172.21.15.94:0/3350986768 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.279+0000 7f20a2259640 1 --2- 172.21.15.94:0/3350986768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfcd370 0x555ecbfcf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f20a2259640 1 -- 172.21.15.94:0/3350986768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 msgr2=0x555ecbfc2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f20a2259640 1 --2- 172.21.15.94:0/3350986768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfc2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f20a2259640 1 -- 172.21.15.94:0/3350986768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfc3b40 msgr2=0x555ecbfc2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f20a2259640 1 --2- 172.21.15.94:0/3350986768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfc3b40 0x555ecbfc2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f20a2259640 1 -- 172.21.15.94:0/3350986768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecbeb43a0 con 0x555ecbfcd370 2024-09-19T04:23:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f20a1a58640 1 --2- 172.21.15.94:0/3350986768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfc3b40 0x555ecbfc2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:23:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f20a2259640 1 --2- 172.21.15.94:0/3350986768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfcd370 0x555ecbfcf760 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2094009f00 tx=0x7f2094011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9829d5e764caae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f2087fff640 1 -- 172.21.15.94:0/3350986768 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094004d20 con 0x555ecbfcd370 2024-09-19T04:23:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f2087fff640 1 -- 172.21.15.94:0/3350986768 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2094004050 con 0x555ecbfcd370 2024-09-19T04:23:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.280+0000 7f2087fff640 1 -- 172.21.15.94:0/3350986768 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094005280 con 0x555ecbfcd370 2024-09-19T04:23:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.281+0000 7f20a6095740 1 -- 172.21.15.94:0/3350986768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfcd370 msgr2=0x555ecbfcf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.281+0000 7f20a6095740 1 --2- 172.21.15.94:0/3350986768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfcd370 0x555ecbfcf760 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2094009f00 tx=0x7f2094011810 comp rx=0 tx=0).stop 2024-09-19T04:23:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.283+0000 7f20a6095740 1 -- 172.21.15.94:0/3350986768 shutdown_connections 2024-09-19T04:23:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.283+0000 7f20a6095740 1 --2- 172.21.15.94:0/3350986768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfcd370 0x555ecbfcf760 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.283+0000 7f20a6095740 1 --2- 172.21.15.94:0/3350986768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfc2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.283+0000 7f20a6095740 1 --2- 172.21.15.94:0/3350986768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfc3b40 0x555ecbfc2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.283+0000 7f20a6095740 1 -- 172.21.15.94:0/3350986768 >> 172.21.15.94:0/3350986768 conn(0x555ecbf8b310 msgr2=0x555ecbfbec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.283+0000 7f20a6095740 1 -- 172.21.15.94:0/3350986768 shutdown_connections 2024-09-19T04:23:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.283+0000 7f20a6095740 1 -- 172.21.15.94:0/3350986768 wait complete. 2024-09-19T04:23:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.284+0000 7f20a6095740 1 -- start start 2024-09-19T04:23:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.284+0000 7f20a6095740 1 Processor -- start 2024-09-19T04:23:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.285+0000 7f20a6095740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfcce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.285+0000 7f20a6095740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.285+0000 7f20a6095740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.285+0000 7f20a6095740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555ecbfd0070 con 0x555ecbfc3b40 2024-09-19T04:23:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.285+0000 7f20a6095740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555ecbee3060 con 0x555ecbfcd370 2024-09-19T04:23:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.285+0000 7f20a6095740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ecbfd01f0 con 0x555ecbfc25d0 2024-09-19T04:23:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.285+0000 7f20a6095740 10 client.0 _ll_register_callbacks cb 0x555ecbc368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.286+0000 7f20a2259640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.286+0000 7f20a2259640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56124/0 (socket says 172.21.15.94:56124) 2024-09-19T04:23:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.286+0000 7f20a2259640 1 -- 172.21.15.94:0/2711676919 learned_addr learned my addr 172.21.15.94:0/2711676919 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.286+0000 7f20a1257640 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.287+0000 7f20a2259640 1 -- 172.21.15.94:0/2711676919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 msgr2=0x555ecbfcce30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:23:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.287+0000 7f20a2259640 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfcce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.287+0000 7f20a2259640 1 -- 172.21.15.94:0/2711676919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 msgr2=0x555ecbfca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.287+0000 7f20a2259640 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.287+0000 7f20a2259640 1 -- 172.21.15.94:0/2711676919 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecbc993c0 con 0x555ecbfcd370 2024-09-19T04:23:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.287+0000 7f20a2259640 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfca870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f20940047f0 tx=0x7f2094004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0c0d2464363e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.287+0000 7f20857fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:23:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.288+0000 7f20857fa640 1 -- 172.21.15.94:0/2711676919 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094029020 con 0x555ecbfcd370 2024-09-19T04:23:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.288+0000 7f20857fa640 1 -- 172.21.15.94:0/2711676919 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f2094005020 con 0x555ecbfcd370 2024-09-19T04:23:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.288+0000 7f20a6095740 1 -- 172.21.15.94:0/2711676919 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2094004050 con 0x555ecbfcd370 2024-09-19T04:23:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20857fa640 1 -- 172.21.15.94:0/2711676919 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2094034050 con 0x555ecbfcd370 2024-09-19T04:23:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20a6095740 10 client.26042 fetch_fsmap learned FSMap version 43 2024-09-19T04:23:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20a6095740 1 -- 172.21.15.94:0/2711676919 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555ecbfdbbe0 con 0x555ecbfcd370 2024-09-19T04:23:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20857fa640 1 -- 172.21.15.94:0/2711676919 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2094004050 con 0x555ecbfcd370 2024-09-19T04:23:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20857fa640 1 -- 172.21.15.94:0/2711676919 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f20940195b0 con 0x555ecbfcd370 2024-09-19T04:23:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20a6095740 10 client.26042 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:23:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20a6095740 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20a6095740 20 client.26042 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20a6095740 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ecbfdcfe0 0x555ecbfdf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20a6095740 4 client.26042 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.289+0000 7f20a6095740 1 -- 172.21.15.94:0/2711676919 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ecbf35860 con 0x555ecbfdcfe0 2024-09-19T04:23:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.290+0000 7f20a1a58640 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ecbfdcfe0 0x555ecbfdf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.291+0000 7f20a1a58640 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ecbfdcfe0 0x555ecbfdf3d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=235249255138c1e server_cookie=8201043a197737a9 in_seq=0 out_seq=0 2024-09-19T04:23:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.291+0000 7f20857fa640 10 client.26042 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.292+0000 7f20857fa640 1 -- 172.21.15.94:0/2711676919 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ecbf35860 con 0x555ecbfdcfe0 2024-09-19T04:23:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.292+0000 7f20857fa640 10 client.26042 handle_command_reply: tid=0 2024-09-19T04:23:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.292+0000 7f20a6095740 1 client.26042 shutdown 2024-09-19T04:23:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 -- 172.21.15.94:0/2711676919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 msgr2=0x555ecbfca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfca870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f20940047f0 tx=0x7f2094004430 comp rx=0 tx=0).stop 2024-09-19T04:23:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 -- 172.21.15.94:0/2711676919 shutdown_connections 2024-09-19T04:23:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ecbfdcfe0 0x555ecbfdf3d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfca870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 --2- 172.21.15.94:0/2711676919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfcce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 -- 172.21.15.94:0/2711676919 >> 172.21.15.94:0/2711676919 conn(0x555ecbf8b310 msgr2=0x555ecbfc4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20857fa640 0 client.26042 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 -- 172.21.15.94:0/2711676919 shutdown_connections 2024-09-19T04:23:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.293+0000 7f20a6095740 1 -- 172.21.15.94:0/2711676919 wait complete. 2024-09-19T04:23:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.294+0000 7f20a6095740 20 client.26042 trim_cache size 0 max 16384 2024-09-19T04:23:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.318+0000 7f20a6095740 1 Processor -- start 2024-09-19T04:23:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.318+0000 7f20a6095740 1 -- start start 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a6095740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfbee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a6095740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfe4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a6095740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfe5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a6095740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f209c1ae510 con 0x555ecbfc3b40 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a6095740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f209c1abdb0 con 0x555ecbfcd370 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a6095740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f209c1a9c20 con 0x555ecbfc25d0 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a1257640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfe4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a1a58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfbee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a1257640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfe4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47302/0 (socket says 172.21.15.94:47302) 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a1257640 1 -- 172.21.15.94:0/3727495147 learned_addr learned my addr 172.21.15.94:0/3727495147 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:44.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a2259640 1 --2- 172.21.15.94:0/3727495147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfe5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a1a58640 1 -- 172.21.15.94:0/3727495147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 msgr2=0x555ecbfe5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:44.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a1a58640 1 --2- 172.21.15.94:0/3727495147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfe5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a1a58640 1 -- 172.21.15.94:0/3727495147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 msgr2=0x555ecbfe4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:44.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a1a58640 1 --2- 172.21.15.94:0/3727495147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfe4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.319+0000 7f20a1a58640 1 -- 172.21.15.94:0/3727495147 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecbfbe4f0 con 0x555ecbfc25d0 2024-09-19T04:23:44.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.320+0000 7f20a1a58640 1 --2- 172.21.15.94:0/3727495147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfbee10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x555ecbfdfab0 tx=0x7f2098018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c9411b707de22dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:44.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.320+0000 7f20877fe640 1 -- 172.21.15.94:0/3727495147 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209800cdb0 con 0x555ecbfc25d0 2024-09-19T04:23:44.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.320+0000 7f20877fe640 1 -- 172.21.15.94:0/3727495147 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20980054d0 con 0x555ecbfc25d0 2024-09-19T04:23:44.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.320+0000 7f20877fe640 1 -- 172.21.15.94:0/3727495147 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209802b400 con 0x555ecbfc25d0 2024-09-19T04:23:44.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.322+0000 7f20a6095740 1 -- 172.21.15.94:0/3727495147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 msgr2=0x555ecbfbee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:44.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.322+0000 7f20a6095740 1 --2- 172.21.15.94:0/3727495147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfbee10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x555ecbfdfab0 tx=0x7f2098018000 comp rx=0 tx=0).stop 2024-09-19T04:23:44.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.322+0000 7f20a6095740 1 -- 172.21.15.94:0/3727495147 shutdown_connections 2024-09-19T04:23:44.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.322+0000 7f20a6095740 1 --2- 172.21.15.94:0/3727495147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfe5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.322+0000 7f20a6095740 1 --2- 172.21.15.94:0/3727495147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfe4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.322+0000 7f20a6095740 1 --2- 172.21.15.94:0/3727495147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfbee10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.323+0000 7f20a6095740 1 -- 172.21.15.94:0/3727495147 >> 172.21.15.94:0/3727495147 conn(0x555ecbf8b310 msgr2=0x555ecbfbe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:44.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.323+0000 7f20a6095740 1 -- 172.21.15.94:0/3727495147 shutdown_connections 2024-09-19T04:23:44.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.323+0000 7f20a6095740 1 -- 172.21.15.94:0/3727495147 wait complete. 2024-09-19T04:23:44.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.324+0000 7f20a6095740 1 -- start start 2024-09-19T04:23:44.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.324+0000 7f20a6095740 1 Processor -- start 2024-09-19T04:23:44.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.325+0000 7f20a6095740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfd5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.325+0000 7f20a6095740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfc6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.325+0000 7f20a6095740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfc7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.325+0000 7f20a6095740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f209c1aa270 con 0x555ecbfc3b40 2024-09-19T04:23:44.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.325+0000 7f20a6095740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f209c1ab6e0 con 0x555ecbfcd370 2024-09-19T04:23:44.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.325+0000 7f20a6095740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f209c1aa940 con 0x555ecbfc25d0 2024-09-19T04:23:44.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.325+0000 7f20a1a58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfd5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.325+0000 7f20a1a58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfd5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47326/0 (socket says 172.21.15.94:47326) 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.325+0000 7f20a1a58640 1 -- 172.21.15.94:0/1040488393 learned_addr learned my addr 172.21.15.94:0/1040488393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.326+0000 7f20a6095740 10 client.0 _ll_register_callbacks cb 0x7f2098009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.326+0000 7f20a2259640 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfc7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.326+0000 7f20a1257640 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfc6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.326+0000 7f20a1a58640 1 -- 172.21.15.94:0/1040488393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 msgr2=0x555ecbfc7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.326+0000 7f20a1a58640 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfc7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.326+0000 7f20a1a58640 1 -- 172.21.15.94:0/1040488393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 msgr2=0x555ecbfc6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.326+0000 7f20a1a58640 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfc6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.326+0000 7f20a1a58640 1 -- 172.21.15.94:0/1040488393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecbfc7770 con 0x555ecbfc25d0 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.327+0000 7f20a1a58640 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfd5950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20980094b0 tx=0x7f2098005680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a115269662de9bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.327+0000 7f20857fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:23:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.327+0000 7f20857fa640 1 -- 172.21.15.94:0/1040488393 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098004be0 con 0x555ecbfc25d0 2024-09-19T04:23:44.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.328+0000 7f20857fa640 1 -- 172.21.15.94:0/1040488393 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f2098009040 con 0x555ecbfc25d0 2024-09-19T04:23:44.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.328+0000 7f20a6095740 1 -- 172.21.15.94:0/1040488393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20980054d0 con 0x555ecbfc25d0 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.328+0000 7f20857fa640 1 -- 172.21.15.94:0/1040488393 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2098049050 con 0x555ecbfc25d0 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.328+0000 7f20a6095740 10 client.26032 fetch_fsmap learned FSMap version 43 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.328+0000 7f20a6095740 1 -- 172.21.15.94:0/1040488393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555ecc0478f0 con 0x555ecbfc25d0 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.328+0000 7f20857fa640 1 -- 172.21.15.94:0/1040488393 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2098005230 con 0x555ecbfc25d0 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.329+0000 7f20857fa640 1 -- 172.21.15.94:0/1040488393 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f2098031030 con 0x555ecbfc25d0 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.329+0000 7f20a6095740 10 client.26032 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.329+0000 7f20a6095740 10 client.26032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.329+0000 7f20a6095740 20 client.26032 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.329+0000 7f20a6095740 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ecc047f60 0x555ecc04a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.329+0000 7f20a6095740 4 client.26032 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:23:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.329+0000 7f20a6095740 1 -- 172.21.15.94:0/1040488393 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f20980054d0 con 0x555ecc047f60 2024-09-19T04:23:44.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.329+0000 7f20a1257640 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ecc047f60 0x555ecc04a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:44.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.330+0000 7f20857fa640 10 client.26032 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:44.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.330+0000 7f20a1257640 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ecc047f60 0x555ecc04a380 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfce16f8875732fa server_cookie=5be7ba4f2553ee5a in_seq=0 out_seq=0 2024-09-19T04:23:44.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20857fa640 1 -- 172.21.15.94:0/1040488393 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f20980054d0 con 0x555ecc047f60 2024-09-19T04:23:44.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20857fa640 10 client.26032 handle_command_reply: tid=0 2024-09-19T04:23:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20a6095740 1 client.26032 shutdown 2024-09-19T04:23:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20a6095740 1 -- 172.21.15.94:0/1040488393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 msgr2=0x555ecbfd5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20a6095740 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfd5950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20980094b0 tx=0x7f2098005680 comp rx=0 tx=0).stop 2024-09-19T04:23:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20a6095740 1 -- 172.21.15.94:0/1040488393 shutdown_connections 2024-09-19T04:23:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20a6095740 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x555ecc047f60 0x555ecc04a380 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20a6095740 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555ecbfcd370 0x555ecbfc7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20a6095740 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ecbfc3b40 0x555ecbfc6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20a6095740 1 --2- 172.21.15.94:0/1040488393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ecbfc25d0 0x555ecbfd5950 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20a6095740 1 -- 172.21.15.94:0/1040488393 >> 172.21.15.94:0/1040488393 conn(0x555ecbf8b310 msgr2=0x555ecbfbd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.331+0000 7f20857fa640 0 client.26032 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.332+0000 7f20a6095740 1 -- 172.21.15.94:0/1040488393 shutdown_connections 2024-09-19T04:23:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.332+0000 7f20a6095740 1 -- 172.21.15.94:0/1040488393 wait complete. 2024-09-19T04:23:44.334 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:23:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.332+0000 7f20a6095740 20 client.26032 trim_cache size 0 max 16384 2024-09-19T04:23:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.333+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20880a3580 msgr2=0x7f20880a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.333+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20880a3580 0x7f20880a5a40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20980096c0 tx=0x7f2098014040 comp rx=0 tx=0).stop 2024-09-19T04:23:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.333+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2599219499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f209c102ef0 msgr2=0x7f209c196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.333+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f209c102ef0 0x7f209c196980 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f208c008000 tx=0x7f208c0040c0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.334+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2599219499 shutdown_connections 2024-09-19T04:23:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.334+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f20880a3580 0x7f20880a5a40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.334+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f209c19ad10 0x7f209c19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.334+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f209c102ef0 0x7f209c196980 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.334+0000 7f20a3cbc640 1 --2- 172.21.15.94:0/2599219499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c100e20 0x7f209c196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.334+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2599219499 >> 172.21.15.94:0/2599219499 conn(0x7f209c0fc7d0 msgr2=0x7f209c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.334+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2599219499 shutdown_connections 2024-09-19T04:23:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:44.334+0000 7f20a3cbc640 1 -- 172.21.15.94:0/2599219499 wait complete. 2024-09-19T04:23:44.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:44 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:23:44.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:44 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:23:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:44 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:23:45.130 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:45 smithi094 ceph-mon[30972]: pgmap v268: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-09-19T04:23:45.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:45 smithi112 ceph-mon[35051]: pgmap v268: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-09-19T04:23:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 450.0 s, 1363.6 tps, lat 73.318 ms stddev 26.621 2024-09-19T04:23:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:45 smithi104 ceph-mon[34145]: pgmap v268: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-09-19T04:23:46.240 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:23:46.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:23:46.663 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.233+0000 7fab21ee5640 1 -- 172.21.15.94:0/1628855013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab1c101e20 msgr2=0x7fab1c1056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.233+0000 7fab21ee5640 1 --2- 172.21.15.94:0/1628855013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab1c101e20 0x7fab1c1056b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab10009860 tx=0x7fab1002a340 comp rx=0 tx=0).stop 2024-09-19T04:23:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.233+0000 7fab21ee5640 1 -- 172.21.15.94:0/1628855013 shutdown_connections 2024-09-19T04:23:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.233+0000 7fab21ee5640 1 --2- 172.21.15.94:0/1628855013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab1c105de0 0x7fab1c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.233+0000 7fab21ee5640 1 --2- 172.21.15.94:0/1628855013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab1c101e20 0x7fab1c1056b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.233+0000 7fab21ee5640 1 --2- 172.21.15.94:0/1628855013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab1c101470 0x7fab1c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.234+0000 7fab21ee5640 1 -- 172.21.15.94:0/1628855013 >> 172.21.15.94:0/1628855013 conn(0x7fab1c077230 msgr2=0x7fab1c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.234+0000 7fab21ee5640 1 -- 172.21.15.94:0/1628855013 shutdown_connections 2024-09-19T04:23:47.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.234+0000 7fab21ee5640 1 -- 172.21.15.94:0/1628855013 wait complete. 2024-09-19T04:23:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.234+0000 7fab21ee5640 1 Processor -- start 2024-09-19T04:23:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.235+0000 7fab21ee5640 1 -- start start 2024-09-19T04:23:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.235+0000 7fab21ee5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab1c101470 0x7fab1c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.235+0000 7fab21ee5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab1c101e20 0x7fab1c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.235+0000 7fab21ee5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab1c105de0 0x7fab1c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab21ee5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c110cb0 con 0x7fab1c101470 2024-09-19T04:23:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab21ee5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c110b30 con 0x7fab1c101e20 2024-09-19T04:23:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab21ee5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c110e30 con 0x7fab1c105de0 2024-09-19T04:23:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab1affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab1c101e20 0x7fab1c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab1affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab1c101e20 0x7fab1c1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56164/0 (socket says 172.21.15.94:56164) 2024-09-19T04:23:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab1affd640 1 -- 172.21.15.94:0/801501253 learned_addr learned my addr 172.21.15.94:0/801501253 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab1c101470 0x7fab1c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab1bfff640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab1c105de0 0x7fab1c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab1affd640 1 -- 172.21.15.94:0/801501253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab1c105de0 msgr2=0x7fab1c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab1affd640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab1c105de0 0x7fab1c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab1affd640 1 -- 172.21.15.94:0/801501253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab1c101470 msgr2=0x7fab1c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.236+0000 7fab1affd640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab1c101470 0x7fab1c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.237+0000 7fab1affd640 1 -- 172.21.15.94:0/801501253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c19d330 con 0x7fab1c101e20 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.237+0000 7fab1bfff640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab1c105de0 0x7fab1c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.237+0000 7fab1affd640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab1c101e20 0x7fab1c1988c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fab100306b0 tx=0x7fab1002af60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.237+0000 7fab18ff9640 1 -- 172.21.15.94:0/801501253 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10041020 con 0x7fab1c101e20 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.237+0000 7fab18ff9640 1 -- 172.21.15.94:0/801501253 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab10030c30 con 0x7fab1c101e20 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.237+0000 7fab21ee5640 1 -- 172.21.15.94:0/801501253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c19d5c0 con 0x7fab1c101e20 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.237+0000 7fab18ff9640 1 -- 172.21.15.94:0/801501253 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10037070 con 0x7fab1c101e20 2024-09-19T04:23:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.237+0000 7fab21ee5640 1 -- 172.21.15.94:0/801501253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c1a4f00 con 0x7fab1c101e20 2024-09-19T04:23:47.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.241+0000 7fab18ff9640 1 -- 172.21.15.94:0/801501253 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fab10044020 con 0x7fab1c101e20 2024-09-19T04:23:47.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.241+0000 7fab242be740 1 Processor -- start 2024-09-19T04:23:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab242be740 1 -- start start 2024-09-19T04:23:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab242be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cdef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab242be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdf480 0x55f5c9ce9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab242be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce9d40 0x55f5c9cec130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab242be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f5c9c44280 con 0x55f5c9cdeb60 2024-09-19T04:23:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab242be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f5c9bebb10 con 0x55f5c9ce9d40 2024-09-19T04:23:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab242be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f5c9c40240 con 0x55f5c9cdf480 2024-09-19T04:23:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdf480 0x55f5c9ce9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdf480 0x55f5c9ce9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47340/0 (socket says 172.21.15.94:47340) 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce9d40 0x55f5c9cec130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1b7fe640 1 -- 172.21.15.94:0/380655298 learned_addr learned my addr 172.21.15.94:0/380655298 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1bfff640 1 --2- 172.21.15.94:0/380655298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cdef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab18ff9640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fab040a3500 0x7fab040a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1b7fe640 1 -- 172.21.15.94:0/380655298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce9d40 msgr2=0x55f5c9cec130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/380655298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce9d40 0x55f5c9cec130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1b7fe640 1 -- 172.21.15.94:0/380655298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9cdeb60 msgr2=0x55f5c9cdef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/380655298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cdef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.242+0000 7fab1b7fe640 1 -- 172.21.15.94:0/380655298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c9bf81f0 con 0x55f5c9cdf480 2024-09-19T04:23:47.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.243+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/380655298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdf480 0x55f5c9ce9800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009d20 tx=0x7fab0c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d6114e94bc439d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.243+0000 7fab1bfff640 1 --2- 172.21.15.94:0/380655298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cdef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:23:47.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.243+0000 7fab18ff9640 1 -- 172.21.15.94:0/801501253 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fab100ef930 con 0x7fab1c101e20 2024-09-19T04:23:47.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.243+0000 7fab01ffb640 1 -- 172.21.15.94:0/380655298 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c012650 con 0x55f5c9cdf480 2024-09-19T04:23:47.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.243+0000 7fab01ffb640 1 -- 172.21.15.94:0/380655298 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab0c009e10 con 0x55f5c9cdf480 2024-09-19T04:23:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab1bfff640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fab040a3500 0x7fab040a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab01ffb640 1 -- 172.21.15.94:0/380655298 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c00fc60 con 0x55f5c9cdf480 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab242be740 1 -- 172.21.15.94:0/380655298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdf480 msgr2=0x55f5c9ce9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab242be740 1 --2- 172.21.15.94:0/380655298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdf480 0x55f5c9ce9800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009d20 tx=0x7fab0c0117b0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab242be740 1 -- 172.21.15.94:0/380655298 shutdown_connections 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab242be740 1 --2- 172.21.15.94:0/380655298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce9d40 0x55f5c9cec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab242be740 1 --2- 172.21.15.94:0/380655298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdf480 0x55f5c9ce9800 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab242be740 1 --2- 172.21.15.94:0/380655298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cdef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab242be740 1 -- 172.21.15.94:0/380655298 >> 172.21.15.94:0/380655298 conn(0x55f5c9cdb0f0 msgr2=0x55f5c9cdb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab1bfff640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fab040a3500 0x7fab040a59c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab14005fd0 tx=0x7fab14004a40 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.244+0000 7fab242be740 1 -- 172.21.15.94:0/380655298 shutdown_connections 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.245+0000 7fab242be740 1 -- 172.21.15.94:0/380655298 wait complete. 2024-09-19T04:23:47.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.245+0000 7fab242be740 1 -- start start 2024-09-19T04:23:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.245+0000 7fab242be740 1 Processor -- start 2024-09-19T04:23:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab242be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab242be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9ce9d40 0x55f5c9ce6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab242be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab242be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f5c9cec9b0 con 0x55f5c9ce6cd0 2024-09-19T04:23:47.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab242be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f5c9bfef10 con 0x55f5c9cdeb60 2024-09-19T04:23:47.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56180/0 (socket says 172.21.15.94:56180) 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab1b7fe640 1 -- 172.21.15.94:0/1195943218 learned_addr learned my addr 172.21.15.94:0/1195943218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab1bfff640 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab242be740 1 -- 172.21.15.94:0/1195943218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f5c9cecb30 con 0x55f5c9ce9d40 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab1affd640 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9ce9d40 0x55f5c9ce6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.246+0000 7fab242be740 10 client.0 _ll_register_callbacks cb 0x55f5c9bc7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.247+0000 7fab1b7fe640 1 -- 172.21.15.94:0/1195943218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9ce9d40 msgr2=0x55f5c9ce6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.247+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9ce9d40 0x55f5c9ce6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.247+0000 7fab1b7fe640 1 -- 172.21.15.94:0/1195943218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce6cd0 msgr2=0x55f5c9ce7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.247+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.247+0000 7fab1b7fe640 1 -- 172.21.15.94:0/1195943218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c99b56a0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.247+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce94e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006010 tx=0x7fab0c0096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d16b2e1e61ab100 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.247+0000 7faafaffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.248+0000 7faafaffd640 1 -- 172.21.15.94:0/1195943218 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c028070 con 0x55f5c9cdeb60 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.248+0000 7fab242be740 1 -- 172.21.15.94:0/1195943218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab0c009e10 con 0x55f5c9cdeb60 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.248+0000 7faafaffd640 1 -- 172.21.15.94:0/1195943218 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fab0c00fdf0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.248+0000 7faafaffd640 1 -- 172.21.15.94:0/1195943218 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fab0c034050 con 0x55f5c9cdeb60 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.248+0000 7fab242be740 10 client.26060 fetch_fsmap learned FSMap version 43 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.248+0000 7fab242be740 1 -- 172.21.15.94:0/1195943218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f5c9cf8630 con 0x55f5c9cdeb60 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.248+0000 7faafaffd640 1 -- 172.21.15.94:0/1195943218 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab0c003aa0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.248+0000 7faafaffd640 1 -- 172.21.15.94:0/1195943218 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7fab0c0124e0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.249+0000 7fab242be740 10 client.26060 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:23:47.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.249+0000 7fab242be740 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:47.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.249+0000 7fab242be740 20 client.26060 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:47.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.249+0000 7fab242be740 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f5c9cf9eb0 0x55f5c9cfc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.249+0000 7fab242be740 4 client.26060 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:47.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.249+0000 7fab242be740 1 -- 172.21.15.94:0/1195943218 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fab0c009e10 con 0x55f5c9cf9eb0 2024-09-19T04:23:47.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.249+0000 7fab1affd640 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f5c9cf9eb0 0x55f5c9cfc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.249+0000 7fab1affd640 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f5c9cf9eb0 0x55f5c9cfc330 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ff7cd6aebe2d78 server_cookie=de4fde33a1ce4b8f in_seq=0 out_seq=0 2024-09-19T04:23:47.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.249+0000 7faafaffd640 10 client.26060 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:47.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.251+0000 7faafaffd640 1 -- 172.21.15.94:0/1195943218 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fab0c009e10 con 0x55f5c9cf9eb0 2024-09-19T04:23:47.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.251+0000 7faafaffd640 10 client.26060 handle_command_reply: tid=0 2024-09-19T04:23:47.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.251+0000 7fab242be740 1 client.26060 shutdown 2024-09-19T04:23:47.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 -- 172.21.15.94:0/1195943218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 msgr2=0x55f5c9ce94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce94e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006010 tx=0x7fab0c0096a0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 -- 172.21.15.94:0/1195943218 shutdown_connections 2024-09-19T04:23:47.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f5c9cf9eb0 0x55f5c9cfc330 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9ce9d40 0x55f5c9ce6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 --2- 172.21.15.94:0/1195943218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce94e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 -- 172.21.15.94:0/1195943218 >> 172.21.15.94:0/1195943218 conn(0x55f5c9cdb0f0 msgr2=0x55f5c9ce05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7faafaffd640 0 client.26060 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:47.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 -- 172.21.15.94:0/1195943218 shutdown_connections 2024-09-19T04:23:47.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.252+0000 7fab242be740 1 -- 172.21.15.94:0/1195943218 wait complete. 2024-09-19T04:23:47.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.253+0000 7fab242be740 20 client.26060 trim_cache size 0 max 16384 2024-09-19T04:23:47.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.283+0000 7fab242be740 1 Processor -- start 2024-09-19T04:23:47.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.283+0000 7fab242be740 1 -- start start 2024-09-19T04:23:47.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.283+0000 7fab242be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.283+0000 7fab242be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.283+0000 7fab242be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 0x55f5c9d01c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.283+0000 7fab242be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab08011230 con 0x55f5c9ce9d40 2024-09-19T04:23:47.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.283+0000 7fab242be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab0800ead0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.283+0000 7fab242be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab0800c940 con 0x55f5c9ce6cd0 2024-09-19T04:23:47.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.283+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 0x55f5c9d01c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 0x55f5c9d01c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47360/0 (socket says 172.21.15.94:47360) 2024-09-19T04:23:47.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab1bfff640 1 -- 172.21.15.94:0/16753899 learned_addr learned my addr 172.21.15.94:0/16753899 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:47.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/16753899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab1b7fe640 1 -- 172.21.15.94:0/16753899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9ce6cd0 msgr2=0x55f5c9ce6590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:23:47.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/16753899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce6590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab1b7fe640 1 -- 172.21.15.94:0/16753899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 msgr2=0x55f5c9d01c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/16753899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 0x55f5c9d01c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab1b7fe640 1 -- 172.21.15.94:0/16753899 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c9ce9850 con 0x55f5c9cdeb60 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/16753899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce6050 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006010 tx=0x7fab0c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=404b021377bf9b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab017fa640 1 -- 172.21.15.94:0/16753899 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c028040 con 0x55f5c9cdeb60 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab017fa640 1 -- 172.21.15.94:0/16753899 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab0c0057f0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.284+0000 7fab017fa640 1 -- 172.21.15.94:0/16753899 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c009930 con 0x55f5c9cdeb60 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.285+0000 7fab242be740 1 -- 172.21.15.94:0/16753899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 msgr2=0x55f5c9ce6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.285+0000 7fab242be740 1 --2- 172.21.15.94:0/16753899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce6050 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006010 tx=0x7fab0c009260 comp rx=0 tx=0).stop 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.286+0000 7fab242be740 1 -- 172.21.15.94:0/16753899 shutdown_connections 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.286+0000 7fab242be740 1 --2- 172.21.15.94:0/16753899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 0x55f5c9d01c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.286+0000 7fab242be740 1 --2- 172.21.15.94:0/16753899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.286+0000 7fab242be740 1 --2- 172.21.15.94:0/16753899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9cdeb60 0x55f5c9ce6050 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.286+0000 7fab242be740 1 -- 172.21.15.94:0/16753899 >> 172.21.15.94:0/16753899 conn(0x55f5c9cdb0f0 msgr2=0x55f5c9ce5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.286+0000 7fab242be740 1 -- 172.21.15.94:0/16753899 shutdown_connections 2024-09-19T04:23:47.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.286+0000 7fab242be740 1 -- 172.21.15.94:0/16753899 wait complete. 2024-09-19T04:23:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.286+0000 7fab242be740 1 -- start start 2024-09-19T04:23:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.286+0000 7fab242be740 1 Processor -- start 2024-09-19T04:23:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab242be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cf23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab242be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab242be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 0x55f5c9ce3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab242be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab0800cf90 con 0x55f5c9ce9d40 2024-09-19T04:23:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cf23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cf23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47364/0 (socket says 172.21.15.94:47364) 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab1b7fe640 1 -- 172.21.15.94:0/1542507686 learned_addr learned my addr 172.21.15.94:0/1542507686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab0800e400 con 0x55f5c9ce6cd0 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab0800d660 con 0x55f5c9cdeb60 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab1bfff640 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 0x55f5c9ce3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab242be740 10 client.0 _ll_register_callbacks cb 0x7fab100f9830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.287+0000 7fab1affd640 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7fab1b7fe640 1 -- 172.21.15.94:0/1542507686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce6cd0 msgr2=0x55f5c9ce3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7fab1b7fe640 1 -- 172.21.15.94:0/1542507686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 msgr2=0x55f5c9ce3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 0x55f5c9ce3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7fab1b7fe640 1 -- 172.21.15.94:0/1542507686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c9ce3fd0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7fab1b7fe640 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cf23a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab0c001eb0 tx=0x7fab0c00f880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f15c40c7fbfbf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7faafaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7faafaffd640 1 -- 172.21.15.94:0/1542507686 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c028040 con 0x55f5c9cdeb60 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7faafaffd640 1 -- 172.21.15.94:0/1542507686 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fab0c012650 con 0x55f5c9cdeb60 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab0c0057f0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7faafaffd640 1 -- 172.21.15.94:0/1542507686 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fab0c009600 con 0x55f5c9cdeb60 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7fab242be740 10 client.26044 fetch_fsmap learned FSMap version 43 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.288+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f5c9d64ac0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.289+0000 7faafaffd640 1 -- 172.21.15.94:0/1542507686 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab0c0054e0 con 0x55f5c9cdeb60 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.289+0000 7faafaffd640 1 -- 172.21.15.94:0/1542507686 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7fab0c003a90 con 0x55f5c9cdeb60 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.289+0000 7fab242be740 10 client.26044 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.289+0000 7fab242be740 10 client.26044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.289+0000 7fab242be740 20 client.26044 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.289+0000 7fab242be740 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f5c9d65180 0x55f5c9d675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.289+0000 7fab242be740 4 client.26044 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:23:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.289+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fab0c0057f0 con 0x55f5c9d65180 2024-09-19T04:23:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.290+0000 7fab1affd640 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f5c9d65180 0x55f5c9d675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.290+0000 7fab1affd640 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f5c9d65180 0x55f5c9d675a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c7c34542ae1f62 server_cookie=7dbb935d54a33abe in_seq=0 out_seq=0 2024-09-19T04:23:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.290+0000 7faafaffd640 10 client.26044 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.291+0000 7faafaffd640 1 -- 172.21.15.94:0/1542507686 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fab0c0057f0 con 0x55f5c9d65180 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.291+0000 7faafaffd640 10 client.26044 handle_command_reply: tid=0 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.291+0000 7fab242be740 1 client.26044 shutdown 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.292+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdeb60 msgr2=0x55f5c9cf23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.292+0000 7fab242be740 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cf23a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab0c001eb0 tx=0x7fab0c00f880 comp rx=0 tx=0).stop 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.292+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 shutdown_connections 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.292+0000 7fab242be740 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f5c9d65180 0x55f5c9d675a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.292+0000 7fab242be740 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9ce9d40 0x55f5c9ce3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.292+0000 7fab242be740 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5c9ce6cd0 0x55f5c9ce3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.292+0000 7fab242be740 1 --2- 172.21.15.94:0/1542507686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5c9cdeb60 0x55f5c9cf23a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.292+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 >> 172.21.15.94:0/1542507686 conn(0x55f5c9cdb0f0 msgr2=0x55f5c9ce5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.292+0000 7faafaffd640 0 client.26044 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.293+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 shutdown_connections 2024-09-19T04:23:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.293+0000 7fab242be740 1 -- 172.21.15.94:0/1542507686 wait complete. 2024-09-19T04:23:47.297 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:23:47.297 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:23:47.297 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "dc472793-62ee-4a3c-b9c9-8494099e40e9", 2024-09-19T04:23:47.297 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:23:47.297 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:23:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.293+0000 7fab242be740 20 client.26044 trim_cache size 0 max 16384 2024-09-19T04:23:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.294+0000 7fab21ee5640 1 -- 172.21.15.94:0/801501253 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fab040a3500 msgr2=0x7fab040a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.294+0000 7fab21ee5640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fab040a3500 0x7fab040a59c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab14005fd0 tx=0x7fab14004a40 comp rx=0 tx=0).stop 2024-09-19T04:23:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.294+0000 7fab21ee5640 1 -- 172.21.15.94:0/801501253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab1c101e20 msgr2=0x7fab1c1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.294+0000 7fab21ee5640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab1c101e20 0x7fab1c1988c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fab100306b0 tx=0x7fab1002af60 comp rx=0 tx=0).stop 2024-09-19T04:23:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.295+0000 7fab21ee5640 1 -- 172.21.15.94:0/801501253 shutdown_connections 2024-09-19T04:23:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.295+0000 7fab21ee5640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fab040a3500 0x7fab040a59c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.295+0000 7fab21ee5640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab1c105de0 0x7fab1c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.295+0000 7fab21ee5640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab1c101e20 0x7fab1c1988c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.295+0000 7fab21ee5640 1 --2- 172.21.15.94:0/801501253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab1c101470 0x7fab1c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.295+0000 7fab21ee5640 1 -- 172.21.15.94:0/801501253 >> 172.21.15.94:0/801501253 conn(0x7fab1c077230 msgr2=0x7fab1c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.296+0000 7fab21ee5640 1 -- 172.21.15.94:0/801501253 shutdown_connections 2024-09-19T04:23:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:47.296+0000 7fab21ee5640 1 -- 172.21.15.94:0/801501253 wait complete. 2024-09-19T04:23:47.405 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:47 smithi104 ceph-mon[34145]: pgmap v269: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-09-19T04:23:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:47 smithi112 ceph-mon[35051]: pgmap v269: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-09-19T04:23:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:47 smithi094 ceph-mon[30972]: pgmap v269: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-09-19T04:23:48.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:23:48.495 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:48 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:48 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:23:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:23:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:23:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:23:48.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:23:48.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:48 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.080+0000 7f00d0903640 1 -- 172.21.15.94:0/2320757822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc10e4f0 msgr2=0x7f00cc110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.080+0000 7f00d0903640 1 --2- 172.21.15.94:0/2320757822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc10e4f0 0x7f00cc110900 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009900 tx=0x7f00bc02a440 comp rx=0 tx=0).stop 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.080+0000 7f00d0903640 1 -- 172.21.15.94:0/2320757822 shutdown_connections 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.080+0000 7f00d0903640 1 --2- 172.21.15.94:0/2320757822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc10e4f0 0x7f00cc110900 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.080+0000 7f00d0903640 1 --2- 172.21.15.94:0/2320757822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00cc101380 0x7f00cc10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.080+0000 7f00d0903640 1 --2- 172.21.15.94:0/2320757822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00cc1009d0 0x7f00cc100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.080+0000 7f00d0903640 1 -- 172.21.15.94:0/2320757822 >> 172.21.15.94:0/2320757822 conn(0x7f00cc0fc7d0 msgr2=0x7f00cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.081+0000 7f00d0903640 1 -- 172.21.15.94:0/2320757822 shutdown_connections 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.081+0000 7f00d0903640 1 -- 172.21.15.94:0/2320757822 wait complete. 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.082+0000 7f00d0903640 1 Processor -- start 2024-09-19T04:23:49.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.082+0000 7f00d0903640 1 -- start start 2024-09-19T04:23:49.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.082+0000 7f00d0903640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc1009d0 0x7f00cc1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00d0903640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00cc101380 0x7f00cc1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00d0903640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00cc10e4f0 0x7f00cc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00d0903640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc114260 con 0x7f00cc1009d0 2024-09-19T04:23:49.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00d0903640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1140e0 con 0x7f00cc101380 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00d0903640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1143e0 con 0x7f00cc10e4f0 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc1009d0 0x7f00cc1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc1009d0 0x7f00cc1a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47394/0 (socket says 172.21.15.94:47394) 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00ca59c640 1 -- 172.21.15.94:0/1781431525 learned_addr learned my addr 172.21.15.94:0/1781431525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00cc101380 0x7f00cc1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00cad9d640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00cc10e4f0 0x7f00cc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00c9d9b640 1 -- 172.21.15.94:0/1781431525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00cc10e4f0 msgr2=0x7f00cc19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00cc10e4f0 0x7f00cc19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00c9d9b640 1 -- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc1009d0 msgr2=0x7f00cc1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc1009d0 0x7f00cc1a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00c9d9b640 1 -- 172.21.15.94:0/1781431525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc19aea0 con 0x7f00cc101380 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.083+0000 7f00ca59c640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc1009d0 0x7f00cc1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.084+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00cc101380 0x7f00cc1a0aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f00c400be20 tx=0x7f00c4009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.084+0000 7f00bb7fe640 1 -- 172.21.15.94:0/1781431525 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4019070 con 0x7f00cc101380 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.084+0000 7f00d0903640 1 -- 172.21.15.94:0/1781431525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc19b190 con 0x7f00cc101380 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.084+0000 7f00bb7fe640 1 -- 172.21.15.94:0/1781431525 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c40148f0 con 0x7f00cc101380 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.084+0000 7f00d0903640 1 -- 172.21.15.94:0/1781431525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc1a74e0 con 0x7f00cc101380 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.084+0000 7f00bb7fe640 1 -- 172.21.15.94:0/1781431525 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c40128f0 con 0x7f00cc101380 2024-09-19T04:23:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.087+0000 7f00bb7fe640 1 -- 172.21.15.94:0/1781431525 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f00c401c070 con 0x7f00cc101380 2024-09-19T04:23:49.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.088+0000 7f00d2cdc740 1 Processor -- start 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.088+0000 7f00d2cdc740 1 -- start start 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ad8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d948a3e2e0 con 0x55d948ad84a0 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d9489e5ce0 con 0x55d948ad8dc0 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d948a3a2a0 con 0x55d948ae3680 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00bb7fe640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f00b40a3430 0x7f00b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00ca59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00ca59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56240/0 (socket says 172.21.15.94:56240) 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00ca59c640 1 -- 172.21.15.94:0/4223628618 learned_addr learned my addr 172.21.15.94:0/4223628618 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00cad9d640 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ad8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.089+0000 7f00bb7fe640 1 -- 172.21.15.94:0/1781431525 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f00c40cf560 con 0x7f00cc101380 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00cad9d640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f00b40a3430 0x7f00b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00ca59c640 1 -- 172.21.15.94:0/4223628618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 msgr2=0x55d948ae5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00ca59c640 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00ca59c640 1 -- 172.21.15.94:0/4223628618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 msgr2=0x55d948ad8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00ca59c640 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ad8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00ca59c640 1 -- 172.21.15.94:0/4223628618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9489ca440 con 0x55d948ad8dc0 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00ca59c640 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae3140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009ed0 tx=0x7f00c000e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b60cc36d27c3f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00b8ff9640 1 -- 172.21.15.94:0/4223628618 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0017070 con 0x55d948ad8dc0 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00b8ff9640 1 -- 172.21.15.94:0/4223628618 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c0013920 con 0x55d948ad8dc0 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.090+0000 7f00b8ff9640 1 -- 172.21.15.94:0/4223628618 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0012980 con 0x55d948ad8dc0 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.091+0000 7f00d2cdc740 1 -- 172.21.15.94:0/4223628618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 msgr2=0x55d948ae3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.091+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae3140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009ed0 tx=0x7f00c000e5a0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.091+0000 7f00cad9d640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f00b40a3430 0x7f00b40a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00bc02a960 tx=0x7f00bc005fd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.092+0000 7f00d2cdc740 1 -- 172.21.15.94:0/4223628618 shutdown_connections 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.092+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.092+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae3140 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.092+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/4223628618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ad8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.092+0000 7f00d2cdc740 1 -- 172.21.15.94:0/4223628618 >> 172.21.15.94:0/4223628618 conn(0x55d948ad4a30 msgr2=0x55d948ad4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.092+0000 7f00d2cdc740 1 -- 172.21.15.94:0/4223628618 shutdown_connections 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.092+0000 7f00d2cdc740 1 -- 172.21.15.94:0/4223628618 wait complete. 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.092+0000 7f00d2cdc740 1 -- start start 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.092+0000 7f00d2cdc740 1 Processor -- start 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ae2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d948ae6250 con 0x55d948ad84a0 2024-09-19T04:23:49.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d9489f9050 con 0x55d948ad8dc0 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d948ae63d0 con 0x55d948ae3680 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00d2cdc740 10 client.0 _ll_register_callbacks cb 0x55d948a39160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ae2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00cad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ae2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47416/0 (socket says 172.21.15.94:47416) 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00ca59c640 1 -- 172.21.15.94:0/3298039321 learned_addr learned my addr 172.21.15.94:0/3298039321 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.093+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00cad9d640 1 -- 172.21.15.94:0/3298039321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 msgr2=0x55d948ae01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00cad9d640 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00cad9d640 1 -- 172.21.15.94:0/3298039321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 msgr2=0x55d948ae2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00cad9d640 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ae2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00cad9d640 1 -- 172.21.15.94:0/3298039321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9487af3c0 con 0x55d948ae3680 2024-09-19T04:23:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00ca59c640 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ae2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:49.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00cad9d640 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae06f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f00bc02aa30 tx=0x7f00bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d3d2e9bc2accea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:23:49.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00b1ffb640 1 -- 172.21.15.94:0/3298039321 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc04f060 con 0x55d948ae3680 2024-09-19T04:23:49.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00b1ffb640 1 -- 172.21.15.94:0/3298039321 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f00bc003a00 con 0x55d948ae3680 2024-09-19T04:23:49.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.094+0000 7f00d2cdc740 1 -- 172.21.15.94:0/3298039321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c0013920 con 0x55d948ae3680 2024-09-19T04:23:49.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.095+0000 7f00b1ffb640 1 -- 172.21.15.94:0/3298039321 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f00bc035040 con 0x55d948ae3680 2024-09-19T04:23:49.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.095+0000 7f00d2cdc740 10 client.26047 fetch_fsmap learned FSMap version 43 2024-09-19T04:23:49.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.095+0000 7f00d2cdc740 1 -- 172.21.15.94:0/3298039321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d948afa320 con 0x55d948ae3680 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.095+0000 7f00b1ffb640 1 -- 172.21.15.94:0/3298039321 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00bc004e50 con 0x55d948ae3680 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.095+0000 7f00b1ffb640 1 -- 172.21.15.94:0/3298039321 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f00bc03a040 con 0x55d948ae3680 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.096+0000 7f00d2cdc740 10 client.26047 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.096+0000 7f00d2cdc740 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.096+0000 7f00d2cdc740 20 client.26047 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.096+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d948afb780 0x55d948afdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.096+0000 7f00d2cdc740 4 client.26047 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.096+0000 7f00d2cdc740 1 -- 172.21.15.94:0/3298039321 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f00c0013920 con 0x55d948afb780 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.097+0000 7f00ca59c640 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d948afb780 0x55d948afdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.097+0000 7f00b1ffb640 10 client.26047 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:49.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.097+0000 7f00ca59c640 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d948afb780 0x55d948afdc00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8def06d0579706e7 server_cookie=93a586d9e1dcec02 in_seq=0 out_seq=0 2024-09-19T04:23:49.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.098+0000 7f00b1ffb640 1 -- 172.21.15.94:0/3298039321 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f00c0013920 con 0x55d948afb780 2024-09-19T04:23:49.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.098+0000 7f00b1ffb640 10 client.26047 handle_command_reply: tid=0 2024-09-19T04:23:49.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.098+0000 7f00d2cdc740 1 client.26047 shutdown 2024-09-19T04:23:49.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 -- 172.21.15.94:0/3298039321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 msgr2=0x55d948ae06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae06f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f00bc02aa30 tx=0x7f00bc009260 comp rx=0 tx=0).stop 2024-09-19T04:23:49.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 -- 172.21.15.94:0/3298039321 shutdown_connections 2024-09-19T04:23:49.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d948afb780 0x55d948afdc00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948ae06f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00b1ffb640 0 client.26047 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:49.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948ae01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/3298039321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad84a0 0x55d948ae2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 -- 172.21.15.94:0/3298039321 >> 172.21.15.94:0/3298039321 conn(0x55d948ad4a30 msgr2=0x55d948ada760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 -- 172.21.15.94:0/3298039321 shutdown_connections 2024-09-19T04:23:49.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 1 -- 172.21.15.94:0/3298039321 wait complete. 2024-09-19T04:23:49.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.099+0000 7f00d2cdc740 20 client.26047 trim_cache size 0 max 16384 2024-09-19T04:23:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00d2cdc740 1 Processor -- start 2024-09-19T04:23:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00d2cdc740 1 -- start start 2024-09-19T04:23:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad84a0 0x55d948ad4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad8dc0 0x55d948ad3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948b03510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac0111e0 con 0x55d948ad8dc0 2024-09-19T04:23:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac00ea80 con 0x55d948ad84a0 2024-09-19T04:23:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac00c8f0 con 0x55d948ae3680 2024-09-19T04:23:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad8dc0 0x55d948ad3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad8dc0 0x55d948ad3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47422/0 (socket says 172.21.15.94:47422) 2024-09-19T04:23:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00c9d9b640 1 -- 172.21.15.94:0/2577067125 learned_addr learned my addr 172.21.15.94:0/2577067125 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.129+0000 7f00ca59c640 1 --2- 172.21.15.94:0/2577067125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad84a0 0x55d948ad4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.130+0000 7f00ca59c640 1 -- 172.21.15.94:0/2577067125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 msgr2=0x55d948b03510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.130+0000 7f00ca59c640 1 --2- 172.21.15.94:0/2577067125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948b03510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.130+0000 7f00ca59c640 1 -- 172.21.15.94:0/2577067125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad8dc0 msgr2=0x55d948ad3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.130+0000 7f00ca59c640 1 --2- 172.21.15.94:0/2577067125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad8dc0 0x55d948ad3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.130+0000 7f00ca59c640 1 -- 172.21.15.94:0/2577067125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d948ad8bf0 con 0x55d948ad84a0 2024-09-19T04:23:49.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.130+0000 7f00ca59c640 1 --2- 172.21.15.94:0/2577067125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad84a0 0x55d948ad4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f00c0000c00 tx=0x7f00c0009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a699c0e6d62a6e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.130+0000 7f00b3fff640 1 -- 172.21.15.94:0/2577067125 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0017040 con 0x55d948ad84a0 2024-09-19T04:23:49.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.130+0000 7f00b3fff640 1 -- 172.21.15.94:0/2577067125 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c0013d50 con 0x55d948ad84a0 2024-09-19T04:23:49.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.130+0000 7f00b3fff640 1 -- 172.21.15.94:0/2577067125 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c00045b0 con 0x55d948ad84a0 2024-09-19T04:23:49.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.131+0000 7f00d2cdc740 1 -- 172.21.15.94:0/2577067125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad84a0 msgr2=0x55d948ad4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.131+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/2577067125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad84a0 0x55d948ad4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f00c0000c00 tx=0x7f00c0009830 comp rx=0 tx=0).stop 2024-09-19T04:23:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.131+0000 7f00d2cdc740 1 -- 172.21.15.94:0/2577067125 shutdown_connections 2024-09-19T04:23:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.131+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/2577067125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ae3680 0x55d948b03510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.131+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/2577067125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ad8dc0 0x55d948ad3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.131+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/2577067125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad84a0 0x55d948ad4e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.132+0000 7f00d2cdc740 1 -- 172.21.15.94:0/2577067125 >> 172.21.15.94:0/2577067125 conn(0x55d948ad4a30 msgr2=0x55d948ad3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.132+0000 7f00d2cdc740 1 -- 172.21.15.94:0/2577067125 shutdown_connections 2024-09-19T04:23:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.132+0000 7f00d2cdc740 1 -- 172.21.15.94:0/2577067125 wait complete. 2024-09-19T04:23:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.132+0000 7f00d2cdc740 1 -- start start 2024-09-19T04:23:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00d2cdc740 1 Processor -- start 2024-09-19T04:23:49.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ad84a0 0x55d948af4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948af1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00d2cdc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ae3680 0x55d948af2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac00cf40 con 0x55d948ae3680 2024-09-19T04:23:49.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00d2cdc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac00e3b0 con 0x55d948ad8dc0 2024-09-19T04:23:49.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ae3680 0x55d948af2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ae3680 0x55d948af2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47432/0 (socket says 172.21.15.94:47432) 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00cad9d640 1 -- 172.21.15.94:0/31387009 learned_addr learned my addr 172.21.15.94:0/31387009 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00d2cdc740 1 -- 172.21.15.94:0/31387009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac00d610 con 0x55d948ad84a0 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948af1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00d2cdc740 10 client.0 _ll_register_callbacks cb 0x7f00bc0040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.133+0000 7f00ca59c640 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ad84a0 0x55d948af4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00c9d9b640 1 -- 172.21.15.94:0/31387009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ad84a0 msgr2=0x55d948af4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ad84a0 0x55d948af4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00c9d9b640 1 -- 172.21.15.94:0/31387009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ae3680 msgr2=0x55d948af2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ae3680 0x55d948af2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00c9d9b640 1 -- 172.21.15.94:0/31387009 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d948ae66c0 con 0x55d948ad8dc0 2024-09-19T04:23:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00cad9d640 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ae3680 0x55d948af2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:49.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00c9d9b640 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948af1b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00c4097dc0 tx=0x7f00c4097600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2741c1bfff53855 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00b1ffb640 1 -- 172.21.15.94:0/31387009 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c40b4060 con 0x55d948ad8dc0 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00b1ffb640 1 -- 172.21.15.94:0/31387009 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f00c409cbe0 con 0x55d948ad8dc0 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.134+0000 7f00d2cdc740 1 -- 172.21.15.94:0/31387009 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c0013d50 con 0x55d948ad8dc0 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.135+0000 7f00b1ffb640 1 -- 172.21.15.94:0/31387009 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f00c40cd090 con 0x55d948ad8dc0 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.135+0000 7f00d2cdc740 10 client.26078 fetch_fsmap learned FSMap version 43 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.135+0000 7f00d2cdc740 1 -- 172.21.15.94:0/31387009 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d948ae79b0 con 0x55d948ad8dc0 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.135+0000 7f00b1ffb640 1 -- 172.21.15.94:0/31387009 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c0013d50 con 0x55d948ad8dc0 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.135+0000 7f00b1ffb640 1 -- 172.21.15.94:0/31387009 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f00c409ae60 con 0x55d948ad8dc0 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.135+0000 7f00d2cdc740 10 client.26078 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.135+0000 7f00d2cdc740 10 client.26078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.135+0000 7f00d2cdc740 20 client.26078 populate_metadata read hostname 'smithi094' 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.135+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d948afb780 0x55d948aea180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.136+0000 7f00d2cdc740 4 client.26078 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:23:49.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.136+0000 7f00d2cdc740 1 -- 172.21.15.94:0/31387009 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d948a4b850 con 0x55d948afb780 2024-09-19T04:23:49.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.136+0000 7f00ca59c640 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d948afb780 0x55d948aea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.136+0000 7f00ca59c640 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d948afb780 0x55d948aea180 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=665ff65bb2cfba34 server_cookie=c5d2a6e97a511767 in_seq=0 out_seq=0 2024-09-19T04:23:49.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.136+0000 7f00b1ffb640 10 client.26078 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:49.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.136+0000 7f00b1ffb640 1 -- 172.21.15.94:0/31387009 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d948a4b850 con 0x55d948afb780 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.136+0000 7f00b1ffb640 10 client.26078 handle_command_reply: tid=0 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.136+0000 7f00d2cdc740 1 client.26078 shutdown 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 -- 172.21.15.94:0/31387009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 msgr2=0x55d948af1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948af1b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00c4097dc0 tx=0x7f00c4097600 comp rx=0 tx=0).stop 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 -- 172.21.15.94:0/31387009 shutdown_connections 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d948afb780 0x55d948aea180 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d948ae3680 0x55d948af2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00b1ffb640 0 client.26078 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d948ad8dc0 0x55d948af1b50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 --2- 172.21.15.94:0/31387009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d948ad84a0 0x55d948af4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 -- 172.21.15.94:0/31387009 >> 172.21.15.94:0/31387009 conn(0x55d948ad4a30 msgr2=0x55d948ae2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 -- 172.21.15.94:0/31387009 shutdown_connections 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.137+0000 7f00d2cdc740 1 -- 172.21.15.94:0/31387009 wait complete. 2024-09-19T04:23:49.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.138+0000 7f00d2cdc740 20 client.26078 trim_cache size 0 max 16384 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout: "dc472793-62ee-4a3c-b9c9-8494099e40e9": { 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "dc472793-62ee-4a3c-b9c9-8494099e40e9", 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:23:49.141 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:23:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 -- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f00b40a3430 msgr2=0x7f00b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f00b40a3430 0x7f00b40a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00bc02a960 tx=0x7f00bc005fd0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 -- 172.21.15.94:0/1781431525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00cc101380 msgr2=0x7f00cc1a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00cc101380 0x7f00cc1a0aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f00c400be20 tx=0x7f00c4009f70 comp rx=0 tx=0).stop 2024-09-19T04:23:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 -- 172.21.15.94:0/1781431525 shutdown_connections 2024-09-19T04:23:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f00b40a3430 0x7f00b40a58f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00cc10e4f0 0x7f00cc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00cc101380 0x7f00cc1a0aa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 --2- 172.21.15.94:0/1781431525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00cc1009d0 0x7f00cc1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 -- 172.21.15.94:0/1781431525 >> 172.21.15.94:0/1781431525 conn(0x7f00cc0fc7d0 msgr2=0x7f00cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 -- 172.21.15.94:0/1781431525 shutdown_connections 2024-09-19T04:23:49.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:23:49.139+0000 7f00d0903640 1 -- 172.21.15.94:0/1781431525 wait complete. 2024-09-19T04:23:49.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:49 smithi094 ceph-mon[30972]: pgmap v270: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:23:49.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:49 smithi112 ceph-mon[35051]: pgmap v270: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:23:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:49 smithi104 ceph-mon[34145]: pgmap v270: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:23:49.895 INFO:tasks.cephfs.filesystem:scrub status for tag:dc472793-62ee-4a3c-b9c9-8494099e40e9 - {'path': '/', 'tag': 'dc472793-62ee-4a3c-b9c9-8494099e40e9', 'options': 'recursive,force'} 2024-09-19T04:23:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 455.0 s, 1343.4 tps, lat 74.424 ms stddev 26.234 2024-09-19T04:23:51.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:51 smithi112 ceph-mon[35051]: pgmap v271: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.5 MiB/s wr, 1.07k op/s 2024-09-19T04:23:51.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:51 smithi094 ceph-mon[30972]: pgmap v271: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.5 MiB/s wr, 1.07k op/s 2024-09-19T04:23:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:51 smithi104 ceph-mon[34145]: pgmap v271: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.5 MiB/s wr, 1.07k op/s 2024-09-19T04:23:53.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:53 smithi094 ceph-mon[30972]: pgmap v272: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.5 MiB/s wr, 1.07k op/s 2024-09-19T04:23:53.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:53 smithi112 ceph-mon[35051]: pgmap v272: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.5 MiB/s wr, 1.07k op/s 2024-09-19T04:23:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:53 smithi104 ceph-mon[34145]: pgmap v272: 97 pgs: 97 active+clean; 3.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.5 MiB/s wr, 1.07k op/s 2024-09-19T04:23:55.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:55 smithi112 ceph-mon[35051]: pgmap v273: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 1.64k op/s 2024-09-19T04:23:55.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:55 smithi094 ceph-mon[30972]: pgmap v273: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 1.64k op/s 2024-09-19T04:23:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 460.0 s, 1498.6 tps, lat 66.835 ms stddev 24.365 2024-09-19T04:23:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:55 smithi104 ceph-mon[34145]: pgmap v273: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 1.64k op/s 2024-09-19T04:23:57.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:57 smithi112 ceph-mon[35051]: pgmap v274: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.2 MiB/s wr, 1.07k op/s 2024-09-19T04:23:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:57 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:23:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:57 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:23:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:57 smithi094 ceph-mon[30972]: pgmap v274: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.2 MiB/s wr, 1.07k op/s 2024-09-19T04:23:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:57 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:23:57.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:57 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:23:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:57 smithi104 ceph-mon[34145]: pgmap v274: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.2 MiB/s wr, 1.07k op/s 2024-09-19T04:23:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:57 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:23:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:57 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:23:58.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:58 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:23:58.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:58 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:23:58.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:58 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:23:59.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:23:59 smithi112 ceph-mon[35051]: pgmap v275: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-09-19T04:23:59.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:23:59 smithi094 ceph-mon[30972]: pgmap v275: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-09-19T04:23:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:23:59 smithi104 ceph-mon[34145]: pgmap v275: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-09-19T04:24:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 465.0 s, 1634.2 tps, lat 61.019 ms stddev 16.795 2024-09-19T04:24:01.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:01 smithi112 ceph-mon[35051]: pgmap v276: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:24:01.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:01 smithi094 ceph-mon[30972]: pgmap v276: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:24:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:01 smithi104 ceph-mon[34145]: pgmap v276: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:24:03.420 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:03 smithi094 ceph-mon[30972]: pgmap v277: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:24:03.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:03 smithi112 ceph-mon[35051]: pgmap v277: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:24:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:03 smithi104 ceph-mon[34145]: pgmap v277: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-09-19T04:24:05.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:05 smithi112 ceph-mon[35051]: pgmap v278: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 17 MiB/s wr, 1.92k op/s 2024-09-19T04:24:05.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:05 smithi094 ceph-mon[30972]: pgmap v278: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 17 MiB/s wr, 1.92k op/s 2024-09-19T04:24:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 470.0 s, 1511.6 tps, lat 66.218 ms stddev 29.731 2024-09-19T04:24:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:05 smithi104 ceph-mon[34145]: pgmap v278: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 17 MiB/s wr, 1.92k op/s 2024-09-19T04:24:07.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:07 smithi112 ceph-mon[35051]: pgmap v279: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:07.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:07 smithi094 ceph-mon[30972]: pgmap v279: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:07 smithi104 ceph-mon[34145]: pgmap v279: 97 pgs: 97 active+clean; 3.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:08.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:08 smithi112 ceph-mon[35051]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:24:08.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:08 smithi094 ceph-mon[30972]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:24:08.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:08 smithi104 ceph-mon[34145]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:24:09.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:09 smithi112 ceph-mon[35051]: pgmap v281: 97 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 86 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 13 MiB/s wr, 1.53k op/s 2024-09-19T04:24:09.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:09 smithi094 ceph-mon[30972]: pgmap v281: 97 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 86 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 13 MiB/s wr, 1.53k op/s 2024-09-19T04:24:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:09 smithi104 ceph-mon[34145]: pgmap v281: 97 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 86 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 13 MiB/s wr, 1.53k op/s 2024-09-19T04:24:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 475.0 s, 1445.4 tps, lat 69.058 ms stddev 29.738 2024-09-19T04:24:11.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:11 smithi112 ceph-mon[35051]: pgmap v282: 97 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 86 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 13 MiB/s wr, 1.53k op/s 2024-09-19T04:24:11.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:11 smithi094 ceph-mon[30972]: pgmap v282: 97 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 86 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 13 MiB/s wr, 1.53k op/s 2024-09-19T04:24:11.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:11 smithi104 ceph-mon[34145]: pgmap v282: 97 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 86 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 13 MiB/s wr, 1.53k op/s 2024-09-19T04:24:13.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:13 smithi112 ceph-mon[35051]: pgmap v283: 97 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 86 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 13 MiB/s wr, 1.53k op/s 2024-09-19T04:24:13.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:13 smithi094 ceph-mon[30972]: pgmap v283: 97 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 86 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 13 MiB/s wr, 1.53k op/s 2024-09-19T04:24:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:13 smithi104 ceph-mon[34145]: pgmap v283: 97 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 86 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 13 MiB/s wr, 1.53k op/s 2024-09-19T04:24:15.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:15 smithi112 ceph-mon[35051]: pgmap v284: 97 pgs: 97 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-09-19T04:24:15.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:15 smithi094 ceph-mon[30972]: pgmap v284: 97 pgs: 97 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-09-19T04:24:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 480.0 s, 1332.2 tps, lat 75.163 ms stddev 45.703 2024-09-19T04:24:15.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:15 smithi104 ceph-mon[34145]: pgmap v284: 97 pgs: 97 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-09-19T04:24:17.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:16 smithi112 ceph-mon[35051]: pgmap v285: 97 pgs: 97 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-09-19T04:24:17.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:16 smithi112 ceph-mon[35051]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:24:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:16 smithi094 ceph-mon[30972]: pgmap v285: 97 pgs: 97 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-09-19T04:24:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:16 smithi094 ceph-mon[30972]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:24:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:16 smithi104 ceph-mon[34145]: pgmap v285: 97 pgs: 97 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-09-19T04:24:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:16 smithi104 ceph-mon[34145]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:24:19.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:18 smithi112 ceph-mon[35051]: pgmap v287: 97 pgs: 97 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:19.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:18 smithi094 ceph-mon[30972]: pgmap v287: 97 pgs: 97 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:19.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:18 smithi104 ceph-mon[34145]: pgmap v287: 97 pgs: 97 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:19.896 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:24:19.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:19 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:24:20.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:19 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:24:20.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:19 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:24:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 485.0 s, 1511.6 tps, lat 66.132 ms stddev 31.008 2024-09-19T04:24:20.773 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:20 smithi094 ceph-mon[30972]: pgmap v288: 97 pgs: 97 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:20.773 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:20 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:24:20.773 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:20 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:24:20.773 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:20 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.907+0000 7efe1fb2a640 1 -- 172.21.15.94:0/1559656966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18102ef0 msgr2=0x7efe1810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.907+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/1559656966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18102ef0 0x7efe1810da40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe10009c10 tx=0x7efe1002a770 comp rx=0 tx=0).stop 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.907+0000 7efe1fb2a640 1 -- 172.21.15.94:0/1559656966 shutdown_connections 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.907+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/1559656966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18102ef0 0x7efe1810da40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.907+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/1559656966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe18102570 0x7efe181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.907+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/1559656966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe18100e20 0x7efe18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.907+0000 7efe1fb2a640 1 -- 172.21.15.94:0/1559656966 >> 172.21.15.94:0/1559656966 conn(0x7efe180fc7d0 msgr2=0x7efe180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.907+0000 7efe1fb2a640 1 -- 172.21.15.94:0/1559656966 shutdown_connections 2024-09-19T04:24:20.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.907+0000 7efe1fb2a640 1 -- 172.21.15.94:0/1559656966 wait complete. 2024-09-19T04:24:20.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.908+0000 7efe1fb2a640 1 Processor -- start 2024-09-19T04:24:20.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.908+0000 7efe1fb2a640 1 -- start start 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1fb2a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe18100e20 0x7efe1819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1fb2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18102570 0x7efe1819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1fb2a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe18102ef0 0x7efe1819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1d8c6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe18100e20 0x7efe1819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1d8c6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe18100e20 0x7efe1819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:43072/0 (socket says 172.21.15.94:43072) 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1d8c6640 1 -- 172.21.15.94:0/3189633409 learned_addr learned my addr 172.21.15.94:0/3189633409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810fc80 con 0x7efe18102ef0 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810fb00 con 0x7efe18100e20 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810fe00 con 0x7efe18102570 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.909+0000 7efe1d0c5640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18102570 0x7efe1819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.910+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe18102ef0 0x7efe1819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.910+0000 7efe1d8c6640 1 -- 172.21.15.94:0/3189633409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18102570 msgr2=0x7efe1819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.910+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18102570 0x7efe1819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.910+0000 7efe1d8c6640 1 -- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe18102ef0 msgr2=0x7efe1819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.910+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe18102ef0 0x7efe1819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.910+0000 7efe1d8c6640 1 -- 172.21.15.94:0/3189633409 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1819f350 con 0x7efe18100e20 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.910+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe18100e20 0x7efe1819a3a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efe1400e9c0 tx=0x7efe1400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.910+0000 7efe0effd640 1 -- 172.21.15.94:0/3189633409 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1400cde0 con 0x7efe18100e20 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.910+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1819f640 con 0x7efe18100e20 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.911+0000 7efe0effd640 1 -- 172.21.15.94:0/3189633409 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe140035f0 con 0x7efe18100e20 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.911+0000 7efe0effd640 1 -- 172.21.15.94:0/3189633409 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14010690 con 0x7efe18100e20 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.911+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe181a6f20 con 0x7efe18100e20 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.914+0000 7efe21f03740 1 Processor -- start 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.914+0000 7efe21f03740 1 -- start start 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.914+0000 7efe0effd640 1 -- 172.21.15.94:0/3189633409 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7efe14010830 con 0x7efe18100e20 2024-09-19T04:24:20.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.914+0000 7efe21f03740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55344c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.914+0000 7efe21f03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e55345f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.914+0000 7efe21f03740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e5534622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.914+0000 7efe21f03740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e5533b22e0 con 0x55e55344c4a0 2024-09-19T04:24:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.914+0000 7efe21f03740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e553359ce0 con 0x55e55345feb0 2024-09-19T04:24:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.914+0000 7efe21f03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e5533ae2a0 con 0x55e55344cdc0 2024-09-19T04:24:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d0c5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e5534622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d0c5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e5534622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:43088/0 (socket says 172.21.15.94:43088) 2024-09-19T04:24:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1e0c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55344c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d0c5640 1 -- 172.21.15.94:0/1015531035 learned_addr learned my addr 172.21.15.94:0/1015531035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e55345f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d0c5640 1 -- 172.21.15.94:0/1015531035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 msgr2=0x55e55345f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d0c5640 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e55345f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d0c5640 1 -- 172.21.15.94:0/1015531035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 msgr2=0x55e55344c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e55345f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d0c5640 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55344c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1d0c5640 1 -- 172.21.15.94:0/1015531035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55333e440 con 0x55e55345feb0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.915+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55344c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.916+0000 7efe0effd640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efe040a3430 0x7efe040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.916+0000 7efe1d0c5640 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e5534622a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efe0800bc80 tx=0x7efe0800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6557b9e92fb333f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.916+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efe040a3430 0x7efe040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.916+0000 7efe0effd640 1 -- 172.21.15.94:0/3189633409 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7efe140d00a0 con 0x7efe18100e20 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.916+0000 7efe03fff640 1 -- 172.21.15.94:0/1015531035 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08015070 con 0x55e55345feb0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.916+0000 7efe03fff640 1 -- 172.21.15.94:0/1015531035 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe0801a910 con 0x55e55345feb0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.916+0000 7efe03fff640 1 -- 172.21.15.94:0/1015531035 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe080118d0 con 0x55e55345feb0 2024-09-19T04:24:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe21f03740 1 -- 172.21.15.94:0/1015531035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 msgr2=0x55e5534622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe21f03740 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e5534622a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efe0800bc80 tx=0x7efe0800f730 comp rx=0 tx=0).stop 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efe040a3430 0x7efe040a58f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe1002ac90 tx=0x7efe1000b040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe21f03740 1 -- 172.21.15.94:0/1015531035 shutdown_connections 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe21f03740 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e5534622a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe21f03740 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e55345f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe21f03740 1 --2- 172.21.15.94:0/1015531035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55344c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe21f03740 1 -- 172.21.15.94:0/1015531035 >> 172.21.15.94:0/1015531035 conn(0x55e553448970 msgr2=0x55e553448d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe21f03740 1 -- 172.21.15.94:0/1015531035 shutdown_connections 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.917+0000 7efe21f03740 1 -- 172.21.15.94:0/1015531035 wait complete. 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.918+0000 7efe21f03740 1 -- start start 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.918+0000 7efe21f03740 1 Processor -- start 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe21f03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344c4a0 0x55e55345f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe21f03740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344cdc0 0x55e55345cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe21f03740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e55345d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe21f03740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e553462a20 con 0x55e55344cdc0 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe21f03740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e55336d050 con 0x55e55345feb0 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe21f03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e553462ba0 con 0x55e55344c4a0 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe21f03740 10 client.0 _ll_register_callbacks cb 0x55e5533ad160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe1e0c7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e55345d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe1e0c7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e55345d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:43094/0 (socket says 172.21.15.94:43094) 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.919+0000 7efe1e0c7640 1 -- 172.21.15.94:0/1215799448 learned_addr learned my addr 172.21.15.94:0/1215799448 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe1e0c7640 1 -- 172.21.15.94:0/1215799448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344c4a0 msgr2=0x55e55345f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344c4a0 0x55e55345f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe1e0c7640 1 -- 172.21.15.94:0/1215799448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344cdc0 msgr2=0x55e55345cb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:24:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344cdc0 0x55e55345cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe1e0c7640 1 -- 172.21.15.94:0/1215799448 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5531233c0 con 0x55e55345feb0 2024-09-19T04:24:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e55345d120 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efe10009c10 tx=0x7efe1002a770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700be4277931e518 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe017fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:24:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe017fa640 1 -- 172.21.15.94:0/1215799448 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10036040 con 0x55e55345feb0 2024-09-19T04:24:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe017fa640 1 -- 172.21.15.94:0/1215799448 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7efe10004450 con 0x55e55345feb0 2024-09-19T04:24:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe21f03740 1 -- 172.21.15.94:0/1215799448 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe0801a910 con 0x55e55345feb0 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe017fa640 1 -- 172.21.15.94:0/1215799448 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efe10059050 con 0x55e55345feb0 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe21f03740 10 client.26090 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.920+0000 7efe21f03740 1 -- 172.21.15.94:0/1215799448 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e55346e4c0 con 0x55e55345feb0 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.921+0000 7efe017fa640 1 -- 172.21.15.94:0/1215799448 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe10050070 con 0x55e55345feb0 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.921+0000 7efe017fa640 1 -- 172.21.15.94:0/1215799448 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7efe1003fcb0 con 0x55e55345feb0 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.921+0000 7efe21f03740 10 client.26090 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.921+0000 7efe21f03740 10 client.26090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.921+0000 7efe21f03740 20 client.26090 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.921+0000 7efe21f03740 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55e55346fa90 0x55e553471f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.921+0000 7efe21f03740 4 client.26090 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.921+0000 7efe21f03740 1 -- 172.21.15.94:0/1215799448 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efe0801a910 con 0x55e55346fa90 2024-09-19T04:24:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.921+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55e55346fa90 0x55e553471f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.922+0000 7efe017fa640 10 client.26090 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.923+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55e55346fa90 0x55e553471f30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c30054bd8992330 server_cookie=11af19ab36c04038 in_seq=0 out_seq=0 2024-09-19T04:24:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.923+0000 7efe017fa640 1 -- 172.21.15.94:0/1215799448 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7efe0801a910 con 0x55e55346fa90 2024-09-19T04:24:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.923+0000 7efe017fa640 10 client.26090 handle_command_reply: tid=0 2024-09-19T04:24:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.923+0000 7efe21f03740 1 client.26090 shutdown 2024-09-19T04:24:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe21f03740 1 -- 172.21.15.94:0/1215799448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 msgr2=0x55e55345d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe21f03740 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e55345d120 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efe10009c10 tx=0x7efe1002a770 comp rx=0 tx=0).stop 2024-09-19T04:24:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe21f03740 1 -- 172.21.15.94:0/1215799448 shutdown_connections 2024-09-19T04:24:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe21f03740 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55e55346fa90 0x55e553471f30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe017fa640 0 client.26090 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe21f03740 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e55345d120 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe21f03740 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344cdc0 0x55e55345cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe21f03740 1 --2- 172.21.15.94:0/1215799448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344c4a0 0x55e55345f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe21f03740 1 -- 172.21.15.94:0/1215799448 >> 172.21.15.94:0/1215799448 conn(0x55e553448970 msgr2=0x55e55344e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.924+0000 7efe21f03740 1 -- 172.21.15.94:0/1215799448 shutdown_connections 2024-09-19T04:24:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.925+0000 7efe21f03740 1 -- 172.21.15.94:0/1215799448 wait complete. 2024-09-19T04:24:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.925+0000 7efe21f03740 20 client.26090 trim_cache size 0 max 16384 2024-09-19T04:24:20.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.956+0000 7efe21f03740 1 Processor -- start 2024-09-19T04:24:20.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe21f03740 1 -- start start 2024-09-19T04:24:20.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe21f03740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55346bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe21f03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e553477840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe21f03740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e553477d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe21f03740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe181ade60 con 0x55e55344c4a0 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe21f03740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe181ab700 con 0x55e55345feb0 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe21f03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe181a95b0 con 0x55e55344cdc0 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe1d8c6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55346bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe1d8c6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55346bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44014/0 (socket says 172.21.15.94:44014) 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe1d8c6640 1 -- 172.21.15.94:0/3930186387 learned_addr learned my addr 172.21.15.94:0/3930186387 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe1d0c5640 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e553477840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.957+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e553477d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.958+0000 7efe1d8c6640 1 -- 172.21.15.94:0/3930186387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 msgr2=0x55e553477840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.958+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e553477840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.958+0000 7efe1d8c6640 1 -- 172.21.15.94:0/3930186387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 msgr2=0x55e553477d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.958+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e553477d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.958+0000 7efe1d8c6640 1 -- 172.21.15.94:0/3930186387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55344cbf0 con 0x55e55344c4a0 2024-09-19T04:24:20.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.958+0000 7efe1d0c5640 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e553477840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.958+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55346bcf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efe140987a0 tx=0x7efe140d25b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7904aa2e5e9c599 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.958+0000 7efe037fe640 1 -- 172.21.15.94:0/3930186387 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1409bca0 con 0x55e55344c4a0 2024-09-19T04:24:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.959+0000 7efe037fe640 1 -- 172.21.15.94:0/3930186387 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe0801a910 con 0x55e55344c4a0 2024-09-19T04:24:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.959+0000 7efe037fe640 1 -- 172.21.15.94:0/3930186387 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe140a4fb0 con 0x55e55344c4a0 2024-09-19T04:24:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.959+0000 7efe21f03740 1 -- 172.21.15.94:0/3930186387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 msgr2=0x55e55346bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.959+0000 7efe21f03740 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55346bcf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efe140987a0 tx=0x7efe140d25b0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.960+0000 7efe21f03740 1 -- 172.21.15.94:0/3930186387 shutdown_connections 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.960+0000 7efe21f03740 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55345feb0 0x55e553477d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.960+0000 7efe21f03740 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e553477840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.960+0000 7efe21f03740 1 --2- 172.21.15.94:0/3930186387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55344c4a0 0x55e55346bcf0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.960+0000 7efe21f03740 1 -- 172.21.15.94:0/3930186387 >> 172.21.15.94:0/3930186387 conn(0x55e553448970 msgr2=0x55e55345b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.960+0000 7efe21f03740 1 -- 172.21.15.94:0/3930186387 shutdown_connections 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.960+0000 7efe21f03740 1 -- 172.21.15.94:0/3930186387 wait complete. 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.960+0000 7efe21f03740 1 -- start start 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.960+0000 7efe21f03740 1 Processor -- start 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe21f03740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55344c4a0 0x55e553467e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe21f03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e5534683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe21f03740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55345feb0 0x55e553464e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe21f03740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe181a9c00 con 0x55e55345feb0 2024-09-19T04:24:20.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe21f03740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe181ab030 con 0x55e55344c4a0 2024-09-19T04:24:20.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe21f03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe181aa290 con 0x55e55344cdc0 2024-09-19T04:24:20.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe21f03740 10 client.0 _ll_register_callbacks cb 0x7efe1002af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:20.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe1e0c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55345feb0 0x55e553464e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe1e0c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55345feb0 0x55e553464e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44028/0 (socket says 172.21.15.94:44028) 2024-09-19T04:24:20.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.961+0000 7efe1e0c7640 1 -- 172.21.15.94:0/1987272824 learned_addr learned my addr 172.21.15.94:0/1987272824 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:20.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55344c4a0 0x55e553467e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1d0c5640 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e5534683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1e0c7640 1 -- 172.21.15.94:0/1987272824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 msgr2=0x55e5534683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e5534683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1e0c7640 1 -- 172.21.15.94:0/1987272824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55344c4a0 msgr2=0x55e553467e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55344c4a0 0x55e553467e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1e0c7640 1 -- 172.21.15.94:0/1987272824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e553468a10 con 0x55e55345feb0 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1d0c5640 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e5534683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55344c4a0 0x55e553467e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe1e0c7640 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55345feb0 0x55e553464e30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe1004b720 tx=0x7efe1004ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e90cc904e9d648 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.962+0000 7efe017fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.963+0000 7efe017fa640 1 -- 172.21.15.94:0/1987272824 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10062060 con 0x55e55345feb0 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.963+0000 7efe017fa640 1 -- 172.21.15.94:0/1987272824 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7efe10004660 con 0x55e55345feb0 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.963+0000 7efe21f03740 1 -- 172.21.15.94:0/1987272824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe0801a910 con 0x55e55345feb0 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.963+0000 7efe017fa640 1 -- 172.21.15.94:0/1987272824 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efe10004860 con 0x55e55345feb0 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.963+0000 7efe21f03740 10 client.16497 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.963+0000 7efe21f03740 1 -- 172.21.15.94:0/1987272824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e5534dada0 con 0x55e55345feb0 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.963+0000 7efe017fa640 1 -- 172.21.15.94:0/1987272824 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe0801a910 con 0x55e55345feb0 2024-09-19T04:24:20.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.964+0000 7efe017fa640 1 -- 172.21.15.94:0/1987272824 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7efe100554e0 con 0x55e55345feb0 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.964+0000 7efe21f03740 10 client.16497 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.964+0000 7efe21f03740 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.964+0000 7efe21f03740 20 client.16497 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.964+0000 7efe21f03740 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55e5534db470 0x55e5534dd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.964+0000 7efe21f03740 4 client.16497 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.964+0000 7efe21f03740 1 -- 172.21.15.94:0/1987272824 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5533bf850 con 0x55e5534db470 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.964+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55e5534db470 0x55e5534dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.964+0000 7efe017fa640 10 client.16497 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe1d8c6640 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55e5534db470 0x55e5534dd890 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b467ed4b547b0091 server_cookie=19a12fd7641376bc in_seq=0 out_seq=0 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe017fa640 1 -- 172.21.15.94:0/1987272824 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e5533bf850 con 0x55e5534db470 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe017fa640 10 client.16497 handle_command_reply: tid=0 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe21f03740 1 client.16497 shutdown 2024-09-19T04:24:20.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe21f03740 1 -- 172.21.15.94:0/1987272824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55345feb0 msgr2=0x55e553464e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe21f03740 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55345feb0 0x55e553464e30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe1004b720 tx=0x7efe1004ba50 comp rx=0 tx=0).stop 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe21f03740 1 -- 172.21.15.94:0/1987272824 shutdown_connections 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe21f03740 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55e5534db470 0x55e5534dd890 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe21f03740 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e55345feb0 0x55e553464e30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe017fa640 0 client.16497 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe21f03740 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e55344cdc0 0x55e5534683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe21f03740 1 --2- 172.21.15.94:0/1987272824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e55344c4a0 0x55e553467e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.965+0000 7efe21f03740 1 -- 172.21.15.94:0/1987272824 >> 172.21.15.94:0/1987272824 conn(0x55e553448970 msgr2=0x55e55345ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.966+0000 7efe21f03740 1 -- 172.21.15.94:0/1987272824 shutdown_connections 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.966+0000 7efe21f03740 1 -- 172.21.15.94:0/1987272824 wait complete. 2024-09-19T04:24:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.966+0000 7efe21f03740 20 client.16497 trim_cache size 0 max 16384 2024-09-19T04:24:20.972 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:24:20.972 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:24:20.972 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:24:20.972 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efe040a3430 msgr2=0x7efe040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efe040a3430 0x7efe040a58f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe1002ac90 tx=0x7efe1000b040 comp rx=0 tx=0).stop 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe18100e20 msgr2=0x7efe1819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe18100e20 0x7efe1819a3a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efe1400e9c0 tx=0x7efe1400ee80 comp rx=0 tx=0).stop 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 shutdown_connections 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efe040a3430 0x7efe040a58f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe18102ef0 0x7efe1819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18102570 0x7efe1819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 --2- 172.21.15.94:0/3189633409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe18100e20 0x7efe1819a3a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.967+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 >> 172.21.15.94:0/3189633409 conn(0x7efe180fc7d0 msgr2=0x7efe18105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.968+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 shutdown_connections 2024-09-19T04:24:20.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:20.968+0000 7efe1fb2a640 1 -- 172.21.15.94:0/3189633409 wait complete. 2024-09-19T04:24:21.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:20 smithi112 ceph-mon[35051]: pgmap v288: 97 pgs: 97 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:20 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:24:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:20 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:24:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:20 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:24:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:20 smithi104 ceph-mon[34145]: pgmap v288: 97 pgs: 97 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:20 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:24:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:20 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:24:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:20 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:24:21.934 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:24:21.934 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:24:22.409 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:22.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:22 smithi094 ceph-mon[30972]: pgmap v289: 97 pgs: 97 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:23.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.004+0000 7fbd0581b640 1 -- 172.21.15.94:0/3019261452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd000ffcd0 msgr2=0x7fbd001000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:23.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.004+0000 7fbd0581b640 1 --2- 172.21.15.94:0/3019261452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd000ffcd0 0x7fbd001000b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009cd0 tx=0x7fbcf002a9a0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.004+0000 7fbd0581b640 1 -- 172.21.15.94:0/3019261452 shutdown_connections 2024-09-19T04:24:23.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.004+0000 7fbd0581b640 1 --2- 172.21.15.94:0/3019261452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd00109f90 0x7fbd0010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.004+0000 7fbd0581b640 1 --2- 172.21.15.94:0/3019261452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd001005f0 0x7fbd00109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.004+0000 7fbd0581b640 1 --2- 172.21.15.94:0/3019261452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd000ffcd0 0x7fbd001000b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.004+0000 7fbd0581b640 1 -- 172.21.15.94:0/3019261452 >> 172.21.15.94:0/3019261452 conn(0x7fbd000fb920 msgr2=0x7fbd000fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:23.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.005+0000 7fbd0581b640 1 -- 172.21.15.94:0/3019261452 shutdown_connections 2024-09-19T04:24:23.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.005+0000 7fbd0581b640 1 -- 172.21.15.94:0/3019261452 wait complete. 2024-09-19T04:24:23.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0581b640 1 Processor -- start 2024-09-19T04:24:23.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0581b640 1 -- start start 2024-09-19T04:24:23.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0581b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd000ffcd0 0x7fbd001a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:23.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0581b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd001005f0 0x7fbd001a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0581b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd00109f90 0x7fbd0019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0581b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd00118310 con 0x7fbd000ffcd0 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0581b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbd00118190 con 0x7fbd00109f90 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0581b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd00118490 con 0x7fbd001005f0 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0501a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd00109f90 0x7fbd0019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbcfffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd001005f0 0x7fbd001a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0501a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd00109f90 0x7fbd0019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48238/0 (socket says 172.21.15.94:48238) 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbcfffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd001005f0 0x7fbd001a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:60086/0 (socket says 172.21.15.94:60086) 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd0501a640 1 -- 172.21.15.94:0/2495891359 learned_addr learned my addr 172.21.15.94:0/2495891359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.006+0000 7fbd04819640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd000ffcd0 0x7fbd001a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbd0501a640 1 -- 172.21.15.94:0/2495891359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd001005f0 msgr2=0x7fbd001a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbd0501a640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd001005f0 0x7fbd001a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbd0501a640 1 -- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd000ffcd0 msgr2=0x7fbd001a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:23.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbd0501a640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd000ffcd0 0x7fbd001a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbd0501a640 1 -- 172.21.15.94:0/2495891359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0019ed50 con 0x7fbd00109f90 2024-09-19T04:24:23.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbd0501a640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd00109f90 0x7fbd0019e620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009fd0 tx=0x7fbcf800b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:23.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbcfdffb640 1 -- 172.21.15.94:0/2495891359 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8011830 con 0x7fbd00109f90 2024-09-19T04:24:23.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbcfdffb640 1 -- 172.21.15.94:0/2495891359 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcf8016070 con 0x7fbd00109f90 2024-09-19T04:24:23.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbcfdffb640 1 -- 172.21.15.94:0/2495891359 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8010e50 con 0x7fbd00109f90 2024-09-19T04:24:23.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.007+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0019f040 con 0x7fbd00109f90 2024-09-19T04:24:23.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.008+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd001ab330 con 0x7fbd00109f90 2024-09-19T04:24:23.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.010+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd00109a50 con 0x7fbd00109f90 2024-09-19T04:24:23.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.011+0000 7fbcfdffb640 1 -- 172.21.15.94:0/2495891359 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fbcf801b4a0 con 0x7fbd00109f90 2024-09-19T04:24:23.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.011+0000 7fbcfdffb640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbcec0a3620 0x7fbcec0a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:23.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.012+0000 7fbcfdffb640 1 -- 172.21.15.94:0/2495891359 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fbcf80cf2a0 con 0x7fbd00109f90 2024-09-19T04:24:23.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.012+0000 7fbd04819640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbcec0a3620 0x7fbcec0a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:23.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.013+0000 7fbd04819640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbcec0a3620 0x7fbcec0a5ae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0005e30 tx=0x7fbcf0004a40 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:23.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.016+0000 7fbcfdffb640 1 -- 172.21.15.94:0/2495891359 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbcf809bcf0 con 0x7fbd00109f90 2024-09-19T04:24:23.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:22 smithi112 ceph-mon[35051]: pgmap v289: 97 pgs: 97 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:23.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:22 smithi104 ceph-mon[34145]: pgmap v289: 97 pgs: 97 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:23.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.397+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd000fd410 con 0x7fbd00109f90 2024-09-19T04:24:23.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.398+0000 7fbcfdffb640 1 -- 172.21.15.94:0/2495891359 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11919 (secure 0 0 0) 0x7fbcf809b5f0 con 0x7fbd00109f90 2024-09-19T04:24:23.400 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:24:23.400 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":43,"btime":"2024-09-19T04:23:43:111703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:23:42.755203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24466,24467,24454]},"id":1}]} 2024-09-19T04:24:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 43 2024-09-19T04:24:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.401+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbcec0a3620 msgr2=0x7fbcec0a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.401+0000 7fbd0581b640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbcec0a3620 0x7fbcec0a5ae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0005e30 tx=0x7fbcf0004a40 comp rx=0 tx=0).stop 2024-09-19T04:24:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.401+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd00109f90 msgr2=0x7fbd0019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.401+0000 7fbd0581b640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd00109f90 0x7fbd0019e620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009fd0 tx=0x7fbcf800b8e0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.402+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 shutdown_connections 2024-09-19T04:24:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.402+0000 7fbd0581b640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbcec0a3620 0x7fbcec0a5ae0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.402+0000 7fbd0581b640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd00109f90 0x7fbd0019e620 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.402+0000 7fbd0581b640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd001005f0 0x7fbd001a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.402+0000 7fbd0581b640 1 --2- 172.21.15.94:0/2495891359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd000ffcd0 0x7fbd001a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.402+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 >> 172.21.15.94:0/2495891359 conn(0x7fbd000fb920 msgr2=0x7fbd00114510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.402+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 shutdown_connections 2024-09-19T04:24:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:23.402+0000 7fbd0581b640 1 -- 172.21.15.94:0/2495891359 wait complete. 2024-09-19T04:24:23.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:23 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2495891359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:24:24.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:23 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2495891359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:24:24.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:23 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2495891359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:24:24.206 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:24:24.635 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:24.945 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:24 smithi094 ceph-mon[30972]: pgmap v290: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:25.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:24 smithi112 ceph-mon[35051]: pgmap v290: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:25.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:24 smithi104 ceph-mon[34145]: pgmap v290: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:25.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.232+0000 7f5150ce7640 1 -- 172.21.15.94:0/4042333391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f514c1011e0 msgr2=0x7f514c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.232+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4042333391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f514c1011e0 0x7f514c105980 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f513c009ca0 tx=0x7f513c02a8d0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.232+0000 7f5150ce7640 1 -- 172.21.15.94:0/4042333391 shutdown_connections 2024-09-19T04:24:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.232+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4042333391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f514c105ec0 0x7f514c106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.232+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4042333391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f514c1011e0 0x7f514c105980 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.232+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4042333391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f514c1008c0 0x7f514c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.233+0000 7f5150ce7640 1 -- 172.21.15.94:0/4042333391 >> 172.21.15.94:0/4042333391 conn(0x7f514c0fc7d0 msgr2=0x7f514c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.233+0000 7f5150ce7640 1 -- 172.21.15.94:0/4042333391 shutdown_connections 2024-09-19T04:24:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.233+0000 7f5150ce7640 1 -- 172.21.15.94:0/4042333391 wait complete. 2024-09-19T04:24:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.234+0000 7f5150ce7640 1 Processor -- start 2024-09-19T04:24:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.234+0000 7f5150ce7640 1 -- start start 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.234+0000 7f5150ce7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f514c1008c0 0x7f514c1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f5150ce7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f514c1011e0 0x7f514c1a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f5150ce7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f514c105ec0 0x7f514c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f514c1011e0 0x7f514c1a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f5150ce7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f514c1185b0 con 0x7f514c1008c0 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f5150ce7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f514c118430 con 0x7f514c1011e0 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f5150ce7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f514c118730 con 0x7f514c105ec0 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f514c1011e0 0x7f514c1a4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48246/0 (socket says 172.21.15.94:48246) 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f5149d9b640 1 -- 172.21.15.94:0/4278252134 learned_addr learned my addr 172.21.15.94:0/4278252134 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f514a59c640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f514c1008c0 0x7f514c1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.235+0000 7f514ad9d640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f514c105ec0 0x7f514c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.236+0000 7f514ad9d640 1 -- 172.21.15.94:0/4278252134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f514c1011e0 msgr2=0x7f514c1a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.236+0000 7f514ad9d640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f514c1011e0 0x7f514c1a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.236+0000 7f514ad9d640 1 -- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f514c1008c0 msgr2=0x7f514c1a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.236+0000 7f514ad9d640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f514c1008c0 0x7f514c1a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.236+0000 7f514ad9d640 1 -- 172.21.15.94:0/4278252134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c19f0b0 con 0x7f514c105ec0 2024-09-19T04:24:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.236+0000 7f514ad9d640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f514c105ec0 0x7f514c19e980 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f514000e970 tx=0x7f514000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.237+0000 7f513b7fe640 1 -- 172.21.15.94:0/4278252134 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514000ccc0 con 0x7f514c105ec0 2024-09-19T04:24:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.237+0000 7f513b7fe640 1 -- 172.21.15.94:0/4278252134 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5140005020 con 0x7f514c105ec0 2024-09-19T04:24:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.237+0000 7f5150ce7640 1 -- 172.21.15.94:0/4278252134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c19f3a0 con 0x7f514c105ec0 2024-09-19T04:24:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.237+0000 7f5150ce7640 1 -- 172.21.15.94:0/4278252134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c1ab690 con 0x7f514c105ec0 2024-09-19T04:24:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.238+0000 7f513b7fe640 1 -- 172.21.15.94:0/4278252134 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140004a00 con 0x7f514c105ec0 2024-09-19T04:24:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.241+0000 7f513b7fe640 1 -- 172.21.15.94:0/4278252134 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5140004ba0 con 0x7f514c105ec0 2024-09-19T04:24:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.242+0000 7f513b7fe640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51340a35c0 0x7f51340a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.242+0000 7f513b7fe640 1 -- 172.21.15.94:0/4278252134 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f51400cfcb0 con 0x7f514c105ec0 2024-09-19T04:24:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.243+0000 7f51530c0740 1 Processor -- start 2024-09-19T04:24:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.243+0000 7f51530c0740 1 -- start start 2024-09-19T04:24:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.243+0000 7f51530c0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c186fb40 0x5642c186e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.243+0000 7f51530c0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c186ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.243+0000 7f514a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c186fb40 0x5642c186e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.243+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c186ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.243+0000 7f514a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c186fb40 0x5642c186e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48250/0 (socket says 172.21.15.94:48250) 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.243+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c186ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41226/0 (socket says 172.21.15.94:41226) 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f51530c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1879370 0x5642c187b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f514a59c640 1 -- 172.21.15.94:0/2898174552 learned_addr learned my addr 172.21.15.94:0/2898174552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f51530c0740 1 -- 172.21.15.94:0/2898174552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5642c17d42f0 con 0x5642c186e5d0 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f51530c0740 1 -- 172.21.15.94:0/2898174552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5642c177bcf0 con 0x5642c186fb40 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f51530c0740 1 -- 172.21.15.94:0/2898174552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5642c17d02b0 con 0x5642c1879370 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f514ad9d640 1 --2- 172.21.15.94:0/2898174552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1879370 0x5642c187b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f514a59c640 1 -- 172.21.15.94:0/2898174552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1879370 msgr2=0x5642c187b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f514a59c640 1 --2- 172.21.15.94:0/2898174552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1879370 0x5642c187b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f514a59c640 1 -- 172.21.15.94:0/2898174552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 msgr2=0x5642c186ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f514a59c640 1 --2- 172.21.15.94:0/2898174552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c186ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f514a59c640 1 -- 172.21.15.94:0/2898174552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c17603a0 con 0x5642c186fb40 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.244+0000 7f514a59c640 1 --2- 172.21.15.94:0/2898174552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c186fb40 0x5642c186e090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f514400a940 tx=0x7f514400ae00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d360942587e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.245+0000 7f5138ff9640 1 -- 172.21.15.94:0/2898174552 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514400fa80 con 0x5642c186fb40 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.245+0000 7f5138ff9640 1 -- 172.21.15.94:0/2898174552 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5144018070 con 0x5642c186fb40 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.245+0000 7f5138ff9640 1 -- 172.21.15.94:0/2898174552 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144012360 con 0x5642c186fb40 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.246+0000 7f51530c0740 1 -- 172.21.15.94:0/2898174552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c186fb40 msgr2=0x5642c186e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.246+0000 7f51530c0740 1 --2- 172.21.15.94:0/2898174552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c186fb40 0x5642c186e090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f514400a940 tx=0x7f514400ae00 comp rx=0 tx=0).stop 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.246+0000 7f51530c0740 1 -- 172.21.15.94:0/2898174552 shutdown_connections 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.246+0000 7f51530c0740 1 --2- 172.21.15.94:0/2898174552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1879370 0x5642c187b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.246+0000 7f51530c0740 1 --2- 172.21.15.94:0/2898174552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c186ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.246+0000 7f51530c0740 1 --2- 172.21.15.94:0/2898174552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c186fb40 0x5642c186e090 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f514400a940 tx=0x7f514400ae00 comp rx=0 tx=0).stop 2024-09-19T04:24:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.246+0000 7f51530c0740 1 -- 172.21.15.94:0/2898174552 >> 172.21.15.94:0/2898174552 conn(0x5642c1837310 msgr2=0x5642c186ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.248+0000 7f514a59c640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51340a35c0 0x7f51340a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.249+0000 7f514a59c640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51340a35c0 0x7f51340a5a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5642c186b860 tx=0x7f514400af90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.250+0000 7f51530c0740 1 -- 172.21.15.94:0/2898174552 shutdown_connections 2024-09-19T04:24:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.250+0000 7f51530c0740 1 -- 172.21.15.94:0/2898174552 wait complete. 2024-09-19T04:24:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.250+0000 7f51530c0740 1 -- start start 2024-09-19T04:24:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.250+0000 7f51530c0740 1 Processor -- start 2024-09-19T04:24:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.250+0000 7f51530c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c1876170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f51530c0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18766b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f51530c0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 0x5642c18855a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f51530c0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5642c187c0c0 con 0x5642c1879370 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f51530c0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5642c178f060 con 0x5642c1876bf0 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f51530c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5642c187c240 con 0x5642c186e5d0 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18766b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41240/0 (socket says 172.21.15.94:41240) 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f5149d9b640 1 -- 172.21.15.94:0/1603851516 learned_addr learned my addr 172.21.15.94:0/1603851516 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f51530c0740 10 client.0 _ll_register_callbacks cb 0x5642c14e28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f514ad9d640 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 0x5642c18855a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f5149d9b640 1 -- 172.21.15.94:0/1603851516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 msgr2=0x5642c1876170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f514a59c640 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c1876170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f5149d9b640 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c1876170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f5149d9b640 1 -- 172.21.15.94:0/1603851516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 msgr2=0x5642c18855a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f5149d9b640 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 0x5642c18855a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f5149d9b640 1 -- 172.21.15.94:0/1603851516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c15453c0 con 0x5642c1879370 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f514ad9d640 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 0x5642c18855a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.251+0000 7f514a59c640 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c1876170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.252+0000 7f5149d9b640 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18766b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f513c006010 tx=0x7f513c031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292e54a09cb3456f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.252+0000 7f5131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.252+0000 7f5131ffb640 1 -- 172.21.15.94:0/1603851516 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c003970 con 0x5642c1879370 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.252+0000 7f5131ffb640 1 -- 172.21.15.94:0/1603851516 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f513c003b10 con 0x5642c1879370 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.252+0000 7f51530c0740 1 -- 172.21.15.94:0/1603851516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5144018070 con 0x5642c1879370 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.252+0000 7f5131ffb640 1 -- 172.21.15.94:0/1603851516 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f513c003d10 con 0x5642c1879370 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.252+0000 7f51530c0740 10 client.16512 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.252+0000 7f51530c0740 1 -- 172.21.15.94:0/1603851516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5642c18880d0 con 0x5642c1879370 2024-09-19T04:24:25.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.253+0000 7f5131ffb640 1 -- 172.21.15.94:0/1603851516 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f513c036040 con 0x5642c1879370 2024-09-19T04:24:25.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.253+0000 7f5131ffb640 1 -- 172.21.15.94:0/1603851516 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f513c0305d0 con 0x5642c1879370 2024-09-19T04:24:25.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.253+0000 7f51530c0740 10 client.16512 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:25.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.253+0000 7f51530c0740 10 client.16512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:24:25.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.253+0000 7f51530c0740 20 client.16512 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:25.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.253+0000 7f51530c0740 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5642c1889590 0x5642c188ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.253+0000 7f51530c0740 4 client.16512 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.253+0000 7f51530c0740 1 -- 172.21.15.94:0/1603851516 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5144018070 con 0x5642c1889590 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.253+0000 7f514a59c640 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5642c1889590 0x5642c188ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.254+0000 7f5131ffb640 10 client.16512 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.255+0000 7f514a59c640 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5642c1889590 0x5642c188ba30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d26e3e70bdd54801 server_cookie=27f1e467d3cda8b0 in_seq=0 out_seq=0 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f5131ffb640 1 -- 172.21.15.94:0/1603851516 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5144018070 con 0x5642c1889590 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f5131ffb640 10 client.16512 handle_command_reply: tid=0 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 client.16512 shutdown 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 -- 172.21.15.94:0/1603851516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 msgr2=0x5642c18766b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18766b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f513c006010 tx=0x7f513c031c70 comp rx=0 tx=0).stop 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 -- 172.21.15.94:0/1603851516 shutdown_connections 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5642c1889590 0x5642c188ba30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 0x5642c18855a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18766b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 --2- 172.21.15.94:0/1603851516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c1876170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 -- 172.21.15.94:0/1603851516 >> 172.21.15.94:0/1603851516 conn(0x5642c1837310 msgr2=0x5642c1870930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f5131ffb640 0 client.16512 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:24:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.256+0000 7f51530c0740 1 -- 172.21.15.94:0/1603851516 shutdown_connections 2024-09-19T04:24:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.257+0000 7f51530c0740 1 -- 172.21.15.94:0/1603851516 wait complete. 2024-09-19T04:24:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.257+0000 7f51530c0740 20 client.16512 trim_cache size 0 max 16384 2024-09-19T04:24:25.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f51530c0740 1 Processor -- start 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f51530c0740 1 -- start start 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f51530c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c186ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f51530c0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 0x5642c1885390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f51530c0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f51530c0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f512c011220 con 0x5642c1879370 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f51530c0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f512c00eac0 con 0x5642c1876bf0 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f51530c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f512c00c930 con 0x5642c186e5d0 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f514a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c186ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f514a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c186ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:60124/0 (socket says 172.21.15.94:60124) 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.282+0000 7f514ad9d640 1 --2- 172.21.15.94:0/452979979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.283+0000 7f5149d9b640 1 --2- 172.21.15.94:0/452979979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 0x5642c1885390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.283+0000 7f514a59c640 1 -- 172.21.15.94:0/452979979 learned_addr learned my addr 172.21.15.94:0/452979979 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.283+0000 7f514ad9d640 1 -- 172.21.15.94:0/452979979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 msgr2=0x5642c186ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.283+0000 7f514ad9d640 1 --2- 172.21.15.94:0/452979979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c186ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.283+0000 7f514ad9d640 1 -- 172.21.15.94:0/452979979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 msgr2=0x5642c1885390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.283+0000 7f514ad9d640 1 --2- 172.21.15.94:0/452979979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 0x5642c1885390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.283+0000 7f514ad9d640 1 -- 172.21.15.94:0/452979979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c1869310 con 0x5642c1879370 2024-09-19T04:24:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.283+0000 7f514ad9d640 1 --2- 172.21.15.94:0/452979979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18858d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f51400cd360 tx=0x7f51400cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23185ee69d9c096d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.284+0000 7f5133fff640 1 -- 172.21.15.94:0/452979979 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514001c070 con 0x5642c1879370 2024-09-19T04:24:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.284+0000 7f5133fff640 1 -- 172.21.15.94:0/452979979 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51400a4b60 con 0x5642c1879370 2024-09-19T04:24:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.284+0000 7f5133fff640 1 -- 172.21.15.94:0/452979979 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514009b5a0 con 0x5642c1879370 2024-09-19T04:24:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.285+0000 7f51530c0740 1 -- 172.21.15.94:0/452979979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 msgr2=0x5642c18858d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.285+0000 7f51530c0740 1 --2- 172.21.15.94:0/452979979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18858d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f51400cd360 tx=0x7f51400cf040 comp rx=0 tx=0).stop 2024-09-19T04:24:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.285+0000 7f51530c0740 1 -- 172.21.15.94:0/452979979 shutdown_connections 2024-09-19T04:24:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.285+0000 7f51530c0740 1 --2- 172.21.15.94:0/452979979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c1879370 0x5642c18858d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.285+0000 7f51530c0740 1 --2- 172.21.15.94:0/452979979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1876bf0 0x5642c1885390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.285+0000 7f51530c0740 1 --2- 172.21.15.94:0/452979979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c186e5d0 0x5642c186ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.285+0000 7f51530c0740 1 -- 172.21.15.94:0/452979979 >> 172.21.15.94:0/452979979 conn(0x5642c1837310 msgr2=0x5642c186a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.286+0000 7f51530c0740 1 -- 172.21.15.94:0/452979979 shutdown_connections 2024-09-19T04:24:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.286+0000 7f51530c0740 1 -- 172.21.15.94:0/452979979 wait complete. 2024-09-19T04:24:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.286+0000 7f51530c0740 1 -- start start 2024-09-19T04:24:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.286+0000 7f51530c0740 1 Processor -- start 2024-09-19T04:24:25.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f51530c0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c18816e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f51530c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1876bf0 0x5642c1881c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f51530c0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1879370 0x5642c1873430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f51530c0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f512c00cf80 con 0x5642c186e5d0 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f51530c0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f512c00e3f0 con 0x5642c1879370 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f51530c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f512c00d650 con 0x5642c1876bf0 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1876bf0 0x5642c1881c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f51530c0740 10 client.0 _ll_register_callbacks cb 0x7f513c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f514ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1879370 0x5642c1873430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f514ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1879370 0x5642c1873430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48278/0 (socket says 172.21.15.94:48278) 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f514ad9d640 1 -- 172.21.15.94:0/862686874 learned_addr learned my addr 172.21.15.94:0/862686874 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f514a59c640 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c18816e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.287+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1876bf0 0x5642c1881c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:60140/0 (socket says 172.21.15.94:60140) 2024-09-19T04:24:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f514ad9d640 1 -- 172.21.15.94:0/862686874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1876bf0 msgr2=0x5642c1881c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f514ad9d640 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1876bf0 0x5642c1881c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f514ad9d640 1 -- 172.21.15.94:0/862686874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 msgr2=0x5642c18816e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f514ad9d640 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c18816e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f514ad9d640 1 -- 172.21.15.94:0/862686874 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c1873af0 con 0x5642c1879370 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f514a59c640 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c18816e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f514ad9d640 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1879370 0x5642c1873430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f51400a4630 tx=0x7f5140005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef6cf9447911e307 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f5131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f5131ffb640 1 -- 172.21.15.94:0/862686874 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51400a8020 con 0x5642c1879370 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f5131ffb640 1 -- 172.21.15.94:0/862686874 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f514009b340 con 0x5642c1879370 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f51530c0740 1 -- 172.21.15.94:0/862686874 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51400a4b60 con 0x5642c1879370 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.288+0000 7f5131ffb640 1 -- 172.21.15.94:0/862686874 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f514001c040 con 0x5642c1879370 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f51530c0740 10 client.26120 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f51530c0740 1 -- 172.21.15.94:0/862686874 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5642c187a470 con 0x5642c1879370 2024-09-19T04:24:25.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f5131ffb640 1 -- 172.21.15.94:0/862686874 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51400a4b60 con 0x5642c1879370 2024-09-19T04:24:25.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f5131ffb640 1 -- 172.21.15.94:0/862686874 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f514009adb0 con 0x5642c1879370 2024-09-19T04:24:25.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f51530c0740 10 client.26120 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:25.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f51530c0740 10 client.26120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:24:25.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f51530c0740 20 client.26120 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:25.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f51530c0740 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5642c18f4360 0x5642c18f6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f51530c0740 4 client.26120 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:24:25.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f51530c0740 1 -- 172.21.15.94:0/862686874 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642c17e1860 con 0x5642c18f4360 2024-09-19T04:24:25.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.289+0000 7f514a59c640 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5642c18f4360 0x5642c18f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.291+0000 7f5131ffb640 10 client.26120 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:24:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.291+0000 7f514a59c640 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5642c18f4360 0x5642c18f6750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d428aaab231a408a server_cookie=b6630fa2e9a5b664 in_seq=0 out_seq=0 2024-09-19T04:24:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.291+0000 7f5131ffb640 1 -- 172.21.15.94:0/862686874 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642c17e1860 con 0x5642c18f4360 2024-09-19T04:24:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.291+0000 7f5131ffb640 10 client.26120 handle_command_reply: tid=0 2024-09-19T04:24:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.291+0000 7f51530c0740 1 client.26120 shutdown 2024-09-19T04:24:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f51530c0740 1 -- 172.21.15.94:0/862686874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1879370 msgr2=0x5642c1873430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f51530c0740 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1879370 0x5642c1873430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f51400a4630 tx=0x7f5140005cc0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f51530c0740 1 -- 172.21.15.94:0/862686874 shutdown_connections 2024-09-19T04:24:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f51530c0740 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5642c18f4360 0x5642c18f6750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f51530c0740 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642c1879370 0x5642c1873430 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f51530c0740 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642c1876bf0 0x5642c1881c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f51530c0740 1 --2- 172.21.15.94:0/862686874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642c186e5d0 0x5642c18816e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f51530c0740 1 -- 172.21.15.94:0/862686874 >> 172.21.15.94:0/862686874 conn(0x5642c1837310 msgr2=0x5642c18779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f5131ffb640 0 client.26120 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:24:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.292+0000 7f51530c0740 1 -- 172.21.15.94:0/862686874 shutdown_connections 2024-09-19T04:24:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.293+0000 7f51530c0740 1 -- 172.21.15.94:0/862686874 wait complete. 2024-09-19T04:24:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.293+0000 7f51530c0740 20 client.26120 trim_cache size 0 max 16384 2024-09-19T04:24:25.297 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 -- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51340a35c0 msgr2=0x7f51340a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51340a35c0 0x7f51340a5a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5642c186b860 tx=0x7f514400af90 comp rx=0 tx=0).stop 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 -- 172.21.15.94:0/4278252134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f514c105ec0 msgr2=0x7f514c19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f514c105ec0 0x7f514c19e980 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f514000e970 tx=0x7f514000ee30 comp rx=0 tx=0).stop 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 -- 172.21.15.94:0/4278252134 shutdown_connections 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51340a35c0 0x7f51340a5a80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f514c105ec0 0x7f514c19e980 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f514c1011e0 0x7f514c1a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 --2- 172.21.15.94:0/4278252134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f514c1008c0 0x7f514c1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.294+0000 7f5150ce7640 1 -- 172.21.15.94:0/4278252134 >> 172.21.15.94:0/4278252134 conn(0x7f514c0fc7d0 msgr2=0x7f514c103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.295+0000 7f5150ce7640 1 -- 172.21.15.94:0/4278252134 shutdown_connections 2024-09-19T04:24:25.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:25.295+0000 7f5150ce7640 1 -- 172.21.15.94:0/4278252134 wait complete. 2024-09-19T04:24:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 490.0 s, 1342.0 tps, lat 74.549 ms stddev 28.291 2024-09-19T04:24:26.335 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:24:26.758 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:26.894 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:26 smithi094 ceph-mon[30972]: pgmap v291: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:27.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:26 smithi112 ceph-mon[35051]: pgmap v291: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:27.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:26 smithi104 ceph-mon[34145]: pgmap v291: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-09-19T04:24:27.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.341+0000 7fe7485bd640 1 -- 172.21.15.94:0/1825512532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7401008c0 msgr2=0x7fe740100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.341+0000 7fe7485bd640 1 --2- 172.21.15.94:0/1825512532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7401008c0 0x7fe740100ca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe734009930 tx=0x7fe73402a420 comp rx=0 tx=0).stop 2024-09-19T04:24:27.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.341+0000 7fe7485bd640 1 -- 172.21.15.94:0/1825512532 shutdown_connections 2024-09-19T04:24:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.341+0000 7fe7485bd640 1 --2- 172.21.15.94:0/1825512532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe740105ec0 0x7fe740106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.341+0000 7fe7485bd640 1 --2- 172.21.15.94:0/1825512532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7401011e0 0x7fe740105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.341+0000 7fe7485bd640 1 --2- 172.21.15.94:0/1825512532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7401008c0 0x7fe740100ca0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.341+0000 7fe7485bd640 1 -- 172.21.15.94:0/1825512532 >> 172.21.15.94:0/1825512532 conn(0x7fe7400fc7d0 msgr2=0x7fe7400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.341+0000 7fe7485bd640 1 -- 172.21.15.94:0/1825512532 shutdown_connections 2024-09-19T04:24:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.341+0000 7fe7485bd640 1 -- 172.21.15.94:0/1825512532 wait complete. 2024-09-19T04:24:27.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.342+0000 7fe7485bd640 1 Processor -- start 2024-09-19T04:24:27.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.342+0000 7fe7485bd640 1 -- start start 2024-09-19T04:24:27.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.342+0000 7fe7485bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7401008c0 0x7fe740073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe7485bd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7401011e0 0x7fe74006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe7485bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe740105ec0 0x7fe74006d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe7485bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe740118530 con 0x7fe740105ec0 2024-09-19T04:24:27.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe7485bd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401183b0 con 0x7fe7401011e0 2024-09-19T04:24:27.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe7485bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401186b0 con 0x7fe7401008c0 2024-09-19T04:24:27.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746359640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7401008c0 0x7fe740073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746b5a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe740105ec0 0x7fe74006d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746359640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7401008c0 0x7fe740073a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:60170/0 (socket says 172.21.15.94:60170) 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746b5a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe740105ec0 0x7fe74006d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41278/0 (socket says 172.21.15.94:41278) 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746359640 1 -- 172.21.15.94:0/488448899 learned_addr learned my addr 172.21.15.94:0/488448899 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe745b58640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7401011e0 0x7fe74006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746359640 1 -- 172.21.15.94:0/488448899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7401011e0 msgr2=0x7fe74006cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746359640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7401011e0 0x7fe74006cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746359640 1 -- 172.21.15.94:0/488448899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe740105ec0 msgr2=0x7fe74006d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746359640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe740105ec0 0x7fe74006d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746359640 1 -- 172.21.15.94:0/488448899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74006d680 con 0x7fe7401008c0 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.343+0000 7fe746359640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7401008c0 0x7fe740073a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe734031440 tx=0x7fe734003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.344+0000 7fe7337fe640 1 -- 172.21.15.94:0/488448899 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73403f070 con 0x7fe7401008c0 2024-09-19T04:24:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.344+0000 7fe7337fe640 1 -- 172.21.15.94:0/488448899 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe734031b50 con 0x7fe7401008c0 2024-09-19T04:24:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.344+0000 7fe7485bd640 1 -- 172.21.15.94:0/488448899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74006d910 con 0x7fe7401008c0 2024-09-19T04:24:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.344+0000 7fe7485bd640 1 -- 172.21.15.94:0/488448899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7400717c0 con 0x7fe7401008c0 2024-09-19T04:24:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.344+0000 7fe7337fe640 1 -- 172.21.15.94:0/488448899 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734049060 con 0x7fe7401008c0 2024-09-19T04:24:27.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.348+0000 7fe7337fe640 1 -- 172.21.15.94:0/488448899 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe734042070 con 0x7fe7401008c0 2024-09-19T04:24:27.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.349+0000 7fe7337fe640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe72c0a3430 0x7fe72c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.350+0000 7fe7337fe640 1 -- 172.21.15.94:0/488448899 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fe7340f0340 con 0x7fe7401008c0 2024-09-19T04:24:27.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.350+0000 7fe745b58640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe72c0a3430 0x7fe72c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.352+0000 7fe74a996740 1 Processor -- start 2024-09-19T04:24:27.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.352+0000 7fe74a996740 1 -- start start 2024-09-19T04:24:27.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe74a996740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 0x55d02b8fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe74a996740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8fd5d0 0x55d02b8fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe74a996740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b90a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe74a996740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d02b8632f0 con 0x55d02b8fd5d0 2024-09-19T04:24:27.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe74a996740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d02b80acf0 con 0x55d02b908370 2024-09-19T04:24:27.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe746359640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 0x55d02b8fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe746359640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 0x55d02b8fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:60172/0 (socket says 172.21.15.94:60172) 2024-09-19T04:24:27.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe746359640 1 -- 172.21.15.94:0/3139452780 learned_addr learned my addr 172.21.15.94:0/3139452780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:27.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe746b5a640 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b90a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.353+0000 7fe74a996740 1 -- 172.21.15.94:0/3139452780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d02b85f2b0 con 0x55d02b8feb40 2024-09-19T04:24:27.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe745b58640 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8fd5d0 0x55d02b8fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe745b58640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe72c0a3430 0x7fe72c0a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe74006e2c0 tx=0x7fe73c005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:27.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe746359640 1 -- 172.21.15.94:0/3139452780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 msgr2=0x55d02b90a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:27.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe746359640 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b90a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe746359640 1 -- 172.21.15.94:0/3139452780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8fd5d0 msgr2=0x55d02b8fda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:27.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe746359640 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8fd5d0 0x55d02b8fda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe746359640 1 -- 172.21.15.94:0/3139452780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d02b7ef3a0 con 0x55d02b8feb40 2024-09-19T04:24:27.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe746b5a640 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b90a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:27.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe746359640 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 0x55d02b8fd090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe734036040 tx=0x7fe7340bbf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7532de0ce8569738 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:27.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.354+0000 7fe730ff9640 1 -- 172.21.15.94:0/3139452780 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340c51f0 con 0x55d02b8feb40 2024-09-19T04:24:27.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.355+0000 7fe730ff9640 1 -- 172.21.15.94:0/3139452780 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7340ba930 con 0x55d02b8feb40 2024-09-19T04:24:27.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.355+0000 7fe730ff9640 1 -- 172.21.15.94:0/3139452780 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340edc70 con 0x55d02b8feb40 2024-09-19T04:24:27.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.356+0000 7fe74a996740 1 -- 172.21.15.94:0/3139452780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 msgr2=0x55d02b8fd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.356+0000 7fe74a996740 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 0x55d02b8fd090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe734036040 tx=0x7fe7340bbf90 comp rx=0 tx=0).stop 2024-09-19T04:24:27.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.356+0000 7fe74a996740 1 -- 172.21.15.94:0/3139452780 shutdown_connections 2024-09-19T04:24:27.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.356+0000 7fe74a996740 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b90a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.356+0000 7fe74a996740 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8fd5d0 0x55d02b8fda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.356+0000 7fe74a996740 1 --2- 172.21.15.94:0/3139452780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 0x55d02b8fd090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.356+0000 7fe74a996740 1 -- 172.21.15.94:0/3139452780 >> 172.21.15.94:0/3139452780 conn(0x55d02b8c6310 msgr2=0x55d02b8f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:27.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.356+0000 7fe74a996740 1 -- 172.21.15.94:0/3139452780 shutdown_connections 2024-09-19T04:24:27.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.357+0000 7fe74a996740 1 -- 172.21.15.94:0/3139452780 wait complete. 2024-09-19T04:24:27.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.357+0000 7fe74a996740 1 -- start start 2024-09-19T04:24:27.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.357+0000 7fe74a996740 1 Processor -- start 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.358+0000 7fe74a996740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b8fd5d0 0x55d02b907df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.358+0000 7fe74a996740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 0x55d02b9050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.358+0000 7fe74a996740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b908370 0x55d02b9057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.358+0000 7fe74a996740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d02b90ae30 con 0x55d02b908370 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.358+0000 7fe74a996740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d02b81e060 con 0x55d02b8fd5d0 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.358+0000 7fe74a996740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d02b90afb0 con 0x55d02b8feb40 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.358+0000 7fe74a996740 10 client.0 _ll_register_callbacks cb 0x55d02b5718c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.359+0000 7fe746359640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b8fd5d0 0x55d02b907df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.359+0000 7fe746359640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b8fd5d0 0x55d02b907df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:48330/0 (socket says 172.21.15.94:48330) 2024-09-19T04:24:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.359+0000 7fe746359640 1 -- 172.21.15.94:0/1748826576 learned_addr learned my addr 172.21.15.94:0/1748826576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.359+0000 7fe746359640 1 -- 172.21.15.94:0/1748826576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 msgr2=0x55d02b9050a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.359+0000 7fe746359640 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 0x55d02b9050a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.359+0000 7fe746359640 1 -- 172.21.15.94:0/1748826576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b908370 msgr2=0x55d02b9057e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.359+0000 7fe746359640 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b908370 0x55d02b9057e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.359+0000 7fe746359640 1 -- 172.21.15.94:0/1748826576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d02b5d43c0 con 0x55d02b8fd5d0 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.360+0000 7fe746359640 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b8fd5d0 0x55d02b907df0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe734005ab0 tx=0x7fe734005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab44237b5ff924c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.360+0000 7fe729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.360+0000 7fe729ffb640 1 -- 172.21.15.94:0/1748826576 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340d9070 con 0x55d02b8fd5d0 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.360+0000 7fe729ffb640 1 -- 172.21.15.94:0/1748826576 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fe7340edc70 con 0x55d02b8fd5d0 2024-09-19T04:24:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.360+0000 7fe74a996740 1 -- 172.21.15.94:0/1748826576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7340ba930 con 0x55d02b8fd5d0 2024-09-19T04:24:27.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.360+0000 7fe729ffb640 1 -- 172.21.15.94:0/1748826576 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe7340ede70 con 0x55d02b8fd5d0 2024-09-19T04:24:27.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.360+0000 7fe74a996740 10 client.26132 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:27.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.360+0000 7fe74a996740 1 -- 172.21.15.94:0/1748826576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d02b90ae30 con 0x55d02b8fd5d0 2024-09-19T04:24:27.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.361+0000 7fe729ffb640 1 -- 172.21.15.94:0/1748826576 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7340c51f0 con 0x55d02b8fd5d0 2024-09-19T04:24:27.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.361+0000 7fe729ffb640 1 -- 172.21.15.94:0/1748826576 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7fe7340bba80 con 0x55d02b8fd5d0 2024-09-19T04:24:27.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.361+0000 7fe74a996740 10 client.26132 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:27.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.361+0000 7fe74a996740 10 client.26132 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:24:27.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.361+0000 7fe74a996740 20 client.26132 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:27.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.361+0000 7fe74a996740 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d02b917b30 0x55d02b919ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.361+0000 7fe74a996740 4 client.26132 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:27.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.361+0000 7fe74a996740 1 -- 172.21.15.94:0/1748826576 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe7340ba930 con 0x55d02b917b30 2024-09-19T04:24:27.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.362+0000 7fe745b58640 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d02b917b30 0x55d02b919ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.363+0000 7fe729ffb640 10 client.26132 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:24:27.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.363+0000 7fe745b58640 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d02b917b30 0x55d02b919ff0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=363b26bcf1eca48f server_cookie=ae18654bcee5ebbd in_seq=0 out_seq=0 2024-09-19T04:24:27.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.364+0000 7fe729ffb640 1 -- 172.21.15.94:0/1748826576 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe7340ba930 con 0x55d02b917b30 2024-09-19T04:24:27.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.364+0000 7fe729ffb640 10 client.26132 handle_command_reply: tid=0 2024-09-19T04:24:27.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.364+0000 7fe74a996740 1 client.26132 shutdown 2024-09-19T04:24:27.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.364+0000 7fe74a996740 1 -- 172.21.15.94:0/1748826576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b8fd5d0 msgr2=0x55d02b907df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.364+0000 7fe74a996740 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b8fd5d0 0x55d02b907df0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe734005ab0 tx=0x7fe734005c00 comp rx=0 tx=0).stop 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe74a996740 1 -- 172.21.15.94:0/1748826576 shutdown_connections 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe74a996740 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d02b917b30 0x55d02b919ff0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe74a996740 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b908370 0x55d02b9057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe729ffb640 0 client.26132 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe74a996740 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8feb40 0x55d02b9050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe74a996740 1 --2- 172.21.15.94:0/1748826576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b8fd5d0 0x55d02b907df0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe74a996740 1 -- 172.21.15.94:0/1748826576 >> 172.21.15.94:0/1748826576 conn(0x55d02b8c6310 msgr2=0x55d02b8ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe74a996740 1 -- 172.21.15.94:0/1748826576 shutdown_connections 2024-09-19T04:24:27.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe74a996740 1 -- 172.21.15.94:0/1748826576 wait complete. 2024-09-19T04:24:27.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.365+0000 7fe74a996740 20 client.26132 trim_cache size 0 max 16384 2024-09-19T04:24:27.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.388+0000 7fe74a996740 1 Processor -- start 2024-09-19T04:24:27.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.388+0000 7fe74a996740 1 -- start start 2024-09-19T04:24:27.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.388+0000 7fe74a996740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b914090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.388+0000 7fe74a996740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 0x55d02b900120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.388+0000 7fe74a996740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b900660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe74a996740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401befb0 con 0x55d02b8feb40 2024-09-19T04:24:27.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe74a996740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401bc850 con 0x55d02b908370 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe74a996740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401ba6c0 con 0x55d02b8fd5d0 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746359640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b914090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746359640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b914090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:60174/0 (socket says 172.21.15.94:60174) 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746359640 1 -- 172.21.15.94:0/1955297007 learned_addr learned my addr 172.21.15.94:0/1955297007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746b5a640 1 --2- 172.21.15.94:0/1955297007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b900660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe745b58640 1 --2- 172.21.15.94:0/1955297007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 0x55d02b900120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746359640 1 -- 172.21.15.94:0/1955297007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 msgr2=0x55d02b900660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746359640 1 --2- 172.21.15.94:0/1955297007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b900660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746359640 1 -- 172.21.15.94:0/1955297007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 msgr2=0x55d02b900120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:27.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746359640 1 --2- 172.21.15.94:0/1955297007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 0x55d02b900120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746359640 1 -- 172.21.15.94:0/1955297007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d02b8f9430 con 0x55d02b8fd5d0 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.389+0000 7fe746359640 1 --2- 172.21.15.94:0/1955297007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b914090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe7340b9000 tx=0x7fe7340bbf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab768f59bba2f970 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.390+0000 7fe72bfff640 1 -- 172.21.15.94:0/1955297007 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340ba6d0 con 0x55d02b8fd5d0 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.390+0000 7fe72bfff640 1 -- 172.21.15.94:0/1955297007 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7340ba930 con 0x55d02b8fd5d0 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.390+0000 7fe72bfff640 1 -- 172.21.15.94:0/1955297007 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340ede80 con 0x55d02b8fd5d0 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.391+0000 7fe74a996740 1 -- 172.21.15.94:0/1955297007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 msgr2=0x55d02b914090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.391+0000 7fe74a996740 1 --2- 172.21.15.94:0/1955297007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b914090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe7340b9000 tx=0x7fe7340bbf90 comp rx=0 tx=0).stop 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.391+0000 7fe74a996740 1 -- 172.21.15.94:0/1955297007 shutdown_connections 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.391+0000 7fe74a996740 1 --2- 172.21.15.94:0/1955297007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b900660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.391+0000 7fe74a996740 1 --2- 172.21.15.94:0/1955297007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 0x55d02b900120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.391+0000 7fe74a996740 1 --2- 172.21.15.94:0/1955297007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b914090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.391+0000 7fe74a996740 1 -- 172.21.15.94:0/1955297007 >> 172.21.15.94:0/1955297007 conn(0x55d02b8c6310 msgr2=0x55d02b8f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:27.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.392+0000 7fe74a996740 1 -- 172.21.15.94:0/1955297007 shutdown_connections 2024-09-19T04:24:27.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.392+0000 7fe74a996740 1 -- 172.21.15.94:0/1955297007 wait complete. 2024-09-19T04:24:27.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.392+0000 7fe74a996740 1 -- start start 2024-09-19T04:24:27.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.392+0000 7fe74a996740 1 Processor -- start 2024-09-19T04:24:27.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.393+0000 7fe74a996740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b910ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.393+0000 7fe74a996740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 0x55d02b901f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.393+0000 7fe746359640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b910ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.393+0000 7fe746359640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b910ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:60188/0 (socket says 172.21.15.94:60188) 2024-09-19T04:24:27.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.393+0000 7fe746359640 1 -- 172.21.15.94:0/1443070122 learned_addr learned my addr 172.21.15.94:0/1443070122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:27.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.393+0000 7fe74a996740 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b9026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.393+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401bad10 con 0x55d02b8feb40 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe746b5a640 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b9026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe745b58640 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 0x55d02b901f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401bc180 con 0x55d02b908370 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401bb3e0 con 0x55d02b8fd5d0 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe74a996740 10 client.0 _ll_register_callbacks cb 0x7fe738009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe746359640 1 -- 172.21.15.94:0/1443070122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 msgr2=0x55d02b9026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe746359640 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b9026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe746359640 1 -- 172.21.15.94:0/1443070122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 msgr2=0x55d02b901f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe746359640 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 0x55d02b901f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe746359640 1 -- 172.21.15.94:0/1443070122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d02b981890 con 0x55d02b8fd5d0 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.394+0000 7fe746b5a640 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b9026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:27.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.395+0000 7fe746359640 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b910ad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7340c0af0 tx=0x7fe73402a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9084116b5c8614 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.395+0000 7fe729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.395+0000 7fe729ffb640 1 -- 172.21.15.94:0/1443070122 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340ede80 con 0x55d02b8fd5d0 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.395+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7340ba930 con 0x55d02b8fd5d0 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.395+0000 7fe729ffb640 1 -- 172.21.15.94:0/1443070122 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fe7340d5500 con 0x55d02b8fd5d0 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.395+0000 7fe729ffb640 1 -- 172.21.15.94:0/1443070122 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe7340bc630 con 0x55d02b8fd5d0 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.395+0000 7fe74a996740 10 client.26098 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.395+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d02b981890 con 0x55d02b8fd5d0 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.395+0000 7fe729ffb640 1 -- 172.21.15.94:0/1443070122 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7340ba930 con 0x55d02b8fd5d0 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.396+0000 7fe729ffb640 1 -- 172.21.15.94:0/1443070122 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7fe7340b8380 con 0x55d02b8fd5d0 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.396+0000 7fe74a996740 10 client.26098 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.396+0000 7fe74a996740 10 client.26098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.396+0000 7fe74a996740 20 client.26098 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.396+0000 7fe74a996740 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d02b917b30 0x55d02b984d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.396+0000 7fe74a996740 4 client.26098 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.396+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d02b870860 con 0x55d02b917b30 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.396+0000 7fe745b58640 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d02b917b30 0x55d02b984d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe729ffb640 10 client.26098 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe745b58640 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d02b917b30 0x55d02b984d20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7aeb87ea0ea3cdf8 server_cookie=19e03760b727013c in_seq=0 out_seq=0 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe729ffb640 1 -- 172.21.15.94:0/1443070122 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d02b870860 con 0x55d02b917b30 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe729ffb640 10 client.26098 handle_command_reply: tid=0 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe74a996740 1 client.26098 shutdown 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 msgr2=0x55d02b910ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe74a996740 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b910ad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7340c0af0 tx=0x7fe73402a420 comp rx=0 tx=0).stop 2024-09-19T04:24:27.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 shutdown_connections 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe74a996740 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d02b917b30 0x55d02b984d20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe74a996740 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d02b908370 0x55d02b9026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe729ffb640 0 client.26098 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe74a996740 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d02b8feb40 0x55d02b901f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe74a996740 1 --2- 172.21.15.94:0/1443070122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02b8fd5d0 0x55d02b910ad0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.397+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 >> 172.21.15.94:0/1443070122 conn(0x55d02b8c6310 msgr2=0x55d02b8f8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.398+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 shutdown_connections 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.398+0000 7fe74a996740 1 -- 172.21.15.94:0/1443070122 wait complete. 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.398+0000 7fe74a996740 20 client.26098 trim_cache size 0 max 16384 2024-09-19T04:24:27.405 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:24:27.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 -- 172.21.15.94:0/488448899 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe72c0a3430 msgr2=0x7fe72c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe72c0a3430 0x7fe72c0a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe74006e2c0 tx=0x7fe73c005f90 comp rx=0 tx=0).stop 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 -- 172.21.15.94:0/488448899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7401008c0 msgr2=0x7fe740073a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7401008c0 0x7fe740073a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe734031440 tx=0x7fe734003b90 comp rx=0 tx=0).stop 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 -- 172.21.15.94:0/488448899 shutdown_connections 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe72c0a3430 0x7fe72c0a58f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe740105ec0 0x7fe74006d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7401011e0 0x7fe74006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 --2- 172.21.15.94:0/488448899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7401008c0 0x7fe740073a70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.399+0000 7fe7485bd640 1 -- 172.21.15.94:0/488448899 >> 172.21.15.94:0/488448899 conn(0x7fe7400fc7d0 msgr2=0x7fe740103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.400+0000 7fe7485bd640 1 -- 172.21.15.94:0/488448899 shutdown_connections 2024-09-19T04:24:27.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:27.400+0000 7fe7485bd640 1 -- 172.21.15.94:0/488448899 wait complete. 2024-09-19T04:24:28.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:24:28.930 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:28 smithi094 ceph-mon[30972]: pgmap v292: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:24:29.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:28 smithi112 ceph-mon[35051]: pgmap v292: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:24:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:28 smithi104 ceph-mon[34145]: pgmap v292: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:24:29.299 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 495.0 s, 1371.0 tps, lat 72.886 ms stddev 28.301 2024-09-19T04:24:31.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:30 smithi112 ceph-mon[35051]: pgmap v293: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.4 MiB/s wr, 1.09k op/s 2024-09-19T04:24:31.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:30 smithi094 ceph-mon[30972]: pgmap v293: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.4 MiB/s wr, 1.09k op/s 2024-09-19T04:24:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:30 smithi104 ceph-mon[34145]: pgmap v293: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.4 MiB/s wr, 1.09k op/s 2024-09-19T04:24:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.287+0000 7fc1b8088640 1 -- 172.21.15.94:0/1264699891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1b0077040 msgr2=0x7fc1b0112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.287+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1264699891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1b0077040 0x7fc1b0112510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009900 tx=0x7fc1a002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.287+0000 7fc1b8088640 1 -- 172.21.15.94:0/1264699891 shutdown_connections 2024-09-19T04:24:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.287+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1264699891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b0112a50 0x7fc1b0114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.287+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1264699891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1b0077040 0x7fc1b0112510 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.287+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1264699891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0076720 0x7fc1b0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.287+0000 7fc1b8088640 1 -- 172.21.15.94:0/1264699891 >> 172.21.15.94:0/1264699891 conn(0x7fc1b00ffa50 msgr2=0x7fc1b0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.287+0000 7fc1b8088640 1 -- 172.21.15.94:0/1264699891 shutdown_connections 2024-09-19T04:24:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.287+0000 7fc1b8088640 1 -- 172.21.15.94:0/1264699891 wait complete. 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.288+0000 7fc1b8088640 1 Processor -- start 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.288+0000 7fc1b8088640 1 -- start start 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b8088640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0076720 0x7fc1b01a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b8088640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b0077040 0x7fc1b01a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b8088640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1b0112a50 0x7fc1b019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b8088640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01184c0 con 0x7fc1b0076720 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b8088640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0118340 con 0x7fc1b0112a50 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b8088640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0118640 con 0x7fc1b0077040 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b5e24640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0076720 0x7fc1b01a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b5e24640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0076720 0x7fc1b01a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44672/0 (socket says 172.21.15.94:44672) 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b5e24640 1 -- 172.21.15.94:0/1757995629 learned_addr learned my addr 172.21.15.94:0/1757995629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b6625640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1b0112a50 0x7fc1b019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b6625640 1 -- 172.21.15.94:0/1757995629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b0077040 msgr2=0x7fc1b01a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b6625640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b0077040 0x7fc1b01a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b6625640 1 -- 172.21.15.94:0/1757995629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0076720 msgr2=0x7fc1b01a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b6625640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0076720 0x7fc1b01a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b6625640 1 -- 172.21.15.94:0/1757995629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b019efa0 con 0x7fc1b0112a50 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.289+0000 7fc1b6625640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1b0112a50 0x7fc1b019e870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc1b007aa10 tx=0x7fc1ac00edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.290+0000 7fc1a6ffd640 1 -- 172.21.15.94:0/1757995629 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00ccc0 con 0x7fc1b0112a50 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.290+0000 7fc1a6ffd640 1 -- 172.21.15.94:0/1757995629 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1ac005020 con 0x7fc1b0112a50 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.290+0000 7fc1a6ffd640 1 -- 172.21.15.94:0/1757995629 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00ce60 con 0x7fc1b0112a50 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.290+0000 7fc1b8088640 1 -- 172.21.15.94:0/1757995629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b019f290 con 0x7fc1b0112a50 2024-09-19T04:24:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.290+0000 7fc1b8088640 1 -- 172.21.15.94:0/1757995629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b01ab580 con 0x7fc1b0112a50 2024-09-19T04:24:32.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.293+0000 7fc1a6ffd640 1 -- 172.21.15.94:0/1757995629 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc1ac0051d0 con 0x7fc1b0112a50 2024-09-19T04:24:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.294+0000 7fc1ba461740 1 Processor -- start 2024-09-19T04:24:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.294+0000 7fc1a6ffd640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc19c0a3500 0x7fc19c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.295+0000 7fc1b5623640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc19c0a3500 0x7fc19c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.295+0000 7fc1a6ffd640 1 -- 172.21.15.94:0/1757995629 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fc1ac099eb0 con 0x7fc1b0112a50 2024-09-19T04:24:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.295+0000 7fc1b5623640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc19c0a3500 0x7fc19c0a59c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc1a002a910 tx=0x7fc1a0031400 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1ba461740 1 -- start start 2024-09-19T04:24:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd1b40 0x558d73fd0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fd0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fdd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1ba461740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558d73f362f0 con 0x558d73fdb370 2024-09-19T04:24:32.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1ba461740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558d73eddcf0 con 0x558d73fd05d0 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1ba461740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d73f322b0 con 0x558d73fd1b40 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1b6625640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fdd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1b5623640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fd0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1b5e24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd1b40 0x558d73fd0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1b6625640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fdd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44684/0 (socket says 172.21.15.94:44684) 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.296+0000 7fc1b6625640 1 -- 172.21.15.94:0/4230104560 learned_addr learned my addr 172.21.15.94:0/4230104560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.297+0000 7fc1b5623640 1 -- 172.21.15.94:0/4230104560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd1b40 msgr2=0x558d73fd0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.297+0000 7fc1b5623640 1 --2- 172.21.15.94:0/4230104560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd1b40 0x558d73fd0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.297+0000 7fc1b5623640 1 -- 172.21.15.94:0/4230104560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 msgr2=0x558d73fdd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.297+0000 7fc1b5623640 1 --2- 172.21.15.94:0/4230104560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fdd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.297+0000 7fc1b5623640 1 -- 172.21.15.94:0/4230104560 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d73ec23a0 con 0x558d73fd05d0 2024-09-19T04:24:32.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.297+0000 7fc1b5623640 1 --2- 172.21.15.94:0/4230104560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fd0a10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009230 tx=0x7fc1a003a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b573e407413287fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:32.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.297+0000 7fc19bfff640 1 -- 172.21.15.94:0/4230104560 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0035040 con 0x558d73fd05d0 2024-09-19T04:24:32.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.298+0000 7fc19bfff640 1 -- 172.21.15.94:0/4230104560 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a004c070 con 0x558d73fd05d0 2024-09-19T04:24:32.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.298+0000 7fc19bfff640 1 -- 172.21.15.94:0/4230104560 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0047440 con 0x558d73fd05d0 2024-09-19T04:24:32.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.298+0000 7fc1ba461740 1 -- 172.21.15.94:0/4230104560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 msgr2=0x558d73fd0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.298+0000 7fc1ba461740 1 --2- 172.21.15.94:0/4230104560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fd0a10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009230 tx=0x7fc1a003a000 comp rx=0 tx=0).stop 2024-09-19T04:24:32.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.298+0000 7fc1ba461740 1 -- 172.21.15.94:0/4230104560 shutdown_connections 2024-09-19T04:24:32.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.299+0000 7fc1ba461740 1 --2- 172.21.15.94:0/4230104560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fdd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.299+0000 7fc1ba461740 1 --2- 172.21.15.94:0/4230104560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fd0a10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.299+0000 7fc1ba461740 1 --2- 172.21.15.94:0/4230104560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd1b40 0x558d73fd0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.299+0000 7fc1ba461740 1 -- 172.21.15.94:0/4230104560 >> 172.21.15.94:0/4230104560 conn(0x558d73f99310 msgr2=0x558d73fccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.299+0000 7fc1ba461740 1 -- 172.21.15.94:0/4230104560 shutdown_connections 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.299+0000 7fc1ba461740 1 -- 172.21.15.94:0/4230104560 wait complete. 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.299+0000 7fc1ba461740 1 -- start start 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.299+0000 7fc1ba461740 1 Processor -- start 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fdad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd1b40 0x558d73fd8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fd8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1b5e24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fdad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1ba461740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558d73fde010 con 0x558d73fdb370 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1ba461740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558d73ef1060 con 0x558d73fd1b40 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1ba461740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d73fde190 con 0x558d73fd05d0 2024-09-19T04:24:32.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1b6625640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fd8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1b5623640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd1b40 0x558d73fd8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1b6625640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fd8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44686/0 (socket says 172.21.15.94:44686) 2024-09-19T04:24:32.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1b6625640 1 -- 172.21.15.94:0/3625472108 learned_addr learned my addr 172.21.15.94:0/3625472108 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:32.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.300+0000 7fc1ba461740 10 client.0 _ll_register_callbacks cb 0x558d73c448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:32.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.301+0000 7fc1b6625640 1 -- 172.21.15.94:0/3625472108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 msgr2=0x558d73fdad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.301+0000 7fc1b6625640 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fdad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.301+0000 7fc1b6625640 1 -- 172.21.15.94:0/3625472108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd1b40 msgr2=0x558d73fd8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.301+0000 7fc1b6625640 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd1b40 0x558d73fd8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.301+0000 7fc1b6625640 1 -- 172.21.15.94:0/3625472108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d73ca73c0 con 0x558d73fdb370 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.301+0000 7fc1b5623640 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd1b40 0x558d73fd8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.301+0000 7fc1b6625640 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fd8610 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac09bda0 tx=0x7fc1ac0cd840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b034a9ffe23cf9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.301+0000 7fc1997fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.301+0000 7fc1997fa640 1 -- 172.21.15.94:0/3625472108 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0b9070 con 0x558d73fdb370 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.302+0000 7fc1997fa640 1 -- 172.21.15.94:0/3625472108 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fc1ac0a4800 con 0x558d73fdb370 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.302+0000 7fc1ba461740 1 -- 172.21.15.94:0/3625472108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a004c070 con 0x558d73fdb370 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.302+0000 7fc1997fa640 1 -- 172.21.15.94:0/3625472108 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc1ac01c040 con 0x558d73fdb370 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.302+0000 7fc1ba461740 10 client.16545 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:32.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.302+0000 7fc1ba461740 1 -- 172.21.15.94:0/3625472108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558d73fe9b10 con 0x558d73fdb370 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.303+0000 7fc1997fa640 1 -- 172.21.15.94:0/3625472108 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1ac0a01c0 con 0x558d73fdb370 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.303+0000 7fc1997fa640 1 -- 172.21.15.94:0/3625472108 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7fc1ac09ad60 con 0x558d73fdb370 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.303+0000 7fc1ba461740 10 client.16545 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.303+0000 7fc1ba461740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.303+0000 7fc1ba461740 20 client.16545 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.303+0000 7fc1ba461740 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d73feb0c0 0x558d73fed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.303+0000 7fc1ba461740 4 client.16545 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.303+0000 7fc1ba461740 1 -- 172.21.15.94:0/3625472108 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc1a004c070 con 0x558d73feb0c0 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.303+0000 7fc1b5e24640 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d73feb0c0 0x558d73fed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.304+0000 7fc1b5e24640 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d73feb0c0 0x558d73fed4e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21dda81a3752802 server_cookie=cc8c5901f22122e2 in_seq=0 out_seq=0 2024-09-19T04:24:32.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.304+0000 7fc1997fa640 10 client.16545 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:32.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.305+0000 7fc1997fa640 1 -- 172.21.15.94:0/3625472108 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc1a004c070 con 0x558d73feb0c0 2024-09-19T04:24:32.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.305+0000 7fc1997fa640 10 client.16545 handle_command_reply: tid=0 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.305+0000 7fc1ba461740 1 client.16545 shutdown 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 -- 172.21.15.94:0/3625472108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 msgr2=0x558d73fd8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fd8610 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac09bda0 tx=0x7fc1ac0cd840 comp rx=0 tx=0).stop 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 -- 172.21.15.94:0/3625472108 shutdown_connections 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d73feb0c0 0x558d73fed4e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fdb370 0x558d73fd8610 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd1b40 0x558d73fd8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 --2- 172.21.15.94:0/3625472108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fdad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 -- 172.21.15.94:0/3625472108 >> 172.21.15.94:0/3625472108 conn(0x558d73f99310 msgr2=0x558d73fd2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1997fa640 0 client.16545 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 -- 172.21.15.94:0/3625472108 shutdown_connections 2024-09-19T04:24:32.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 1 -- 172.21.15.94:0/3625472108 wait complete. 2024-09-19T04:24:32.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.306+0000 7fc1ba461740 20 client.16545 trim_cache size 0 max 16384 2024-09-19T04:24:32.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.326+0000 7fc1ba461740 1 Processor -- start 2024-09-19T04:24:32.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.326+0000 7fc1ba461740 1 -- start start 2024-09-19T04:24:32.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fe72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 0x558d73fd3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fdb370 0x558d73fd3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1b5e24640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fe72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1b5623640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 0x558d73fd3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1b5e24640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fe72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52926/0 (socket says 172.21.15.94:52926) 2024-09-19T04:24:32.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1b5623640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 0x558d73fd3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44702/0 (socket says 172.21.15.94:44702) 2024-09-19T04:24:32.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1b5e24640 1 -- 172.21.15.94:0/16386021 learned_addr learned my addr 172.21.15.94:0/16386021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1ba461740 1 -- 172.21.15.94:0/16386021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b6870 con 0x558d73fd1b40 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1ba461740 1 -- 172.21.15.94:0/16386021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b4110 con 0x558d73fd05d0 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1ba461740 1 -- 172.21.15.94:0/16386021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b1f80 con 0x558d73fdb370 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.327+0000 7fc1b6625640 1 --2- 172.21.15.94:0/16386021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fdb370 0x558d73fd3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.328+0000 7fc1b5e24640 1 -- 172.21.15.94:0/16386021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fdb370 msgr2=0x558d73fd3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.328+0000 7fc1b5e24640 1 --2- 172.21.15.94:0/16386021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fdb370 0x558d73fd3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.328+0000 7fc1b5e24640 1 -- 172.21.15.94:0/16386021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 msgr2=0x558d73fd3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.328+0000 7fc1b5e24640 1 --2- 172.21.15.94:0/16386021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 0x558d73fd3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.328+0000 7fc1b5e24640 1 -- 172.21.15.94:0/16386021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d73fd6570 con 0x558d73fd05d0 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.328+0000 7fc1b5e24640 1 --2- 172.21.15.94:0/16386021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fe72b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x558d73fedd20 tx=0x7fc1a8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=681653250301aab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.328+0000 7fc19b7fe640 1 -- 172.21.15.94:0/16386021 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8004040 con 0x558d73fd05d0 2024-09-19T04:24:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.328+0000 7fc19b7fe640 1 -- 172.21.15.94:0/16386021 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a8004690 con 0x558d73fd05d0 2024-09-19T04:24:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.328+0000 7fc19b7fe640 1 -- 172.21.15.94:0/16386021 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8010c60 con 0x558d73fd05d0 2024-09-19T04:24:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.329+0000 7fc1ba461740 1 -- 172.21.15.94:0/16386021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 msgr2=0x558d73fe72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.329+0000 7fc1ba461740 1 --2- 172.21.15.94:0/16386021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fe72b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x558d73fedd20 tx=0x7fc1a8009500 comp rx=0 tx=0).stop 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.330+0000 7fc1ba461740 1 -- 172.21.15.94:0/16386021 shutdown_connections 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.330+0000 7fc1ba461740 1 --2- 172.21.15.94:0/16386021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fdb370 0x558d73fd3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.330+0000 7fc1ba461740 1 --2- 172.21.15.94:0/16386021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 0x558d73fd3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.330+0000 7fc1ba461740 1 --2- 172.21.15.94:0/16386021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fd05d0 0x558d73fe72b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.330+0000 7fc1ba461740 1 -- 172.21.15.94:0/16386021 >> 172.21.15.94:0/16386021 conn(0x558d73f99310 msgr2=0x558d73fd7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.330+0000 7fc1ba461740 1 -- 172.21.15.94:0/16386021 shutdown_connections 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.330+0000 7fc1ba461740 1 -- 172.21.15.94:0/16386021 wait complete. 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.331+0000 7fc1ba461740 1 -- start start 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.331+0000 7fc1ba461740 1 Processor -- start 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fe3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 0x558d73fd4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1ba461740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fdb370 0x558d73fd4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1ba461740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b25d0 con 0x558d73fd1b40 2024-09-19T04:24:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1ba461740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b3a40 con 0x558d73fdb370 2024-09-19T04:24:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1ba461740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b2ca0 con 0x558d73fd05d0 2024-09-19T04:24:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1ba461740 10 client.0 _ll_register_callbacks cb 0x7fc1ac005540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b5e24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fe3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b5e24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fe3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:33366/0 (socket says 172.21.15.94:33366) 2024-09-19T04:24:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b5e24640 1 -- 172.21.15.94:0/2503942179 learned_addr learned my addr 172.21.15.94:0/2503942179 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b5623640 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 0x558d73fd4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b6625640 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fdb370 0x558d73fd4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b5e24640 1 -- 172.21.15.94:0/2503942179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fdb370 msgr2=0x558d73fd4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b5e24640 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fdb370 0x558d73fd4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b5e24640 1 -- 172.21.15.94:0/2503942179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 msgr2=0x558d73fd4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b5e24640 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 0x558d73fd4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.332+0000 7fc1b5e24640 1 -- 172.21.15.94:0/2503942179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d73fd5510 con 0x558d73fd05d0 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.333+0000 7fc1b5e24640 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fe3880 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009cd0 tx=0x7fc1a80121a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff80ef9cd620a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.333+0000 7fc1997fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.333+0000 7fc1997fa640 1 -- 172.21.15.94:0/2503942179 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8010430 con 0x558d73fd05d0 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.333+0000 7fc1ba461740 1 -- 172.21.15.94:0/2503942179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a8004690 con 0x558d73fd05d0 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.333+0000 7fc1997fa640 1 -- 172.21.15.94:0/2503942179 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fc1a80105d0 con 0x558d73fd05d0 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.333+0000 7fc1997fa640 1 -- 172.21.15.94:0/2503942179 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc1a80109e0 con 0x558d73fd05d0 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.333+0000 7fc1ba461740 10 client.26113 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.333+0000 7fc1ba461740 1 -- 172.21.15.94:0/2503942179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558d74055ac0 con 0x558d73fd05d0 2024-09-19T04:24:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.334+0000 7fc1997fa640 1 -- 172.21.15.94:0/2503942179 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a8004690 con 0x558d73fd05d0 2024-09-19T04:24:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.334+0000 7fc1997fa640 1 -- 172.21.15.94:0/2503942179 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7fc1a80315e0 con 0x558d73fd05d0 2024-09-19T04:24:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.334+0000 7fc1ba461740 10 client.26113 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.334+0000 7fc1ba461740 10 client.26113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.334+0000 7fc1ba461740 20 client.26113 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.334+0000 7fc1ba461740 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d740561e0 0x558d74058600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.334+0000 7fc1ba461740 4 client.26113 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:24:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.334+0000 7fc1ba461740 1 -- 172.21.15.94:0/2503942179 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d73f43860 con 0x558d740561e0 2024-09-19T04:24:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.334+0000 7fc1b5623640 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d740561e0 0x558d74058600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.335+0000 7fc1997fa640 10 client.26113 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.335+0000 7fc1b5623640 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d740561e0 0x558d74058600 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3bf492e57c0b882 server_cookie=4d57f2915c8de87e in_seq=0 out_seq=0 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.335+0000 7fc1997fa640 1 -- 172.21.15.94:0/2503942179 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d73f43860 con 0x558d740561e0 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.335+0000 7fc1997fa640 10 client.26113 handle_command_reply: tid=0 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.335+0000 7fc1ba461740 1 client.26113 shutdown 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.335+0000 7fc1ba461740 1 -- 172.21.15.94:0/2503942179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 msgr2=0x558d73fe3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.335+0000 7fc1ba461740 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fe3880 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009cd0 tx=0x7fc1a80121a0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1ba461740 1 -- 172.21.15.94:0/2503942179 shutdown_connections 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1ba461740 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558d740561e0 0x558d74058600 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1ba461740 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558d73fdb370 0x558d73fd4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1ba461740 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d73fd1b40 0x558d73fd4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1997fa640 0 client.26113 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1ba461740 1 --2- 172.21.15.94:0/2503942179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d73fd05d0 0x558d73fe3880 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1ba461740 1 -- 172.21.15.94:0/2503942179 >> 172.21.15.94:0/2503942179 conn(0x558d73f99310 msgr2=0x558d73fd9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1ba461740 1 -- 172.21.15.94:0/2503942179 shutdown_connections 2024-09-19T04:24:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1ba461740 1 -- 172.21.15.94:0/2503942179 wait complete. 2024-09-19T04:24:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.336+0000 7fc1ba461740 20 client.26113 trim_cache size 0 max 16384 2024-09-19T04:24:32.342 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:24:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 -- 172.21.15.94:0/1757995629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc19c0a3500 msgr2=0x7fc19c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc19c0a3500 0x7fc19c0a59c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc1a002a910 tx=0x7fc1a0031400 comp rx=0 tx=0).stop 2024-09-19T04:24:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 -- 172.21.15.94:0/1757995629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1b0112a50 msgr2=0x7fc1b019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1b0112a50 0x7fc1b019e870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc1b007aa10 tx=0x7fc1ac00edd0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 -- 172.21.15.94:0/1757995629 shutdown_connections 2024-09-19T04:24:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc19c0a3500 0x7fc19c0a59c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1b0112a50 0x7fc1b019e870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b0077040 0x7fc1b01a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 --2- 172.21.15.94:0/1757995629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0076720 0x7fc1b01a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 -- 172.21.15.94:0/1757995629 >> 172.21.15.94:0/1757995629 conn(0x7fc1b00ffa50 msgr2=0x7fc1b0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 -- 172.21.15.94:0/1757995629 shutdown_connections 2024-09-19T04:24:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:32.337+0000 7fc1b8088640 1 -- 172.21.15.94:0/1757995629 wait complete. 2024-09-19T04:24:32.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:32 smithi094 ceph-mon[30972]: pgmap v294: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.4 MiB/s wr, 1.09k op/s 2024-09-19T04:24:33.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:32 smithi112 ceph-mon[35051]: pgmap v294: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.4 MiB/s wr, 1.09k op/s 2024-09-19T04:24:33.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:32 smithi104 ceph-mon[34145]: pgmap v294: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.4 MiB/s wr, 1.09k op/s 2024-09-19T04:24:34.028 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:24:34.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:24:34.451 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:34.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:34 smithi094 ceph-mon[30972]: pgmap v295: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:24:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.024+0000 7f6da3fa1640 1 -- 172.21.15.94:0/2413859328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 msgr2=0x7f6d9c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.024+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/2413859328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 0x7f6d9c107b90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009930 tx=0x7f6d8c02a420 comp rx=0 tx=0).stop 2024-09-19T04:24:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.024+0000 7f6da3fa1640 1 -- 172.21.15.94:0/2413859328 shutdown_connections 2024-09-19T04:24:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.024+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/2413859328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d9c1082a0 0x7f6d9c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.024+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/2413859328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 0x7f6d9c107b90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.024+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/2413859328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c102d30 0x7f6d9c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.024+0000 7f6da3fa1640 1 -- 172.21.15.94:0/2413859328 >> 172.21.15.94:0/2413859328 conn(0x7f6d9c0fc7d0 msgr2=0x7f6d9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:35.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.024+0000 7f6da3fa1640 1 -- 172.21.15.94:0/2413859328 shutdown_connections 2024-09-19T04:24:35.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.024+0000 7f6da3fa1640 1 -- 172.21.15.94:0/2413859328 wait complete. 2024-09-19T04:24:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.025+0000 7f6da3fa1640 1 Processor -- start 2024-09-19T04:24:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.025+0000 7f6da3fa1640 1 -- start start 2024-09-19T04:24:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da3fa1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d9c102d30 0x7f6d9c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da3fa1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 0x7f6d9c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da3fa1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c1082a0 0x7f6d9c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da3fa1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c111f40 con 0x7f6d9c1036e0 2024-09-19T04:24:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da3fa1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c111dc0 con 0x7f6d9c1082a0 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da3fa1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c1120c0 con 0x7f6d9c102d30 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da153c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 0x7f6d9c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da1d3d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d9c102d30 0x7f6d9c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da153c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 0x7f6d9c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44734/0 (socket says 172.21.15.94:44734) 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da153c640 1 -- 172.21.15.94:0/3770370861 learned_addr learned my addr 172.21.15.94:0/3770370861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da253e640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c1082a0 0x7f6d9c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da253e640 1 -- 172.21.15.94:0/3770370861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d9c102d30 msgr2=0x7f6d9c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da253e640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d9c102d30 0x7f6d9c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da253e640 1 -- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 msgr2=0x7f6d9c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da253e640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 0x7f6d9c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da253e640 1 -- 172.21.15.94:0/3770370861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9c19f6b0 con 0x7f6d9c1082a0 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da153c640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 0x7f6d9c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:35.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.026+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d9c102d30 0x7f6d9c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:35.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.027+0000 7f6da253e640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c1082a0 0x7f6d9c19efd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400e940 tx=0x7f6d9400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:35.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.027+0000 7f6d92ffd640 1 -- 172.21.15.94:0/3770370861 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94004d20 con 0x7f6d9c1082a0 2024-09-19T04:24:35.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.027+0000 7f6d92ffd640 1 -- 172.21.15.94:0/3770370861 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d94005020 con 0x7f6d9c1082a0 2024-09-19T04:24:35.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.027+0000 7f6d92ffd640 1 -- 172.21.15.94:0/3770370861 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d940116d0 con 0x7f6d9c1082a0 2024-09-19T04:24:35.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.027+0000 7f6da3fa1640 1 -- 172.21.15.94:0/3770370861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9c19f9a0 con 0x7f6d9c1082a0 2024-09-19T04:24:35.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.027+0000 7f6da3fa1640 1 -- 172.21.15.94:0/3770370861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d9c103b50 con 0x7f6d9c1082a0 2024-09-19T04:24:35.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.030+0000 7f6d92ffd640 1 -- 172.21.15.94:0/3770370861 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6d9400cb20 con 0x7f6d9c1082a0 2024-09-19T04:24:35.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.031+0000 7f6d92ffd640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6d880a3400 0x7f6d880a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.031+0000 7f6da637a740 1 Processor -- start 2024-09-19T04:24:35.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.031+0000 7f6d92ffd640 1 -- 172.21.15.94:0/3770370861 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f6d940cfda0 con 0x7f6d9c1082a0 2024-09-19T04:24:35.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da637a740 1 -- start start 2024-09-19T04:24:35.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6d880a3400 0x7f6d880a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da637a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 0x559f15866f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da637a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15867480 0x559f15871800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da637a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15874130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da637a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559f157cc280 con 0x559f15866b60 2024-09-19T04:24:35.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da637a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559f15773b10 con 0x559f15871d40 2024-09-19T04:24:35.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da637a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f157c8240 con 0x559f15867480 2024-09-19T04:24:35.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:34 smithi112 ceph-mon[35051]: pgmap v295: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:24:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da1d3d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 0x559f15866f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da153c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15867480 0x559f15871800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da153c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15867480 0x559f15871800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:33398/0 (socket says 172.21.15.94:33398) 2024-09-19T04:24:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da153c640 1 -- 172.21.15.94:0/3739914048 learned_addr learned my addr 172.21.15.94:0/3739914048 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da253e640 1 --2- 172.21.15.94:0/3739914048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15874130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.032+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6d880a3400 0x7f6d880a58c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d98011490 tx=0x7f6d98011890 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.033+0000 7f6da1d3d640 1 -- 172.21.15.94:0/3739914048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15867480 msgr2=0x559f15871800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.033+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/3739914048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15867480 0x559f15871800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.033+0000 7f6da1d3d640 1 -- 172.21.15.94:0/3739914048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 msgr2=0x559f15874130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.033+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/3739914048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15874130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.033+0000 7f6da1d3d640 1 -- 172.21.15.94:0/3739914048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f157801f0 con 0x559f15866b60 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.033+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/3739914048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 0x559f15866f40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6d98011d90 tx=0x7f6d9801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dedfe0acfcc66b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.033+0000 7f6d87fff640 1 -- 172.21.15.94:0/3739914048 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98017040 con 0x559f15866b60 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.033+0000 7f6d87fff640 1 -- 172.21.15.94:0/3739914048 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d98019050 con 0x559f15866b60 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.033+0000 7f6d87fff640 1 -- 172.21.15.94:0/3739914048 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9801d6a0 con 0x559f15866b60 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.034+0000 7f6da637a740 1 -- 172.21.15.94:0/3739914048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 msgr2=0x559f15866f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.034+0000 7f6da637a740 1 --2- 172.21.15.94:0/3739914048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 0x559f15866f40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6d98011d90 tx=0x7f6d9801c000 comp rx=0 tx=0).stop 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.034+0000 7f6da637a740 1 -- 172.21.15.94:0/3739914048 shutdown_connections 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.034+0000 7f6da637a740 1 --2- 172.21.15.94:0/3739914048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15874130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.034+0000 7f6da637a740 1 --2- 172.21.15.94:0/3739914048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15867480 0x559f15871800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.034+0000 7f6da637a740 1 --2- 172.21.15.94:0/3739914048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 0x559f15866f40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.034+0000 7f6da637a740 1 -- 172.21.15.94:0/3739914048 >> 172.21.15.94:0/3739914048 conn(0x559f158630f0 msgr2=0x559f158634d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.035+0000 7f6da637a740 1 -- 172.21.15.94:0/3739914048 shutdown_connections 2024-09-19T04:24:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.035+0000 7f6da637a740 1 -- 172.21.15.94:0/3739914048 wait complete. 2024-09-19T04:24:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.035+0000 7f6da637a740 1 -- start start 2024-09-19T04:24:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.035+0000 7f6da637a740 1 Processor -- start 2024-09-19T04:24:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da637a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 0x559f1586ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da637a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15867480 0x559f1586f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da637a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f1586f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da637a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559f15874960 con 0x559f15867480 2024-09-19T04:24:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da637a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559f15786f10 con 0x559f15871d40 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da637a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f15874ae0 con 0x559f15866b60 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da1d3d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 0x559f1586ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da153c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15867480 0x559f1586f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da1d3d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 0x559f1586ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:33414/0 (socket says 172.21.15.94:33414) 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da153c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15867480 0x559f1586f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44746/0 (socket says 172.21.15.94:44746) 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da1d3d640 1 -- 172.21.15.94:0/4051638351 learned_addr learned my addr 172.21.15.94:0/4051638351 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.036+0000 7f6da637a740 10 client.0 _ll_register_callbacks cb 0x559f1574fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6da1d3d640 1 -- 172.21.15.94:0/4051638351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 msgr2=0x559f1586f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6da253e640 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f1586f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f1586f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6da1d3d640 1 -- 172.21.15.94:0/4051638351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15867480 msgr2=0x559f1586f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15867480 0x559f1586f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6da1d3d640 1 -- 172.21.15.94:0/4051638351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f1553d6a0 con 0x559f15866b60 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6da253e640 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f1586f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 0x559f1586ec20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801d670 tx=0x7f6d9801d6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda3c80e20bb1600 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6d857fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6d857fa640 1 -- 172.21.15.94:0/4051638351 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98034060 con 0x559f15866b60 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6d857fa640 1 -- 172.21.15.94:0/4051638351 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f6d98009040 con 0x559f15866b60 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.037+0000 7f6da637a740 1 -- 172.21.15.94:0/4051638351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d98019050 con 0x559f15866b60 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6d857fa640 1 -- 172.21.15.94:0/4051638351 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6d98017050 con 0x559f15866b60 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6da637a740 10 client.26125 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6da637a740 1 -- 172.21.15.94:0/4051638351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559f15880520 con 0x559f15866b60 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6d857fa640 1 -- 172.21.15.94:0/4051638351 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d98019050 con 0x559f15866b60 2024-09-19T04:24:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6d857fa640 1 -- 172.21.15.94:0/4051638351 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f6d98003da0 con 0x559f15866b60 2024-09-19T04:24:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6da637a740 10 client.26125 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6da637a740 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6da637a740 20 client.26125 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6da637a740 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559f158818d0 0x559f15883cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6da637a740 4 client.26125 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6da637a740 1 -- 172.21.15.94:0/4051638351 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f157d97f0 con 0x559f158818d0 2024-09-19T04:24:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.038+0000 7f6da153c640 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559f158818d0 0x559f15883cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.039+0000 7f6da153c640 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559f158818d0 0x559f15883cc0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97026281df1b6605 server_cookie=18692b23df7a3f19 in_seq=0 out_seq=0 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.039+0000 7f6d857fa640 10 client.26125 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.040+0000 7f6d857fa640 1 -- 172.21.15.94:0/4051638351 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f157d97f0 con 0x559f158818d0 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.040+0000 7f6d857fa640 10 client.26125 handle_command_reply: tid=0 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.040+0000 7f6da637a740 1 client.26125 shutdown 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.040+0000 7f6da637a740 1 -- 172.21.15.94:0/4051638351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 msgr2=0x559f1586ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.040+0000 7f6da637a740 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 0x559f1586ec20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801d670 tx=0x7f6d9801d6a0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.040+0000 7f6da637a740 1 -- 172.21.15.94:0/4051638351 shutdown_connections 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.040+0000 7f6da637a740 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559f158818d0 0x559f15883cc0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.041+0000 7f6da637a740 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f1586f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.041+0000 7f6d857fa640 0 client.26125 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.041+0000 7f6da637a740 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15867480 0x559f1586f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.041+0000 7f6da637a740 1 --2- 172.21.15.94:0/4051638351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 0x559f1586ec20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.041+0000 7f6da637a740 1 -- 172.21.15.94:0/4051638351 >> 172.21.15.94:0/4051638351 conn(0x559f158630f0 msgr2=0x559f15862270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:35.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.041+0000 7f6da637a740 1 -- 172.21.15.94:0/4051638351 shutdown_connections 2024-09-19T04:24:35.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.041+0000 7f6da637a740 1 -- 172.21.15.94:0/4051638351 wait complete. 2024-09-19T04:24:35.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.042+0000 7f6da637a740 20 client.26125 trim_cache size 0 max 16384 2024-09-19T04:24:35.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.073+0000 7f6da637a740 1 Processor -- start 2024-09-19T04:24:35.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.073+0000 7f6da637a740 1 -- start start 2024-09-19T04:24:35.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.073+0000 7f6da637a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 0x559f158695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.073+0000 7f6da637a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15867480 0x559f15889320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da637a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15871d40 0x559f15889860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da637a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c1b68c0 con 0x559f15866b60 2024-09-19T04:24:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da637a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c1b4160 con 0x559f15867480 2024-09-19T04:24:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da637a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c1b1ff0 con 0x559f15871d40 2024-09-19T04:24:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da153c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15867480 0x559f15889320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da153c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15867480 0x559f15889320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52976/0 (socket says 172.21.15.94:52976) 2024-09-19T04:24:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da153c640 1 -- 172.21.15.94:0/267142369 learned_addr learned my addr 172.21.15.94:0/267142369 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da253e640 1 --2- 172.21.15.94:0/267142369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15871d40 0x559f15889860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da153c640 1 -- 172.21.15.94:0/267142369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15871d40 msgr2=0x559f15889860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:35.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da153c640 1 --2- 172.21.15.94:0/267142369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15871d40 0x559f15889860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da153c640 1 -- 172.21.15.94:0/267142369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 msgr2=0x559f158695c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:24:35.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da153c640 1 --2- 172.21.15.94:0/267142369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 0x559f158695c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.074+0000 7f6da153c640 1 -- 172.21.15.94:0/267142369 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f15871220 con 0x559f15867480 2024-09-19T04:24:35.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.075+0000 7f6da153c640 1 --2- 172.21.15.94:0/267142369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15867480 0x559f15889320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009850 tx=0x7f6d8c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac3c23348f78f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:35.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.075+0000 7f6d877fe640 1 -- 172.21.15.94:0/267142369 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c035040 con 0x559f15867480 2024-09-19T04:24:35.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.075+0000 7f6d877fe640 1 -- 172.21.15.94:0/267142369 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d8c004c10 con 0x559f15867480 2024-09-19T04:24:35.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.076+0000 7f6d877fe640 1 -- 172.21.15.94:0/267142369 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c047ad0 con 0x559f15867480 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.076+0000 7f6da637a740 1 -- 172.21.15.94:0/267142369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15867480 msgr2=0x559f15889320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.076+0000 7f6da637a740 1 --2- 172.21.15.94:0/267142369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15867480 0x559f15889320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009850 tx=0x7f6d8c005fd0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.076+0000 7f6da637a740 1 -- 172.21.15.94:0/267142369 shutdown_connections 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.076+0000 7f6da637a740 1 --2- 172.21.15.94:0/267142369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15871d40 0x559f15889860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.076+0000 7f6da637a740 1 --2- 172.21.15.94:0/267142369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15867480 0x559f15889320 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009850 tx=0x7f6d8c005fd0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.076+0000 7f6da637a740 1 --2- 172.21.15.94:0/267142369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f15866b60 0x559f158695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.076+0000 7f6da637a740 1 -- 172.21.15.94:0/267142369 >> 172.21.15.94:0/267142369 conn(0x559f158630f0 msgr2=0x559f1586dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.076+0000 7f6da637a740 1 -- 172.21.15.94:0/267142369 shutdown_connections 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.077+0000 7f6da637a740 1 -- 172.21.15.94:0/267142369 wait complete. 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.077+0000 7f6da637a740 1 -- start start 2024-09-19T04:24:35.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.077+0000 7f6da637a740 1 Processor -- start 2024-09-19T04:24:35.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da637a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 0x559f15876200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da637a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15876740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da637a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f158818d0 0x559f158eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da637a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c1b2640 con 0x559f158818d0 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da637a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c1b3a90 con 0x559f15871d40 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da637a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c1b2cf0 con 0x559f15866b60 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da637a740 10 client.0 _ll_register_callbacks cb 0x7f6d94005b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da153c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15876740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da153c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15876740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52992/0 (socket says 172.21.15.94:52992) 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da253e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f158818d0 0x559f158eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.078+0000 7f6da153c640 1 -- 172.21.15.94:0/620577287 learned_addr learned my addr 172.21.15.94:0/620577287 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.079+0000 7f6da153c640 1 -- 172.21.15.94:0/620577287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 msgr2=0x559f15876200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.079+0000 7f6da153c640 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 0x559f15876200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.079+0000 7f6da153c640 1 -- 172.21.15.94:0/620577287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f158818d0 msgr2=0x559f158eb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.079+0000 7f6da153c640 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f158818d0 0x559f158eb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.079+0000 7f6da153c640 1 -- 172.21.15.94:0/620577287 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f158672b0 con 0x559f15871d40 2024-09-19T04:24:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.079+0000 7f6da153c640 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15876740 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c005ab0 tx=0x7f6d8c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45edaed8d72ea268 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.079+0000 7f6d857fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:24:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.079+0000 7f6d857fa640 1 -- 172.21.15.94:0/620577287 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c035040 con 0x559f15871d40 2024-09-19T04:24:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6da637a740 1 -- 172.21.15.94:0/620577287 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d8c004c10 con 0x559f15871d40 2024-09-19T04:24:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6d857fa640 1 -- 172.21.15.94:0/620577287 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f6d8c004dc0 con 0x559f15871d40 2024-09-19T04:24:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6d857fa640 1 -- 172.21.15.94:0/620577287 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6d8c005480 con 0x559f15871d40 2024-09-19T04:24:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6da637a740 10 client.26171 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6da637a740 1 -- 172.21.15.94:0/620577287 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559f158eca80 con 0x559f15871d40 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6d857fa640 1 -- 172.21.15.94:0/620577287 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d8c004c10 con 0x559f15871d40 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6d857fa640 1 -- 172.21.15.94:0/620577287 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f6d8c060070 con 0x559f15871d40 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6da637a740 10 client.26171 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6da637a740 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6da637a740 20 client.26171 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6da637a740 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559f158ed190 0x559f158ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6da637a740 4 client.26171 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.080+0000 7f6da637a740 1 -- 172.21.15.94:0/620577287 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f157d3d00 con 0x559f158ed190 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.081+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559f158ed190 0x559f158ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.081+0000 7f6d857fa640 10 client.26171 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.081+0000 7f6da1d3d640 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559f158ed190 0x559f158ef5b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1034d5ae389e2092 server_cookie=f3ab7f96b1e0f2bb in_seq=0 out_seq=0 2024-09-19T04:24:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.082+0000 7f6d857fa640 1 -- 172.21.15.94:0/620577287 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f157d3d00 con 0x559f158ed190 2024-09-19T04:24:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.082+0000 7f6d857fa640 10 client.26171 handle_command_reply: tid=0 2024-09-19T04:24:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.082+0000 7f6da637a740 1 client.26171 shutdown 2024-09-19T04:24:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.082+0000 7f6da637a740 1 -- 172.21.15.94:0/620577287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 msgr2=0x559f15876740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.082+0000 7f6da637a740 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15876740 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c005ab0 tx=0x7f6d8c004610 comp rx=0 tx=0).stop 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.082+0000 7f6da637a740 1 -- 172.21.15.94:0/620577287 shutdown_connections 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.082+0000 7f6da637a740 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559f158ed190 0x559f158ef5b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.082+0000 7f6da637a740 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f158818d0 0x559f158eb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.083+0000 7f6da637a740 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559f15871d40 0x559f15876740 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c005ab0 tx=0x7f6d8c004610 comp rx=0 tx=0).stop 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.083+0000 7f6da637a740 1 --2- 172.21.15.94:0/620577287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f15866b60 0x559f15876200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.083+0000 7f6da637a740 1 -- 172.21.15.94:0/620577287 >> 172.21.15.94:0/620577287 conn(0x559f158630f0 msgr2=0x559f1586d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.083+0000 7f6d857fa640 0 client.26171 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.083+0000 7f6da637a740 1 -- 172.21.15.94:0/620577287 shutdown_connections 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.083+0000 7f6da637a740 1 -- 172.21.15.94:0/620577287 wait complete. 2024-09-19T04:24:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.084+0000 7f6da637a740 20 client.26171 trim_cache size 0 max 16384 2024-09-19T04:24:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:24:35.087 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:24:35.087 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "afe570b5-73d8-40fd-8eae-b69a4758ec0d", 2024-09-19T04:24:35.087 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:24:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:24:35.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 -- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6d880a3400 msgr2=0x7f6d880a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:35.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6d880a3400 0x7f6d880a58c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d98011490 tx=0x7f6d98011890 comp rx=0 tx=0).stop 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 -- 172.21.15.94:0/3770370861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c1082a0 msgr2=0x7f6d9c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c1082a0 0x7f6d9c19efd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400e940 tx=0x7f6d9400ee00 comp rx=0 tx=0).stop 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 -- 172.21.15.94:0/3770370861 shutdown_connections 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6d880a3400 0x7f6d880a58c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c1082a0 0x7f6d9c19efd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9c1036e0 0x7f6d9c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 --2- 172.21.15.94:0/3770370861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d9c102d30 0x7f6d9c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 -- 172.21.15.94:0/3770370861 >> 172.21.15.94:0/3770370861 conn(0x7f6d9c0fc7d0 msgr2=0x7f6d9c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 -- 172.21.15.94:0/3770370861 shutdown_connections 2024-09-19T04:24:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:35.085+0000 7f6da3fa1640 1 -- 172.21.15.94:0/3770370861 wait complete. 2024-09-19T04:24:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:34 smithi104 ceph-mon[34145]: pgmap v295: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 14 MiB/s wr, 1.65k op/s 2024-09-19T04:24:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 500.0 s, 1450.2 tps, lat 69.009 ms stddev 29.873 2024-09-19T04:24:35.824 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:24:36.226 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:36.653 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:36 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:24:36.653 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:36 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:36.653 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:36 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:24:36.653 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:36 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:24:36.653 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:36 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:24:36.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:36 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:36.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:36 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:36.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:36 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:36.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:36 smithi094 ceph-mon[30972]: pgmap v296: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 9.3 MiB/s wr, 1.08k op/s 2024-09-19T04:24:36.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.834+0000 7f816959a640 1 -- 172.21.15.94:0/3564611989 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160030820 con 0x7f816c100a80 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.835+0000 7f817083d640 1 -- 172.21.15.94:0/3564611989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c100a80 msgr2=0x7f816c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.835+0000 7f817083d640 1 --2- 172.21.15.94:0/3564611989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c100a80 0x7f816c100ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8160009890 tx=0x7f816002a360 comp rx=0 tx=0).stop 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.835+0000 7f817083d640 1 -- 172.21.15.94:0/3564611989 shutdown_connections 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.835+0000 7f817083d640 1 --2- 172.21.15.94:0/3564611989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101420 0x7f816c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.835+0000 7f817083d640 1 --2- 172.21.15.94:0/3564611989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c100a80 0x7f816c100ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.835+0000 7f817083d640 1 --2- 172.21.15.94:0/3564611989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f816c106a80 0x7f816c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.835+0000 7f817083d640 1 -- 172.21.15.94:0/3564611989 >> 172.21.15.94:0/3564611989 conn(0x7f816c0fc7a0 msgr2=0x7f816c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.835+0000 7f817083d640 1 -- 172.21.15.94:0/3564611989 shutdown_connections 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.835+0000 7f817083d640 1 -- 172.21.15.94:0/3564611989 wait complete. 2024-09-19T04:24:36.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.836+0000 7f817083d640 1 Processor -- start 2024-09-19T04:24:36.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.836+0000 7f817083d640 1 -- start start 2024-09-19T04:24:36.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.836+0000 7f817083d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f816c100a80 0x7f816c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f817083d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101420 0x7f816c1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f817083d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c106a80 0x7f816c19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f817083d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f816c110d40 con 0x7f816c101420 2024-09-19T04:24:36.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f817083d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f816c110bc0 con 0x7f816c100a80 2024-09-19T04:24:36.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f817083d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f816c110ec0 con 0x7f816c106a80 2024-09-19T04:24:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f816ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c106a80 0x7f816c19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f816ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c106a80 0x7f816c19bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:33438/0 (socket says 172.21.15.94:33438) 2024-09-19T04:24:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f816ad9d640 1 -- 172.21.15.94:0/818963023 learned_addr learned my addr 172.21.15.94:0/818963023 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f8169d9b640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101420 0x7f816c1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f816ad9d640 1 -- 172.21.15.94:0/818963023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f816c100a80 msgr2=0x7f816c197290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:24:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f816ad9d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f816c100a80 0x7f816c197290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f816ad9d640 1 -- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101420 msgr2=0x7f816c1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f816ad9d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101420 0x7f816c1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.837+0000 7f816ad9d640 1 -- 172.21.15.94:0/818963023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c19c260 con 0x7f816c106a80 2024-09-19T04:24:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.838+0000 7f8169d9b640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101420 0x7f816c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.838+0000 7f816ad9d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c106a80 0x7f816c19bb60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f816400eb10 tx=0x7f816400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.838+0000 7f815b7fe640 1 -- 172.21.15.94:0/818963023 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164004b20 con 0x7f816c106a80 2024-09-19T04:24:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.838+0000 7f817083d640 1 -- 172.21.15.94:0/818963023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c19c550 con 0x7f816c106a80 2024-09-19T04:24:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.838+0000 7f815b7fe640 1 -- 172.21.15.94:0/818963023 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8164004cc0 con 0x7f816c106a80 2024-09-19T04:24:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.838+0000 7f815b7fe640 1 -- 172.21.15.94:0/818963023 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164010690 con 0x7f816c106a80 2024-09-19T04:24:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.838+0000 7f817083d640 1 -- 172.21.15.94:0/818963023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c06a790 con 0x7f816c106a80 2024-09-19T04:24:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.842+0000 7f815b7fe640 1 -- 172.21.15.94:0/818963023 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f8164010830 con 0x7f816c106a80 2024-09-19T04:24:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.842+0000 7f8172c16740 1 Processor -- start 2024-09-19T04:24:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.842+0000 7f8172c16740 1 -- start start 2024-09-19T04:24:36.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f8172c16740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c26640 0x55ab68c26a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f8172c16740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c21010 0x55ab68c21490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f8172c16740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c219d0 0x55ab68c2d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f8172c16740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ab68b7d2e0 con 0x55ab68c219d0 2024-09-19T04:24:36.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f8172c16740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab68b24ce0 con 0x55ab68c21010 2024-09-19T04:24:36.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f8172c16740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab68b792a0 con 0x55ab68c26640 2024-09-19T04:24:36.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f815b7fe640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f81540a3430 0x7f81540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c219d0 0x55ab68c2d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f816a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c21010 0x55ab68c21490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c26640 0x55ab68c26a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f816a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c21010 0x55ab68c21490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:53008/0 (socket says 172.21.15.94:53008) 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.843+0000 7f816a59c640 1 -- 172.21.15.94:0/1789228331 learned_addr learned my addr 172.21.15.94:0/1789228331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.844+0000 7f816ad9d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f81540a3430 0x7f81540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.844+0000 7f815b7fe640 1 -- 172.21.15.94:0/818963023 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f81640d0230 con 0x7f816c106a80 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.844+0000 7f8169d9b640 1 -- 172.21.15.94:0/1789228331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c26640 msgr2=0x55ab68c26a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.844+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1789228331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c26640 0x55ab68c26a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.844+0000 7f8169d9b640 1 -- 172.21.15.94:0/1789228331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c21010 msgr2=0x55ab68c21490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.844+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1789228331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c21010 0x55ab68c21490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.844+0000 7f8169d9b640 1 -- 172.21.15.94:0/1789228331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab68b09440 con 0x55ab68c219d0 2024-09-19T04:24:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.845+0000 7f816ad9d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f81540a3430 0x7f81540a58f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81640049d0 tx=0x7f8164017000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.845+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1789228331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c219d0 0x55ab68c2d440 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8160035040 tx=0x7f8160000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b81d73404a880f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.845+0000 7f8158ff9640 1 -- 172.21.15.94:0/1789228331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160045070 con 0x55ab68c219d0 2024-09-19T04:24:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.845+0000 7f8158ff9640 1 -- 172.21.15.94:0/1789228331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8160030e70 con 0x55ab68c219d0 2024-09-19T04:24:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.846+0000 7f8158ff9640 1 -- 172.21.15.94:0/1789228331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160037040 con 0x55ab68c219d0 2024-09-19T04:24:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.846+0000 7f8172c16740 1 -- 172.21.15.94:0/1789228331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c219d0 msgr2=0x55ab68c2d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.846+0000 7f8172c16740 1 --2- 172.21.15.94:0/1789228331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c219d0 0x55ab68c2d440 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8160035040 tx=0x7f8160000bc0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.847+0000 7f8172c16740 1 -- 172.21.15.94:0/1789228331 shutdown_connections 2024-09-19T04:24:36.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.847+0000 7f8172c16740 1 --2- 172.21.15.94:0/1789228331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c219d0 0x55ab68c2d440 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.847+0000 7f8172c16740 1 --2- 172.21.15.94:0/1789228331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c21010 0x55ab68c21490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.847+0000 7f8172c16740 1 --2- 172.21.15.94:0/1789228331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c26640 0x55ab68c26a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.847+0000 7f8172c16740 1 -- 172.21.15.94:0/1789228331 >> 172.21.15.94:0/1789228331 conn(0x55ab68c13500 msgr2=0x55ab68c138e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.847+0000 7f8172c16740 1 -- 172.21.15.94:0/1789228331 shutdown_connections 2024-09-19T04:24:36.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.847+0000 7f8172c16740 1 -- 172.21.15.94:0/1789228331 wait complete. 2024-09-19T04:24:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.848+0000 7f8172c16740 1 -- start start 2024-09-19T04:24:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.848+0000 7f8172c16740 1 Processor -- start 2024-09-19T04:24:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.849+0000 7f8172c16740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c2a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.849+0000 7f8172c16740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 0x55ab68c2ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.849+0000 7f8172c16740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c2b220 0x55ab68c36f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.849+0000 7f8172c16740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ab68c2da90 con 0x55ab68c21010 2024-09-19T04:24:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.849+0000 7f8172c16740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab68b38050 con 0x55ab68c26640 2024-09-19T04:24:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.849+0000 7f8172c16740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab68c2dc10 con 0x55ab68c2b220 2024-09-19T04:24:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.849+0000 7f8172c16740 10 client.0 _ll_register_callbacks cb 0x55ab68b78160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.849+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c2a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c2a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44786/0 (socket says 172.21.15.94:44786) 2024-09-19T04:24:36.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f8169d9b640 1 -- 172.21.15.94:0/2703511393 learned_addr learned my addr 172.21.15.94:0/2703511393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f816a59c640 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 0x55ab68c2ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f8169d9b640 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c2b220 0x55ab68c36f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f8169d9b640 1 -- 172.21.15.94:0/2703511393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c2b220 msgr2=0x55ab68c36f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f8169d9b640 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c2b220 0x55ab68c36f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f8169d9b640 1 -- 172.21.15.94:0/2703511393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 msgr2=0x55ab68c2ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f8169d9b640 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 0x55ab68c2ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f8169d9b640 1 -- 172.21.15.94:0/2703511393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab688ee3c0 con 0x55ab68c21010 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.850+0000 7f816a59c640 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 0x55ab68c2ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.851+0000 7f8169d9b640 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c2a7a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8160005e00 tx=0x7f8160030760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f77feeda023ed24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.851+0000 7f8151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.851+0000 7f8151ffb640 1 -- 172.21.15.94:0/2703511393 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160045040 con 0x55ab68c21010 2024-09-19T04:24:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.851+0000 7f8151ffb640 1 -- 172.21.15.94:0/2703511393 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f8160004230 con 0x55ab68c21010 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.851+0000 7f8172c16740 1 -- 172.21.15.94:0/2703511393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab68b8a850 con 0x55ab68c21010 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.851+0000 7f8151ffb640 1 -- 172.21.15.94:0/2703511393 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8160037040 con 0x55ab68c21010 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.851+0000 7f8172c16740 10 client.16578 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.851+0000 7f8172c16740 1 -- 172.21.15.94:0/2703511393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ab68c39a70 con 0x55ab68c21010 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.852+0000 7f8151ffb640 1 -- 172.21.15.94:0/2703511393 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ab68b8a850 con 0x55ab68c21010 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.852+0000 7f8151ffb640 1 -- 172.21.15.94:0/2703511393 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f816003d040 con 0x55ab68c21010 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.852+0000 7f8172c16740 10 client.16578 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.852+0000 7f8172c16740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.852+0000 7f8172c16740 20 client.16578 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.852+0000 7f8172c16740 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ab68c3adc0 0x55ab68c3d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.852+0000 7f8172c16740 4 client.16578 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:36.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.852+0000 7f8172c16740 1 -- 172.21.15.94:0/2703511393 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab68b84d60 con 0x55ab68c3adc0 2024-09-19T04:24:36.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.853+0000 7f816a59c640 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ab68c3adc0 0x55ab68c3d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.853+0000 7f816a59c640 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ab68c3adc0 0x55ab68c3d1b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e253f5cee311737b server_cookie=671a199828a5dff0 in_seq=0 out_seq=0 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.853+0000 7f8151ffb640 10 client.16578 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.854+0000 7f8151ffb640 1 -- 172.21.15.94:0/2703511393 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab68b84d60 con 0x55ab68c3adc0 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.854+0000 7f8151ffb640 10 client.16578 handle_command_reply: tid=0 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.854+0000 7f8172c16740 1 client.16578 shutdown 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.854+0000 7f8172c16740 1 -- 172.21.15.94:0/2703511393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 msgr2=0x55ab68c2a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.854+0000 7f8172c16740 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c2a7a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8160005e00 tx=0x7f8160030760 comp rx=0 tx=0).stop 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8172c16740 1 -- 172.21.15.94:0/2703511393 shutdown_connections 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8172c16740 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ab68c3adc0 0x55ab68c3d1b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8172c16740 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c2b220 0x55ab68c36f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8172c16740 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 0x55ab68c2ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8172c16740 1 --2- 172.21.15.94:0/2703511393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c2a7a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8172c16740 1 -- 172.21.15.94:0/2703511393 >> 172.21.15.94:0/2703511393 conn(0x55ab68c13500 msgr2=0x55ab68c20820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8151ffb640 0 client.16578 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:36.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8172c16740 1 -- 172.21.15.94:0/2703511393 shutdown_connections 2024-09-19T04:24:36.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8172c16740 1 -- 172.21.15.94:0/2703511393 wait complete. 2024-09-19T04:24:36.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.855+0000 7f8172c16740 20 client.16578 trim_cache size 0 max 16384 2024-09-19T04:24:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.884+0000 7f8172c16740 1 Processor -- start 2024-09-19T04:24:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.884+0000 7f8172c16740 1 -- start start 2024-09-19T04:24:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.884+0000 7f8172c16740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c13500 0x55ab68c28340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.884+0000 7f8172c16740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c21010 0x55ab68c36ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.884+0000 7f8172c16740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c26640 0x55ab68c373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8172c16740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f814c0110d0 con 0x55ab68c26640 2024-09-19T04:24:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8172c16740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f814c00e970 con 0x55ab68c13500 2024-09-19T04:24:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8172c16740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f814c00c7e0 con 0x55ab68c21010 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f816a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c21010 0x55ab68c36ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c13500 0x55ab68c28340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c13500 0x55ab68c28340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:53028/0 (socket says 172.21.15.94:53028) 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 -- 172.21.15.94:0/645309636 learned_addr learned my addr 172.21.15.94:0/645309636 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 --2- 172.21.15.94:0/645309636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c26640 0x55ab68c373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 -- 172.21.15.94:0/645309636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c21010 msgr2=0x55ab68c36ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 --2- 172.21.15.94:0/645309636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c21010 0x55ab68c36ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 -- 172.21.15.94:0/645309636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c13500 msgr2=0x55ab68c28340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 --2- 172.21.15.94:0/645309636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c13500 0x55ab68c28340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 -- 172.21.15.94:0/645309636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab68c28d10 con 0x55ab68c26640 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.885+0000 7f8169d9b640 1 --2- 172.21.15.94:0/645309636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c26640 0x55ab68c373e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8160030a50 tx=0x7f8160032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd7d08344185213 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.886+0000 7f8153fff640 1 -- 172.21.15.94:0/645309636 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160045030 con 0x55ab68c26640 2024-09-19T04:24:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.886+0000 7f8153fff640 1 -- 172.21.15.94:0/645309636 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81600054f0 con 0x55ab68c26640 2024-09-19T04:24:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.886+0000 7f8153fff640 1 -- 172.21.15.94:0/645309636 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160004be0 con 0x55ab68c26640 2024-09-19T04:24:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 -- 172.21.15.94:0/645309636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c26640 msgr2=0x55ab68c373e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 --2- 172.21.15.94:0/645309636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c26640 0x55ab68c373e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8160030a50 tx=0x7f8160032d30 comp rx=0 tx=0).stop 2024-09-19T04:24:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 -- 172.21.15.94:0/645309636 shutdown_connections 2024-09-19T04:24:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 --2- 172.21.15.94:0/645309636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c26640 0x55ab68c373e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 --2- 172.21.15.94:0/645309636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c21010 0x55ab68c36ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 --2- 172.21.15.94:0/645309636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c13500 0x55ab68c28340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 -- 172.21.15.94:0/645309636 >> 172.21.15.94:0/645309636 conn(0x55ab68c2ae40 msgr2=0x55ab68c29090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 -- 172.21.15.94:0/645309636 shutdown_connections 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 -- 172.21.15.94:0/645309636 wait complete. 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 -- start start 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.887+0000 7f8172c16740 1 Processor -- start 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f8172c16740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c13500 0x55ab68c3bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f8172c16740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c3c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f8172c16740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 0x55ab68c3c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f8172c16740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f814c00ce30 con 0x55ab68c21010 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f8172c16740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f814c00e2a0 con 0x55ab68c26640 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f8172c16740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f814c00d500 con 0x55ab68c13500 2024-09-19T04:24:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f8172c16740 10 client.0 _ll_register_callbacks cb 0x7f8160043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f816a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c3c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c13500 0x55ab68c3bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f816a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c3c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44792/0 (socket says 172.21.15.94:44792) 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f816a59c640 1 -- 172.21.15.94:0/1914927663 learned_addr learned my addr 172.21.15.94:0/1914927663 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.888+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 0x55ab68c3c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8169d9b640 1 -- 172.21.15.94:0/1914927663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 msgr2=0x55ab68c3c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 0x55ab68c3c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8169d9b640 1 -- 172.21.15.94:0/1914927663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 msgr2=0x55ab68c3c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c3c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8169d9b640 1 -- 172.21.15.94:0/1914927663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab68ca4890 con 0x55ab68c13500 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f816a59c640 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c3c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c13500 0x55ab68c3bd00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55ab68c17760 tx=0x7f8160032df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d608b32e537748 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8151ffb640 1 -- 172.21.15.94:0/1914927663 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160045030 con 0x55ab68c13500 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8151ffb640 1 -- 172.21.15.94:0/1914927663 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f8160005240 con 0x55ab68c13500 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8172c16740 1 -- 172.21.15.94:0/1914927663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81600054f0 con 0x55ab68c13500 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8151ffb640 1 -- 172.21.15.94:0/1914927663 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8160032a00 con 0x55ab68c13500 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8172c16740 10 client.26146 fetch_fsmap learned FSMap version 43 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.889+0000 7f8172c16740 1 -- 172.21.15.94:0/1914927663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ab68ca5b80 con 0x55ab68c13500 2024-09-19T04:24:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.890+0000 7f8151ffb640 1 -- 172.21.15.94:0/1914927663 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8160004d10 con 0x55ab68c13500 2024-09-19T04:24:36.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.890+0000 7f8151ffb640 1 -- 172.21.15.94:0/1914927663 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f816003d990 con 0x55ab68c13500 2024-09-19T04:24:36.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.890+0000 7f8172c16740 10 client.26146 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:24:36.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.890+0000 7f8172c16740 10 client.26146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:36.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.890+0000 7f8172c16740 20 client.26146 populate_metadata read hostname 'smithi094' 2024-09-19T04:24:36.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.890+0000 7f8172c16740 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ab68c2b220 0x55ab68ca81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.890+0000 7f8172c16740 4 client.26146 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:24:36.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.890+0000 7f8172c16740 1 -- 172.21.15.94:0/1914927663 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f81600054f0 con 0x55ab68c2b220 2024-09-19T04:24:36.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.890+0000 7f816a59c640 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ab68c2b220 0x55ab68ca81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.891+0000 7f8151ffb640 10 client.26146 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.891+0000 7f816a59c640 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ab68c2b220 0x55ab68ca81b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f49714f603733b2 server_cookie=79ac87b1f5511481 in_seq=0 out_seq=0 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.891+0000 7f8151ffb640 1 -- 172.21.15.94:0/1914927663 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f81600054f0 con 0x55ab68c2b220 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.891+0000 7f8151ffb640 10 client.26146 handle_command_reply: tid=0 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.891+0000 7f8172c16740 1 client.26146 shutdown 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.891+0000 7f8172c16740 1 -- 172.21.15.94:0/1914927663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c13500 msgr2=0x55ab68c3bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.891+0000 7f8172c16740 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c13500 0x55ab68c3bd00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55ab68c17760 tx=0x7f8160032df0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.891+0000 7f8172c16740 1 -- 172.21.15.94:0/1914927663 shutdown_connections 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.892+0000 7f8172c16740 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ab68c2b220 0x55ab68ca81b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.892+0000 7f8172c16740 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab68c26640 0x55ab68c3c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.892+0000 7f8172c16740 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab68c21010 0x55ab68c3c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.892+0000 7f8172c16740 1 --2- 172.21.15.94:0/1914927663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab68c13500 0x55ab68c3bd00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.892+0000 7f8151ffb640 0 client.26146 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.892+0000 7f8172c16740 1 -- 172.21.15.94:0/1914927663 >> 172.21.15.94:0/1914927663 conn(0x55ab68c2ae40 msgr2=0x55ab68c12350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.892+0000 7f8172c16740 1 -- 172.21.15.94:0/1914927663 shutdown_connections 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.892+0000 7f8172c16740 1 -- 172.21.15.94:0/1914927663 wait complete. 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.892+0000 7f8172c16740 20 client.26146 trim_cache size 0 max 16384 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stdout: "afe570b5-73d8-40fd-8eae-b69a4758ec0d": { 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:24:36.898 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "afe570b5-73d8-40fd-8eae-b69a4758ec0d", 2024-09-19T04:24:36.899 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:24:36.899 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:24:36.899 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:24:36.899 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 -- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f81540a3430 msgr2=0x7f81540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f81540a3430 0x7f81540a58f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81640049d0 tx=0x7f8164017000 comp rx=0 tx=0).stop 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 -- 172.21.15.94:0/818963023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c106a80 msgr2=0x7f816c19bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c106a80 0x7f816c19bb60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f816400eb10 tx=0x7f816400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 -- 172.21.15.94:0/818963023 shutdown_connections 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f81540a3430 0x7f81540a58f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c106a80 0x7f816c19bb60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101420 0x7f816c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 --2- 172.21.15.94:0/818963023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f816c100a80 0x7f816c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 -- 172.21.15.94:0/818963023 >> 172.21.15.94:0/818963023 conn(0x7f816c0fc7a0 msgr2=0x7f816c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 -- 172.21.15.94:0/818963023 shutdown_connections 2024-09-19T04:24:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:24:36.893+0000 7f817083d640 1 -- 172.21.15.94:0/818963023 wait complete. 2024-09-19T04:24:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:36 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:24:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:36 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:36 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:24:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:36 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:24:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:36 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:24:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:36 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:36 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:36 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:36 smithi112 ceph-mon[35051]: pgmap v296: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 9.3 MiB/s wr, 1.08k op/s 2024-09-19T04:24:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:36 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:24:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:36 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:36 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:24:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:36 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:24:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:36 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:24:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:36 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:36 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:36 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:36 smithi104 ceph-mon[34145]: pgmap v296: 97 pgs: 97 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 9.3 MiB/s wr, 1.08k op/s 2024-09-19T04:24:37.642 INFO:tasks.cephfs.filesystem:scrub status for tag:afe570b5-73d8-40fd-8eae-b69a4758ec0d - {'path': '/', 'tag': 'afe570b5-73d8-40fd-8eae-b69a4758ec0d', 'options': 'recursive,force'} 2024-09-19T04:24:39.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:38 smithi112 ceph-mon[35051]: pgmap v297: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 14 MiB/s wr, 1.68k op/s 2024-09-19T04:24:39.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:38 smithi094 ceph-mon[30972]: pgmap v297: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 14 MiB/s wr, 1.68k op/s 2024-09-19T04:24:39.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:38 smithi104 ceph-mon[34145]: pgmap v297: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 14 MiB/s wr, 1.68k op/s 2024-09-19T04:24:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 505.0 s, 1482.4 tps, lat 67.246 ms stddev 22.331 2024-09-19T04:24:41.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:40 smithi112 ceph-mon[35051]: pgmap v298: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.9 MiB/s wr, 1.16k op/s 2024-09-19T04:24:41.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:40 smithi094 ceph-mon[30972]: pgmap v298: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.9 MiB/s wr, 1.16k op/s 2024-09-19T04:24:41.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:40 smithi104 ceph-mon[34145]: pgmap v298: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.9 MiB/s wr, 1.16k op/s 2024-09-19T04:24:43.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:42 smithi112 ceph-mon[35051]: pgmap v299: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.9 MiB/s wr, 1.16k op/s 2024-09-19T04:24:43.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:42 smithi094 ceph-mon[30972]: pgmap v299: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.9 MiB/s wr, 1.16k op/s 2024-09-19T04:24:43.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:42 smithi104 ceph-mon[34145]: pgmap v299: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.9 MiB/s wr, 1.16k op/s 2024-09-19T04:24:45.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:44 smithi112 ceph-mon[35051]: pgmap v300: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 16 MiB/s wr, 1.78k op/s 2024-09-19T04:24:45.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:44 smithi094 ceph-mon[30972]: pgmap v300: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 16 MiB/s wr, 1.78k op/s 2024-09-19T04:24:45.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:44 smithi104 ceph-mon[34145]: pgmap v300: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 16 MiB/s wr, 1.78k op/s 2024-09-19T04:24:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 510.0 s, 1554.0 tps, lat 64.597 ms stddev 22.990 2024-09-19T04:24:47.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:46 smithi112 ceph-mon[35051]: pgmap v301: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-19T04:24:47.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:46 smithi094 ceph-mon[30972]: pgmap v301: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-19T04:24:47.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:46 smithi104 ceph-mon[34145]: pgmap v301: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-19T04:24:48.005 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:47 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:24:48.005 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:47 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:24:48.005 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:47 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:24:48.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:47 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:24:48.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:47 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:24:48.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:47 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:24:48.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:47 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:24:48.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:47 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:24:48.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:47 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:24:49.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:48 smithi112 ceph-mon[35051]: pgmap v302: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 1.83k op/s 2024-09-19T04:24:49.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:48 smithi094 ceph-mon[30972]: pgmap v302: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 1.83k op/s 2024-09-19T04:24:49.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:48 smithi104 ceph-mon[34145]: pgmap v302: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 1.83k op/s 2024-09-19T04:24:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 515.0 s, 1388.4 tps, lat 71.939 ms stddev 33.963 2024-09-19T04:24:51.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:50 smithi112 ceph-mon[35051]: pgmap v303: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:24:51.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:50 smithi094 ceph-mon[30972]: pgmap v303: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:24:51.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:50 smithi104 ceph-mon[34145]: pgmap v303: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:24:53.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:52 smithi112 ceph-mon[35051]: pgmap v304: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:24:53.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:52 smithi094 ceph-mon[30972]: pgmap v304: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:24:53.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:52 smithi104 ceph-mon[34145]: pgmap v304: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T04:24:55.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:54 smithi112 ceph-mon[35051]: pgmap v305: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 16 MiB/s wr, 1.80k op/s 2024-09-19T04:24:55.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:54 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:24:55.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:54 smithi094 ceph-mon[30972]: pgmap v305: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 16 MiB/s wr, 1.80k op/s 2024-09-19T04:24:55.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:54 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:24:55.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:54 smithi104 ceph-mon[34145]: pgmap v305: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 16 MiB/s wr, 1.80k op/s 2024-09-19T04:24:55.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:54 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:24:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 520.0 s, 1338.0 tps, lat 74.688 ms stddev 22.444 2024-09-19T04:24:57.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:56 smithi112 ceph-mon[35051]: pgmap v306: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-09-19T04:24:57.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:56 smithi094 ceph-mon[30972]: pgmap v306: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-09-19T04:24:57.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:56 smithi104 ceph-mon[34145]: pgmap v306: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-09-19T04:24:59.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:24:58 smithi112 ceph-mon[35051]: pgmap v307: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-09-19T04:24:59.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:24:58 smithi094 ceph-mon[30972]: pgmap v307: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-09-19T04:24:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:24:58 smithi104 ceph-mon[34145]: pgmap v307: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-09-19T04:25:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 525.0 s, 1386.4 tps, lat 72.146 ms stddev 21.846 2024-09-19T04:25:01.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:00 smithi112 ceph-mon[35051]: pgmap v308: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.4 MiB/s wr, 1.08k op/s 2024-09-19T04:25:01.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:00 smithi094 ceph-mon[30972]: pgmap v308: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.4 MiB/s wr, 1.08k op/s 2024-09-19T04:25:01.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:00 smithi104 ceph-mon[34145]: pgmap v308: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.4 MiB/s wr, 1.08k op/s 2024-09-19T04:25:03.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:02 smithi112 ceph-mon[35051]: pgmap v309: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.4 MiB/s wr, 1.08k op/s 2024-09-19T04:25:03.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:02 smithi094 ceph-mon[30972]: pgmap v309: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.4 MiB/s wr, 1.08k op/s 2024-09-19T04:25:03.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:02 smithi104 ceph-mon[34145]: pgmap v309: 97 pgs: 97 active+clean; 3.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.4 MiB/s wr, 1.08k op/s 2024-09-19T04:25:05.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:04 smithi112 ceph-mon[35051]: pgmap v310: 97 pgs: 97 active+clean; 4.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-09-19T04:25:05.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:04 smithi094 ceph-mon[30972]: pgmap v310: 97 pgs: 97 active+clean; 4.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-09-19T04:25:05.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:04 smithi104 ceph-mon[34145]: pgmap v310: 97 pgs: 97 active+clean; 4.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-09-19T04:25:05.603 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 530.0 s, 1472.8 tps, lat 67.851 ms stddev 28.267 2024-09-19T04:25:07.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:06 smithi112 ceph-mon[35051]: pgmap v311: 97 pgs: 97 active+clean; 4.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.5 MiB/s wr, 1.13k op/s 2024-09-19T04:25:07.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:06 smithi094 ceph-mon[30972]: pgmap v311: 97 pgs: 97 active+clean; 4.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.5 MiB/s wr, 1.13k op/s 2024-09-19T04:25:07.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:06 smithi104 ceph-mon[34145]: pgmap v311: 97 pgs: 97 active+clean; 4.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.5 MiB/s wr, 1.13k op/s 2024-09-19T04:25:07.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:25:08.176 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:08.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.768+0000 7fc6b6cae640 1 -- 172.21.15.94:0/2411098011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6a8093c60 msgr2=0x7fc6a8094040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:08.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.768+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/2411098011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6a8093c60 0x7fc6a8094040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009900 tx=0x7fc6ac02a410 comp rx=0 tx=0).stop 2024-09-19T04:25:08.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.769+0000 7fc6b6cae640 1 -- 172.21.15.94:0/2411098011 shutdown_connections 2024-09-19T04:25:08.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.769+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/2411098011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a809fdb0 0x7fc6a80a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.769+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/2411098011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a8094580 0x7fc6a809f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.769+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/2411098011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6a8093c60 0x7fc6a8094040 secure :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009900 tx=0x7fc6ac02a410 comp rx=0 tx=0).stop 2024-09-19T04:25:08.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.769+0000 7fc6b6cae640 1 -- 172.21.15.94:0/2411098011 >> 172.21.15.94:0/2411098011 conn(0x7fc6a8009520 msgr2=0x7fc6a8009930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:08.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.769+0000 7fc6b6cae640 1 -- 172.21.15.94:0/2411098011 shutdown_connections 2024-09-19T04:25:08.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.769+0000 7fc6b6cae640 1 -- 172.21.15.94:0/2411098011 wait complete. 2024-09-19T04:25:08.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.770+0000 7fc6b6cae640 1 Processor -- start 2024-09-19T04:25:08.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.770+0000 7fc6b6cae640 1 -- start start 2024-09-19T04:25:08.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.771+0000 7fc6b6cae640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6a8094580 0x7fc6a812bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.771+0000 7fc6b6cae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a809fdb0 0x7fc6a812c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.771+0000 7fc6b5cac640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6a8094580 0x7fc6a812bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.771+0000 7fc6b5cac640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6a8094580 0x7fc6a812bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:57638/0 (socket says 172.21.15.94:57638) 2024-09-19T04:25:08.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.771+0000 7fc6b5cac640 1 -- 172.21.15.94:0/335483159 learned_addr learned my addr 172.21.15.94:0/335483159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:08.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.771+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a8130800 0x7fc6a8130cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.772+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a809fdb0 0x7fc6a812c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.772+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a80a57f0 con 0x7fc6a8130800 2024-09-19T04:25:08.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.772+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a80a5670 con 0x7fc6a8094580 2024-09-19T04:25:08.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.772+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a80a5970 con 0x7fc6a809fdb0 2024-09-19T04:25:08.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.772+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a8130800 0x7fc6a8130cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.772+0000 7fc6b54ab640 1 -- 172.21.15.94:0/335483159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6a8094580 msgr2=0x7fc6a812bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6a8094580 0x7fc6a812bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6b54ab640 1 -- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a8130800 msgr2=0x7fc6a8130cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a8130800 0x7fc6a8130cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6b54ab640 1 -- 172.21.15.94:0/335483159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a81312c0 con 0x7fc6a809fdb0 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a809fdb0 0x7fc6a812c470 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009e60 tx=0x7fc6a000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6a6ffd640 1 -- 172.21.15.94:0/335483159 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0020070 con 0x7fc6a809fdb0 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6a6ffd640 1 -- 172.21.15.94:0/335483159 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6a000ccc0 con 0x7fc6a809fdb0 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6a6ffd640 1 -- 172.21.15.94:0/335483159 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a001bb10 con 0x7fc6a809fdb0 2024-09-19T04:25:08.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a81315b0 con 0x7fc6a809fdb0 2024-09-19T04:25:08.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.773+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a8095890 con 0x7fc6a809fdb0 2024-09-19T04:25:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.776+0000 7fc6a6ffd640 1 -- 172.21.15.94:0/335483159 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc6a0023070 con 0x7fc6a809fdb0 2024-09-19T04:25:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.777+0000 7fc6ba2e9740 1 Processor -- start 2024-09-19T04:25:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.778+0000 7fc6a6ffd640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc69c0a34c0 0x7fc69c0a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.778+0000 7fc6ba2e9740 1 -- start start 2024-09-19T04:25:08.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.778+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9670 0x56040e3f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.778+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9f90 0x56040e404310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.778+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e404850 0x56040e406c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.778+0000 7fc6ba2e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56040e35f2e0 con 0x56040e404850 2024-09-19T04:25:08.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.778+0000 7fc6ba2e9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56040e306ce0 con 0x56040e3f9f90 2024-09-19T04:25:08.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.778+0000 7fc6ba2e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56040e35b2a0 con 0x56040e3f9670 2024-09-19T04:25:08.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.778+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc69c0a34c0 0x7fc69c0a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b64ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e404850 0x56040e406c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b5cac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9670 0x56040e3f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b64ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e404850 0x56040e406c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35018/0 (socket says 172.21.15.94:35018) 2024-09-19T04:25:08.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b64ad640 1 -- 172.21.15.94:0/64057269 learned_addr learned my addr 172.21.15.94:0/64057269 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:08.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/64057269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9f90 0x56040e404310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6a6ffd640 1 -- 172.21.15.94:0/335483159 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7fc6a00d3ed0 con 0x7fc6a809fdb0 2024-09-19T04:25:08.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b64ad640 1 -- 172.21.15.94:0/64057269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9670 msgr2=0x56040e3f9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:08.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/64057269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9670 0x56040e3f9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b64ad640 1 -- 172.21.15.94:0/64057269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9f90 msgr2=0x56040e404310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:08.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/64057269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9f90 0x56040e404310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.779+0000 7fc6b64ad640 1 -- 172.21.15.94:0/64057269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56040e2eb440 con 0x56040e404850 2024-09-19T04:25:08.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.780+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc69c0a34c0 0x7fc69c0a5980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0065360 tx=0x7fc6b0068660 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:08.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.780+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/64057269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e404850 0x56040e406c40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00679f0 tx=0x7fc6b008c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19b1769f3d36a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:08.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.780+0000 7fc69bfff640 1 -- 172.21.15.94:0/64057269 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b007f810 con 0x56040e404850 2024-09-19T04:25:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.781+0000 7fc69bfff640 1 -- 172.21.15.94:0/64057269 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b0091020 con 0x56040e404850 2024-09-19T04:25:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.781+0000 7fc69bfff640 1 -- 172.21.15.94:0/64057269 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b008d9a0 con 0x56040e404850 2024-09-19T04:25:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.782+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/64057269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e404850 msgr2=0x56040e406c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.782+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/64057269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e404850 0x56040e406c40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00679f0 tx=0x7fc6b008c000 comp rx=0 tx=0).stop 2024-09-19T04:25:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.782+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/64057269 shutdown_connections 2024-09-19T04:25:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.782+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/64057269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e404850 0x56040e406c40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.782+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/64057269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9f90 0x56040e404310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.782+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/64057269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9670 0x56040e3f9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.782+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/64057269 >> 172.21.15.94:0/64057269 conn(0x56040e3f5c00 msgr2=0x56040e3f5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.783+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/64057269 shutdown_connections 2024-09-19T04:25:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.783+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/64057269 wait complete. 2024-09-19T04:25:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.783+0000 7fc6ba2e9740 1 -- start start 2024-09-19T04:25:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.783+0000 7fc6ba2e9740 1 Processor -- start 2024-09-19T04:25:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.784+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9670 0x56040e4041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.784+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9f90 0x56040e401450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.784+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e404850 0x56040e4019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.784+0000 7fc6ba2e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56040e407420 con 0x56040e3f9f90 2024-09-19T04:25:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.785+0000 7fc6ba2e9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56040e31a050 con 0x56040e3f9670 2024-09-19T04:25:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.785+0000 7fc6b5cac640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9670 0x56040e4041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.785+0000 7fc6b5cac640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9670 0x56040e4041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:57658/0 (socket says 172.21.15.94:57658) 2024-09-19T04:25:08.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.785+0000 7fc6b5cac640 1 -- 172.21.15.94:0/1344482807 learned_addr learned my addr 172.21.15.94:0/1344482807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:08.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.785+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/1344482807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56040e4075a0 con 0x56040e404850 2024-09-19T04:25:08.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.785+0000 7fc6ba2e9740 10 client.0 _ll_register_callbacks cb 0x56040e35a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.785+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9f90 0x56040e401450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.785+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e404850 0x56040e4019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.786+0000 7fc6b64ad640 1 -- 172.21.15.94:0/1344482807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9670 msgr2=0x56040e4041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.786+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9670 0x56040e4041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.786+0000 7fc6b64ad640 1 -- 172.21.15.94:0/1344482807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9f90 msgr2=0x56040e401450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.786+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9f90 0x56040e401450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.786+0000 7fc6b64ad640 1 -- 172.21.15.94:0/1344482807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56040e0d03c0 con 0x56040e404850 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.786+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e404850 0x56040e4019b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0068f20 tx=0x7fc6b0068eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80a8de7d9ae2b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.786+0000 7fc6997fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.786+0000 7fc6997fa640 1 -- 172.21.15.94:0/1344482807 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b008dab0 con 0x56040e404850 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.786+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/1344482807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6b0091020 con 0x56040e404850 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6997fa640 1 -- 172.21.15.94:0/1344482807 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7fc6b0092040 con 0x56040e404850 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6997fa640 1 -- 172.21.15.94:0/1344482807 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc6b00a1050 con 0x56040e404850 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6ba2e9740 10 client.26158 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/1344482807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56040e41b630 con 0x56040e404850 2024-09-19T04:25:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6997fa640 1 -- 172.21.15.94:0/1344482807 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b007fcc0 con 0x56040e404850 2024-09-19T04:25:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6997fa640 1 -- 172.21.15.94:0/1344482807 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc6b00957a0 con 0x56040e404850 2024-09-19T04:25:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6ba2e9740 10 client.26158 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6ba2e9740 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6ba2e9740 20 client.26158 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56040e41cdc0 0x56040e41f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.787+0000 7fc6ba2e9740 4 client.26158 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.788+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/1344482807 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc6b0091020 con 0x56040e41cdc0 2024-09-19T04:25:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.788+0000 7fc6b5cac640 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56040e41cdc0 0x56040e41f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.788+0000 7fc6997fa640 10 client.26158 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.788+0000 7fc6b5cac640 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56040e41cdc0 0x56040e41f1e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e5f2be07afb0ba server_cookie=dc63ccb9c69fce52 in_seq=0 out_seq=0 2024-09-19T04:25:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.789+0000 7fc6997fa640 1 -- 172.21.15.94:0/1344482807 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc6b0091020 con 0x56040e41cdc0 2024-09-19T04:25:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.789+0000 7fc6997fa640 10 client.26158 handle_command_reply: tid=0 2024-09-19T04:25:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.790+0000 7fc6ba2e9740 1 client.26158 shutdown 2024-09-19T04:25:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.790+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/1344482807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e404850 msgr2=0x56040e4019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.790+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e404850 0x56040e4019b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0068f20 tx=0x7fc6b0068eb0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.790+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/1344482807 shutdown_connections 2024-09-19T04:25:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.790+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56040e41cdc0 0x56040e41f1e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.790+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e404850 0x56040e4019b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.791+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9f90 0x56040e401450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.791+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/1344482807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e3f9670 0x56040e4041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.791+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/1344482807 >> 172.21.15.94:0/1344482807 conn(0x56040e3f5c00 msgr2=0x56040e3fb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.791+0000 7fc6997fa640 0 client.26158 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.791+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/1344482807 shutdown_connections 2024-09-19T04:25:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.791+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/1344482807 wait complete. 2024-09-19T04:25:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.791+0000 7fc6ba2e9740 20 client.26158 trim_cache size 0 max 16384 2024-09-19T04:25:08.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.820+0000 7fc6ba2e9740 1 Processor -- start 2024-09-19T04:25:08.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.821+0000 7fc6ba2e9740 1 -- start start 2024-09-19T04:25:08.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.821+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 0x56040e418f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.821+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e3fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.821+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e424af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.821+0000 7fc6b5cac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 0x56040e418f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.822+0000 7fc6b5cac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 0x56040e418f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35032/0 (socket says 172.21.15.94:35032) 2024-09-19T04:25:08.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.822+0000 7fc6b5cac640 1 -- 172.21.15.94:0/223720621 learned_addr learned my addr 172.21.15.94:0/223720621 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:08.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.822+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/223720621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc694011190 con 0x56040e3f9670 2024-09-19T04:25:08.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.822+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e424af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.822+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/223720621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc69400ea00 con 0x56040e404850 2024-09-19T04:25:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.822+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e3fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.822+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/223720621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc69400c870 con 0x56040e3f9f90 2024-09-19T04:25:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc6b64ad640 1 -- 172.21.15.94:0/223720621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 msgr2=0x56040e3fc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e3fc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc6b64ad640 1 -- 172.21.15.94:0/223720621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 msgr2=0x56040e418f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 0x56040e418f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc6b64ad640 1 -- 172.21.15.94:0/223720621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56040e404510 con 0x56040e404850 2024-09-19T04:25:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e3fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:25:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc6b64ad640 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e424af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0051220 tx=0x7fc6b0068300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f693f2b2fddc70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc69b7fe640 1 -- 172.21.15.94:0/223720621 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b008d630 con 0x56040e404850 2024-09-19T04:25:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc69b7fe640 1 -- 172.21.15.94:0/223720621 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b0091070 con 0x56040e404850 2024-09-19T04:25:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.823+0000 7fc69b7fe640 1 -- 172.21.15.94:0/223720621 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0095810 con 0x56040e404850 2024-09-19T04:25:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.824+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/223720621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 msgr2=0x56040e424af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.824+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e424af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0051220 tx=0x7fc6b0068300 comp rx=0 tx=0).stop 2024-09-19T04:25:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.825+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/223720621 shutdown_connections 2024-09-19T04:25:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.825+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e424af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.825+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e3fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.825+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/223720621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 0x56040e418f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.825+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/223720621 >> 172.21.15.94:0/223720621 conn(0x56040e3f5c00 msgr2=0x56040e3fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.825+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/223720621 shutdown_connections 2024-09-19T04:25:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.825+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/223720621 wait complete. 2024-09-19T04:25:08.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.825+0000 7fc6ba2e9740 1 -- start start 2024-09-19T04:25:08.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.826+0000 7fc6ba2e9740 1 Processor -- start 2024-09-19T04:25:08.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 0x56040e4107f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e410d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6ba2e9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e47fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6ba2e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc69400cec0 con 0x56040e3f9670 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6ba2e9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc69400e330 con 0x56040e404850 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6ba2e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc69400d590 con 0x56040e3f9f90 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6ba2e9740 10 client.0 _ll_register_callbacks cb 0x7fc6a009d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6b54ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e410d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6b64ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e47fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6b64ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e47fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:57678/0 (socket says 172.21.15.94:57678) 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6b64ad640 1 -- 172.21.15.94:0/2912685108 learned_addr learned my addr 172.21.15.94:0/2912685108 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.827+0000 7fc6b5cac640 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 0x56040e4107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.828+0000 7fc6b54ab640 1 -- 172.21.15.94:0/2912685108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 msgr2=0x56040e47fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.828+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e47fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.828+0000 7fc6b54ab640 1 -- 172.21.15.94:0/2912685108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 msgr2=0x56040e4107f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.828+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 0x56040e4107f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.828+0000 7fc6b54ab640 1 -- 172.21.15.94:0/2912685108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56040e480120 con 0x56040e3f9f90 2024-09-19T04:25:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.828+0000 7fc6b54ab640 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e410d30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00d1660 tx=0x7fc6a00d7440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295ea686e493e5c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.828+0000 7fc6997fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:25:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.828+0000 7fc6997fa640 1 -- 172.21.15.94:0/2912685108 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a001bcb0 con 0x56040e3f9f90 2024-09-19T04:25:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.828+0000 7fc6997fa640 1 -- 172.21.15.94:0/2912685108 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7fc6a009d780 con 0x56040e3f9f90 2024-09-19T04:25:08.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.829+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/2912685108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6b0091070 con 0x56040e3f9f90 2024-09-19T04:25:08.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6997fa640 1 -- 172.21.15.94:0/2912685108 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc6a00a1810 con 0x56040e3f9f90 2024-09-19T04:25:08.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6ba2e9740 10 client.26161 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:08.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/2912685108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56040e487450 con 0x56040e3f9f90 2024-09-19T04:25:08.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6997fa640 1 -- 172.21.15.94:0/2912685108 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6a00a1110 con 0x56040e3f9f90 2024-09-19T04:25:08.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6997fa640 1 -- 172.21.15.94:0/2912685108 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc6a009ce60 con 0x56040e3f9f90 2024-09-19T04:25:08.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6ba2e9740 10 client.26161 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:08.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6ba2e9740 10 client.26161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:08.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6ba2e9740 20 client.26161 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:08.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56040e487be0 0x56040e48a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:08.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6ba2e9740 4 client.26161 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:25:08.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.830+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/2912685108 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc6b0091070 con 0x56040e487be0 2024-09-19T04:25:08.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.831+0000 7fc6b5cac640 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56040e487be0 0x56040e48a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.832+0000 7fc6b5cac640 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56040e487be0 0x56040e48a000 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da7de1fb930a75ce server_cookie=138fbc1834d0d57 in_seq=0 out_seq=0 2024-09-19T04:25:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.832+0000 7fc6997fa640 10 client.26161 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.832+0000 7fc6997fa640 1 -- 172.21.15.94:0/2912685108 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc6b0091070 con 0x56040e487be0 2024-09-19T04:25:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.832+0000 7fc6997fa640 10 client.26161 handle_command_reply: tid=0 2024-09-19T04:25:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.832+0000 7fc6ba2e9740 1 client.26161 shutdown 2024-09-19T04:25:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.834+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/2912685108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 msgr2=0x56040e410d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.834+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e410d30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00d1660 tx=0x7fc6a00d7440 comp rx=0 tx=0).stop 2024-09-19T04:25:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.835+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/2912685108 shutdown_connections 2024-09-19T04:25:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.835+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56040e487be0 0x56040e48a000 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.835+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56040e404850 0x56040e47fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.835+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56040e3f9f90 0x56040e410d30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.835+0000 7fc6ba2e9740 1 --2- 172.21.15.94:0/2912685108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e3f9670 0x56040e4107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.835+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/2912685108 >> 172.21.15.94:0/2912685108 conn(0x56040e3f5c00 msgr2=0x56040e3ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:08.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.835+0000 7fc6997fa640 0 client.26161 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:08.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.835+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/2912685108 shutdown_connections 2024-09-19T04:25:08.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.835+0000 7fc6ba2e9740 1 -- 172.21.15.94:0/2912685108 wait complete. 2024-09-19T04:25:08.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.837+0000 7fc6ba2e9740 20 client.26161 trim_cache size 0 max 16384 2024-09-19T04:25:08.838 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:25:08.839 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:25:08.839 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:25:08.839 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:25:08.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.838+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc69c0a34c0 msgr2=0x7fc69c0a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:08.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.838+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc69c0a34c0 0x7fc69c0a5980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0065360 tx=0x7fc6b0068660 comp rx=0 tx=0).stop 2024-09-19T04:25:08.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.838+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a809fdb0 msgr2=0x7fc6a812c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:08.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.838+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a809fdb0 0x7fc6a812c470 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009e60 tx=0x7fc6a000edc0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.839+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 shutdown_connections 2024-09-19T04:25:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.839+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc69c0a34c0 0x7fc69c0a5980 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.839+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a8130800 0x7fc6a8130cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.839+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a809fdb0 0x7fc6a812c470 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.839+0000 7fc6b6cae640 1 --2- 172.21.15.94:0/335483159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6a8094580 0x7fc6a812bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.839+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 >> 172.21.15.94:0/335483159 conn(0x7fc6a8009520 msgr2=0x7fc6a8090f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.839+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 shutdown_connections 2024-09-19T04:25:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:08.839+0000 7fc6b6cae640 1 -- 172.21.15.94:0/335483159 wait complete. 2024-09-19T04:25:08.859 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:08 smithi094 ceph-mon[30972]: pgmap v312: 97 pgs: 97 active+clean; 4.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.74k op/s 2024-09-19T04:25:08.859 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:08 smithi094 ceph-mon[30972]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:25:09.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:08 smithi112 ceph-mon[35051]: pgmap v312: 97 pgs: 97 active+clean; 4.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.74k op/s 2024-09-19T04:25:09.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:08 smithi112 ceph-mon[35051]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:25:09.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:08 smithi104 ceph-mon[34145]: pgmap v312: 97 pgs: 97 active+clean; 4.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.74k op/s 2024-09-19T04:25:09.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:08 smithi104 ceph-mon[34145]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:25:09.776 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:25:09.776 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:25:10.205 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 535.0 s, 1174.6 tps, lat 84.859 ms stddev 46.348 2024-09-19T04:25:10.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.787+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1618316900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8095c70 msgr2=0x7fe8c8096050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:10.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.787+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1618316900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8095c70 0x7fe8c8096050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009890 tx=0x7fe8c002a360 comp rx=0 tx=0).stop 2024-09-19T04:25:10.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.788+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1618316900 shutdown_connections 2024-09-19T04:25:10.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.788+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1618316900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c809a990 0x7fe8c80a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:10.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.788+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1618316900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8c8096620 0x7fe8c809a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:10.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.788+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1618316900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8095c70 0x7fe8c8096050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:10.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.788+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1618316900 >> 172.21.15.94:0/1618316900 conn(0x7fe8c800c560 msgr2=0x7fe8c800c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:10.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.788+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1618316900 shutdown_connections 2024-09-19T04:25:10.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.788+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1618316900 wait complete. 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.789+0000 7fe8cf59e640 1 Processor -- start 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.789+0000 7fe8cf59e640 1 -- start start 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.789+0000 7fe8cf59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c8095c70 0x7fe8c8132a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.789+0000 7fe8cf59e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8096620 0x7fe8c8132f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.789+0000 7fe8cf59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8c809a990 0x7fe8c80a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.789+0000 7fe8cf59e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c80a5840 con 0x7fe8c8095c70 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.789+0000 7fe8cf59e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c80a56c0 con 0x7fe8c8096620 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.789+0000 7fe8cf59e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c80a59c0 con 0x7fe8c809a990 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.790+0000 7fe8ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8c809a990 0x7fe8c80a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.790+0000 7fe8ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8c809a990 0x7fe8c80a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44986/0 (socket says 172.21.15.94:44986) 2024-09-19T04:25:10.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.790+0000 7fe8ced9d640 1 -- 172.21.15.94:0/1895287562 learned_addr learned my addr 172.21.15.94:0/1895287562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:10.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.790+0000 7fe8ced9d640 1 -- 172.21.15.94:0/1895287562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8096620 msgr2=0x7fe8c8132f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:25:10.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.790+0000 7fe8ced9d640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8096620 0x7fe8c8132f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:10.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.790+0000 7fe8ced9d640 1 -- 172.21.15.94:0/1895287562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c8095c70 msgr2=0x7fe8c8132a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:25:10.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.791+0000 7fe8ced9d640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c8095c70 0x7fe8c8132a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:10.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.791+0000 7fe8ced9d640 1 -- 172.21.15.94:0/1895287562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c80a20d0 con 0x7fe8c809a990 2024-09-19T04:25:10.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.791+0000 7fe8ced9d640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8c809a990 0x7fe8c80a1810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00518b0 tx=0x7fe8d00747f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:10.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.792+0000 7fe8bf7fe640 1 -- 172.21.15.94:0/1895287562 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0074e50 con 0x7fe8c809a990 2024-09-19T04:25:10.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.792+0000 7fe8bf7fe640 1 -- 172.21.15.94:0/1895287562 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d0084020 con 0x7fe8c809a990 2024-09-19T04:25:10.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.792+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c80a23c0 con 0x7fe8c809a990 2024-09-19T04:25:10.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.793+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c812e5c0 con 0x7fe8c809a990 2024-09-19T04:25:10.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.793+0000 7fe8bf7fe640 1 -- 172.21.15.94:0/1895287562 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d007e6f0 con 0x7fe8c809a990 2024-09-19T04:25:10.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.794+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b0005290 con 0x7fe8c809a990 2024-09-19T04:25:10.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.796+0000 7fe8bf7fe640 1 -- 172.21.15.94:0/1895287562 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe8d007e890 con 0x7fe8c809a990 2024-09-19T04:25:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.797+0000 7fe8bf7fe640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe8b80a3660 0x7fe8b80a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:10.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.797+0000 7fe8bf7fe640 1 -- 172.21.15.94:0/1895287562 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7fe8d013d550 con 0x7fe8c809a990 2024-09-19T04:25:10.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.798+0000 7fe8ce59c640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe8b80a3660 0x7fe8b80a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:10.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.801+0000 7fe8bf7fe640 1 -- 172.21.15.94:0/1895287562 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8d008e020 con 0x7fe8c809a990 2024-09-19T04:25:10.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:10.801+0000 7fe8ce59c640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe8b80a3660 0x7fe8b80a5b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8c002a880 tx=0x7fe8c0005c90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:11.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:10 smithi112 ceph-mon[35051]: pgmap v314: 97 pgs: 97 active+clean; 4.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-09-19T04:25:11.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:10 smithi094 ceph-mon[30972]: pgmap v314: 97 pgs: 97 active+clean; 4.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-09-19T04:25:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:10 smithi104 ceph-mon[34145]: pgmap v314: 97 pgs: 97 active+clean; 4.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-09-19T04:25:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.191+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8b0005580 con 0x7fe8c809a990 2024-09-19T04:25:11.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.194+0000 7fe8bf7fe640 1 -- 172.21.15.94:0/1895287562 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11920 (secure 0 0 0) 0x7fe8d0109e60 con 0x7fe8c809a990 2024-09-19T04:25:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:25:11.197 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":44,"btime":"2024-09-19T04:24:53:683818+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:24:52.788666+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:25:11.197 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 44 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.198+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe8b80a3660 msgr2=0x7fe8b80a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.198+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe8b80a3660 0x7fe8b80a5b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8c002a880 tx=0x7fe8c0005c90 comp rx=0 tx=0).stop 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.198+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8c809a990 msgr2=0x7fe8c80a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.198+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8c809a990 0x7fe8c80a1810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00518b0 tx=0x7fe8d00747f0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.199+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 shutdown_connections 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.199+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe8b80a3660 0x7fe8b80a5b20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.199+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8c809a990 0x7fe8c80a1810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.199+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8096620 0x7fe8c8132f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.199+0000 7fe8cf59e640 1 --2- 172.21.15.94:0/1895287562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c8095c70 0x7fe8c8132a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.199+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 >> 172.21.15.94:0/1895287562 conn(0x7fe8c800c560 msgr2=0x7fe8c8099180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.199+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 shutdown_connections 2024-09-19T04:25:11.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:11.199+0000 7fe8cf59e640 1 -- 172.21.15.94:0/1895287562 wait complete. 2024-09-19T04:25:11.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:11 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1895287562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:25:11.956 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:25:12.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:11 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1895287562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:25:12.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:11 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1895287562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:25:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:12.794 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:12 smithi094 ceph-mon[30972]: pgmap v315: 97 pgs: 97 active+clean; 4.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-09-19T04:25:12.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.924+0000 7f89e13b3640 1 -- 172.21.15.94:0/182375212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d40952d0 msgr2=0x7f89d4004cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:12.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.924+0000 7f89e13b3640 1 --2- 172.21.15.94:0/182375212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d40952d0 0x7f89d4004cb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f89dc050280 tx=0x7f89dc096c50 comp rx=0 tx=0).stop 2024-09-19T04:25:12.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.925+0000 7f89e13b3640 1 -- 172.21.15.94:0/182375212 shutdown_connections 2024-09-19T04:25:12.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.925+0000 7f89e13b3640 1 --2- 172.21.15.94:0/182375212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d40952d0 0x7f89d4004cb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.925+0000 7f89e13b3640 1 --2- 172.21.15.94:0/182375212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d4094930 0x7f89d4094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.925+0000 7f89e13b3640 1 --2- 172.21.15.94:0/182375212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d409a550 0x7f89d409a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.925+0000 7f89e13b3640 1 -- 172.21.15.94:0/182375212 >> 172.21.15.94:0/182375212 conn(0x7f89d400a1f0 msgr2=0x7f89d400a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:12.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.925+0000 7f89e13b3640 1 -- 172.21.15.94:0/182375212 shutdown_connections 2024-09-19T04:25:12.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.925+0000 7f89e13b3640 1 -- 172.21.15.94:0/182375212 wait complete. 2024-09-19T04:25:12.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.926+0000 7f89e13b3640 1 Processor -- start 2024-09-19T04:25:12.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.926+0000 7f89e13b3640 1 -- start start 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89e13b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d4094930 0x7f89d412de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89e13b3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d40952d0 0x7f89d412e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89e13b3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d409a550 0x7f89d4132750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89e13b3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89d40a7ac0 con 0x7f89d40952d0 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89e13b3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89d40a7940 con 0x7f89d409a550 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89e0bb2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d409a550 0x7f89d4132750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89e0bb2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d409a550 0x7f89d4132750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:40866/0 (socket says 172.21.15.94:40866) 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89e0bb2640 1 -- 172.21.15.94:0/97970094 learned_addr learned my addr 172.21.15.94:0/97970094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89e13b3640 1 -- 172.21.15.94:0/97970094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89d40a7c40 con 0x7f89d4094930 2024-09-19T04:25:12.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.927+0000 7f89dbfff640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d4094930 0x7f89d412de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89db7fe640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d40952d0 0x7f89d412e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89e0bb2640 1 -- 172.21.15.94:0/97970094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d4094930 msgr2=0x7f89d412de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d4094930 0x7f89d412de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89e0bb2640 1 -- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d40952d0 msgr2=0x7f89d412e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d40952d0 0x7f89d412e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89e0bb2640 1 -- 172.21.15.94:0/97970094 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d4132e30 con 0x7f89d409a550 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89dbfff640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d4094930 0x7f89d412de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d409a550 0x7f89d4132750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f89dc000b60 tx=0x7f89dc0a0790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89d97fa640 1 -- 172.21.15.94:0/97970094 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0a0d90 con 0x7f89d409a550 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.928+0000 7f89d97fa640 1 -- 172.21.15.94:0/97970094 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89dc0634e0 con 0x7f89d409a550 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.929+0000 7f89d97fa640 1 -- 172.21.15.94:0/97970094 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0a9460 con 0x7f89d409a550 2024-09-19T04:25:12.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.929+0000 7f89e13b3640 1 -- 172.21.15.94:0/97970094 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d41330c0 con 0x7f89d409a550 2024-09-19T04:25:12.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.929+0000 7f89e13b3640 1 -- 172.21.15.94:0/97970094 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d4133580 con 0x7f89d409a550 2024-09-19T04:25:12.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.933+0000 7f89e49ee740 1 Processor -- start 2024-09-19T04:25:12.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.933+0000 7f89d97fa640 1 -- 172.21.15.94:0/97970094 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f89dc09d030 con 0x7f89d409a550 2024-09-19T04:25:12.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.933+0000 7f89e49ee740 1 -- start start 2024-09-19T04:25:12.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe334240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe334780 0x562cbe334bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 0x562cbe341910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89e49ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562cbe29a2f0 con 0x562cbe33f520 2024-09-19T04:25:12.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89e49ee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562cbe241cf0 con 0x562cbe334780 2024-09-19T04:25:12.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89e49ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562cbe2962b0 con 0x562cbe335cf0 2024-09-19T04:25:12.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89db7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe334240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89db7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe334240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45442/0 (socket says 172.21.15.94:45442) 2024-09-19T04:25:12.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89db7fe640 1 -- 172.21.15.94:0/569339552 learned_addr learned my addr 172.21.15.94:0/569339552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:12.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.934+0000 7f89db7fe640 1 --2- 172.21.15.94:0/569339552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 0x562cbe341910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89db7fe640 1 -- 172.21.15.94:0/569339552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe334780 msgr2=0x562cbe334bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:12.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89db7fe640 1 --2- 172.21.15.94:0/569339552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe334780 0x562cbe334bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89db7fe640 1 -- 172.21.15.94:0/569339552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 msgr2=0x562cbe341910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:12.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89db7fe640 1 --2- 172.21.15.94:0/569339552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 0x562cbe341910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89db7fe640 1 -- 172.21.15.94:0/569339552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbe2263a0 con 0x562cbe335cf0 2024-09-19T04:25:12.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89db7fe640 1 --2- 172.21.15.94:0/569339552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe334240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89c8004650 tx=0x7f89c800c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb288d3a5a008f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:12.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89c67fc640 1 -- 172.21.15.94:0/569339552 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c800f760 con 0x562cbe335cf0 2024-09-19T04:25:12.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89c67fc640 1 -- 172.21.15.94:0/569339552 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89c800fdb0 con 0x562cbe335cf0 2024-09-19T04:25:12.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89c67fc640 1 -- 172.21.15.94:0/569339552 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8012b60 con 0x562cbe335cf0 2024-09-19T04:25:12.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.935+0000 7f89d97fa640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f89cc0a3440 0x7f89cc0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.936+0000 7f89d97fa640 1 -- 172.21.15.94:0/97970094 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7f89dc15c710 con 0x7f89d409a550 2024-09-19T04:25:12.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.936+0000 7f89dbfff640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f89cc0a3440 0x7f89cc0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.936+0000 7f89e49ee740 1 -- 172.21.15.94:0/569339552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 msgr2=0x562cbe334240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:12.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.936+0000 7f89e49ee740 1 --2- 172.21.15.94:0/569339552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe334240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89c8004650 tx=0x7f89c800c4f0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.937+0000 7f89dbfff640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f89cc0a3440 0x7f89cc0a5900 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562cbe331c70 tx=0x7f89d00094f0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:12.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.937+0000 7f89e49ee740 1 -- 172.21.15.94:0/569339552 shutdown_connections 2024-09-19T04:25:12.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.937+0000 7f89e49ee740 1 --2- 172.21.15.94:0/569339552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 0x562cbe341910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.937+0000 7f89e49ee740 1 --2- 172.21.15.94:0/569339552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe334780 0x562cbe334bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.937+0000 7f89e49ee740 1 --2- 172.21.15.94:0/569339552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe334240 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.937+0000 7f89e49ee740 1 -- 172.21.15.94:0/569339552 >> 172.21.15.94:0/569339552 conn(0x562cbe2fd310 msgr2=0x562cbe330e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:12.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.938+0000 7f89e49ee740 1 -- 172.21.15.94:0/569339552 shutdown_connections 2024-09-19T04:25:12.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.938+0000 7f89e49ee740 1 -- 172.21.15.94:0/569339552 wait complete. 2024-09-19T04:25:12.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.938+0000 7f89e49ee740 1 -- start start 2024-09-19T04:25:12.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.938+0000 7f89e49ee740 1 Processor -- start 2024-09-19T04:25:12.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe334780 0x562cbe33ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe335cf0 0x562cbe33c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe33c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89e49ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562cbe3421c0 con 0x562cbe335cf0 2024-09-19T04:25:12.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89e49ee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562cbe255060 con 0x562cbe33f520 2024-09-19T04:25:12.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89e49ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562cbe342340 con 0x562cbe334780 2024-09-19T04:25:12.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89e49ee740 10 client.0 _ll_register_callbacks cb 0x562cbdfa88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:12.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe334780 0x562cbe33ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe334780 0x562cbe33ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45452/0 (socket says 172.21.15.94:45452) 2024-09-19T04:25:12.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89dbfff640 1 -- 172.21.15.94:0/2583396180 learned_addr learned my addr 172.21.15.94:0/2583396180 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:12.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.939+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe33c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89dbfff640 1 -- 172.21.15.94:0/2583396180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 msgr2=0x562cbe33c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:12.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89dbfff640 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe33c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89dbfff640 1 -- 172.21.15.94:0/2583396180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe335cf0 msgr2=0x562cbe33c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:12.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89db7fe640 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe335cf0 0x562cbe33c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89dbfff640 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe335cf0 0x562cbe33c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89dbfff640 1 -- 172.21.15.94:0/2583396180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbe00b3c0 con 0x562cbe334780 2024-09-19T04:25:12.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89db7fe640 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe335cf0 0x562cbe33c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:25:12.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe33c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:12.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89dbfff640 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe334780 0x562cbe33ee50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f89d0012fd0 tx=0x7f89d0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac73d26f1e56f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:12.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:25:12.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2583396180 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d001c900 con 0x562cbe334780 2024-09-19T04:25:12.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2583396180 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7f89d001caa0 con 0x562cbe334780 2024-09-19T04:25:12.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.940+0000 7f89e49ee740 1 -- 172.21.15.94:0/2583396180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c800fdb0 con 0x562cbe334780 2024-09-19T04:25:12.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.941+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2583396180 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f89d001cca0 con 0x562cbe334780 2024-09-19T04:25:12.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.941+0000 7f89e49ee740 10 client.26179 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:12.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.941+0000 7f89e49ee740 1 -- 172.21.15.94:0/2583396180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562cbe3564b0 con 0x562cbe334780 2024-09-19T04:25:12.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.942+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2583396180 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89c800fdb0 con 0x562cbe334780 2024-09-19T04:25:12.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.942+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2583396180 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f89d001d5c0 con 0x562cbe334780 2024-09-19T04:25:12.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.942+0000 7f89e49ee740 10 client.26179 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:12.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.942+0000 7f89e49ee740 10 client.26179 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:25:12.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.942+0000 7f89e49ee740 20 client.26179 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:12.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.942+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x562cbe357970 0x562cbe359e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.942+0000 7f89e49ee740 4 client.26179 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:12.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.942+0000 7f89e49ee740 1 -- 172.21.15.94:0/2583396180 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cbe2a7860 con 0x562cbe357970 2024-09-19T04:25:12.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.942+0000 7f89db7fe640 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x562cbe357970 0x562cbe359e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.943+0000 7f89db7fe640 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x562cbe357970 0x562cbe359e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a965eca41fa79f7 server_cookie=2ad7c378cb692f1 in_seq=0 out_seq=0 2024-09-19T04:25:12.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.943+0000 7f89bb7fe640 10 client.26179 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.944+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2583396180 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cbe2a7860 con 0x562cbe357970 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.944+0000 7f89bb7fe640 10 client.26179 handle_command_reply: tid=0 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.944+0000 7f89e49ee740 1 client.26179 shutdown 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.945+0000 7f89e49ee740 1 -- 172.21.15.94:0/2583396180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe334780 msgr2=0x562cbe33ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.945+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe334780 0x562cbe33ee50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f89d0012fd0 tx=0x7f89d0005f90 comp rx=0 tx=0).stop 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.945+0000 7f89e49ee740 1 -- 172.21.15.94:0/2583396180 shutdown_connections 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.946+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x562cbe357970 0x562cbe359e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.946+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe33c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.946+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe335cf0 0x562cbe33c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.946+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2583396180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe334780 0x562cbe33ee50 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f89d0012fd0 tx=0x7f89d0005f90 comp rx=0 tx=0).stop 2024-09-19T04:25:12.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.946+0000 7f89bb7fe640 0 client.26179 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:25:12.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.946+0000 7f89e49ee740 1 -- 172.21.15.94:0/2583396180 >> 172.21.15.94:0/2583396180 conn(0x562cbe2fd310 msgr2=0x562cbe336ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:12.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.946+0000 7f89e49ee740 1 -- 172.21.15.94:0/2583396180 shutdown_connections 2024-09-19T04:25:12.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.946+0000 7f89e49ee740 1 -- 172.21.15.94:0/2583396180 wait complete. 2024-09-19T04:25:12.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.946+0000 7f89e49ee740 20 client.26179 trim_cache size 0 max 16384 2024-09-19T04:25:12.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.969+0000 7f89e49ee740 1 Processor -- start 2024-09-19T04:25:12.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.969+0000 7f89e49ee740 1 -- start start 2024-09-19T04:25:12.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe353d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 0x562cbe333d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe357970 0x562cbe35f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e49ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4141970 con 0x562cbe33f520 2024-09-19T04:25:12.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e49ee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89d413f1e0 con 0x562cbe357970 2024-09-19T04:25:12.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e49ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89d413d070 con 0x562cbe335cf0 2024-09-19T04:25:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e0bb2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe357970 0x562cbe35f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89db7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 0x562cbe333d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e0bb2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe357970 0x562cbe35f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:40886/0 (socket says 172.21.15.94:40886) 2024-09-19T04:25:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e0bb2640 1 -- 172.21.15.94:0/3033720347 learned_addr learned my addr 172.21.15.94:0/3033720347 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89db7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 0x562cbe333d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53668/0 (socket says 172.21.15.94:53668) 2024-09-19T04:25:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e0bb2640 1 -- 172.21.15.94:0/3033720347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 msgr2=0x562cbe353d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:25:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/3033720347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe353d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e0bb2640 1 -- 172.21.15.94:0/3033720347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 msgr2=0x562cbe333d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:12.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.970+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/3033720347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 0x562cbe333d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.971+0000 7f89e0bb2640 1 -- 172.21.15.94:0/3033720347 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbe33a650 con 0x562cbe357970 2024-09-19T04:25:12.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.971+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/3033720347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe357970 0x562cbe35f700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f89dc065a40 tx=0x7f89dc063690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da4f575cb4a7484e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:12.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.971+0000 7f89c5ffb640 1 -- 172.21.15.94:0/3033720347 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc141070 con 0x562cbe357970 2024-09-19T04:25:12.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.971+0000 7f89c5ffb640 1 -- 172.21.15.94:0/3033720347 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89dc09b030 con 0x562cbe357970 2024-09-19T04:25:12.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.971+0000 7f89c5ffb640 1 -- 172.21.15.94:0/3033720347 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0b6040 con 0x562cbe357970 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.972+0000 7f89e49ee740 1 -- 172.21.15.94:0/3033720347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe357970 msgr2=0x562cbe35f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.972+0000 7f89e49ee740 1 --2- 172.21.15.94:0/3033720347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe357970 0x562cbe35f700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f89dc065a40 tx=0x7f89dc063690 comp rx=0 tx=0).stop 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.972+0000 7f89e49ee740 1 -- 172.21.15.94:0/3033720347 shutdown_connections 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.972+0000 7f89e49ee740 1 --2- 172.21.15.94:0/3033720347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe357970 0x562cbe35f700 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.972+0000 7f89e49ee740 1 --2- 172.21.15.94:0/3033720347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe33f520 0x562cbe333d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.972+0000 7f89e49ee740 1 --2- 172.21.15.94:0/3033720347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe353d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.972+0000 7f89e49ee740 1 -- 172.21.15.94:0/3033720347 >> 172.21.15.94:0/3033720347 conn(0x562cbe2fd310 msgr2=0x562cbe33a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.973+0000 7f89e49ee740 1 -- 172.21.15.94:0/3033720347 shutdown_connections 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.973+0000 7f89e49ee740 1 -- 172.21.15.94:0/3033720347 wait complete. 2024-09-19T04:25:12.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.973+0000 7f89e49ee740 1 -- start start 2024-09-19T04:25:12.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.974+0000 7f89e49ee740 1 Processor -- start 2024-09-19T04:25:12.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.974+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe34fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.974+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe350430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e49ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe357970 0x562cbe34cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe34fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe34fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45474/0 (socket says 172.21.15.94:45474) 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89db7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe350430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e49ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89d413d6c0 con 0x562cbe357970 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89dbfff640 1 -- 172.21.15.94:0/2624529354 learned_addr learned my addr 172.21.15.94:0/2624529354 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89d413eb10 con 0x562cbe33f520 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89d413dd70 con 0x562cbe335cf0 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe357970 0x562cbe34cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e49ee740 10 client.0 _ll_register_callbacks cb 0x7f89d0009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e0bb2640 1 -- 172.21.15.94:0/2624529354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 msgr2=0x562cbe34fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:12.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe34fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e0bb2640 1 -- 172.21.15.94:0/2624529354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 msgr2=0x562cbe350430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe350430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89e0bb2640 1 -- 172.21.15.94:0/2624529354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbe350aa0 con 0x562cbe357970 2024-09-19T04:25:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.975+0000 7f89db7fe640 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe350430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.976+0000 7f89dbfff640 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe34fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.976+0000 7f89e0bb2640 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe357970 0x562cbe34cec0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f89dc12c000 tx=0x7f89dc096c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127ec489373780df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.976+0000 7f89bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:25:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.976+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2624529354 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0b6040 con 0x562cbe357970 2024-09-19T04:25:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.976+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2624529354 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7f89dc127e80 con 0x562cbe357970 2024-09-19T04:25:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.977+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89dc09b030 con 0x562cbe357970 2024-09-19T04:25:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.977+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2624529354 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f89dc141040 con 0x562cbe357970 2024-09-19T04:25:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.978+0000 7f89e49ee740 10 client.16620 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.978+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562cbe3c2c30 con 0x562cbe357970 2024-09-19T04:25:12.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.978+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2624529354 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89dc09b030 con 0x562cbe357970 2024-09-19T04:25:12.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.979+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2624529354 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f89dc0b9060 con 0x562cbe357970 2024-09-19T04:25:12.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.979+0000 7f89e49ee740 10 client.16620 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:12.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.979+0000 7f89e49ee740 10 client.16620 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:25:12.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.979+0000 7f89e49ee740 20 client.16620 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:12.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.979+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x562cbe3c31f0 0x562cbe3c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:12.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.979+0000 7f89e49ee740 4 client.16620 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:25:12.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.979+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cbe2a1d70 con 0x562cbe3c31f0 2024-09-19T04:25:12.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.981+0000 7f89dbfff640 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x562cbe3c31f0 0x562cbe3c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:12.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.982+0000 7f89dbfff640 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x562cbe3c31f0 0x562cbe3c55e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21a5fd8eaabbdc81 server_cookie=e291e04023ef4d0a in_seq=0 out_seq=0 2024-09-19T04:25:12.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.982+0000 7f89bb7fe640 10 client.16620 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:25:12.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.982+0000 7f89bb7fe640 1 -- 172.21.15.94:0/2624529354 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cbe2a1d70 con 0x562cbe3c31f0 2024-09-19T04:25:12.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.982+0000 7f89bb7fe640 10 client.16620 handle_command_reply: tid=0 2024-09-19T04:25:12.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.982+0000 7f89e49ee740 1 client.16620 shutdown 2024-09-19T04:25:12.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.982+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe357970 msgr2=0x562cbe34cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:12.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.982+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe357970 0x562cbe34cec0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f89dc12c000 tx=0x7f89dc096c50 comp rx=0 tx=0).stop 2024-09-19T04:25:12.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.983+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 shutdown_connections 2024-09-19T04:25:12.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.983+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x562cbe3c31f0 0x562cbe3c55e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.983+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562cbe357970 0x562cbe34cec0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.983+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cbe33f520 0x562cbe350430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.983+0000 7f89e49ee740 1 --2- 172.21.15.94:0/2624529354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562cbe335cf0 0x562cbe34fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.983+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 >> 172.21.15.94:0/2624529354 conn(0x562cbe2fd310 msgr2=0x562cbe33d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:12.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.983+0000 7f89bb7fe640 0 client.16620 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:25:12.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.983+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 shutdown_connections 2024-09-19T04:25:12.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.983+0000 7f89e49ee740 1 -- 172.21.15.94:0/2624529354 wait complete. 2024-09-19T04:25:12.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.984+0000 7f89e49ee740 20 client.16620 trim_cache size 0 max 16384 2024-09-19T04:25:12.987 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 -- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f89cc0a3440 msgr2=0x7f89cc0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f89cc0a3440 0x7f89cc0a5900 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562cbe331c70 tx=0x7f89d00094f0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 -- 172.21.15.94:0/97970094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d409a550 msgr2=0x7f89d4132750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d409a550 0x7f89d4132750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f89dc000b60 tx=0x7f89dc0a0790 comp rx=0 tx=0).stop 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 -- 172.21.15.94:0/97970094 shutdown_connections 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f89cc0a3440 0x7f89cc0a5900 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89d409a550 0x7f89d4132750 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d40952d0 0x7f89d412e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 --2- 172.21.15.94:0/97970094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d4094930 0x7f89d412de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:12.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 -- 172.21.15.94:0/97970094 >> 172.21.15.94:0/97970094 conn(0x7f89d400a1f0 msgr2=0x7f89d4096040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:12.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 -- 172.21.15.94:0/97970094 shutdown_connections 2024-09-19T04:25:12.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:12.985+0000 7f89e13b3640 1 -- 172.21.15.94:0/97970094 wait complete. 2024-09-19T04:25:13.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:12 smithi112 ceph-mon[35051]: pgmap v315: 97 pgs: 97 active+clean; 4.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-09-19T04:25:13.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:12 smithi104 ceph-mon[34145]: pgmap v315: 97 pgs: 97 active+clean; 4.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-09-19T04:25:13.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:25:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:15.033 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:14 smithi094 ceph-mon[30972]: pgmap v316: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-19T04:25:15.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:14 smithi112 ceph-mon[35051]: pgmap v316: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-19T04:25:15.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:14 smithi104 ceph-mon[34145]: pgmap v316: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-19T04:25:15.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.158+0000 7fcacc82c640 1 -- 172.21.15.94:0/1828475656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac8109b10 msgr2=0x7fcac8114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.158+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1828475656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac8109b10 0x7fcac8114570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009860 tx=0x7fcab002a3d0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.158+0000 7fcacc82c640 1 -- 172.21.15.94:0/1828475656 shutdown_connections 2024-09-19T04:25:15.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.158+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1828475656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8114ab0 0x7fcac8116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.158+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1828475656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac8109b10 0x7fcac8114570 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.158+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1828475656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac81091f0 0x7fcac81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.158+0000 7fcacc82c640 1 -- 172.21.15.94:0/1828475656 >> 172.21.15.94:0/1828475656 conn(0x7fcac806e140 msgr2=0x7fcac806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.158+0000 7fcacc82c640 1 -- 172.21.15.94:0/1828475656 shutdown_connections 2024-09-19T04:25:15.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.158+0000 7fcacc82c640 1 -- 172.21.15.94:0/1828475656 wait complete. 2024-09-19T04:25:15.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.159+0000 7fcacc82c640 1 Processor -- start 2024-09-19T04:25:15.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcacc82c640 1 -- start start 2024-09-19T04:25:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcacc82c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac81091f0 0x7fcac81aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcacc82c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac8109b10 0x7fcac81a7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcac659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac81091f0 0x7fcac81aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcacc82c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8114ab0 0x7fcac81a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcac659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac81091f0 0x7fcac81aa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45488/0 (socket says 172.21.15.94:45488) 2024-09-19T04:25:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcacc82c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810b970 con 0x7fcac8109b10 2024-09-19T04:25:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcacc82c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810b7f0 con 0x7fcac8114ab0 2024-09-19T04:25:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcacc82c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810baf0 con 0x7fcac81091f0 2024-09-19T04:25:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.160+0000 7fcac659c640 1 -- 172.21.15.94:0/1907862023 learned_addr learned my addr 172.21.15.94:0/1907862023 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:15.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8114ab0 0x7fcac81a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcac659c640 1 -- 172.21.15.94:0/1907862023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8114ab0 msgr2=0x7fcac81a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcac659c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8114ab0 0x7fcac81a76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcac659c640 1 -- 172.21.15.94:0/1907862023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac8109b10 msgr2=0x7fcac81a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcac659c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac8109b10 0x7fcac81a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcac659c640 1 -- 172.21.15.94:0/1907862023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac81a7df0 con 0x7fcac81091f0 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8114ab0 0x7fcac81a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcac659c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac81091f0 0x7fcac81aa0d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcab800e9c0 tx=0x7fcab800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcabf7fe640 1 -- 172.21.15.94:0/1907862023 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab800ccc0 con 0x7fcac81091f0 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcabf7fe640 1 -- 172.21.15.94:0/1907862023 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab80054d0 con 0x7fcac81091f0 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcabf7fe640 1 -- 172.21.15.94:0/1907862023 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8010430 con 0x7fcac81091f0 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.161+0000 7fcacc82c640 1 -- 172.21.15.94:0/1907862023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac8113180 con 0x7fcac81091f0 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.162+0000 7fcacc82c640 1 -- 172.21.15.94:0/1907862023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac8113670 con 0x7fcac81091f0 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.166+0000 7fcacec05740 1 Processor -- start 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.166+0000 7fcabf7fe640 1 -- 172.21.15.94:0/1907862023 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fcab80046c0 con 0x7fcac81091f0 2024-09-19T04:25:15.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.166+0000 7fcacec05740 1 -- start start 2024-09-19T04:25:15.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.167+0000 7fcacec05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d178524090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.167+0000 7fcacec05740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d178524a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.167+0000 7fcacec05740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178531760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.167+0000 7fcacec05740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d17848a2f0 con 0x55d1785245d0 2024-09-19T04:25:15.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.167+0000 7fcacec05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d178431cf0 con 0x55d178525b40 2024-09-19T04:25:15.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.167+0000 7fcacec05740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d1784862b0 con 0x55d17852f370 2024-09-19T04:25:15.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.167+0000 7fcac5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178531760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.167+0000 7fcac5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178531760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45490/0 (socket says 172.21.15.94:45490) 2024-09-19T04:25:15.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.168+0000 7fcac5d9b640 1 -- 172.21.15.94:0/3958240362 learned_addr learned my addr 172.21.15.94:0/3958240362 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:15.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.168+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d178524090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.168+0000 7fcac659c640 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d178524a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.168+0000 7fcac6d9d640 1 -- 172.21.15.94:0/3958240362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 msgr2=0x55d178531760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.168+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178531760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.168+0000 7fcac6d9d640 1 -- 172.21.15.94:0/3958240362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 msgr2=0x55d178524a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.168+0000 7fcac5d9b640 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178531760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.168+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d178524a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.168+0000 7fcac6d9d640 1 -- 172.21.15.94:0/3958240362 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1784163a0 con 0x55d178525b40 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.169+0000 7fcabf7fe640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcab40a34b0 0x7fcab40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.169+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d178524090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcac80743f0 tx=0x7fcac00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f46f51e5a29bf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.169+0000 7fcac659c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcab40a34b0 0x7fcab40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.169+0000 7fcabf7fe640 1 -- 172.21.15.94:0/1907862023 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7fcab80d0140 con 0x7fcac81091f0 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.170+0000 7fcabcff9640 1 -- 172.21.15.94:0/3958240362 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac001c680 con 0x55d178525b40 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.170+0000 7fcabcff9640 1 -- 172.21.15.94:0/3958240362 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac001ccd0 con 0x55d178525b40 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.171+0000 7fcabcff9640 1 -- 172.21.15.94:0/3958240362 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0011d90 con 0x55d178525b40 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.171+0000 7fcacec05740 1 -- 172.21.15.94:0/3958240362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 msgr2=0x55d178524090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.171+0000 7fcacec05740 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d178524090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcac80743f0 tx=0x7fcac00107a0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.171+0000 7fcac659c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcab40a34b0 0x7fcab40a5970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55d178521ac0 tx=0x7fcab80d5040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.171+0000 7fcacec05740 1 -- 172.21.15.94:0/3958240362 shutdown_connections 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.171+0000 7fcacec05740 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178531760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.171+0000 7fcacec05740 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d178524a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.171+0000 7fcacec05740 1 --2- 172.21.15.94:0/3958240362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d178524090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.171+0000 7fcacec05740 1 -- 172.21.15.94:0/3958240362 >> 172.21.15.94:0/3958240362 conn(0x55d1784ed310 msgr2=0x55d178520c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.172+0000 7fcacec05740 1 -- 172.21.15.94:0/3958240362 shutdown_connections 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.172+0000 7fcacec05740 1 -- 172.21.15.94:0/3958240362 wait complete. 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.172+0000 7fcacec05740 1 -- start start 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcacec05740 1 Processor -- start 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcacec05740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d17852ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcacec05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcacec05740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d17852c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcacec05740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d1785320c0 con 0x55d1785245d0 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcacec05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d178445060 con 0x55d178525b40 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcacec05740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d178532240 con 0x55d17852f370 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcacec05740 10 client.0 _ll_register_callbacks cb 0x55d1781988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcac5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcac5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:40940/0 (socket says 172.21.15.94:40940) 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.173+0000 7fcac5d9b640 1 -- 172.21.15.94:0/4267331526 learned_addr learned my addr 172.21.15.94:0/4267331526 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.174+0000 7fcac5d9b640 1 -- 172.21.15.94:0/4267331526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 msgr2=0x55d17852c5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.174+0000 7fcac5d9b640 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d17852c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.174+0000 7fcac5d9b640 1 -- 172.21.15.94:0/4267331526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 msgr2=0x55d17852ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.174+0000 7fcac5d9b640 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d17852ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.174+0000 7fcac5d9b640 1 -- 172.21.15.94:0/4267331526 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1781fb3c0 con 0x55d178525b40 2024-09-19T04:25:15.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.175+0000 7fcac5d9b640 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852bfe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005b60 tx=0x7fcab0030ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a830868df2e448c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.175+0000 7fcaadffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:25:15.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.175+0000 7fcaadffb640 1 -- 172.21.15.94:0/4267331526 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0005c90 con 0x55d178525b40 2024-09-19T04:25:15.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.175+0000 7fcaadffb640 1 -- 172.21.15.94:0/4267331526 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7fcab0005230 con 0x55d178525b40 2024-09-19T04:25:15.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.175+0000 7fcacec05740 1 -- 172.21.15.94:0/4267331526 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac001ccd0 con 0x55d178525b40 2024-09-19T04:25:15.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.175+0000 7fcaadffb640 1 -- 172.21.15.94:0/4267331526 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcab0005430 con 0x55d178525b40 2024-09-19T04:25:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.175+0000 7fcacec05740 10 client.26234 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.175+0000 7fcacec05740 1 -- 172.21.15.94:0/4267331526 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d17853db60 con 0x55d178525b40 2024-09-19T04:25:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.176+0000 7fcaadffb640 1 -- 172.21.15.94:0/4267331526 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab0041070 con 0x55d178525b40 2024-09-19T04:25:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.176+0000 7fcaadffb640 1 -- 172.21.15.94:0/4267331526 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fcab003d690 con 0x55d178525b40 2024-09-19T04:25:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.176+0000 7fcacec05740 10 client.26234 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.176+0000 7fcacec05740 10 client.26234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:25:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.176+0000 7fcacec05740 20 client.26234 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.176+0000 7fcacec05740 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d17853f570 0x55d178541a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.176+0000 7fcacec05740 4 client.26234 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.176+0000 7fcacec05740 1 -- 172.21.15.94:0/4267331526 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcac001ccd0 con 0x55d17853f570 2024-09-19T04:25:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.177+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d17853f570 0x55d178541a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.178+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d17853f570 0x55d178541a30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dc059d40793e0a3 server_cookie=4e843eac5e4dba80 in_seq=0 out_seq=0 2024-09-19T04:25:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.178+0000 7fcaadffb640 10 client.26234 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:25:15.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.179+0000 7fcaadffb640 1 -- 172.21.15.94:0/4267331526 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcac001ccd0 con 0x55d17853f570 2024-09-19T04:25:15.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.179+0000 7fcaadffb640 10 client.26234 handle_command_reply: tid=0 2024-09-19T04:25:15.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.179+0000 7fcacec05740 1 client.26234 shutdown 2024-09-19T04:25:15.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.179+0000 7fcacec05740 1 -- 172.21.15.94:0/4267331526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 msgr2=0x55d17852bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcacec05740 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852bfe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005b60 tx=0x7fcab0030ba0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcacec05740 1 -- 172.21.15.94:0/4267331526 shutdown_connections 2024-09-19T04:25:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcacec05740 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d17853f570 0x55d178541a30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcacec05740 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d17852c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcaadffb640 0 client.26234 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:25:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcacec05740 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852bfe0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcacec05740 1 --2- 172.21.15.94:0/4267331526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d17852ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcacec05740 1 -- 172.21.15.94:0/4267331526 >> 172.21.15.94:0/4267331526 conn(0x55d1784ed310 msgr2=0x55d178526930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcacec05740 1 -- 172.21.15.94:0/4267331526 shutdown_connections 2024-09-19T04:25:15.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.180+0000 7fcacec05740 1 -- 172.21.15.94:0/4267331526 wait complete. 2024-09-19T04:25:15.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.181+0000 7fcacec05740 20 client.26234 trim_cache size 0 max 16384 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.205+0000 7fcacec05740 1 Processor -- start 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcacec05740 1 -- start start 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcacec05740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d17853b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcacec05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcacec05740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178547340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcacec05740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81c78f0 con 0x55d1785245d0 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcacec05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81c5160 con 0x55d178525b40 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcacec05740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81c2fd0 con 0x55d17852f370 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcac5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d17853b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcac5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:40942/0 (socket says 172.21.15.94:40942) 2024-09-19T04:25:15.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d17853b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53738/0 (socket says 172.21.15.94:53738) 2024-09-19T04:25:15.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcac5d9b640 1 -- 172.21.15.94:0/702260564 learned_addr learned my addr 172.21.15.94:0/702260564 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:15.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.206+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/702260564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178547340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcac5d9b640 1 -- 172.21.15.94:0/702260564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 msgr2=0x55d178547340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcac5d9b640 1 --2- 172.21.15.94:0/702260564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178547340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcac5d9b640 1 -- 172.21.15.94:0/702260564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 msgr2=0x55d17853b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcac5d9b640 1 --2- 172.21.15.94:0/702260564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d17853b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcac5d9b640 1 -- 172.21.15.94:0/702260564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d17852a530 con 0x55d178525b40 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/702260564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178547340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcac5d9b640 1 --2- 172.21.15.94:0/702260564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852b8a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcab003dc50 tx=0x7fcab0005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a8353181db80e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcaaffff640 1 -- 172.21.15.94:0/702260564 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0051070 con 0x55d178525b40 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcaaffff640 1 -- 172.21.15.94:0/702260564 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab0041040 con 0x55d178525b40 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.207+0000 7fcaaffff640 1 -- 172.21.15.94:0/702260564 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0004360 con 0x55d178525b40 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.208+0000 7fcacec05740 1 -- 172.21.15.94:0/702260564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 msgr2=0x55d17852b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.208+0000 7fcacec05740 1 --2- 172.21.15.94:0/702260564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852b8a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcab003dc50 tx=0x7fcab0005f10 comp rx=0 tx=0).stop 2024-09-19T04:25:15.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.208+0000 7fcacec05740 1 -- 172.21.15.94:0/702260564 shutdown_connections 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.208+0000 7fcacec05740 1 --2- 172.21.15.94:0/702260564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d17852f370 0x55d178547340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.208+0000 7fcacec05740 1 --2- 172.21.15.94:0/702260564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d17852b8a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.209+0000 7fcacec05740 1 --2- 172.21.15.94:0/702260564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d1785245d0 0x55d17853b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.209+0000 7fcacec05740 1 -- 172.21.15.94:0/702260564 >> 172.21.15.94:0/702260564 conn(0x55d1784ed310 msgr2=0x55d17852a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.209+0000 7fcacec05740 1 -- 172.21.15.94:0/702260564 shutdown_connections 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.209+0000 7fcacec05740 1 -- 172.21.15.94:0/702260564 wait complete. 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.209+0000 7fcacec05740 1 -- start start 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.209+0000 7fcacec05740 1 Processor -- start 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcacec05740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1785245d0 0x55d178537b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcacec05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d178532e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcacec05740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17852f370 0x55d1785333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcacec05740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81c3620 con 0x55d17852f370 2024-09-19T04:25:15.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcacec05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81c4a90 con 0x55d178525b40 2024-09-19T04:25:15.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcacec05740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81c3cf0 con 0x55d1785245d0 2024-09-19T04:25:15.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcacec05740 10 client.0 _ll_register_callbacks cb 0x7fcac00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:15.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d178532e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1785245d0 0x55d178537b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1785245d0 0x55d178537b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45514/0 (socket says 172.21.15.94:45514) 2024-09-19T04:25:15.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 -- 172.21.15.94:0/413291431 learned_addr learned my addr 172.21.15.94:0/413291431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:15.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17852f370 0x55d1785333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 -- 172.21.15.94:0/413291431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 msgr2=0x55d178532e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d178532e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 -- 172.21.15.94:0/413291431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17852f370 msgr2=0x55d1785333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17852f370 0x55d1785333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 -- 172.21.15.94:0/413291431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d178533a90 con 0x55d1785245d0 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.210+0000 7fcac6d9d640 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1785245d0 0x55d178537b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcac00107e0 tx=0x7fcac0009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=159efc741b24ad5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.211+0000 7fcaadffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.211+0000 7fcaadffb640 1 -- 172.21.15.94:0/413291431 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0011cb0 con 0x55d1785245d0 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.211+0000 7fcaadffb640 1 -- 172.21.15.94:0/413291431 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8629+0+0 (secure 0 0 0) 0x7fcac001c480 con 0x55d1785245d0 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.211+0000 7fcacec05740 1 -- 172.21.15.94:0/413291431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab0041040 con 0x55d1785245d0 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcaadffb640 1 -- 172.21.15.94:0/413291431 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcac003b050 con 0x55d1785245d0 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcacec05740 10 client.26197 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:15.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcacec05740 1 -- 172.21.15.94:0/413291431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d1785aa1d0 con 0x55d1785245d0 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcaadffb640 1 -- 172.21.15.94:0/413291431 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab0041040 con 0x55d1785245d0 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcaadffb640 1 -- 172.21.15.94:0/413291431 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fcac00274c0 con 0x55d1785245d0 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcacec05740 10 client.26197 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcacec05740 10 client.26197 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcacec05740 20 client.26197 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcacec05740 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d17853f570 0x55d1785ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcacec05740 4 client.26197 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.212+0000 7fcacec05740 1 -- 172.21.15.94:0/413291431 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d178497860 con 0x55d17853f570 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.213+0000 7fcac5d9b640 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d17853f570 0x55d1785ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.213+0000 7fcac5d9b640 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d17853f570 0x55d1785ac6a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2fc2db0b846ae81d server_cookie=6b31d2977b2c1c7e in_seq=0 out_seq=0 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.213+0000 7fcaadffb640 10 client.26197 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.213+0000 7fcaadffb640 1 -- 172.21.15.94:0/413291431 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d178497860 con 0x55d17853f570 2024-09-19T04:25:15.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.213+0000 7fcaadffb640 10 client.26197 handle_command_reply: tid=0 2024-09-19T04:25:15.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.214+0000 7fcacec05740 1 client.26197 shutdown 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.214+0000 7fcacec05740 1 -- 172.21.15.94:0/413291431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1785245d0 msgr2=0x55d178537b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.214+0000 7fcacec05740 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1785245d0 0x55d178537b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcac00107e0 tx=0x7fcac0009fd0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.215+0000 7fcacec05740 1 -- 172.21.15.94:0/413291431 shutdown_connections 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.215+0000 7fcacec05740 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d17853f570 0x55d1785ac6a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.215+0000 7fcacec05740 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17852f370 0x55d1785333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.215+0000 7fcaadffb640 0 client.26197 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.215+0000 7fcacec05740 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d178525b40 0x55d178532e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.215+0000 7fcacec05740 1 --2- 172.21.15.94:0/413291431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1785245d0 0x55d178537b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.215+0000 7fcacec05740 1 -- 172.21.15.94:0/413291431 >> 172.21.15.94:0/413291431 conn(0x55d1784ed310 msgr2=0x55d17852a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.216+0000 7fcacec05740 1 -- 172.21.15.94:0/413291431 shutdown_connections 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.216+0000 7fcacec05740 1 -- 172.21.15.94:0/413291431 wait complete. 2024-09-19T04:25:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.216+0000 7fcacec05740 20 client.26197 trim_cache size 0 max 16384 2024-09-19T04:25:15.221 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.218+0000 7fcacc82c640 1 -- 172.21.15.94:0/1907862023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcab40a34b0 msgr2=0x7fcab40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.218+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcab40a34b0 0x7fcab40a5970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55d178521ac0 tx=0x7fcab80d5040 comp rx=0 tx=0).stop 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.218+0000 7fcacc82c640 1 -- 172.21.15.94:0/1907862023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac81091f0 msgr2=0x7fcac81aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.218+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac81091f0 0x7fcac81aa0d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcab800e9c0 tx=0x7fcab800ee80 comp rx=0 tx=0).stop 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.218+0000 7fcacc82c640 1 -- 172.21.15.94:0/1907862023 shutdown_connections 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.219+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fcab40a34b0 0x7fcab40a5970 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.219+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8114ab0 0x7fcac81a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.219+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac8109b10 0x7fcac81a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.219+0000 7fcacc82c640 1 --2- 172.21.15.94:0/1907862023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcac81091f0 0x7fcac81aa0d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.219+0000 7fcacc82c640 1 -- 172.21.15.94:0/1907862023 >> 172.21.15.94:0/1907862023 conn(0x7fcac806e140 msgr2=0x7fcac8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.219+0000 7fcacc82c640 1 -- 172.21.15.94:0/1907862023 shutdown_connections 2024-09-19T04:25:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:15.219+0000 7fcacc82c640 1 -- 172.21.15.94:0/1907862023 wait complete. 2024-09-19T04:25:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 540.0 s, 1214.6 tps, lat 82.582 ms stddev 29.234 2024-09-19T04:25:16.091 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:25:16.501 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:17.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:16 smithi112 ceph-mon[35051]: pgmap v317: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-19T04:25:17.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:16 smithi112 ceph-mon[35051]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:25:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:16 smithi094 ceph-mon[30972]: pgmap v317: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-19T04:25:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:16 smithi094 ceph-mon[30972]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:25:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:16 smithi104 ceph-mon[34145]: pgmap v317: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-19T04:25:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:16 smithi104 ceph-mon[34145]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:25:17.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.749+0000 7f1e0534f640 1 -- 172.21.15.94:0/1465531007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0010e540 msgr2=0x7f1e00110930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.749+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1465531007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0010e540 0x7f1e00110930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009ca0 tx=0x7f1df802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.749+0000 7f1e0534f640 1 -- 172.21.15.94:0/1465531007 shutdown_connections 2024-09-19T04:25:17.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.749+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1465531007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0010e540 0x7f1e00110930 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.749+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1465531007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e00101480 0x7f1e0010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.749+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1465531007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e00100b60 0x7f1e00100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.749+0000 7f1e0534f640 1 -- 172.21.15.94:0/1465531007 >> 172.21.15.94:0/1465531007 conn(0x7f1e000fc7d0 msgr2=0x7f1e000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.750+0000 7f1e0534f640 1 -- 172.21.15.94:0/1465531007 shutdown_connections 2024-09-19T04:25:17.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.750+0000 7f1e0534f640 1 -- 172.21.15.94:0/1465531007 wait complete. 2024-09-19T04:25:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.751+0000 7f1e0534f640 1 Processor -- start 2024-09-19T04:25:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.751+0000 7f1e0534f640 1 -- start start 2024-09-19T04:25:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.751+0000 7f1e0534f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e00100b60 0x7f1e0010ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1e0534f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e00101480 0x7f1e0010cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1e0534f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e0010e540 0x7f1e001a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1e0534f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00103bc0 con 0x7f1e00100b60 2024-09-19T04:25:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1e0534f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00103a40 con 0x7f1e0010e540 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1e0534f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00103d40 con 0x7f1e00101480 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dfe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e00101480 0x7f1e0010cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dfe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e00101480 0x7f1e0010cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45524/0 (socket says 172.21.15.94:45524) 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dff7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e0010e540 0x7f1e001a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dfe7fc640 1 -- 172.21.15.94:0/1998445273 learned_addr learned my addr 172.21.15.94:0/1998445273 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e00100b60 0x7f1e0010ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dff7fe640 1 -- 172.21.15.94:0/1998445273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e00101480 msgr2=0x7f1e0010cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dff7fe640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e00101480 0x7f1e0010cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dff7fe640 1 -- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e00100b60 msgr2=0x7f1e0010ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dff7fe640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e00100b60 0x7f1e0010ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dff7fe640 1 -- 172.21.15.94:0/1998445273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e001a4cd0 con 0x7f1e0010e540 2024-09-19T04:25:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.752+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e00101480 0x7f1e0010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.753+0000 7f1dff7fe640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e0010e540 0x7f1e001a45f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1df802aeb0 tx=0x7f1df80093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.753+0000 7f1de7fff640 1 -- 172.21.15.94:0/1998445273 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8003c70 con 0x7f1e0010e540 2024-09-19T04:25:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.753+0000 7f1de7fff640 1 -- 172.21.15.94:0/1998445273 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df8003e10 con 0x7f1e0010e540 2024-09-19T04:25:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.753+0000 7f1de7fff640 1 -- 172.21.15.94:0/1998445273 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8030c10 con 0x7f1e0010e540 2024-09-19T04:25:17.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.753+0000 7f1e0534f640 1 -- 172.21.15.94:0/1998445273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e001a4f60 con 0x7f1e0010e540 2024-09-19T04:25:17.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.753+0000 7f1e0534f640 1 -- 172.21.15.94:0/1998445273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e001a5420 con 0x7f1e0010e540 2024-09-19T04:25:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.757+0000 7f1de7fff640 1 -- 172.21.15.94:0/1998445273 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1df804e440 con 0x7f1e0010e540 2024-09-19T04:25:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.757+0000 7f1e07728740 1 Processor -- start 2024-09-19T04:25:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.757+0000 7f1e07728740 1 -- start start 2024-09-19T04:25:17.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1e07728740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c72b40 0x55ae70c71090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1e07728740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 0x55ae70c71a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1e07728740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c7c370 0x55ae70c7e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1e07728740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ae70bd72f0 con 0x55ae70c715d0 2024-09-19T04:25:17.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1e07728740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ae70b7ecf0 con 0x55ae70c72b40 2024-09-19T04:25:17.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1e07728740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae70bd32b0 con 0x55ae70c7c370 2024-09-19T04:25:17.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1dfe7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c72b40 0x55ae70c71090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1dfe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c7c370 0x55ae70c7e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1dfe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c7c370 0x55ae70c7e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45540/0 (socket says 172.21.15.94:45540) 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1dfe7fc640 1 -- 172.21.15.94:0/3194441464 learned_addr learned my addr 172.21.15.94:0/3194441464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1de7fff640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1de80a34b0 0x7f1de80a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.758+0000 7f1de7fff640 1 -- 172.21.15.94:0/1998445273 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f1df80f0ed0 con 0x7f1e0010e540 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/3194441464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 0x55ae70c71a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1dfe7fc640 1 -- 172.21.15.94:0/3194441464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c72b40 msgr2=0x55ae70c71090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3194441464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c72b40 0x55ae70c71090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1dfe7fc640 1 -- 172.21.15.94:0/3194441464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 msgr2=0x55ae70c71a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3194441464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 0x55ae70c71a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1dfe7fc640 1 -- 172.21.15.94:0/3194441464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae70b633a0 con 0x55ae70c7c370 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1dff7fe640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1de80a34b0 0x7f1de80a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3194441464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c7c370 0x55ae70c7e760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1df4005bc0 tx=0x7f1df4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df1748e5e70c39cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1de57fa640 1 -- 172.21.15.94:0/3194441464 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df400fbf0 con 0x55ae70c7c370 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1de57fa640 1 -- 172.21.15.94:0/3194441464 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df400fd90 con 0x55ae70c7c370 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.759+0000 7f1de57fa640 1 -- 172.21.15.94:0/3194441464 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df40047a0 con 0x55ae70c7c370 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.760+0000 7f1e07728740 1 -- 172.21.15.94:0/3194441464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c7c370 msgr2=0x55ae70c7e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.760+0000 7f1e07728740 1 --2- 172.21.15.94:0/3194441464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c7c370 0x55ae70c7e760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1df4005bc0 tx=0x7f1df4004730 comp rx=0 tx=0).stop 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.761+0000 7f1e07728740 1 -- 172.21.15.94:0/3194441464 shutdown_connections 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.761+0000 7f1e07728740 1 --2- 172.21.15.94:0/3194441464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c7c370 0x55ae70c7e760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.761+0000 7f1e07728740 1 --2- 172.21.15.94:0/3194441464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 0x55ae70c71a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.761+0000 7f1e07728740 1 --2- 172.21.15.94:0/3194441464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c72b40 0x55ae70c71090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.761+0000 7f1e07728740 1 -- 172.21.15.94:0/3194441464 >> 172.21.15.94:0/3194441464 conn(0x55ae70c3a310 msgr2=0x55ae70c6dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.761+0000 7f1dff7fe640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1de80a34b0 0x7f1de80a5970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1df80b9680 tx=0x7f1df80095c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.761+0000 7f1e07728740 1 -- 172.21.15.94:0/3194441464 shutdown_connections 2024-09-19T04:25:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.761+0000 7f1e07728740 1 -- 172.21.15.94:0/3194441464 wait complete. 2024-09-19T04:25:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1e07728740 1 -- start start 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1e07728740 1 Processor -- start 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1e07728740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 0x55ae70c7bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1e07728740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c79070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1e07728740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 0x55ae70c79800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1e07728740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ae70c7f070 con 0x55ae70c7c370 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1e07728740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ae70b92060 con 0x55ae70c715d0 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1e07728740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae70c7f1f0 con 0x55ae70c72b40 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1e07728740 10 client.0 _ll_register_callbacks cb 0x55ae708e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1dfeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c79070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1dfeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c79070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45544/0 (socket says 172.21.15.94:45544) 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.762+0000 7f1dfeffd640 1 -- 172.21.15.94:0/3030643826 learned_addr learned my addr 172.21.15.94:0/3030643826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 0x55ae70c7bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 0x55ae70c79800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1dfeffd640 1 -- 172.21.15.94:0/3030643826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 msgr2=0x55ae70c7bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 0x55ae70c7bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1dfeffd640 1 -- 172.21.15.94:0/3030643826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 msgr2=0x55ae70c79800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 0x55ae70c79800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1dfeffd640 1 -- 172.21.15.94:0/3030643826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae709483c0 con 0x55ae70c72b40 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c79070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1df0013710 tx=0x7f1df0013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826a73f52c1e3523 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1de2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:25:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3030643826 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0020060 con 0x55ae70c72b40 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1e07728740 1 -- 172.21.15.94:0/3030643826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df400fd90 con 0x55ae70c72b40 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.763+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3030643826 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f1df001b430 con 0x55ae70c72b40 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3030643826 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1df001b630 con 0x55ae70c72b40 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1e07728740 10 client.26209 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1e07728740 1 -- 172.21.15.94:0/3030643826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ae70c8ac40 con 0x55ae70c72b40 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3030643826 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df400fd90 con 0x55ae70c72b40 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3030643826 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f1df000f4e0 con 0x55ae70c72b40 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1e07728740 10 client.26209 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1e07728740 10 client.26209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1e07728740 20 client.26209 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1e07728740 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ae70c8c1d0 0x55ae70c8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1e07728740 4 client.26209 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:17.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.764+0000 7f1e07728740 1 -- 172.21.15.94:0/3030643826 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae70be4860 con 0x55ae70c8c1d0 2024-09-19T04:25:17.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.766+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ae70c8c1d0 0x55ae70c8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.770+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ae70c8c1d0 0x55ae70c8e5f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68494a6d2646fdb9 server_cookie=8b0e3d5d7b86efb5 in_seq=0 out_seq=0 2024-09-19T04:25:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.771+0000 7f1de2ff5640 10 client.26209 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.771+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3030643826 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae70be4860 con 0x55ae70c8c1d0 2024-09-19T04:25:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.771+0000 7f1de2ff5640 10 client.26209 handle_command_reply: tid=0 2024-09-19T04:25:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.771+0000 7f1e07728740 1 client.26209 shutdown 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.772+0000 7f1e07728740 1 -- 172.21.15.94:0/3030643826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 msgr2=0x55ae70c79070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c79070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1df0013710 tx=0x7f1df0013bd0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 1 -- 172.21.15.94:0/3030643826 shutdown_connections 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ae70c8c1d0 0x55ae70c8e5f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 0x55ae70c79800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c79070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 1 --2- 172.21.15.94:0/3030643826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 0x55ae70c7bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 1 -- 172.21.15.94:0/3030643826 >> 172.21.15.94:0/3030643826 conn(0x55ae70c3a310 msgr2=0x55ae70c73930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1de2ff5640 0 client.26209 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 1 -- 172.21.15.94:0/3030643826 shutdown_connections 2024-09-19T04:25:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 1 -- 172.21.15.94:0/3030643826 wait complete. 2024-09-19T04:25:17.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.773+0000 7f1e07728740 20 client.26209 trim_cache size 0 max 16384 2024-09-19T04:25:17.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.794+0000 7f1e07728740 1 Processor -- start 2024-09-19T04:25:17.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.794+0000 7f1e07728740 1 -- start start 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1e07728740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 0x55ae70c6de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1e07728740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c78930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1e07728740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 0x55ae70c93ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1e07728740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001b6700 con 0x55ae70c7c370 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1e07728740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001b3f70 con 0x55ae70c715d0 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1e07728740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001b1de0 con 0x55ae70c72b40 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 0x55ae70c93ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c78930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 0x55ae70c93ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53816/0 (socket says 172.21.15.94:53816) 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfe7fc640 1 -- 172.21.15.94:0/2773388540 learned_addr learned my addr 172.21.15.94:0/2773388540 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/2773388540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 0x55ae70c6de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfeffd640 1 -- 172.21.15.94:0/2773388540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 msgr2=0x55ae70c6de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/2773388540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 0x55ae70c6de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfeffd640 1 -- 172.21.15.94:0/2773388540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 msgr2=0x55ae70c93ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/2773388540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 0x55ae70c93ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.795+0000 7f1dfeffd640 1 -- 172.21.15.94:0/2773388540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae70c78020 con 0x55ae70c72b40 2024-09-19T04:25:17.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.796+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/2773388540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c78930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1df000f980 tx=0x7f1df0005690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c4d7c66b50fbdfe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.796+0000 7f1de4ff9640 1 -- 172.21.15.94:0/2773388540 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0030070 con 0x55ae70c72b40 2024-09-19T04:25:17.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.796+0000 7f1de4ff9640 1 -- 172.21.15.94:0/2773388540 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df0011d50 con 0x55ae70c72b40 2024-09-19T04:25:17.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.796+0000 7f1de4ff9640 1 -- 172.21.15.94:0/2773388540 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0005350 con 0x55ae70c72b40 2024-09-19T04:25:17.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.797+0000 7f1e07728740 1 -- 172.21.15.94:0/2773388540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 msgr2=0x55ae70c78930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.797+0000 7f1e07728740 1 --2- 172.21.15.94:0/2773388540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c78930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1df000f980 tx=0x7f1df0005690 comp rx=0 tx=0).stop 2024-09-19T04:25:17.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.798+0000 7f1e07728740 1 -- 172.21.15.94:0/2773388540 shutdown_connections 2024-09-19T04:25:17.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.798+0000 7f1e07728740 1 --2- 172.21.15.94:0/2773388540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c7c370 0x55ae70c93ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.798+0000 7f1e07728740 1 --2- 172.21.15.94:0/2773388540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c78930 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.798+0000 7f1e07728740 1 --2- 172.21.15.94:0/2773388540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c715d0 0x55ae70c6de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.798+0000 7f1e07728740 1 -- 172.21.15.94:0/2773388540 >> 172.21.15.94:0/2773388540 conn(0x55ae70c3a310 msgr2=0x55ae70c6d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.798+0000 7f1e07728740 1 -- 172.21.15.94:0/2773388540 shutdown_connections 2024-09-19T04:25:17.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.798+0000 7f1e07728740 1 -- 172.21.15.94:0/2773388540 wait complete. 2024-09-19T04:25:17.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.799+0000 7f1e07728740 1 -- start start 2024-09-19T04:25:17.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.799+0000 7f1e07728740 1 Processor -- start 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.799+0000 7f1e07728740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 0x55ae70c84940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.800+0000 7f1e07728740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c75d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.800+0000 7f1dfe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 0x55ae70c84940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.800+0000 7f1e07728740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c7c370 0x55ae70c762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.800+0000 7f1dfe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 0x55ae70c84940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53832/0 (socket says 172.21.15.94:53832) 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.800+0000 7f1dfe7fc640 1 -- 172.21.15.94:0/3828232293 learned_addr learned my addr 172.21.15.94:0/3828232293 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.800+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001b2430 con 0x55ae70c715d0 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.800+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001b38a0 con 0x55ae70c7c370 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.800+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001b2b00 con 0x55ae70c72b40 2024-09-19T04:25:17.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.800+0000 7f1e07728740 10 client.0 _ll_register_callbacks cb 0x7f1df0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:17.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.801+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c75d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.801+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c7c370 0x55ae70c762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.801+0000 7f1dfeffd640 1 -- 172.21.15.94:0/3828232293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c7c370 msgr2=0x55ae70c762a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.801+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c7c370 0x55ae70c762a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.801+0000 7f1dfeffd640 1 -- 172.21.15.94:0/3828232293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 msgr2=0x55ae70c84940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.801+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 0x55ae70c84940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.801+0000 7f1dfeffd640 1 -- 172.21.15.94:0/3828232293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae70c7c130 con 0x55ae70c72b40 2024-09-19T04:25:17.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.801+0000 7f1dfeffd640 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c75d60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1df00136e0 tx=0x7f1df0005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=688345a9629ba03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.801+0000 7f1de2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.802+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3828232293 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0020070 con 0x55ae70c72b40 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.802+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3828232293 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f1df001b7d0 con 0x55ae70c72b40 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.802+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df0011d50 con 0x55ae70c72b40 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.803+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3828232293 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1df0030070 con 0x55ae70c72b40 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.803+0000 7f1e07728740 10 client.26215 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.803+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ae70cf6a90 con 0x55ae70c72b40 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.803+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3828232293 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df0011d50 con 0x55ae70c72b40 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.803+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3828232293 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f1df0003aa0 con 0x55ae70c72b40 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.803+0000 7f1e07728740 10 client.26215 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:17.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.804+0000 7f1e07728740 10 client.26215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:17.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.804+0000 7f1e07728740 20 client.26215 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:17.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.804+0000 7f1e07728740 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ae70cf7180 0x55ae70cf95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.804+0000 7f1e07728740 4 client.26215 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:25:17.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.804+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae70bded70 con 0x55ae70cf7180 2024-09-19T04:25:17.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.804+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ae70cf7180 0x55ae70cf95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.804+0000 7f1dfe7fc640 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ae70cf7180 0x55ae70cf95a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f4db8cb321b5c0 server_cookie=59b86b624e43d49a in_seq=0 out_seq=0 2024-09-19T04:25:17.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.804+0000 7f1de2ff5640 10 client.26215 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:17.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1de2ff5640 1 -- 172.21.15.94:0/3828232293 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae70bded70 con 0x55ae70cf7180 2024-09-19T04:25:17.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1de2ff5640 10 client.26215 handle_command_reply: tid=0 2024-09-19T04:25:17.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 client.26215 shutdown 2024-09-19T04:25:17.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 msgr2=0x55ae70c75d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c75d60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1df00136e0 tx=0x7f1df0005850 comp rx=0 tx=0).stop 2024-09-19T04:25:17.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 shutdown_connections 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55ae70cf7180 0x55ae70cf95a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae70c7c370 0x55ae70c762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae70c72b40 0x55ae70c75d60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 --2- 172.21.15.94:0/3828232293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae70c715d0 0x55ae70c84940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1de2ff5640 0 client.26215 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 >> 172.21.15.94:0/3828232293 conn(0x55ae70c3a310 msgr2=0x55ae70c7b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 shutdown_connections 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 1 -- 172.21.15.94:0/3828232293 wait complete. 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.805+0000 7f1e07728740 20 client.26215 trim_cache size 0 max 16384 2024-09-19T04:25:17.810 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.806+0000 7f1e0534f640 1 -- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1de80a34b0 msgr2=0x7f1de80a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.806+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1de80a34b0 0x7f1de80a5970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1df80b9680 tx=0x7f1df80095c0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.806+0000 7f1e0534f640 1 -- 172.21.15.94:0/1998445273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e0010e540 msgr2=0x7f1e001a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.806+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e0010e540 0x7f1e001a45f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1df802aeb0 tx=0x7f1df80093d0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.806+0000 7f1e0534f640 1 -- 172.21.15.94:0/1998445273 shutdown_connections 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.807+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1de80a34b0 0x7f1de80a5970 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.807+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e0010e540 0x7f1e001a45f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.807+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e00101480 0x7f1e0010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.807+0000 7f1e0534f640 1 --2- 172.21.15.94:0/1998445273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e00100b60 0x7f1e0010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.807+0000 7f1e0534f640 1 -- 172.21.15.94:0/1998445273 >> 172.21.15.94:0/1998445273 conn(0x7f1e000fc7d0 msgr2=0x7f1e000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.807+0000 7f1e0534f640 1 -- 172.21.15.94:0/1998445273 shutdown_connections 2024-09-19T04:25:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:17.807+0000 7f1e0534f640 1 -- 172.21.15.94:0/1998445273 wait complete. 2024-09-19T04:25:18.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:18 smithi104 ceph-mon[34145]: pgmap v319: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.9 MiB/s wr, 1.15k op/s 2024-09-19T04:25:19.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:18 smithi112 ceph-mon[35051]: pgmap v319: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.9 MiB/s wr, 1.15k op/s 2024-09-19T04:25:19.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:18 smithi094 ceph-mon[30972]: pgmap v319: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.9 MiB/s wr, 1.15k op/s 2024-09-19T04:25:19.618 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:25:19.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:25:20.050 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 545.0 s, 1331.2 tps, lat 74.875 ms stddev 38.595 2024-09-19T04:25:20.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.644+0000 7f22b5871640 1 -- 172.21.15.94:0/1482251310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b0108020 msgr2=0x7f22b0108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.644+0000 7f22b5871640 1 --2- 172.21.15.94:0/1482251310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b0108020 0x7f22b0108440 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009900 tx=0x7f22a402a400 comp rx=0 tx=0).stop 2024-09-19T04:25:20.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.644+0000 7f22b5871640 1 -- 172.21.15.94:0/1482251310 shutdown_connections 2024-09-19T04:25:20.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.644+0000 7f22b5871640 1 --2- 172.21.15.94:0/1482251310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22b0108980 0x7f22b0114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.644+0000 7f22b5871640 1 --2- 172.21.15.94:0/1482251310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b0108020 0x7f22b0108440 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.644+0000 7f22b5871640 1 --2- 172.21.15.94:0/1482251310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22b010e060 0x7f22b010e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.644+0000 7f22b5871640 1 -- 172.21.15.94:0/1482251310 >> 172.21.15.94:0/1482251310 conn(0x7f22b0077980 msgr2=0x7f22b0079da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.644+0000 7f22b5871640 1 -- 172.21.15.94:0/1482251310 shutdown_connections 2024-09-19T04:25:20.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.644+0000 7f22b5871640 1 -- 172.21.15.94:0/1482251310 wait complete. 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.645+0000 7f22b5871640 1 Processor -- start 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.645+0000 7f22b5871640 1 -- start start 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.645+0000 7f22b5871640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22b0108020 0x7f22b0082830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.645+0000 7f22b5871640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b0108980 0x7f22b007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.645+0000 7f22b5871640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22b010e060 0x7f22b007be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.645+0000 7f22b5871640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01182e0 con 0x7f22b0108020 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.645+0000 7f22b5871640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0118160 con 0x7f22b0108980 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.645+0000 7f22b5871640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0118460 con 0x7f22b010e060 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.646+0000 7f22b5070640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22b010e060 0x7f22b007be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.646+0000 7f22b5070640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22b010e060 0x7f22b007be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45592/0 (socket says 172.21.15.94:45592) 2024-09-19T04:25:20.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.646+0000 7f22b5070640 1 -- 172.21.15.94:0/1859961741 learned_addr learned my addr 172.21.15.94:0/1859961741 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:20.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.646+0000 7f22b5070640 1 -- 172.21.15.94:0/1859961741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b0108980 msgr2=0x7f22b007b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.646+0000 7f22affff640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b0108980 0x7f22b007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.646+0000 7f22b5070640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b0108980 0x7f22b007b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.646+0000 7f22b5070640 1 -- 172.21.15.94:0/1859961741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22b0108020 msgr2=0x7f22b0082830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.646+0000 7f22b5070640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22b0108020 0x7f22b0082830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.646+0000 7f22b5070640 1 -- 172.21.15.94:0/1859961741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b007c410 con 0x7f22b010e060 2024-09-19T04:25:20.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.647+0000 7f22b5070640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22b010e060 0x7f22b007be50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d680 tx=0x7f22a000db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.647+0000 7f22adffb640 1 -- 172.21.15.94:0/1859961741 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0017070 con 0x7f22b010e060 2024-09-19T04:25:20.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.647+0000 7f22adffb640 1 -- 172.21.15.94:0/1859961741 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a0009820 con 0x7f22b010e060 2024-09-19T04:25:20.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.647+0000 7f22adffb640 1 -- 172.21.15.94:0/1859961741 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0004c40 con 0x7f22b010e060 2024-09-19T04:25:20.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.647+0000 7f22b5871640 1 -- 172.21.15.94:0/1859961741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b007c700 con 0x7f22b010e060 2024-09-19T04:25:20.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.647+0000 7f22b5871640 1 -- 172.21.15.94:0/1859961741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b0080680 con 0x7f22b010e060 2024-09-19T04:25:20.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.651+0000 7f22adffb640 1 -- 172.21.15.94:0/1859961741 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f22a0010460 con 0x7f22b010e060 2024-09-19T04:25:20.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.651+0000 7f22b8eac740 1 Processor -- start 2024-09-19T04:25:20.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.651+0000 7f22b8eac740 1 -- start start 2024-09-19T04:25:20.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41b2690 0x55d5d41bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 0x55d5d41bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22b8eac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d5d4117280 con 0x55d5d41bcf50 2024-09-19T04:25:20.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22b8eac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d5d40beb10 con 0x55d5d41b2690 2024-09-19T04:25:20.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22b8eac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d5d4113240 con 0x55d5d41b1d70 2024-09-19T04:25:20.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45598/0 (socket says 172.21.15.94:45598) 2024-09-19T04:25:20.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22affff640 1 -- 172.21.15.94:0/202703446 learned_addr learned my addr 172.21.15.94:0/202703446 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:20.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22affff640 1 --2- 172.21.15.94:0/202703446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 0x55d5d41bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.652+0000 7f22b486f640 1 --2- 172.21.15.94:0/202703446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41b2690 0x55d5d41bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.653+0000 7f22affff640 1 -- 172.21.15.94:0/202703446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41b2690 msgr2=0x55d5d41bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.653+0000 7f22affff640 1 --2- 172.21.15.94:0/202703446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41b2690 0x55d5d41bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.653+0000 7f22affff640 1 -- 172.21.15.94:0/202703446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 msgr2=0x55d5d41bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.653+0000 7f22affff640 1 --2- 172.21.15.94:0/202703446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 0x55d5d41bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.653+0000 7f22affff640 1 -- 172.21.15.94:0/202703446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d40cb1f0 con 0x55d5d41b1d70 2024-09-19T04:25:20.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.653+0000 7f22adffb640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22980a35c0 0x7f22980a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.653+0000 7f22affff640 1 --2- 172.21.15.94:0/202703446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b2150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3192ad42e3101484 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.656 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:20 smithi094 ceph-mon[30972]: pgmap v320: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-09-19T04:25:20.656 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:20 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:25:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.653+0000 7f22adffb640 1 -- 172.21.15.94:0/1859961741 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f22a00cf650 con 0x7f22b010e060 2024-09-19T04:25:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.654+0000 7f2296ffd640 1 -- 172.21.15.94:0/202703446 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4035040 con 0x55d5d41b1d70 2024-09-19T04:25:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.654+0000 7f2296ffd640 1 -- 172.21.15.94:0/202703446 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a4003910 con 0x55d5d41b1d70 2024-09-19T04:25:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.654+0000 7f2296ffd640 1 -- 172.21.15.94:0/202703446 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4004920 con 0x55d5d41b1d70 2024-09-19T04:25:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.654+0000 7f22b486f640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22980a35c0 0x7f22980a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.655+0000 7f22b8eac740 1 -- 172.21.15.94:0/202703446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 msgr2=0x55d5d41b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.655+0000 7f22b8eac740 1 --2- 172.21.15.94:0/202703446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b2150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a4000bc0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.655+0000 7f22b486f640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22980a35c0 0x7f22980a5a80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55d5d41af160 tx=0x7f22a800d040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.655+0000 7f22b8eac740 1 -- 172.21.15.94:0/202703446 shutdown_connections 2024-09-19T04:25:20.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.655+0000 7f22b8eac740 1 --2- 172.21.15.94:0/202703446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 0x55d5d41bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.655+0000 7f22b8eac740 1 --2- 172.21.15.94:0/202703446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41b2690 0x55d5d41bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.655+0000 7f22b8eac740 1 --2- 172.21.15.94:0/202703446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b2150 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.655+0000 7f22b8eac740 1 -- 172.21.15.94:0/202703446 >> 172.21.15.94:0/202703446 conn(0x55d5d41ae300 msgr2=0x55d5d41ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.656+0000 7f22b8eac740 1 -- 172.21.15.94:0/202703446 shutdown_connections 2024-09-19T04:25:20.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.656+0000 7f22b8eac740 1 -- 172.21.15.94:0/202703446 wait complete. 2024-09-19T04:25:20.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.656+0000 7f22b8eac740 1 -- start start 2024-09-19T04:25:20.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.656+0000 7f22b8eac740 1 Processor -- start 2024-09-19T04:25:20.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.657+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.657+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b2690 0x55d5d41ba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.658+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41ba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.658+0000 7f22b8eac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d5d41bfb20 con 0x55d5d41b2690 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.658+0000 7f22b8eac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d5d40d1f10 con 0x55d5d41bcf50 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.658+0000 7f22b8eac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d5d41bfca0 con 0x55d5d41b1d70 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.658+0000 7f22b8eac740 10 client.0 _ll_register_callbacks cb 0x55d5d409aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.658+0000 7f22affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b2690 0x55d5d41ba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.658+0000 7f22affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b2690 0x55d5d41ba420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53852/0 (socket says 172.21.15.94:53852) 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.658+0000 7f22affff640 1 -- 172.21.15.94:0/2391572826 learned_addr learned my addr 172.21.15.94:0/2391572826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.658+0000 7f22b486f640 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.659+0000 7f22b5070640 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41ba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.659+0000 7f22b5070640 1 -- 172.21.15.94:0/2391572826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 msgr2=0x55d5d41b9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.659+0000 7f22b5070640 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.659+0000 7f22b5070640 1 -- 172.21.15.94:0/2391572826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b2690 msgr2=0x55d5d41ba420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.659+0000 7f22b5070640 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b2690 0x55d5d41ba420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.659+0000 7f22b5070640 1 -- 172.21.15.94:0/2391572826 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d3e886a0 con 0x55d5d41bcf50 2024-09-19T04:25:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.659+0000 7f22b5070640 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41ba960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22a00ccd00 tx=0x7f22a009fc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59f83aea0822a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.659+0000 7f228ffff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:25:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.660+0000 7f228ffff640 1 -- 172.21.15.94:0/2391572826 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0020040 con 0x55d5d41bcf50 2024-09-19T04:25:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.660+0000 7f228ffff640 1 -- 172.21.15.94:0/2391572826 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f22a009cbc0 con 0x55d5d41bcf50 2024-09-19T04:25:20.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.660+0000 7f22b8eac740 1 -- 172.21.15.94:0/2391572826 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a4003910 con 0x55d5d41bcf50 2024-09-19T04:25:20.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.661+0000 7f228ffff640 1 -- 172.21.15.94:0/2391572826 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f22a00cd180 con 0x55d5d41bcf50 2024-09-19T04:25:20.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.661+0000 7f22b8eac740 10 client.26264 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:20.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.661+0000 7f22b8eac740 1 -- 172.21.15.94:0/2391572826 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d5d41cb860 con 0x55d5d41bcf50 2024-09-19T04:25:20.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.661+0000 7f228ffff640 1 -- 172.21.15.94:0/2391572826 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a0099c50 con 0x55d5d41bcf50 2024-09-19T04:25:20.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.661+0000 7f228ffff640 1 -- 172.21.15.94:0/2391572826 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f22a00975e0 con 0x55d5d41bcf50 2024-09-19T04:25:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.661+0000 7f22b8eac740 10 client.26264 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.661+0000 7f22b8eac740 10 client.26264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.661+0000 7f22b8eac740 20 client.26264 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.661+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d5d41cd210 0x55d5d41cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.662+0000 7f22b8eac740 4 client.26264 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.662+0000 7f22b8eac740 1 -- 172.21.15.94:0/2391572826 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22a4003910 con 0x55d5d41cd210 2024-09-19T04:25:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.662+0000 7f22b486f640 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d5d41cd210 0x55d5d41cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.662+0000 7f228ffff640 10 client.26264 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.662+0000 7f22b486f640 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d5d41cd210 0x55d5d41cf650 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16474a4b6ba074d4 server_cookie=6e1d1d8aba8a22ba in_seq=0 out_seq=0 2024-09-19T04:25:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.663+0000 7f228ffff640 1 -- 172.21.15.94:0/2391572826 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f22a4003910 con 0x55d5d41cd210 2024-09-19T04:25:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.663+0000 7f228ffff640 10 client.26264 handle_command_reply: tid=0 2024-09-19T04:25:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.663+0000 7f22b8eac740 1 client.26264 shutdown 2024-09-19T04:25:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f22b8eac740 1 -- 172.21.15.94:0/2391572826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 msgr2=0x55d5d41ba960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41ba960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22a00ccd00 tx=0x7f22a009fc40 comp rx=0 tx=0).stop 2024-09-19T04:25:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f22b8eac740 1 -- 172.21.15.94:0/2391572826 shutdown_connections 2024-09-19T04:25:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d5d41cd210 0x55d5d41cf650 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41ba960 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f228ffff640 0 client.26264 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b2690 0x55d5d41ba420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2391572826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b1d70 0x55d5d41b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f22b8eac740 1 -- 172.21.15.94:0/2391572826 >> 172.21.15.94:0/2391572826 conn(0x55d5d41ae300 msgr2=0x55d5d41b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.664+0000 7f22b8eac740 1 -- 172.21.15.94:0/2391572826 shutdown_connections 2024-09-19T04:25:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.665+0000 7f22b8eac740 1 -- 172.21.15.94:0/2391572826 wait complete. 2024-09-19T04:25:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.665+0000 7f22b8eac740 20 client.26264 trim_cache size 0 max 16384 2024-09-19T04:25:20.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.696+0000 7f22b8eac740 1 Processor -- start 2024-09-19T04:25:20.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.696+0000 7f22b8eac740 1 -- start start 2024-09-19T04:25:20.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.696+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41b1d70 0x55d5d41b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.696+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41d4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.696+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 0x55d5d41d54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.696+0000 7f22b8eac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f229c011250 con 0x55d5d41bcf50 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.696+0000 7f22b8eac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00eac0 con 0x55d5d41b1d70 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.696+0000 7f22b8eac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00c930 con 0x55d5d41b2690 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41d4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41d4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45606/0 (socket says 172.21.15.94:45606) 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22affff640 1 -- 172.21.15.94:0/2403663729 learned_addr learned my addr 172.21.15.94:0/2403663729 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22b5070640 1 --2- 172.21.15.94:0/2403663729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 0x55d5d41d54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22affff640 1 -- 172.21.15.94:0/2403663729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41b1d70 msgr2=0x55d5d41b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22affff640 1 --2- 172.21.15.94:0/2403663729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41b1d70 0x55d5d41b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22affff640 1 -- 172.21.15.94:0/2403663729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 msgr2=0x55d5d41d54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22affff640 1 --2- 172.21.15.94:0/2403663729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 0x55d5d41d54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22affff640 1 -- 172.21.15.94:0/2403663729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d41b8e90 con 0x55d5d41b2690 2024-09-19T04:25:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.697+0000 7f22affff640 1 --2- 172.21.15.94:0/2403663729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41d4f60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f22a40099f0 tx=0x7f22a402a920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=586b8ff619c3e067 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.698+0000 7f22967fc640 1 -- 172.21.15.94:0/2403663729 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a404d020 con 0x55d5d41b2690 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.698+0000 7f22967fc640 1 -- 172.21.15.94:0/2403663729 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a4042070 con 0x55d5d41b2690 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.698+0000 7f22967fc640 1 -- 172.21.15.94:0/2403663729 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4035040 con 0x55d5d41b2690 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.699+0000 7f22b8eac740 1 -- 172.21.15.94:0/2403663729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 msgr2=0x55d5d41d4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.699+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2403663729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41d4f60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f22a40099f0 tx=0x7f22a402a920 comp rx=0 tx=0).stop 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.699+0000 7f22b8eac740 1 -- 172.21.15.94:0/2403663729 shutdown_connections 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.699+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2403663729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41bcf50 0x55d5d41d54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.699+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2403663729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41d4f60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.699+0000 7f22b8eac740 1 --2- 172.21.15.94:0/2403663729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41b1d70 0x55d5d41b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.699+0000 7f22b8eac740 1 -- 172.21.15.94:0/2403663729 >> 172.21.15.94:0/2403663729 conn(0x55d5d41ae300 msgr2=0x55d5d41b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.700+0000 7f22b8eac740 1 -- 172.21.15.94:0/2403663729 shutdown_connections 2024-09-19T04:25:20.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.700+0000 7f22b8eac740 1 -- 172.21.15.94:0/2403663729 wait complete. 2024-09-19T04:25:20.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.701+0000 7f22b8eac740 1 -- start start 2024-09-19T04:25:20.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.701+0000 7f22b8eac740 1 Processor -- start 2024-09-19T04:25:20.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.701+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b1d70 0x55d5d41c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.701+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.701+0000 7f22b8eac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41c1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.701+0000 7f22b8eac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00cf80 con 0x55d5d41b1d70 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.701+0000 7f22b8eac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00e3f0 con 0x55d5d41bcf50 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.701+0000 7f22b8eac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00d650 con 0x55d5d41b2690 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22b8eac740 10 client.0 _ll_register_callbacks cb 0x7f22a009b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:45612/0 (socket says 172.21.15.94:45612) 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22b486f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b1d70 0x55d5d41c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22affff640 1 -- 172.21.15.94:0/1907715793 learned_addr learned my addr 172.21.15.94:0/1907715793 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22b5070640 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22affff640 1 -- 172.21.15.94:0/1907715793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 msgr2=0x55d5d41c1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22affff640 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41c1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22affff640 1 -- 172.21.15.94:0/1907715793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b1d70 msgr2=0x55d5d41c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22affff640 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b1d70 0x55d5d41c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22affff640 1 -- 172.21.15.94:0/1907715793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d41c1cb0 con 0x55d5d41b2690 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22b5070640 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.702+0000 7f22b486f640 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b1d70 0x55d5d41c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.703+0000 7f22affff640 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41c5580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f22a402a9f0 tx=0x7f22a4003af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffcded62d2042025 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.703+0000 7f228ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.703+0000 7f228ffff640 1 -- 172.21.15.94:0/1907715793 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4035070 con 0x55d5d41b2690 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.703+0000 7f228ffff640 1 -- 172.21.15.94:0/1907715793 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f22a4005430 con 0x55d5d41b2690 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.703+0000 7f22b8eac740 1 -- 172.21.15.94:0/1907715793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a4042070 con 0x55d5d41b2690 2024-09-19T04:25:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.704+0000 7f228ffff640 1 -- 172.21.15.94:0/1907715793 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f22a404d040 con 0x55d5d41b2690 2024-09-19T04:25:20.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.704+0000 7f22b8eac740 10 client.26233 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:20.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.704+0000 7f22b8eac740 1 -- 172.21.15.94:0/1907715793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d5d4238a60 con 0x55d5d41b2690 2024-09-19T04:25:20.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.704+0000 7f228ffff640 1 -- 172.21.15.94:0/1907715793 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a4042070 con 0x55d5d41b2690 2024-09-19T04:25:20.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.705+0000 7f228ffff640 1 -- 172.21.15.94:0/1907715793 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f22a405a030 con 0x55d5d41b2690 2024-09-19T04:25:20.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.705+0000 7f22b8eac740 10 client.26233 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.705+0000 7f22b8eac740 10 client.26233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.705+0000 7f22b8eac740 20 client.26233 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.705+0000 7f22b8eac740 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d5d41cd210 0x55d5d423acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.705+0000 7f22b8eac740 4 client.26233 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:25:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.705+0000 7f22b8eac740 1 -- 172.21.15.94:0/1907715793 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d5d41247f0 con 0x55d5d41cd210 2024-09-19T04:25:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.706+0000 7f22b486f640 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d5d41cd210 0x55d5d423acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.706+0000 7f228ffff640 10 client.26233 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.706+0000 7f22b486f640 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d5d41cd210 0x55d5d423acf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69742697eb54a45a server_cookie=7987a0fe08984853 in_seq=0 out_seq=0 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.708+0000 7f228ffff640 1 -- 172.21.15.94:0/1907715793 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d5d41247f0 con 0x55d5d41cd210 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.709+0000 7f228ffff640 10 client.26233 handle_command_reply: tid=0 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.709+0000 7f22b8eac740 1 client.26233 shutdown 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.709+0000 7f22b8eac740 1 -- 172.21.15.94:0/1907715793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 msgr2=0x55d5d41c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.709+0000 7f22b8eac740 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41c5580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f22a402a9f0 tx=0x7f22a4003af0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.710+0000 7f22b8eac740 1 -- 172.21.15.94:0/1907715793 shutdown_connections 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.710+0000 7f22b8eac740 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d5d41cd210 0x55d5d423acf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.710+0000 7f22b8eac740 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5d41bcf50 0x55d5d41c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.710+0000 7f228ffff640 0 client.26233 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:20.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.710+0000 7f22b8eac740 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5d41b2690 0x55d5d41c5580 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.710+0000 7f22b8eac740 1 --2- 172.21.15.94:0/1907715793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5d41b1d70 0x55d5d41c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.710+0000 7f22b8eac740 1 -- 172.21.15.94:0/1907715793 >> 172.21.15.94:0/1907715793 conn(0x55d5d41ae300 msgr2=0x55d5d41b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.711+0000 7f22b8eac740 1 -- 172.21.15.94:0/1907715793 shutdown_connections 2024-09-19T04:25:20.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.711+0000 7f22b8eac740 1 -- 172.21.15.94:0/1907715793 wait complete. 2024-09-19T04:25:20.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.711+0000 7f22b8eac740 20 client.26233 trim_cache size 0 max 16384 2024-09-19T04:25:20.715 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:25:20.715 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:25:20.715 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "b464808a-56b6-46d6-b003-dd8bb739a31d", 2024-09-19T04:25:20.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:25:20.715 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:25:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 -- 172.21.15.94:0/1859961741 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22980a35c0 msgr2=0x7f22980a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22980a35c0 0x7f22980a5a80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55d5d41af160 tx=0x7f22a800d040 comp rx=0 tx=0).stop 2024-09-19T04:25:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 -- 172.21.15.94:0/1859961741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22b010e060 msgr2=0x7f22b007be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22b010e060 0x7f22b007be50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d680 tx=0x7f22a000db40 comp rx=0 tx=0).stop 2024-09-19T04:25:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 -- 172.21.15.94:0/1859961741 shutdown_connections 2024-09-19T04:25:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22980a35c0 0x7f22980a5a80 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55d5d41af160 tx=0x7f22a800d040 comp rx=0 tx=0).stop 2024-09-19T04:25:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22b010e060 0x7f22b007be50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b0108980 0x7f22b007b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 --2- 172.21.15.94:0/1859961741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22b0108020 0x7f22b0082830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.714+0000 7f22b5871640 1 -- 172.21.15.94:0/1859961741 >> 172.21.15.94:0/1859961741 conn(0x7f22b0077980 msgr2=0x7f22b0079200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.715+0000 7f22b5871640 1 -- 172.21.15.94:0/1859961741 shutdown_connections 2024-09-19T04:25:20.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:20.715+0000 7f22b5871640 1 -- 172.21.15.94:0/1859961741 wait complete. 2024-09-19T04:25:20.934 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:20 smithi104 ceph-mon[34145]: pgmap v320: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-09-19T04:25:20.934 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:20 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:25:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:20 smithi112 ceph-mon[35051]: pgmap v320: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-09-19T04:25:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:20 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:25:21.438 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:25:22.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:25:22.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:25:22.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:25:22.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:25:22.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:25:22.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:22.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:21 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:22.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:25:22.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:25:22.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:25:22.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:25:22.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:25:22.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:22.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:21 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:22.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:25:22.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:25:22.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:25:22.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:25:22.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:25:22.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:22.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:22.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:21 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:23.020 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:23.020 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: pgmap v321: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-09-19T04:25:23.020 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:23.020 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: Unable to set osd_memory_target on smithi104 to 184576716: error parsing value: Value '184576716' is below minimum 939524096 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.021 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-19T04:25:23.022 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:22 smithi094 ceph-mon[30972]: Unable to set osd_memory_target on smithi112 to 184553779: error parsing value: Value '184553779' is below minimum 939524096 2024-09-19T04:25:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: pgmap v321: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-09-19T04:25:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-19T04:25:23.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: Unable to set osd_memory_target on smithi104 to 184576716: error parsing value: Value '184576716' is below minimum 939524096 2024-09-19T04:25:23.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-19T04:25:23.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:22 smithi112 ceph-mon[35051]: Unable to set osd_memory_target on smithi112 to 184553779: error parsing value: Value '184553779' is below minimum 939524096 2024-09-19T04:25:23.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:23.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: pgmap v321: 97 pgs: 97 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: Unable to set osd_memory_target on smithi104 to 184576716: error parsing value: Value '184576716' is below minimum 939524096 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:23.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-19T04:25:23.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:22 smithi104 ceph-mon[34145]: Unable to set osd_memory_target on smithi112 to 184553779: error parsing value: Value '184553779' is below minimum 939524096 2024-09-19T04:25:25.024 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:25.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:24 smithi112 ceph-mon[35051]: pgmap v322: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-19T04:25:25.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:24 smithi094 ceph-mon[30972]: pgmap v322: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-19T04:25:25.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:24 smithi104 ceph-mon[34145]: pgmap v322: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-19T04:25:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 550.0 s, 1231.8 tps, lat 81.393 ms stddev 33.388 2024-09-19T04:25:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.606+0000 7f427ff7d640 1 -- 172.21.15.94:0/671095647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4278100090 msgr2=0x7f4278100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.606+0000 7f427ff7d640 1 --2- 172.21.15.94:0/671095647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4278100090 0x7f4278100510 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4268009900 tx=0x7f426802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.606+0000 7f427ff7d640 1 -- 172.21.15.94:0/671095647 shutdown_connections 2024-09-19T04:25:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.606+0000 7f427ff7d640 1 --2- 172.21.15.94:0/671095647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4278100a50 0x7f42781057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.606+0000 7f427ff7d640 1 --2- 172.21.15.94:0/671095647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4278100090 0x7f4278100510 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.606+0000 7f427ff7d640 1 --2- 172.21.15.94:0/671095647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4278106b70 0x7f4278106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.606+0000 7f427ff7d640 1 -- 172.21.15.94:0/671095647 >> 172.21.15.94:0/671095647 conn(0x7f4278077210 msgr2=0x7f42780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.607+0000 7f427ff7d640 1 -- 172.21.15.94:0/671095647 shutdown_connections 2024-09-19T04:25:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.607+0000 7f427ff7d640 1 -- 172.21.15.94:0/671095647 wait complete. 2024-09-19T04:25:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.607+0000 7f427ff7d640 1 Processor -- start 2024-09-19T04:25:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427ff7d640 1 -- start start 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427ff7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4278100090 0x7f42781a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427ff7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4278100a50 0x7f42781a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427ff7d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4278106b70 0x7f427819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427ff7d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4278118480 con 0x7f4278100a50 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427ff7d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4278118300 con 0x7f4278106b70 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427ff7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4278118600 con 0x7f4278100090 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427dd19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4278100090 0x7f42781a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427dd19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4278100090 0x7f42781a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:38184/0 (socket says 172.21.15.94:38184) 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427dd19640 1 -- 172.21.15.94:0/2284726411 learned_addr learned my addr 172.21.15.94:0/2284726411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427d518640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4278100a50 0x7f42781a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427dd19640 1 -- 172.21.15.94:0/2284726411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4278106b70 msgr2=0x7f427819e750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427dd19640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4278106b70 0x7f427819e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427dd19640 1 -- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4278100a50 msgr2=0x7f42781a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427dd19640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4278100a50 0x7f42781a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.608+0000 7f427dd19640 1 -- 172.21.15.94:0/2284726411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427819efc0 con 0x7f4278100090 2024-09-19T04:25:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.609+0000 7f427dd19640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4278100090 0x7f42781a4680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f427400d950 tx=0x7f427400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.609+0000 7f426effd640 1 -- 172.21.15.94:0/2284726411 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274017070 con 0x7f4278100090 2024-09-19T04:25:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.609+0000 7f426effd640 1 -- 172.21.15.94:0/2284726411 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4274009c30 con 0x7f4278100090 2024-09-19T04:25:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.609+0000 7f427ff7d640 1 -- 172.21.15.94:0/2284726411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427819f2b0 con 0x7f4278100090 2024-09-19T04:25:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.609+0000 7f427ff7d640 1 -- 172.21.15.94:0/2284726411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42781ab600 con 0x7f4278100090 2024-09-19T04:25:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.610+0000 7f426effd640 1 -- 172.21.15.94:0/2284726411 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427400f620 con 0x7f4278100090 2024-09-19T04:25:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.613+0000 7f426effd640 1 -- 172.21.15.94:0/2284726411 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f427400f7c0 con 0x7f4278100090 2024-09-19T04:25:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.613+0000 7f4282356740 1 Processor -- start 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.614+0000 7f4282356740 1 -- start start 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f4282356740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca607f4a0 0x55fca607f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f4282356740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca608a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f4282356740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca608a680 0x55fca608ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f4282356740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fca5fe52e0 con 0x55fca607f4a0 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f4282356740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fca5f8cce0 con 0x55fca607fdc0 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f4282356740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fca5fe12a0 con 0x55fca608a680 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f427e51a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca607f4a0 0x55fca607f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f427e51a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca607f4a0 0x55fca607f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59396/0 (socket says 172.21.15.94:59396) 2024-09-19T04:25:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f427e51a640 1 -- 172.21.15.94:0/4032052830 learned_addr learned my addr 172.21.15.94:0/4032052830 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f427dd19640 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca608a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f427d518640 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca608a680 0x55fca608ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.615+0000 7f426effd640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42640a3580 0x7f42640a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f427dd19640 1 -- 172.21.15.94:0/4032052830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca608a680 msgr2=0x55fca608ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f427dd19640 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca608a680 0x55fca608ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f427dd19640 1 -- 172.21.15.94:0/4032052830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca607f4a0 msgr2=0x55fca607f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f427dd19640 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca607f4a0 0x55fca607f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f427dd19640 1 -- 172.21.15.94:0/4032052830 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca5f71440 con 0x55fca607fdc0 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f427e51a640 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca607f4a0 0x55fca607f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f426effd640 1 -- 172.21.15.94:0/2284726411 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f42740cf820 con 0x7f4278100090 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f427dd19640 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca608a140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42740cced0 tx=0x7f4274067540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6b9f1fbf897b60f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f4263fff640 1 -- 172.21.15.94:0/4032052830 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274020070 con 0x55fca607fdc0 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f427e51a640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42640a3580 0x7f42640a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f4263fff640 1 -- 172.21.15.94:0/4032052830 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f427409c160 con 0x55fca607fdc0 2024-09-19T04:25:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.616+0000 7f4263fff640 1 -- 172.21.15.94:0/4032052830 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427409ba60 con 0x55fca607fdc0 2024-09-19T04:25:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f427e51a640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42640a3580 0x7f42640a5a40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4270005c40 tx=0x7f4270004ab0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f4282356740 1 -- 172.21.15.94:0/4032052830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 msgr2=0x55fca608a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f4282356740 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca608a140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42740cced0 tx=0x7f4274067540 comp rx=0 tx=0).stop 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f4282356740 1 -- 172.21.15.94:0/4032052830 shutdown_connections 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f4282356740 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca608a680 0x55fca608ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f4282356740 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca608a140 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f4282356740 1 --2- 172.21.15.94:0/4032052830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca607f4a0 0x55fca607f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f4282356740 1 -- 172.21.15.94:0/4032052830 >> 172.21.15.94:0/4032052830 conn(0x55fca607ba30 msgr2=0x55fca607be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f4282356740 1 -- 172.21.15.94:0/4032052830 shutdown_connections 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.618+0000 7f4282356740 1 -- 172.21.15.94:0/4032052830 wait complete. 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.619+0000 7f4282356740 1 -- start start 2024-09-19T04:25:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.619+0000 7f4282356740 1 Processor -- start 2024-09-19T04:25:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f4282356740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca6089f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f4282356740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f4282356740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f427dd19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca6089f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f427dd19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca6089f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:38188/0 (socket says 172.21.15.94:38188) 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f427d518640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f427dd19640 1 -- 172.21.15.94:0/3119712996 learned_addr learned my addr 172.21.15.94:0/3119712996 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f427e51a640 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fca609e200 con 0x55fca608a680 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fca5fa0050 con 0x55fca607fdc0 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.620+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fca609e380 con 0x55fca607f4a0 2024-09-19T04:25:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.621+0000 7f4282356740 10 client.0 _ll_register_callbacks cb 0x55fca5fe0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:25.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.621+0000 7f427d518640 1 -- 172.21.15.94:0/3119712996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 msgr2=0x55fca6089f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.621+0000 7f427d518640 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca6089f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.621+0000 7f427d518640 1 -- 172.21.15.94:0/3119712996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 msgr2=0x55fca60876f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.621+0000 7f427d518640 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60876f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.621+0000 7f427d518640 1 -- 172.21.15.94:0/3119712996 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca5d563c0 con 0x55fca607fdc0 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.621+0000 7f427d518640 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60871b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f426802a9e0 tx=0x7f42680043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4287125959702390 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.622+0000 7f42617fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.622+0000 7f42617fa640 1 -- 172.21.15.94:0/3119712996 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268035040 con 0x55fca607fdc0 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.622+0000 7f42617fa640 1 -- 172.21.15.94:0/3119712996 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f4268005450 con 0x55fca607fdc0 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.622+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f427409c160 con 0x55fca607fdc0 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.622+0000 7f42617fa640 1 -- 172.21.15.94:0/3119712996 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4268004770 con 0x55fca607fdc0 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f4282356740 10 client.26276 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fca60a9b40 con 0x55fca607fdc0 2024-09-19T04:25:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f42617fa640 1 -- 172.21.15.94:0/3119712996 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f427409c160 con 0x55fca607fdc0 2024-09-19T04:25:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f42617fa640 1 -- 172.21.15.94:0/3119712996 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f4268003880 con 0x55fca607fdc0 2024-09-19T04:25:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f4282356740 10 client.26276 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f4282356740 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f4282356740 20 client.26276 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f4282356740 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fca60ab450 0x55fca60ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f4282356740 4 client.26276 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.623+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca5ff2850 con 0x55fca60ab450 2024-09-19T04:25:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.624+0000 7f427dd19640 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fca60ab450 0x55fca60ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.624+0000 7f427dd19640 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fca60ab450 0x55fca60ad870 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5693b8329e2fa52 server_cookie=47536c2fce7ae3fd in_seq=0 out_seq=0 2024-09-19T04:25:25.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.624+0000 7f42617fa640 10 client.26276 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:25.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.625+0000 7f42617fa640 1 -- 172.21.15.94:0/3119712996 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fca5ff2850 con 0x55fca60ab450 2024-09-19T04:25:25.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.625+0000 7f42617fa640 10 client.26276 handle_command_reply: tid=0 2024-09-19T04:25:25.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.625+0000 7f4282356740 1 client.26276 shutdown 2024-09-19T04:25:25.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 msgr2=0x55fca60871b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60871b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f426802a9e0 tx=0x7f42680043b0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 shutdown_connections 2024-09-19T04:25:25.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fca60ab450 0x55fca60ad870 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60871b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 --2- 172.21.15.94:0/3119712996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca6089f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 >> 172.21.15.94:0/3119712996 conn(0x55fca607ba30 msgr2=0x55fca6081710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f42617fa640 0 client.26276 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:25.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 shutdown_connections 2024-09-19T04:25:25.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 1 -- 172.21.15.94:0/3119712996 wait complete. 2024-09-19T04:25:25.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.626+0000 7f4282356740 20 client.26276 trim_cache size 0 max 16384 2024-09-19T04:25:25.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f4282356740 1 Processor -- start 2024-09-19T04:25:25.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f4282356740 1 -- start start 2024-09-19T04:25:25.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f4282356740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca60a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f4282356740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f4282356740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f427dd19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca60a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f427dd19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca60a7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:38202/0 (socket says 172.21.15.94:38202) 2024-09-19T04:25:25.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f427dd19640 1 -- 172.21.15.94:0/497779075 learned_addr learned my addr 172.21.15.94:0/497779075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:25.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f4282356740 1 -- 172.21.15.94:0/497779075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f425c011210 con 0x55fca608a680 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f4282356740 1 -- 172.21.15.94:0/497779075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f425c00ea80 con 0x55fca607fdc0 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.658+0000 7f4282356740 1 -- 172.21.15.94:0/497779075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f425c00c8f0 con 0x55fca607f4a0 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.659+0000 7f427dd19640 1 -- 172.21.15.94:0/497779075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 msgr2=0x55fca60b3160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.659+0000 7f427dd19640 1 --2- 172.21.15.94:0/497779075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60b3160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.659+0000 7f427dd19640 1 -- 172.21.15.94:0/497779075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 msgr2=0x55fca60b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.659+0000 7f427dd19640 1 --2- 172.21.15.94:0/497779075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.659+0000 7f427dd19640 1 -- 172.21.15.94:0/497779075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca6087f70 con 0x55fca607f4a0 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.660+0000 7f427dd19640 1 --2- 172.21.15.94:0/497779075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca60a7490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4274097ec0 tx=0x7f42740bbc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52f4d5efc4410311 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.660+0000 7f42637fe640 1 -- 172.21.15.94:0/497779075 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274020040 con 0x55fca607f4a0 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.660+0000 7f42637fe640 1 -- 172.21.15.94:0/497779075 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f427409c160 con 0x55fca607f4a0 2024-09-19T04:25:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.660+0000 7f42637fe640 1 -- 172.21.15.94:0/497779075 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42740cfbd0 con 0x55fca607f4a0 2024-09-19T04:25:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.662+0000 7f4282356740 1 -- 172.21.15.94:0/497779075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 msgr2=0x55fca60a7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.662+0000 7f4282356740 1 --2- 172.21.15.94:0/497779075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca60a7490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4274097ec0 tx=0x7f42740bbc10 comp rx=0 tx=0).stop 2024-09-19T04:25:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.662+0000 7f4282356740 1 -- 172.21.15.94:0/497779075 shutdown_connections 2024-09-19T04:25:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.662+0000 7f4282356740 1 --2- 172.21.15.94:0/497779075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.662+0000 7f4282356740 1 --2- 172.21.15.94:0/497779075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.662+0000 7f4282356740 1 --2- 172.21.15.94:0/497779075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca60a7490 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.662+0000 7f4282356740 1 -- 172.21.15.94:0/497779075 >> 172.21.15.94:0/497779075 conn(0x55fca607ba30 msgr2=0x55fca6089230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.663+0000 7f4282356740 1 -- 172.21.15.94:0/497779075 shutdown_connections 2024-09-19T04:25:25.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.663+0000 7f4282356740 1 -- 172.21.15.94:0/497779075 wait complete. 2024-09-19T04:25:25.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.663+0000 7f4282356740 1 -- start start 2024-09-19T04:25:25.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.663+0000 7f4282356740 1 Processor -- start 2024-09-19T04:25:25.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.664+0000 7f4282356740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca61117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.664+0000 7f4282356740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60a3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f4282356740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f427d518640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f427d518640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33468/0 (socket says 172.21.15.94:33468) 2024-09-19T04:25:25.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f427dd19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca61117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f427d518640 1 -- 172.21.15.94:0/3657939693 learned_addr learned my addr 172.21.15.94:0/3657939693 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:25.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f427e51a640 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f425c00cf40 con 0x55fca608a680 2024-09-19T04:25:25.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f425c00e3b0 con 0x55fca607fdc0 2024-09-19T04:25:25.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f425c00d610 con 0x55fca607f4a0 2024-09-19T04:25:25.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.665+0000 7f4282356740 10 client.0 _ll_register_callbacks cb 0x7f4268009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:25.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.666+0000 7f427e51a640 1 -- 172.21.15.94:0/3657939693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 msgr2=0x55fca61117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.666+0000 7f427e51a640 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca61117a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.666+0000 7f427e51a640 1 -- 172.21.15.94:0/3657939693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 msgr2=0x55fca60a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.666+0000 7f427e51a640 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.666+0000 7f427e51a640 1 -- 172.21.15.94:0/3657939693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca60a4100 con 0x55fca608a680 2024-09-19T04:25:25.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.666+0000 7f427dd19640 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca61117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:25.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.667+0000 7f427e51a640 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60a3b40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4270012010 tx=0x7f4270009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18387089e21253b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.667+0000 7f42617fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.667+0000 7f42617fa640 1 -- 172.21.15.94:0/3657939693 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270004b20 con 0x55fca608a680 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.667+0000 7f42617fa640 1 -- 172.21.15.94:0/3657939693 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f4270004cc0 con 0x55fca608a680 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.667+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f427409c160 con 0x55fca608a680 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.667+0000 7f42617fa640 1 -- 172.21.15.94:0/3657939693 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f427001c6e0 con 0x55fca608a680 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.667+0000 7f4282356740 10 client.16674 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.667+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fca6127140 con 0x55fca608a680 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.668+0000 7f42617fa640 1 -- 172.21.15.94:0/3657939693 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f427409c160 con 0x55fca608a680 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.668+0000 7f42617fa640 1 -- 172.21.15.94:0/3657939693 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f4270026a20 con 0x55fca608a680 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.668+0000 7f4282356740 10 client.16674 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.668+0000 7f4282356740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:25.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.668+0000 7f4282356740 20 client.16674 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:25.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.668+0000 7f4282356740 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fca61277f0 0x55fca6129c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.668+0000 7f4282356740 4 client.16674 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:25:25.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.668+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fca5fecd60 con 0x55fca61277f0 2024-09-19T04:25:25.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.668+0000 7f427dd19640 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fca61277f0 0x55fca6129c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.669+0000 7f42617fa640 10 client.16674 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:25.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.669+0000 7f427dd19640 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fca61277f0 0x55fca6129c10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff49ca3a8defee7 server_cookie=9faf62abe832f90f in_seq=0 out_seq=0 2024-09-19T04:25:25.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f42617fa640 1 -- 172.21.15.94:0/3657939693 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fca5fecd60 con 0x55fca61277f0 2024-09-19T04:25:25.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f42617fa640 10 client.16674 handle_command_reply: tid=0 2024-09-19T04:25:25.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f4282356740 1 client.16674 shutdown 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 msgr2=0x55fca60a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f4282356740 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60a3b40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4270012010 tx=0x7f4270009500 comp rx=0 tx=0).stop 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 shutdown_connections 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f4282356740 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55fca61277f0 0x55fca6129c10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f4282356740 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fca608a680 0x55fca60a3b40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f4282356740 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fca607fdc0 0x55fca60a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f4282356740 1 --2- 172.21.15.94:0/3657939693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca607f4a0 0x55fca61117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 >> 172.21.15.94:0/3657939693 conn(0x55fca607ba30 msgr2=0x55fca607a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.670+0000 7f42617fa640 0 client.16674 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.671+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 shutdown_connections 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.671+0000 7f4282356740 1 -- 172.21.15.94:0/3657939693 wait complete. 2024-09-19T04:25:25.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.671+0000 7f4282356740 20 client.16674 trim_cache size 0 max 16384 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout: "b464808a-56b6-46d6-b003-dd8bb739a31d": { 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "b464808a-56b6-46d6-b003-dd8bb739a31d", 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:25:25.674 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:25:25.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 -- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42640a3580 msgr2=0x7f42640a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42640a3580 0x7f42640a5a40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4270005c40 tx=0x7f4270004ab0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 -- 172.21.15.94:0/2284726411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4278100090 msgr2=0x7f42781a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4278100090 0x7f42781a4680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f427400d950 tx=0x7f427400de10 comp rx=0 tx=0).stop 2024-09-19T04:25:25.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 -- 172.21.15.94:0/2284726411 shutdown_connections 2024-09-19T04:25:25.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f42640a3580 0x7f42640a5a40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4278106b70 0x7f427819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4278100a50 0x7f42781a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 --2- 172.21.15.94:0/2284726411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4278100090 0x7f42781a4680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.672+0000 7f427ff7d640 1 -- 172.21.15.94:0/2284726411 >> 172.21.15.94:0/2284726411 conn(0x7f4278077210 msgr2=0x7f42781037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.673+0000 7f427ff7d640 1 -- 172.21.15.94:0/2284726411 shutdown_connections 2024-09-19T04:25:25.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:25.673+0000 7f427ff7d640 1 -- 172.21.15.94:0/2284726411 wait complete. 2024-09-19T04:25:26.400 INFO:tasks.cephfs.filesystem:scrub status for tag:b464808a-56b6-46d6-b003-dd8bb739a31d - {'path': '/', 'tag': 'b464808a-56b6-46d6-b003-dd8bb739a31d', 'options': 'recursive,force'} 2024-09-19T04:25:27.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:26 smithi112 ceph-mon[35051]: pgmap v323: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-19T04:25:27.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:26 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:27.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:26 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:27.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:26 smithi094 ceph-mon[30972]: pgmap v323: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-19T04:25:27.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:26 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:27.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:26 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:27.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:26 smithi104 ceph-mon[34145]: pgmap v323: 97 pgs: 97 active+clean; 3.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-19T04:25:27.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:26 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:27.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:26 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:28.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:27 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:27 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:27 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:27 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:27 smithi112 ceph-mon[35051]: Adjusting osd_memory_target on smithi094 to 176.0M 2024-09-19T04:25:28.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:27 smithi112 ceph-mon[35051]: Unable to set osd_memory_target on smithi094 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-19T04:25:28.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:27 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:25:28.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:27 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:25:28.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:27 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:27 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:27 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:27 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:27 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:27 smithi094 ceph-mon[30972]: Adjusting osd_memory_target on smithi094 to 176.0M 2024-09-19T04:25:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:27 smithi094 ceph-mon[30972]: Unable to set osd_memory_target on smithi094 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-19T04:25:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:27 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:25:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:27 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:25:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:27 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:27 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:27 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:27 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:27 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:25:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:27 smithi104 ceph-mon[34145]: Adjusting osd_memory_target on smithi094 to 176.0M 2024-09-19T04:25:28.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:27 smithi104 ceph-mon[34145]: Unable to set osd_memory_target on smithi094 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-19T04:25:28.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:27 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:25:28.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:27 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:25:28.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:27 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:25:29.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:29 smithi112 ceph-mon[35051]: pgmap v324: 97 pgs: 97 active+clean; 3.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:25:29.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:29 smithi094 ceph-mon[30972]: pgmap v324: 97 pgs: 97 active+clean; 3.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:25:29.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:29 smithi104 ceph-mon[34145]: pgmap v324: 97 pgs: 97 active+clean; 3.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-09-19T04:25:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 555.0 s, 1345.8 tps, lat 74.410 ms stddev 41.509 2024-09-19T04:25:31.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:31 smithi112 ceph-mon[35051]: pgmap v325: 97 pgs: 97 active+clean; 3.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-09-19T04:25:31.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:31 smithi094 ceph-mon[30972]: pgmap v325: 97 pgs: 97 active+clean; 3.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-09-19T04:25:31.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:31 smithi104 ceph-mon[34145]: pgmap v325: 97 pgs: 97 active+clean; 3.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-09-19T04:25:32.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:32 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:25:32.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:32 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:25:32.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:32 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:25:32.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:32 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:25:32.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:32 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:25:32.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:32 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:25:32.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:32 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:25:32.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:32 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:25:32.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:32 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:25:33.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:33 smithi112 ceph-mon[35051]: pgmap v326: 97 pgs: 97 active+clean; 3.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-09-19T04:25:33.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:33 smithi094 ceph-mon[30972]: pgmap v326: 97 pgs: 97 active+clean; 3.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-09-19T04:25:33.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:33 smithi104 ceph-mon[34145]: pgmap v326: 97 pgs: 97 active+clean; 3.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-09-19T04:25:35.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:35 smithi094 ceph-mon[30972]: pgmap v327: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 1.62k op/s 2024-09-19T04:25:35.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:35 smithi112 ceph-mon[35051]: pgmap v327: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 1.62k op/s 2024-09-19T04:25:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 560.0 s, 1444.4 tps, lat 69.342 ms stddev 26.118 2024-09-19T04:25:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:35 smithi104 ceph-mon[34145]: pgmap v327: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 1.62k op/s 2024-09-19T04:25:37.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:37 smithi112 ceph-mon[35051]: pgmap v328: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-09-19T04:25:37.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:37 smithi094 ceph-mon[30972]: pgmap v328: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-09-19T04:25:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:37 smithi104 ceph-mon[34145]: pgmap v328: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-09-19T04:25:39.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:39 smithi112 ceph-mon[35051]: pgmap v329: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 21 MiB/s wr, 1.76k op/s 2024-09-19T04:25:39.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:39 smithi094 ceph-mon[30972]: pgmap v329: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 21 MiB/s wr, 1.76k op/s 2024-09-19T04:25:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:39 smithi104 ceph-mon[34145]: pgmap v329: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 21 MiB/s wr, 1.76k op/s 2024-09-19T04:25:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 565.0 s, 1528.2 tps, lat 65.317 ms stddev 26.277 2024-09-19T04:25:41.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:41 smithi112 ceph-mon[35051]: pgmap v330: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:25:41.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:41 smithi094 ceph-mon[30972]: pgmap v330: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:25:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:41 smithi104 ceph-mon[34145]: pgmap v330: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:25:43.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:43 smithi112 ceph-mon[35051]: pgmap v331: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:25:43.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:43 smithi094 ceph-mon[30972]: pgmap v331: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:25:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:43 smithi104 ceph-mon[34145]: pgmap v331: 97 pgs: 97 active+clean; 3.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:25:45.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:45 smithi112 ceph-mon[35051]: pgmap v332: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 27 MiB/s wr, 1.89k op/s 2024-09-19T04:25:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:45 smithi094 ceph-mon[30972]: pgmap v332: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 27 MiB/s wr, 1.89k op/s 2024-09-19T04:25:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 570.0 s, 1510.4 tps, lat 66.057 ms stddev 29.919 2024-09-19T04:25:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:45 smithi104 ceph-mon[34145]: pgmap v332: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 27 MiB/s wr, 1.89k op/s 2024-09-19T04:25:47.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:47 smithi112 ceph-mon[35051]: pgmap v333: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 21 MiB/s wr, 1.30k op/s 2024-09-19T04:25:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:47 smithi094 ceph-mon[30972]: pgmap v333: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 21 MiB/s wr, 1.30k op/s 2024-09-19T04:25:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:47 smithi104 ceph-mon[34145]: pgmap v333: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 21 MiB/s wr, 1.30k op/s 2024-09-19T04:25:49.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:49 smithi112 ceph-mon[35051]: pgmap v334: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 30 MiB/s wr, 1.93k op/s 2024-09-19T04:25:49.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:49 smithi094 ceph-mon[30972]: pgmap v334: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 30 MiB/s wr, 1.93k op/s 2024-09-19T04:25:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:49 smithi104 ceph-mon[34145]: pgmap v334: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 30 MiB/s wr, 1.93k op/s 2024-09-19T04:25:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 575.0 s, 1372.6 tps, lat 72.503 ms stddev 44.579 2024-09-19T04:25:51.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:51 smithi112 ceph-mon[35051]: pgmap v335: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 20 MiB/s wr, 1.27k op/s 2024-09-19T04:25:51.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:51 smithi094 ceph-mon[30972]: pgmap v335: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 20 MiB/s wr, 1.27k op/s 2024-09-19T04:25:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:51 smithi104 ceph-mon[34145]: pgmap v335: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 20 MiB/s wr, 1.27k op/s 2024-09-19T04:25:53.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:53 smithi112 ceph-mon[35051]: pgmap v336: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 20 MiB/s wr, 1.27k op/s 2024-09-19T04:25:53.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:53 smithi094 ceph-mon[30972]: pgmap v336: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 20 MiB/s wr, 1.27k op/s 2024-09-19T04:25:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:53 smithi104 ceph-mon[34145]: pgmap v336: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 20 MiB/s wr, 1.27k op/s 2024-09-19T04:25:55.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:55 smithi112 ceph-mon[35051]: pgmap v337: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 29 MiB/s wr, 1.93k op/s 2024-09-19T04:25:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:55 smithi094 ceph-mon[30972]: pgmap v337: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 29 MiB/s wr, 1.93k op/s 2024-09-19T04:25:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 580.0 s, 1400.3 tps, lat 71.831 ms stddev 28.025 2024-09-19T04:25:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:55 smithi104 ceph-mon[34145]: pgmap v337: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 29 MiB/s wr, 1.93k op/s 2024-09-19T04:25:56.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:25:56.811 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:57.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.377+0000 7f55088f8640 1 -- 172.21.15.94:0/747376871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5504109130 msgr2=0x7f5504110cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:57.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.378+0000 7f55088f8640 1 --2- 172.21.15.94:0/747376871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5504109130 0x7f5504110cb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009be0 tx=0x7f54fc02a6c0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.378+0000 7f55088f8640 1 -- 172.21.15.94:0/747376871 shutdown_connections 2024-09-19T04:25:57.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.378+0000 7f55088f8640 1 --2- 172.21.15.94:0/747376871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5504109130 0x7f5504110cb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.378+0000 7f55088f8640 1 --2- 172.21.15.94:0/747376871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55041049b0 0x7f5504108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.378+0000 7f55088f8640 1 --2- 172.21.15.94:0/747376871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5504104000 0x7f55041043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.378+0000 7f55088f8640 1 -- 172.21.15.94:0/747376871 >> 172.21.15.94:0/747376871 conn(0x7f55040ffa20 msgr2=0x7f5504101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:57.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.379+0000 7f55088f8640 1 -- 172.21.15.94:0/747376871 shutdown_connections 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.379+0000 7f55088f8640 1 -- 172.21.15.94:0/747376871 wait complete. 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f55088f8640 1 Processor -- start 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f55088f8640 1 -- start start 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f55088f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5504104000 0x7f5504076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f55088f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55041049b0 0x7f5504076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f55088f8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5504109130 0x7f5504077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f55088f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5504114010 con 0x7f55041049b0 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f55088f8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5504113e90 con 0x7f5504109130 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f55088f8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5504114190 con 0x7f5504104000 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f5502d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5504109130 0x7f5504077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f54fbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55041049b0 0x7f5504076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f5502d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5504109130 0x7f5504077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:53290/0 (socket says 172.21.15.94:53290) 2024-09-19T04:25:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f5502d9d640 1 -- 172.21.15.94:0/4036064203 learned_addr learned my addr 172.21.15.94:0/4036064203 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.380+0000 7f550259c640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5504104000 0x7f5504076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.381+0000 7f5502d9d640 1 -- 172.21.15.94:0/4036064203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5504104000 msgr2=0x7f5504076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.381+0000 7f5502d9d640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5504104000 0x7f5504076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.381+0000 7f5502d9d640 1 -- 172.21.15.94:0/4036064203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55041049b0 msgr2=0x7f5504076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.381+0000 7f5502d9d640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55041049b0 0x7f5504076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.381+0000 7f5502d9d640 1 -- 172.21.15.94:0/4036064203 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5504072b30 con 0x7f5504109130 2024-09-19T04:25:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.381+0000 7f5502d9d640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5504109130 0x7f5504077180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f54fc030c20 tx=0x7f54fc030fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.381+0000 7f54fb7fe640 1 -- 172.21.15.94:0/4036064203 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc041070 con 0x7f5504109130 2024-09-19T04:25:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.381+0000 7f55088f8640 1 -- 172.21.15.94:0/4036064203 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5504072dc0 con 0x7f5504109130 2024-09-19T04:25:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.382+0000 7f55088f8640 1 -- 172.21.15.94:0/4036064203 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55040732f0 con 0x7f5504109130 2024-09-19T04:25:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.381+0000 7f54fb7fe640 1 -- 172.21.15.94:0/4036064203 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54fc003e20 con 0x7f5504109130 2024-09-19T04:25:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.382+0000 7f54fb7fe640 1 -- 172.21.15.94:0/4036064203 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc036070 con 0x7f5504109130 2024-09-19T04:25:57.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.386+0000 7f54fb7fe640 1 -- 172.21.15.94:0/4036064203 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f54fc044070 con 0x7f5504109130 2024-09-19T04:25:57.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.386+0000 7f550acd1740 1 Processor -- start 2024-09-19T04:25:57.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.387+0000 7f550acd1740 1 -- start start 2024-09-19T04:25:57.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.387+0000 7f550acd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef19880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.387+0000 7f550acd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef19dc0 0x5654cef24140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.387+0000 7f550259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef19dc0 0x5654cef24140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.387+0000 7f550259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef19dc0 0x5654cef24140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51906/0 (socket says 172.21.15.94:51906) 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.387+0000 7f550acd1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef24680 0x5654cef26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.387+0000 7f550259c640 1 -- 172.21.15.94:0/2313322906 learned_addr learned my addr 172.21.15.94:0/2313322906 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.387+0000 7f54fbfff640 1 --2- 172.21.15.94:0/2313322906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef19880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f54fb7fe640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f54f00a34b0 0x7f54f00a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f550acd1740 1 -- 172.21.15.94:0/2313322906 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5654cee7f2e0 con 0x5654cef19dc0 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f550acd1740 1 -- 172.21.15.94:0/2313322906 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5654cee26ce0 con 0x5654cef24680 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f550acd1740 1 -- 172.21.15.94:0/2313322906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654cee7b2a0 con 0x5654cef194a0 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f54fbfff640 1 --2- 172.21.15.94:0/2313322906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef24680 0x5654cef26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f54fb7fe640 1 -- 172.21.15.94:0/4036064203 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f54fc0f0100 con 0x7f5504109130 2024-09-19T04:25:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f54fbfff640 1 -- 172.21.15.94:0/2313322906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef24680 msgr2=0x5654cef26a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f54fbfff640 1 --2- 172.21.15.94:0/2313322906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef24680 0x5654cef26a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f54fbfff640 1 -- 172.21.15.94:0/2313322906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef19dc0 msgr2=0x5654cef24140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f54fbfff640 1 --2- 172.21.15.94:0/2313322906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef19dc0 0x5654cef24140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f54fbfff640 1 -- 172.21.15.94:0/2313322906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cee0b440 con 0x5654cef194a0 2024-09-19T04:25:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.388+0000 7f5502d9d640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f54f00a34b0 0x7f54f00a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.389+0000 7f54fbfff640 1 --2- 172.21.15.94:0/2313322906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef19880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009670 tx=0x7f54f4009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a8136cde866f70a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:57.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.389+0000 7f54f8ff9640 1 -- 172.21.15.94:0/2313322906 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4013130 con 0x5654cef194a0 2024-09-19T04:25:57.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.389+0000 7f54f8ff9640 1 -- 172.21.15.94:0/2313322906 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f4013c70 con 0x5654cef194a0 2024-09-19T04:25:57.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.389+0000 7f54f8ff9640 1 -- 172.21.15.94:0/2313322906 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4005580 con 0x5654cef194a0 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.390+0000 7f550acd1740 1 -- 172.21.15.94:0/2313322906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 msgr2=0x5654cef19880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.390+0000 7f550acd1740 1 --2- 172.21.15.94:0/2313322906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef19880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009670 tx=0x7f54f4009600 comp rx=0 tx=0).stop 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.390+0000 7f5502d9d640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f54f00a34b0 0x7f54f00a5970 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f54fc032fd0 tx=0x7f54fc030b00 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.391+0000 7f550acd1740 1 -- 172.21.15.94:0/2313322906 shutdown_connections 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.391+0000 7f550acd1740 1 --2- 172.21.15.94:0/2313322906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef24680 0x5654cef26a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.391+0000 7f550acd1740 1 --2- 172.21.15.94:0/2313322906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef19dc0 0x5654cef24140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.391+0000 7f550acd1740 1 --2- 172.21.15.94:0/2313322906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef19880 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.391+0000 7f550acd1740 1 -- 172.21.15.94:0/2313322906 >> 172.21.15.94:0/2313322906 conn(0x5654cef15a30 msgr2=0x5654cef15e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.391+0000 7f550acd1740 1 -- 172.21.15.94:0/2313322906 shutdown_connections 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.391+0000 7f550acd1740 1 -- 172.21.15.94:0/2313322906 wait complete. 2024-09-19T04:25:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.391+0000 7f550acd1740 1 -- start start 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.392+0000 7f550acd1740 1 Processor -- start 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.392+0000 7f550acd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef23f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.392+0000 7f550acd1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef211f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.392+0000 7f550acd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef24680 0x5654cef21730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.392+0000 7f550acd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5654cef272a0 con 0x5654cef24680 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.392+0000 7f550acd1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5654cee3a050 con 0x5654cef19dc0 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f550acd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654cef27420 con 0x5654cef194a0 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f550acd1740 10 client.0 _ll_register_callbacks cb 0x5654cee7a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f550259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f550259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef211f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:53304/0 (socket says 172.21.15.94:53304) 2024-09-19T04:25:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f550259c640 1 -- 172.21.15.94:0/372002885 learned_addr learned my addr 172.21.15.94:0/372002885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:57.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f54fbfff640 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef23f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f550259c640 1 -- 172.21.15.94:0/372002885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 msgr2=0x5654cef23f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:57.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f550259c640 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef23f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f550259c640 1 -- 172.21.15.94:0/372002885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef24680 msgr2=0x5654cef21730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.393+0000 7f550259c640 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef24680 0x5654cef21730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.394+0000 7f550259c640 1 -- 172.21.15.94:0/372002885 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cebf03c0 con 0x5654cef19dc0 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.394+0000 7f550259c640 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef211f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54ec010a20 tx=0x7f54ec010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ded75b869e33a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.394+0000 7f54e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.394+0000 7f54e9ffb640 1 -- 172.21.15.94:0/372002885 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec01e060 con 0x5654cef19dc0 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.395+0000 7f54e9ffb640 1 -- 172.21.15.94:0/372002885 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f54ec00fc50 con 0x5654cef19dc0 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.395+0000 7f550acd1740 1 -- 172.21.15.94:0/372002885 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f4013c70 con 0x5654cef19dc0 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.395+0000 7f54e9ffb640 1 -- 172.21.15.94:0/372002885 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f54ec00fe50 con 0x5654cef19dc0 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.395+0000 7f550acd1740 10 client.26288 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.395+0000 7f550acd1740 1 -- 172.21.15.94:0/372002885 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5654cef32d10 con 0x5654cef19dc0 2024-09-19T04:25:57.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.395+0000 7f54e9ffb640 1 -- 172.21.15.94:0/372002885 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f4013c70 con 0x5654cef19dc0 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.396+0000 7f54e9ffb640 1 -- 172.21.15.94:0/372002885 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f54ec021580 con 0x5654cef19dc0 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.396+0000 7f550acd1740 10 client.26288 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.396+0000 7f550acd1740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.396+0000 7f550acd1740 20 client.26288 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.396+0000 7f550acd1740 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654cef34650 0x5654cef36a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.396+0000 7f550acd1740 4 client.26288 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.396+0000 7f550acd1740 1 -- 172.21.15.94:0/372002885 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654cee8c850 con 0x5654cef34650 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.396+0000 7f54fbfff640 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654cef34650 0x5654cef36a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.397+0000 7f54fbfff640 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654cef34650 0x5654cef36a70 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e0ca61ce8dc232 server_cookie=8615bf25631b84c6 in_seq=0 out_seq=0 2024-09-19T04:25:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.397+0000 7f54e9ffb640 10 client.26288 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.397+0000 7f54e9ffb640 1 -- 172.21.15.94:0/372002885 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654cee8c850 con 0x5654cef34650 2024-09-19T04:25:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.397+0000 7f54e9ffb640 10 client.26288 handle_command_reply: tid=0 2024-09-19T04:25:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.397+0000 7f550acd1740 1 client.26288 shutdown 2024-09-19T04:25:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 -- 172.21.15.94:0/372002885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 msgr2=0x5654cef211f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef211f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54ec010a20 tx=0x7f54ec010ee0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 -- 172.21.15.94:0/372002885 shutdown_connections 2024-09-19T04:25:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654cef34650 0x5654cef36a70 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef24680 0x5654cef21730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f54e9ffb640 0 client.26288 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef211f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 --2- 172.21.15.94:0/372002885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef23f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 -- 172.21.15.94:0/372002885 >> 172.21.15.94:0/372002885 conn(0x5654cef15a30 msgr2=0x5654cef14b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 -- 172.21.15.94:0/372002885 shutdown_connections 2024-09-19T04:25:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.398+0000 7f550acd1740 1 -- 172.21.15.94:0/372002885 wait complete. 2024-09-19T04:25:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.399+0000 7f550acd1740 20 client.26288 trim_cache size 0 max 16384 2024-09-19T04:25:57.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.432+0000 7f550acd1740 1 Processor -- start 2024-09-19T04:25:57.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.432+0000 7f550acd1740 1 -- start start 2024-09-19T04:25:57.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f550acd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef15a30 0x5654cef30590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f550acd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef194a0 0x5654cef211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f550acd1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef1c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f550acd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54e4011170 con 0x5654cef194a0 2024-09-19T04:25:57.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f550acd1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f54e400e9e0 con 0x5654cef19dc0 2024-09-19T04:25:57.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f550acd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54e400c850 con 0x5654cef15a30 2024-09-19T04:25:57.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f550259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef194a0 0x5654cef211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f550259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef194a0 0x5654cef211b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51914/0 (socket says 172.21.15.94:51914) 2024-09-19T04:25:57.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f550259c640 1 -- 172.21.15.94:0/47892264 learned_addr learned my addr 172.21.15.94:0/47892264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:57.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f54fbfff640 1 --2- 172.21.15.94:0/47892264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef15a30 0x5654cef30590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.433+0000 7f54fbfff640 1 --2- 172.21.15.94:0/47892264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef1c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.434+0000 7f54fbfff640 1 -- 172.21.15.94:0/47892264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 msgr2=0x5654cef1c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:57.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.434+0000 7f54fbfff640 1 --2- 172.21.15.94:0/47892264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef1c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.434+0000 7f54fbfff640 1 -- 172.21.15.94:0/47892264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef194a0 msgr2=0x5654cef211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:57.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.434+0000 7f54fbfff640 1 --2- 172.21.15.94:0/47892264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef194a0 0x5654cef211b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.434+0000 7f54fbfff640 1 -- 172.21.15.94:0/47892264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cef242b0 con 0x5654cef15a30 2024-09-19T04:25:57.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.434+0000 7f54fbfff640 1 --2- 172.21.15.94:0/47892264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef15a30 0x5654cef30590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5654cef24650 tx=0x7f54f40092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f59b5f3b80a8c7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:57.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.435+0000 7f54ebfff640 1 -- 172.21.15.94:0/47892264 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f400fb90 con 0x5654cef15a30 2024-09-19T04:25:57.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.435+0000 7f54ebfff640 1 -- 172.21.15.94:0/47892264 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f4017040 con 0x5654cef15a30 2024-09-19T04:25:57.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.435+0000 7f54ebfff640 1 -- 172.21.15.94:0/47892264 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4020db0 con 0x5654cef15a30 2024-09-19T04:25:57.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.436+0000 7f550acd1740 1 -- 172.21.15.94:0/47892264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef15a30 msgr2=0x5654cef30590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:57.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.436+0000 7f550acd1740 1 --2- 172.21.15.94:0/47892264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef15a30 0x5654cef30590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5654cef24650 tx=0x7f54f40092d0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.436+0000 7f550acd1740 1 -- 172.21.15.94:0/47892264 shutdown_connections 2024-09-19T04:25:57.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.436+0000 7f550acd1740 1 --2- 172.21.15.94:0/47892264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef1c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.436+0000 7f550acd1740 1 --2- 172.21.15.94:0/47892264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef194a0 0x5654cef211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.436+0000 7f550acd1740 1 --2- 172.21.15.94:0/47892264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef15a30 0x5654cef30590 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.436+0000 7f550acd1740 1 -- 172.21.15.94:0/47892264 >> 172.21.15.94:0/47892264 conn(0x5654cef21890 msgr2=0x5654cef1c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.436+0000 7f550acd1740 1 -- 172.21.15.94:0/47892264 shutdown_connections 2024-09-19T04:25:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.436+0000 7f550acd1740 1 -- 172.21.15.94:0/47892264 wait complete. 2024-09-19T04:25:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550acd1740 1 -- start start 2024-09-19T04:25:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550acd1740 1 Processor -- start 2024-09-19T04:25:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550acd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef15a30 0x5654cef2cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550acd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef27f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550acd1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef28450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550acd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54e400cea0 con 0x5654cef15a30 2024-09-19T04:25:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550acd1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f54e400e310 con 0x5654cef19dc0 2024-09-19T04:25:57.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550acd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54e400d570 con 0x5654cef194a0 2024-09-19T04:25:57.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550acd1740 10 client.0 _ll_register_callbacks cb 0x7f54ec00a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:57.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef27f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef27f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:57694/0 (socket says 172.21.15.94:57694) 2024-09-19T04:25:57.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.437+0000 7f550259c640 1 -- 172.21.15.94:0/3849027246 learned_addr learned my addr 172.21.15.94:0/3849027246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:57.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.438+0000 7f54fbfff640 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef15a30 0x5654cef2cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.438+0000 7f54fbfff640 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef28450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.439+0000 7f550259c640 1 -- 172.21.15.94:0/3849027246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 msgr2=0x5654cef28450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:57.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.439+0000 7f550259c640 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef28450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.439+0000 7f550259c640 1 -- 172.21.15.94:0/3849027246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef15a30 msgr2=0x5654cef2cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:57.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.439+0000 7f550259c640 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef15a30 0x5654cef2cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.439+0000 7f550259c640 1 -- 172.21.15.94:0/3849027246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cef28a10 con 0x5654cef194a0 2024-09-19T04:25:57.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.440+0000 7f550259c640 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef27f10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00ae50 tx=0x7f54ec00f880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35ceef56724b2cff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:57.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.440+0000 7f54e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:25:57.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.440+0000 7f54e9ffb640 1 -- 172.21.15.94:0/3849027246 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec021600 con 0x5654cef194a0 2024-09-19T04:25:57.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.440+0000 7f550acd1740 1 -- 172.21.15.94:0/3849027246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f4017040 con 0x5654cef194a0 2024-09-19T04:25:57.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.440+0000 7f54e9ffb640 1 -- 172.21.15.94:0/3849027246 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f54ec005040 con 0x5654cef194a0 2024-09-19T04:25:57.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.440+0000 7f54e9ffb640 1 -- 172.21.15.94:0/3849027246 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f54ec005240 con 0x5654cef194a0 2024-09-19T04:25:57.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.440+0000 7f550acd1740 10 client.26269 fetch_fsmap learned FSMap version 44 2024-09-19T04:25:57.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.440+0000 7f550acd1740 1 -- 172.21.15.94:0/3849027246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5654cef9f180 con 0x5654cef194a0 2024-09-19T04:25:57.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.441+0000 7f54e9ffb640 1 -- 172.21.15.94:0/3849027246 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f4017040 con 0x5654cef194a0 2024-09-19T04:25:57.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.441+0000 7f54e9ffb640 1 -- 172.21.15.94:0/3849027246 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f54ec004ca0 con 0x5654cef194a0 2024-09-19T04:25:57.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.441+0000 7f550acd1740 10 client.26269 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:25:57.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.441+0000 7f550acd1740 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:57.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.441+0000 7f550acd1740 20 client.26269 populate_metadata read hostname 'smithi094' 2024-09-19T04:25:57.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.441+0000 7f550acd1740 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654cef24680 0x5654cefa1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:57.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.441+0000 7f550acd1740 4 client.26269 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:25:57.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.441+0000 7f550acd1740 1 -- 172.21.15.94:0/3849027246 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654cee86d60 con 0x5654cef24680 2024-09-19T04:25:57.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.442+0000 7f54fbfff640 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654cef24680 0x5654cefa1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:57.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.442+0000 7f54e9ffb640 10 client.26269 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:57.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.442+0000 7f54fbfff640 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654cef24680 0x5654cefa1a20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5862aaa109d716cc server_cookie=402276d92cc8ae1c in_seq=0 out_seq=0 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f54e9ffb640 1 -- 172.21.15.94:0/3849027246 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5654cee86d60 con 0x5654cef24680 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f54e9ffb640 10 client.26269 handle_command_reply: tid=0 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f550acd1740 1 client.26269 shutdown 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f550acd1740 1 -- 172.21.15.94:0/3849027246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 msgr2=0x5654cef27f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f550acd1740 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef27f10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00ae50 tx=0x7f54ec00f880 comp rx=0 tx=0).stop 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f550acd1740 1 -- 172.21.15.94:0/3849027246 shutdown_connections 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f550acd1740 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5654cef24680 0x5654cefa1a20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f550acd1740 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654cef19dc0 0x5654cef28450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f550acd1740 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654cef194a0 0x5654cef27f10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f550acd1740 1 --2- 172.21.15.94:0/3849027246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654cef15a30 0x5654cef2cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f550acd1740 1 -- 172.21.15.94:0/3849027246 >> 172.21.15.94:0/3849027246 conn(0x5654cef21890 msgr2=0x5654cef1bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:57.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.443+0000 7f54e9ffb640 0 client.26269 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:25:57.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.444+0000 7f550acd1740 1 -- 172.21.15.94:0/3849027246 shutdown_connections 2024-09-19T04:25:57.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.444+0000 7f550acd1740 1 -- 172.21.15.94:0/3849027246 wait complete. 2024-09-19T04:25:57.447 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:25:57.447 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:25:57.447 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:25:57.447 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:25:57.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.444+0000 7f550acd1740 20 client.26269 trim_cache size 0 max 16384 2024-09-19T04:25:57.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 -- 172.21.15.94:0/4036064203 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f54f00a34b0 msgr2=0x7f54f00a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:57.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f54f00a34b0 0x7f54f00a5970 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f54fc032fd0 tx=0x7f54fc030b00 comp rx=0 tx=0).stop 2024-09-19T04:25:57.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 -- 172.21.15.94:0/4036064203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5504109130 msgr2=0x7f5504077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:57.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5504109130 0x7f5504077180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f54fc030c20 tx=0x7f54fc030fa0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 -- 172.21.15.94:0/4036064203 shutdown_connections 2024-09-19T04:25:57.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f54f00a34b0 0x7f54f00a5970 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5504109130 0x7f5504077180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55041049b0 0x7f5504076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 --2- 172.21.15.94:0/4036064203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5504104000 0x7f5504076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:57.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 -- 172.21.15.94:0/4036064203 >> 172.21.15.94:0/4036064203 conn(0x7f55040ffa20 msgr2=0x7f5504100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:57.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 -- 172.21.15.94:0/4036064203 shutdown_connections 2024-09-19T04:25:57.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:57.445+0000 7f55088f8640 1 -- 172.21.15.94:0/4036064203 wait complete. 2024-09-19T04:25:57.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:57 smithi112 ceph-mon[35051]: pgmap v338: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 19 MiB/s wr, 1.29k op/s 2024-09-19T04:25:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:57 smithi094 ceph-mon[30972]: pgmap v338: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 19 MiB/s wr, 1.29k op/s 2024-09-19T04:25:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:57 smithi104 ceph-mon[34145]: pgmap v338: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 19 MiB/s wr, 1.29k op/s 2024-09-19T04:25:58.306 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:25:58.306 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:25:58.675 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.256+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2175196740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc109e00 msgr2=0x7f8cdc10e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.256+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2175196740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc109e00 0x7f8cdc10e350 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009ca0 tx=0x7f8cd402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.256+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2175196740 shutdown_connections 2024-09-19T04:25:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.256+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2175196740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc10ea80 0x7f8cdc1154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.256+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2175196740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc109e00 0x7f8cdc10e350 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.256+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2175196740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cdc109450 0x7f8cdc109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.256+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2175196740 >> 172.21.15.94:0/2175196740 conn(0x7f8cdc06e2d0 msgr2=0x7f8cdc06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:59.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.256+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2175196740 shutdown_connections 2024-09-19T04:25:59.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.256+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2175196740 wait complete. 2024-09-19T04:25:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.257+0000 7f8ce1bff640 1 Processor -- start 2024-09-19T04:25:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.258+0000 7f8ce1bff640 1 -- start start 2024-09-19T04:25:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.258+0000 7f8ce1bff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc109450 0x7f8cdc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.258+0000 7f8ce1bff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc109e00 0x7f8cdc19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.258+0000 7f8ce1bff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cdc10ea80 0x7f8cdc19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.258+0000 7f8ce1bff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc118440 con 0x7f8cdc109450 2024-09-19T04:25:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.258+0000 7f8ce1bff640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc1182c0 con 0x7f8cdc10ea80 2024-09-19T04:25:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.258+0000 7f8ce1bff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc1185c0 con 0x7f8cdc109e00 2024-09-19T04:25:59.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.259+0000 7f8cdaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc109e00 0x7f8cdc19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.259+0000 7f8cdbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cdc10ea80 0x7f8cdc19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.259+0000 7f8cdbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cdc10ea80 0x7f8cdc19f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:53336/0 (socket says 172.21.15.94:53336) 2024-09-19T04:25:59.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.259+0000 7f8cdbfff640 1 -- 172.21.15.94:0/2740293642 learned_addr learned my addr 172.21.15.94:0/2740293642 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:25:59.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.259+0000 7f8cdb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc109450 0x7f8cdc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.259+0000 7f8cdaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc109e00 0x7f8cdc19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:57718/0 (socket says 172.21.15.94:57718) 2024-09-19T04:25:59.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.260+0000 7f8cdb7fe640 1 -- 172.21.15.94:0/2740293642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc109e00 msgr2=0x7f8cdc19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.260+0000 7f8cdb7fe640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc109e00 0x7f8cdc19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.260+0000 7f8cdb7fe640 1 -- 172.21.15.94:0/2740293642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cdc10ea80 msgr2=0x7f8cdc19f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.260+0000 7f8cdb7fe640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cdc10ea80 0x7f8cdc19f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.260+0000 7f8cdb7fe640 1 -- 172.21.15.94:0/2740293642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc1a2b10 con 0x7f8cdc109450 2024-09-19T04:25:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.260+0000 7f8cdb7fe640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc109450 0x7f8cdc19e790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000e9f0 tx=0x7f8cd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.261+0000 7f8cdbfff640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cdc10ea80 0x7f8cdc19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.261+0000 7f8cd8ff9640 1 -- 172.21.15.94:0/2740293642 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd000ccc0 con 0x7f8cdc109450 2024-09-19T04:25:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.261+0000 7f8cd8ff9640 1 -- 172.21.15.94:0/2740293642 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cd00108e0 con 0x7f8cdc109450 2024-09-19T04:25:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.261+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc1a2e00 con 0x7f8cdc109450 2024-09-19T04:25:59.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.262+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc1a3310 con 0x7f8cdc109450 2024-09-19T04:25:59.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.262+0000 7f8cd8ff9640 1 -- 172.21.15.94:0/2740293642 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0018840 con 0x7f8cdc109450 2024-09-19T04:25:59.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.264+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc073f40 con 0x7f8cdc109450 2024-09-19T04:25:59.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.266+0000 7f8cd8ff9640 1 -- 172.21.15.94:0/2740293642 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f8cd0010430 con 0x7f8cdc109450 2024-09-19T04:25:59.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.267+0000 7f8cd8ff9640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8cc40a33b0 0x7f8cc40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.267+0000 7f8cd8ff9640 1 -- 172.21.15.94:0/2740293642 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f8cd00d02a0 con 0x7f8cdc109450 2024-09-19T04:25:59.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.267+0000 7f8cdaffd640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8cc40a33b0 0x7f8cc40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.268+0000 7f8cdaffd640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8cc40a33b0 0x7f8cc40a5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc19ffe0 tx=0x7f8cd4005bd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:59.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.272+0000 7f8cd8ff9640 1 -- 172.21.15.94:0/2740293642 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8cd009ccc0 con 0x7f8cdc109450 2024-09-19T04:25:59.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.585+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cdc19fcf0 con 0x7f8cdc109450 2024-09-19T04:25:59.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:59 smithi094 ceph-mon[30972]: pgmap v339: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 27 MiB/s wr, 1.87k op/s 2024-09-19T04:25:59.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:25:59 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:25:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.587+0000 7f8cd8ff9640 1 -- 172.21.15.94:0/2740293642 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11919 (secure 0 0 0) 0x7f8cdc19fcf0 con 0x7f8cdc109450 2024-09-19T04:25:59.589 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:25:59.589 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":45,"btime":"2024-09-19T04:25:58:286510+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:25:57.757342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24466,24467,24454]},"id":1}]} 2024-09-19T04:25:59.589 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 45 2024-09-19T04:25:59.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.590+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8cc40a33b0 msgr2=0x7f8cc40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.590+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8cc40a33b0 0x7f8cc40a5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc19ffe0 tx=0x7f8cd4005bd0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.590+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc109450 msgr2=0x7f8cdc19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.590+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc109450 0x7f8cdc19e790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000e9f0 tx=0x7f8cd000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.592+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 shutdown_connections 2024-09-19T04:25:59.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.592+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8cc40a33b0 0x7f8cc40a5870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.592+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cdc10ea80 0x7f8cdc19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.592+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc109e00 0x7f8cdc19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.592+0000 7f8ce1bff640 1 --2- 172.21.15.94:0/2740293642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc109450 0x7f8cdc19e790 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.592+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 >> 172.21.15.94:0/2740293642 conn(0x7f8cdc06e2d0 msgr2=0x7f8cdc10c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:59.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.595+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 shutdown_connections 2024-09-19T04:25:59.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:25:59.595+0000 7f8ce1bff640 1 -- 172.21.15.94:0/2740293642 wait complete. 2024-09-19T04:25:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:59 smithi104 ceph-mon[34145]: pgmap v339: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 27 MiB/s wr, 1.87k op/s 2024-09-19T04:25:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:25:59 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:25:59.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:59 smithi112 ceph-mon[35051]: pgmap v339: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 27 MiB/s wr, 1.87k op/s 2024-09-19T04:25:59.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:25:59 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:00.259 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:26:00.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:00 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2740293642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 585.0 s, 1097.8 tps, lat 90.883 ms stddev 113.488 2024-09-19T04:26:00.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:00 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2740293642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:00.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:00 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2740293642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:01.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.245+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/578928125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84102570 msgr2=0x7f3d841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:01.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.245+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/578928125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84102570 0x7f3d841029b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009900 tx=0x7f3d7002a480 comp rx=0 tx=0).stop 2024-09-19T04:26:01.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.245+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/578928125 shutdown_connections 2024-09-19T04:26:01.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.245+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/578928125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84102ef0 0x7f3d8410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.245+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/578928125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84102570 0x7f3d841029b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.245+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/578928125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d84100e20 0x7f3d84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.245+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/578928125 >> 172.21.15.94:0/578928125 conn(0x7f3d840fc7d0 msgr2=0x7f3d840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:01.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.245+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/578928125 shutdown_connections 2024-09-19T04:26:01.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.245+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/578928125 wait complete. 2024-09-19T04:26:01.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d8ac4d640 1 Processor -- start 2024-09-19T04:26:01.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d8ac4d640 1 -- start start 2024-09-19T04:26:01.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d8ac4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84100e20 0x7f3d84198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d8ac4d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d84102570 0x7f3d841989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d8ac4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84102ef0 0x7f3d8419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d8ac4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84110e30 con 0x7f3d84100e20 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d8ac4d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84110cb0 con 0x7f3d84102570 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d8ac4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84110fb0 con 0x7f3d84102ef0 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d889e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84100e20 0x7f3d84198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d84102570 0x7f3d841989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d889e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84100e20 0x7f3d84198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51956/0 (socket says 172.21.15.94:51956) 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d84102570 0x7f3d841989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:53356/0 (socket says 172.21.15.94:53356) 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d889e9640 1 -- 172.21.15.94:0/4047490288 learned_addr learned my addr 172.21.15.94:0/4047490288 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.246+0000 7f3d891ea640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84102ef0 0x7f3d8419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d889e9640 1 -- 172.21.15.94:0/4047490288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84102ef0 msgr2=0x7f3d8419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d889e9640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84102ef0 0x7f3d8419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d889e9640 1 -- 172.21.15.94:0/4047490288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d84102570 msgr2=0x7f3d841989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:01.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d889e9640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d84102570 0x7f3d841989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d889e9640 1 -- 172.21.15.94:0/4047490288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8419d3a0 con 0x7f3d84100e20 2024-09-19T04:26:01.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d891ea640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84102ef0 0x7f3d8419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:01.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d83fff640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d84102570 0x7f3d841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:01.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d889e9640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84100e20 0x7f3d84198490 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009c20 tx=0x7f3d7800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:01.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d81ffb640 1 -- 172.21.15.94:0/4047490288 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7801e070 con 0x7f3d84100e20 2024-09-19T04:26:01.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/4047490288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8419d690 con 0x7f3d84100e20 2024-09-19T04:26:01.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d81ffb640 1 -- 172.21.15.94:0/4047490288 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d780198c0 con 0x7f3d84100e20 2024-09-19T04:26:01.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d81ffb640 1 -- 172.21.15.94:0/4047490288 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78018710 con 0x7f3d84100e20 2024-09-19T04:26:01.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.247+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/4047490288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d840fcbb0 con 0x7f3d84100e20 2024-09-19T04:26:01.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.250+0000 7f3d81ffb640 1 -- 172.21.15.94:0/4047490288 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3d78021070 con 0x7f3d84100e20 2024-09-19T04:26:01.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.250+0000 7f3d8d026740 1 Processor -- start 2024-09-19T04:26:01.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.250+0000 7f3d8d026740 1 -- start start 2024-09-19T04:26:01.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646ede9ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 0x5646edea9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d8d026740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646ede022f0 con 0x5646ede9c5d0 2024-09-19T04:26:01.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d8d026740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5646edda9cf0 con 0x5646ede9db40 2024-09-19T04:26:01.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d8d026740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646eddfe2b0 con 0x5646edea7370 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d891ea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d891ea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede9c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:53372/0 (socket says 172.21.15.94:53372) 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d891ea640 1 -- 172.21.15.94:0/2286010949 learned_addr learned my addr 172.21.15.94:0/2286010949 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d83fff640 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 0x5646edea9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.251+0000 7f3d889e9640 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646ede9ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d83fff640 1 -- 172.21.15.94:0/2286010949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 msgr2=0x5646ede9c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d83fff640 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede9c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d83fff640 1 -- 172.21.15.94:0/2286010949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 msgr2=0x5646ede9ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d83fff640 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646ede9ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d83fff640 1 -- 172.21.15.94:0/2286010949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646edd8e3a0 con 0x5646edea7370 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d81ffb640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3d740a33b0 0x7f3d740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d889e9640 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646ede9ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:01.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d83fff640 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 0x5646edea9760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005940 tx=0x7f3d70032e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ae89ca5cf68a46d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:01.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d81ffb640 1 -- 172.21.15.94:0/4047490288 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f3d780d35d0 con 0x7f3d84100e20 2024-09-19T04:26:01.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d891ea640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3d740a33b0 0x7f3d740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d6effd640 1 -- 172.21.15.94:0/2286010949 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70035040 con 0x5646edea7370 2024-09-19T04:26:01.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d6effd640 1 -- 172.21.15.94:0/2286010949 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d700316b0 con 0x5646edea7370 2024-09-19T04:26:01.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.252+0000 7f3d6effd640 1 -- 172.21.15.94:0/2286010949 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70003dc0 con 0x5646edea7370 2024-09-19T04:26:01.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d8d026740 1 -- 172.21.15.94:0/2286010949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 msgr2=0x5646edea9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:01.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 0x5646edea9760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005940 tx=0x7f3d70032e60 comp rx=0 tx=0).stop 2024-09-19T04:26:01.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d891ea640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3d740a33b0 0x7f3d740a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005c40 tx=0x7f3d7c009500 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:01.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d8d026740 1 -- 172.21.15.94:0/2286010949 shutdown_connections 2024-09-19T04:26:01.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 0x5646edea9760 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646ede9ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2286010949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d8d026740 1 -- 172.21.15.94:0/2286010949 >> 172.21.15.94:0/2286010949 conn(0x5646ede65310 msgr2=0x5646ede98c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d8d026740 1 -- 172.21.15.94:0/2286010949 shutdown_connections 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.253+0000 7f3d8d026740 1 -- 172.21.15.94:0/2286010949 wait complete. 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.254+0000 7f3d8d026740 1 -- start start 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.254+0000 7f3d8d026740 1 Processor -- start 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.254+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.254+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.254+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646edea44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.254+0000 7f3d8d026740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646edeaa0b0 con 0x5646edea7370 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.254+0000 7f3d8d026740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5646eddbd060 con 0x5646ede9db40 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.254+0000 7f3d8d026740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646edeaa230 con 0x5646ede9c5d0 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.254+0000 7f3d8d026740 10 client.0 _ll_register_callbacks cb 0x5646edb108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:01.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d889e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d891ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646edea44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d889e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:57742/0 (socket says 172.21.15.94:57742) 2024-09-19T04:26:01.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d889e9640 1 -- 172.21.15.94:0/2992953782 learned_addr learned my addr 172.21.15.94:0/2992953782 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:01.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d889e9640 1 -- 172.21.15.94:0/2992953782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 msgr2=0x5646edea3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:01.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d889e9640 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d889e9640 1 -- 172.21.15.94:0/2992953782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 msgr2=0x5646edea44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:01.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d889e9640 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646edea44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d889e9640 1 -- 172.21.15.94:0/2992953782 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646edb733c0 con 0x5646ede9c5d0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d891ea640 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646edea44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d889e9640 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea6ce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009bf0 tx=0x7f3d7809f220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2c75d9b128b1a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d63fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d63fff640 1 -- 172.21.15.94:0/2992953782 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780a3bc0 con 0x5646ede9c5d0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d8d026740 1 -- 172.21.15.94:0/2992953782 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d700316b0 con 0x5646ede9c5d0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.255+0000 7f3d63fff640 1 -- 172.21.15.94:0/2992953782 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f3d780d0f00 con 0x5646ede9c5d0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d63fff640 1 -- 172.21.15.94:0/2992953782 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3d780d1100 con 0x5646ede9c5d0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d8d026740 10 client.26287 fetch_fsmap learned FSMap version 45 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d8d026740 1 -- 172.21.15.94:0/2992953782 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5646edeb5b70 con 0x5646ede9c5d0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d63fff640 1 -- 172.21.15.94:0/2992953782 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d700316b0 con 0x5646ede9c5d0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d63fff640 1 -- 172.21.15.94:0/2992953782 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f3d7809ed10 con 0x5646ede9c5d0 2024-09-19T04:26:01.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d8d026740 10 client.26287 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d8d026740 10 client.26287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d8d026740 20 client.26287 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5646edeb6ec0 0x5646edeb92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d8d026740 4 client.26287 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d8d026740 1 -- 172.21.15.94:0/2992953782 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646ede0f860 con 0x5646edeb6ec0 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.256+0000 7f3d83fff640 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5646edeb6ec0 0x5646edeb92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.257+0000 7f3d63fff640 10 client.26287 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.257+0000 7f3d83fff640 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5646edeb6ec0 0x5646edeb92b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3306e95d496b052b server_cookie=1d36d3a10024ef28 in_seq=0 out_seq=0 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.257+0000 7f3d63fff640 1 -- 172.21.15.94:0/2992953782 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646ede0f860 con 0x5646edeb6ec0 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.257+0000 7f3d63fff640 10 client.26287 handle_command_reply: tid=0 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.257+0000 7f3d8d026740 1 client.26287 shutdown 2024-09-19T04:26:01.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.257+0000 7f3d8d026740 1 -- 172.21.15.94:0/2992953782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 msgr2=0x5646edea6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.257+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea6ce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009bf0 tx=0x7f3d7809f220 comp rx=0 tx=0).stop 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d8d026740 1 -- 172.21.15.94:0/2992953782 shutdown_connections 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5646edeb6ec0 0x5646edeb92b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646edea44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2992953782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea6ce0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d63fff640 0 client.26287 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d8d026740 1 -- 172.21.15.94:0/2992953782 >> 172.21.15.94:0/2992953782 conn(0x5646ede65310 msgr2=0x5646ede9e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d8d026740 1 -- 172.21.15.94:0/2992953782 shutdown_connections 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d8d026740 1 -- 172.21.15.94:0/2992953782 wait complete. 2024-09-19T04:26:01.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.258+0000 7f3d8d026740 20 client.26287 trim_cache size 0 max 16384 2024-09-19T04:26:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d8d026740 1 Processor -- start 2024-09-19T04:26:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d8d026740 1 -- start start 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede97bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646ede9f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d8d026740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841adea0 con 0x5646edea7370 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d8d026740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841ab740 con 0x5646ede9db40 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d8d026740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841a95b0 con 0x5646ede9c5d0 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d891ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646ede9f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede97bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede97bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:53396/0 (socket says 172.21.15.94:53396) 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d889e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d83fff640 1 -- 172.21.15.94:0/1445832202 learned_addr learned my addr 172.21.15.94:0/1445832202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d891ea640 1 -- 172.21.15.94:0/1445832202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 msgr2=0x5646edea3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d891ea640 1 --2- 172.21.15.94:0/1445832202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d891ea640 1 -- 172.21.15.94:0/1445832202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 msgr2=0x5646ede97bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d891ea640 1 --2- 172.21.15.94:0/1445832202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede97bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d891ea640 1 -- 172.21.15.94:0/1445832202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646edea2f40 con 0x5646edea7370 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.280+0000 7f3d889e9640 1 --2- 172.21.15.94:0/1445832202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.281+0000 7f3d891ea640 1 --2- 172.21.15.94:0/1445832202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646ede9f120 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005fd0 tx=0x7f3d7c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b54322ffc3042e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.281+0000 7f3d6e7fc640 1 -- 172.21.15.94:0/1445832202 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c010d40 con 0x5646edea7370 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.281+0000 7f3d6e7fc640 1 -- 172.21.15.94:0/1445832202 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d7c005020 con 0x5646edea7370 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.281+0000 7f3d6e7fc640 1 -- 172.21.15.94:0/1445832202 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c01c650 con 0x5646edea7370 2024-09-19T04:26:01.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 -- 172.21.15.94:0/1445832202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 msgr2=0x5646ede9f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 --2- 172.21.15.94:0/1445832202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646ede9f120 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005fd0 tx=0x7f3d7c019000 comp rx=0 tx=0).stop 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 -- 172.21.15.94:0/1445832202 shutdown_connections 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 --2- 172.21.15.94:0/1445832202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646edea7370 0x5646ede9f120 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 --2- 172.21.15.94:0/1445832202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646ede97bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 --2- 172.21.15.94:0/1445832202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ede9c5d0 0x5646edea3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 -- 172.21.15.94:0/1445832202 >> 172.21.15.94:0/1445832202 conn(0x5646ede65310 msgr2=0x5646ede974e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 -- 172.21.15.94:0/1445832202 shutdown_connections 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 -- 172.21.15.94:0/1445832202 wait complete. 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 -- start start 2024-09-19T04:26:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.282+0000 7f3d8d026740 1 Processor -- start 2024-09-19T04:26:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646edeafc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d8d026740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 0x5646edea12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d8d026740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841a9c00 con 0x5646ede9c5d0 2024-09-19T04:26:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d8d026740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841ab070 con 0x5646ede9db40 2024-09-19T04:26:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d8d026740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841aa2d0 con 0x5646edea7370 2024-09-19T04:26:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d8d026740 10 client.0 _ll_register_callbacks cb 0x7f3d78009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:53410/0 (socket says 172.21.15.94:53410) 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d83fff640 1 -- 172.21.15.94:0/2836726515 learned_addr learned my addr 172.21.15.94:0/2836726515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d889e9640 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646edeafc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d891ea640 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 0x5646edea12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d83fff640 1 -- 172.21.15.94:0/2836726515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 msgr2=0x5646edea12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d83fff640 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 0x5646edea12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d83fff640 1 -- 172.21.15.94:0/2836726515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 msgr2=0x5646edeafc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d83fff640 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646edeafc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.283+0000 7f3d83fff640 1 -- 172.21.15.94:0/2836726515 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646edeb9620 con 0x5646ede9db40 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.284+0000 7f3d889e9640 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646edeafc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.284+0000 7f3d83fff640 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea0d70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005940 tx=0x7f3d70031790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5edb5374271c2513 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.284+0000 7f3d63fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:26:01.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.284+0000 7f3d63fff640 1 -- 172.21.15.94:0/2836726515 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70035040 con 0x5646ede9db40 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.284+0000 7f3d63fff640 1 -- 172.21.15.94:0/2836726515 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8585+0+0 (secure 0 0 0) 0x7f3d70004440 con 0x5646ede9db40 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.284+0000 7f3d8d026740 1 -- 172.21.15.94:0/2836726515 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7c005020 con 0x5646ede9db40 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.284+0000 7f3d63fff640 1 -- 172.21.15.94:0/2836726515 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3d70060050 con 0x5646ede9db40 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.284+0000 7f3d8d026740 10 client.26318 fetch_fsmap learned FSMap version 45 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.284+0000 7f3d8d026740 1 -- 172.21.15.94:0/2836726515 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5646edf21a70 con 0x5646ede9db40 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.285+0000 7f3d63fff640 1 -- 172.21.15.94:0/2836726515 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d7c005020 con 0x5646ede9db40 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.285+0000 7f3d63fff640 1 -- 172.21.15.94:0/2836726515 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f3d70050040 con 0x5646ede9db40 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.285+0000 7f3d8d026740 10 client.26318 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.285+0000 7f3d8d026740 10 client.26318 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.285+0000 7f3d8d026740 20 client.26318 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.285+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5646edf22180 0x5646edf245a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.285+0000 7f3d8d026740 4 client.26318 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.285+0000 7f3d8d026740 1 -- 172.21.15.94:0/2836726515 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646ede09d70 con 0x5646edf22180 2024-09-19T04:26:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.285+0000 7f3d889e9640 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5646edf22180 0x5646edf245a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d63fff640 10 client.26318 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d889e9640 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5646edf22180 0x5646edf245a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bda9d52ebb35137 server_cookie=c82f706e244227a0 in_seq=0 out_seq=0 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d63fff640 1 -- 172.21.15.94:0/2836726515 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646ede09d70 con 0x5646edf22180 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d63fff640 10 client.26318 handle_command_reply: tid=0 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 client.26318 shutdown 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 -- 172.21.15.94:0/2836726515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 msgr2=0x5646edea0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea0d70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005940 tx=0x7f3d70031790 comp rx=0 tx=0).stop 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 -- 172.21.15.94:0/2836726515 shutdown_connections 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x5646edf22180 0x5646edf245a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646edea7370 0x5646edea12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ede9db40 0x5646edea0d70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 --2- 172.21.15.94:0/2836726515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646ede9c5d0 0x5646edeafc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 -- 172.21.15.94:0/2836726515 >> 172.21.15.94:0/2836726515 conn(0x5646ede65310 msgr2=0x5646ede97d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d63fff640 0 client.26318 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:26:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 -- 172.21.15.94:0/2836726515 shutdown_connections 2024-09-19T04:26:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 1 -- 172.21.15.94:0/2836726515 wait complete. 2024-09-19T04:26:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.286+0000 7f3d8d026740 20 client.26318 trim_cache size 0 max 16384 2024-09-19T04:26:01.293 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3d740a33b0 msgr2=0x7f3d740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3d740a33b0 0x7f3d740a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005c40 tx=0x7f3d7c009500 comp rx=0 tx=0).stop 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84100e20 msgr2=0x7f3d84198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84100e20 0x7f3d84198490 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009c20 tx=0x7f3d7800b690 comp rx=0 tx=0).stop 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/4047490288 shutdown_connections 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3d740a33b0 0x7f3d740a5870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84102ef0 0x7f3d8419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d84102570 0x7f3d841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 --2- 172.21.15.94:0/4047490288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d84100e20 0x7f3d84198490 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/4047490288 >> 172.21.15.94:0/4047490288 conn(0x7f3d840fc7d0 msgr2=0x7f3d84069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/4047490288 shutdown_connections 2024-09-19T04:26:01.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:01.287+0000 7f3d8ac4d640 1 -- 172.21.15.94:0/4047490288 wait complete. 2024-09-19T04:26:01.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:01 smithi094 ceph-mon[30972]: pgmap v340: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:26:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:01 smithi104 ceph-mon[34145]: pgmap v340: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:26:01.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:01 smithi112 ceph-mon[35051]: pgmap v340: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:26:02.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:26:02.665 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:03.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.279+0000 7f3dd5e27640 1 -- 172.21.15.94:0/2788109406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 msgr2=0x7f3dd01087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.279+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/2788109406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 0x7f3dd01087d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009ca0 tx=0x7f3dc802a8b0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.280+0000 7f3dd5e27640 1 -- 172.21.15.94:0/2788109406 shutdown_connections 2024-09-19T04:26:03.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.280+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/2788109406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd010d520 0x7f3dd01150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.280+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/2788109406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd0108da0 0x7f3dd010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.280+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/2788109406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 0x7f3dd01087d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.280+0000 7f3dd5e27640 1 -- 172.21.15.94:0/2788109406 >> 172.21.15.94:0/2788109406 conn(0x7f3dd0077af0 msgr2=0x7f3dd0079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.280+0000 7f3dd5e27640 1 -- 172.21.15.94:0/2788109406 shutdown_connections 2024-09-19T04:26:03.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.280+0000 7f3dd5e27640 1 -- 172.21.15.94:0/2788109406 wait complete. 2024-09-19T04:26:03.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.281+0000 7f3dd5e27640 1 Processor -- start 2024-09-19T04:26:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dd5e27640 1 -- start start 2024-09-19T04:26:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dd5e27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 0x7f3dd01a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dd5e27640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd0108da0 0x7f3dd01a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dd5e27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd010d520 0x7f3dd019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dd5e27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0118440 con 0x7f3dd010d520 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dcf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 0x7f3dd01a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dd5e27640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd01182c0 con 0x7f3dd0108da0 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dcf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 0x7f3dd01a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44328/0 (socket says 172.21.15.94:44328) 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/3523807385 learned_addr learned my addr 172.21.15.94:0/3523807385 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dceffd640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd0108da0 0x7f3dd01a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.282+0000 7f3dd5e27640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd01185c0 con 0x7f3dd01083f0 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.283+0000 7f3dcffff640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd010d520 0x7f3dd019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.283+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/3523807385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd0108da0 msgr2=0x7f3dd01a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.283+0000 7f3dcf7fe640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd0108da0 0x7f3dd01a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.283+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd010d520 msgr2=0x7f3dd019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.283+0000 7f3dcf7fe640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd010d520 0x7f3dd019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.283+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/3523807385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd019ecc0 con 0x7f3dd01083f0 2024-09-19T04:26:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.283+0000 7f3dcffff640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd010d520 0x7f3dd019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:26:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.283+0000 7f3dcf7fe640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 0x7f3dd01a43d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3dc802aea0 tx=0x7f3dc8003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.283+0000 7f3dccff9640 1 -- 172.21.15.94:0/3523807385 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8004240 con 0x7f3dd01083f0 2024-09-19T04:26:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.284+0000 7f3dccff9640 1 -- 172.21.15.94:0/3523807385 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dc8040070 con 0x7f3dd01083f0 2024-09-19T04:26:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.284+0000 7f3dccff9640 1 -- 172.21.15.94:0/3523807385 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8045d60 con 0x7f3dd01083f0 2024-09-19T04:26:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.284+0000 7f3dd5e27640 1 -- 172.21.15.94:0/3523807385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd019ef50 con 0x7f3dd01083f0 2024-09-19T04:26:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.284+0000 7f3dd5e27640 1 -- 172.21.15.94:0/3523807385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd01ab2a0 con 0x7f3dd01083f0 2024-09-19T04:26:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dd8200740 1 Processor -- start 2024-09-19T04:26:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dd8200740 1 -- start start 2024-09-19T04:26:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f73887090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f73887a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73892370 0x556f73894760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dd8200740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556f737ed2f0 con 0x556f738875d0 2024-09-19T04:26:03.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dd8200740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f73794cf0 con 0x556f73888b40 2024-09-19T04:26:03.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dd8200740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f737e92b0 con 0x556f73892370 2024-09-19T04:26:03.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dccff9640 1 -- 172.21.15.94:0/3523807385 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3dc8031440 con 0x7f3dd01083f0 2024-09-19T04:26:03.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dcffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f73887090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dcffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f73887090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:49646/0 (socket says 172.21.15.94:49646) 2024-09-19T04:26:03.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.288+0000 7f3dcffff640 1 -- 172.21.15.94:0/1158281985 learned_addr learned my addr 172.21.15.94:0/1158281985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:03.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.289+0000 7f3dcf7fe640 1 --2- 172.21.15.94:0/1158281985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f73887a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.289+0000 7f3dcffff640 1 -- 172.21.15.94:0/1158281985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73892370 msgr2=0x556f73894760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.289+0000 7f3dcffff640 1 --2- 172.21.15.94:0/1158281985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73892370 0x556f73894760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.289+0000 7f3dcffff640 1 -- 172.21.15.94:0/1158281985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 msgr2=0x556f73887a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.289+0000 7f3dcffff640 1 --2- 172.21.15.94:0/1158281985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f73887a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.289+0000 7f3dcffff640 1 -- 172.21.15.94:0/1158281985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f737793a0 con 0x556f73888b40 2024-09-19T04:26:03.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.290+0000 7f3dcffff640 1 --2- 172.21.15.94:0/1158281985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f73887090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009b50 tx=0x7f3dc000e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5497556c6476943b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.290+0000 7f3dccff9640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3db80a35c0 0x7f3db80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.290+0000 7f3dcf7fe640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3db80a35c0 0x7f3db80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.290+0000 7f3dccff9640 1 -- 172.21.15.94:0/3523807385 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f3dc80f10d0 con 0x7f3dd01083f0 2024-09-19T04:26:03.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.291+0000 7f3db5ffb640 1 -- 172.21.15.94:0/1158281985 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0016070 con 0x556f73888b40 2024-09-19T04:26:03.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.291+0000 7f3db5ffb640 1 -- 172.21.15.94:0/1158281985 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dc0009e40 con 0x556f73888b40 2024-09-19T04:26:03.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.291+0000 7f3db5ffb640 1 -- 172.21.15.94:0/1158281985 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0010790 con 0x556f73888b40 2024-09-19T04:26:03.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.291+0000 7f3dcf7fe640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3db80a35c0 0x7f3db80a5a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x556f73884ac0 tx=0x7f3dc80b9040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 -- 172.21.15.94:0/1158281985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 msgr2=0x556f73887090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 --2- 172.21.15.94:0/1158281985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f73887090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009b50 tx=0x7f3dc000e6c0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 -- 172.21.15.94:0/1158281985 shutdown_connections 2024-09-19T04:26:03.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 --2- 172.21.15.94:0/1158281985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73892370 0x556f73894760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 --2- 172.21.15.94:0/1158281985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f73887a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 --2- 172.21.15.94:0/1158281985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f73887090 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 -- 172.21.15.94:0/1158281985 >> 172.21.15.94:0/1158281985 conn(0x556f73850310 msgr2=0x556f73883c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 -- 172.21.15.94:0/1158281985 shutdown_connections 2024-09-19T04:26:03.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 -- 172.21.15.94:0/1158281985 wait complete. 2024-09-19T04:26:03.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.292+0000 7f3dd8200740 1 -- start start 2024-09-19T04:26:03.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.293+0000 7f3dd8200740 1 Processor -- start 2024-09-19T04:26:03.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.293+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f73891dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.293+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73888b40 0x556f7388f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.293+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73892370 0x556f7388f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dd8200740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556f738a6080 con 0x556f738875d0 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dd8200740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f737a8060 con 0x556f73892370 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dd8200740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f738a6200 con 0x556f73888b40 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73892370 0x556f7388f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73892370 0x556f7388f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:49656/0 (socket says 172.21.15.94:49656) 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 -- 172.21.15.94:0/3582388727 learned_addr learned my addr 172.21.15.94:0/3582388727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f73891dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dd8200740 10 client.0 _ll_register_callbacks cb 0x556f734fb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dceffd640 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73888b40 0x556f7388f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 -- 172.21.15.94:0/3582388727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73888b40 msgr2=0x556f7388f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73888b40 0x556f7388f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 -- 172.21.15.94:0/3582388727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 msgr2=0x556f73891dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f73891dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 -- 172.21.15.94:0/3582388727 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7355e3c0 con 0x556f73892370 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dceffd640 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73888b40 0x556f7388f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.294+0000 7f3dcffff640 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73892370 0x556f7388f800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009430 tx=0x7f3dc0011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e1779b1360f6254 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.295+0000 7f3daeffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.295+0000 7f3daeffd640 1 -- 172.21.15.94:0/3582388727 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0016070 con 0x556f73892370 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.295+0000 7f3daeffd640 1 -- 172.21.15.94:0/3582388727 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f3dc0009c40 con 0x556f73892370 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.295+0000 7f3dd8200740 1 -- 172.21.15.94:0/3582388727 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f737fa860 con 0x556f73892370 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.295+0000 7f3daeffd640 1 -- 172.21.15.94:0/3582388727 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3dc002f050 con 0x556f73892370 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.295+0000 7f3dd8200740 10 client.26330 fetch_fsmap learned FSMap version 45 2024-09-19T04:26:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.295+0000 7f3dd8200740 1 -- 172.21.15.94:0/3582388727 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556f738b1ba0 con 0x556f73892370 2024-09-19T04:26:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.295+0000 7f3daeffd640 1 -- 172.21.15.94:0/3582388727 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556f737fa860 con 0x556f73892370 2024-09-19T04:26:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.296+0000 7f3daeffd640 1 -- 172.21.15.94:0/3582388727 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f3dc000f040 con 0x556f73892370 2024-09-19T04:26:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.296+0000 7f3dd8200740 10 client.26330 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:26:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.296+0000 7f3dd8200740 10 client.26330 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:26:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.296+0000 7f3dd8200740 20 client.26330 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.296+0000 7f3dd8200740 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x556f738b2fb0 0x556f738b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.296+0000 7f3dd8200740 4 client.26330 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.296+0000 7f3dd8200740 1 -- 172.21.15.94:0/3582388727 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f737f4d70 con 0x556f738b2fb0 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.296+0000 7f3dceffd640 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x556f738b2fb0 0x556f738b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.297+0000 7f3dceffd640 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x556f738b2fb0 0x556f738b53a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55efef071ead0bc5 server_cookie=f3d02c7280ef3cd3 in_seq=0 out_seq=0 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.297+0000 7f3daeffd640 10 client.26330 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3daeffd640 1 -- 172.21.15.94:0/3582388727 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f737f4d70 con 0x556f738b2fb0 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3daeffd640 10 client.26330 handle_command_reply: tid=0 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3dd8200740 1 client.26330 shutdown 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3dd8200740 1 -- 172.21.15.94:0/3582388727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73892370 msgr2=0x556f7388f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3dd8200740 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73892370 0x556f7388f800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009430 tx=0x7f3dc0011f90 comp rx=0 tx=0).stop 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3dd8200740 1 -- 172.21.15.94:0/3582388727 shutdown_connections 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3dd8200740 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x556f738b2fb0 0x556f738b53a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3dd8200740 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73892370 0x556f7388f800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3dd8200740 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73888b40 0x556f7388f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3dd8200740 1 --2- 172.21.15.94:0/3582388727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f73891dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3dd8200740 1 -- 172.21.15.94:0/3582388727 >> 172.21.15.94:0/3582388727 conn(0x556f73850310 msgr2=0x556f73889930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.298+0000 7f3daeffd640 0 client.26330 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:26:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.299+0000 7f3dd8200740 1 -- 172.21.15.94:0/3582388727 shutdown_connections 2024-09-19T04:26:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.299+0000 7f3dd8200740 1 -- 172.21.15.94:0/3582388727 wait complete. 2024-09-19T04:26:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.299+0000 7f3dd8200740 20 client.26330 trim_cache size 0 max 16384 2024-09-19T04:26:03.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.322+0000 7f3dd8200740 1 Processor -- start 2024-09-19T04:26:03.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.322+0000 7f3dd8200740 1 -- start start 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f7388f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738bab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73892370 0x556f738bb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dd8200740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd01add40 con 0x556f738875d0 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dd8200740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd01ab5e0 con 0x556f73888b40 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dd8200740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0074dd0 con 0x556f73892370 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dcffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f7388f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dcffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f7388f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33588/0 (socket says 172.21.15.94:33588) 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dcffff640 1 -- 172.21.15.94:0/7414561 learned_addr learned my addr 172.21.15.94:0/7414561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dcffff640 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73892370 0x556f738bb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.323+0000 7f3dceffd640 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738bab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.324+0000 7f3dcffff640 1 -- 172.21.15.94:0/7414561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73892370 msgr2=0x556f738bb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.324+0000 7f3dcffff640 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73892370 0x556f738bb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.324+0000 7f3dcffff640 1 -- 172.21.15.94:0/7414561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 msgr2=0x556f738bab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.324+0000 7f3dcffff640 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738bab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.324+0000 7f3dcffff640 1 -- 172.21.15.94:0/7414561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f73894ad0 con 0x556f738875d0 2024-09-19T04:26:03.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.324+0000 7f3dceffd640 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738bab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:03.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.324+0000 7f3dcffff640 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f7388f040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0006010 tx=0x7f3dc00105e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e37900a6afa57a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.325+0000 7f3db57fa640 1 -- 172.21.15.94:0/7414561 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0016040 con 0x556f738875d0 2024-09-19T04:26:03.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.325+0000 7f3db57fa640 1 -- 172.21.15.94:0/7414561 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dc0009720 con 0x556f738875d0 2024-09-19T04:26:03.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.325+0000 7f3db57fa640 1 -- 172.21.15.94:0/7414561 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0004ae0 con 0x556f738875d0 2024-09-19T04:26:03.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.326+0000 7f3dd8200740 1 -- 172.21.15.94:0/7414561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 msgr2=0x556f7388f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.326+0000 7f3dd8200740 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f7388f040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0006010 tx=0x7f3dc00105e0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.326+0000 7f3dd8200740 1 -- 172.21.15.94:0/7414561 shutdown_connections 2024-09-19T04:26:03.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.326+0000 7f3dd8200740 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f73892370 0x556f738bb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.326+0000 7f3dd8200740 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738bab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.326+0000 7f3dd8200740 1 --2- 172.21.15.94:0/7414561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f738875d0 0x556f7388f040 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.326+0000 7f3dd8200740 1 -- 172.21.15.94:0/7414561 >> 172.21.15.94:0/7414561 conn(0x556f73850310 msgr2=0x556f7388e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.327+0000 7f3dd8200740 1 -- 172.21.15.94:0/7414561 shutdown_connections 2024-09-19T04:26:03.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.327+0000 7f3dd8200740 1 -- 172.21.15.94:0/7414561 wait complete. 2024-09-19T04:26:03.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.327+0000 7f3dd8200740 1 -- start start 2024-09-19T04:26:03.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.328+0000 7f3dd8200740 1 Processor -- start 2024-09-19T04:26:03.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.328+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f738875d0 0x556f7388cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.328+0000 7f3dd8200740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.328+0000 7f3dcffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f738875d0 0x556f7388cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.328+0000 7f3dcffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f738875d0 0x556f7388cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44378/0 (socket says 172.21.15.94:44378) 2024-09-19T04:26:03.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dcffff640 1 -- 172.21.15.94:0/519968260 learned_addr learned my addr 172.21.15.94:0/519968260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:03.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dceffd640 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dd8200740 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f73892370 0x556f738abc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00818f0 con 0x556f73892370 2024-09-19T04:26:03.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0082d20 con 0x556f73888b40 2024-09-19T04:26:03.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0081f80 con 0x556f738875d0 2024-09-19T04:26:03.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dd8200740 10 client.0 _ll_register_callbacks cb 0x7f3dc00099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:03.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dcffff640 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f73892370 0x556f738abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dceffd640 1 -- 172.21.15.94:0/519968260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f738875d0 msgr2=0x556f7388cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dceffd640 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f738875d0 0x556f7388cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dceffd640 1 -- 172.21.15.94:0/519968260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f73892370 msgr2=0x556f738abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dceffd640 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f73892370 0x556f738abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dceffd640 1 -- 172.21.15.94:0/519968260 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7388b420 con 0x556f73888b40 2024-09-19T04:26:03.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.329+0000 7f3dcffff640 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f73892370 0x556f738abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.330+0000 7f3dceffd640 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738ab4c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x556f73888070 tx=0x7f3dc4009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9fe3835da43b42d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.330+0000 7f3daeffd640 10 client.26336 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.330+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dc0009720 con 0x556f73888b40 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.330+0000 7f3daeffd640 1 -- 172.21.15.94:0/519968260 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4005220 con 0x556f73888b40 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.330+0000 7f3daeffd640 1 -- 172.21.15.94:0/519968260 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f3dc40053c0 con 0x556f73888b40 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.330+0000 7f3daeffd640 1 -- 172.21.15.94:0/519968260 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3dc402d050 con 0x556f73888b40 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.331+0000 7f3dd8200740 10 client.26336 fetch_fsmap learned FSMap version 45 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.331+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=45}) -- 0x556f7391ced0 con 0x556f73888b40 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.331+0000 7f3daeffd640 1 -- 172.21.15.94:0/519968260 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f3dc40047a0 con 0x556f73888b40 2024-09-19T04:26:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.331+0000 7f3dd8200740 10 client.26336 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:26:03.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.331+0000 7f3dd8200740 10 client.26336 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:26:03.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.332+0000 7f3dd8200740 20 client.26336 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:03.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.332+0000 7f3dd8200740 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x556f7391d4c0 0x556f7391f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.332+0000 7f3dd8200740 4 client.26336 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:03.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.332+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f73771170 con 0x556f7391d4c0 2024-09-19T04:26:03.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.332+0000 7f3dcffff640 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x556f7391d4c0 0x556f7391f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3daeffd640 10 client.26336 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dcffff640 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x556f7391d4c0 0x556f7391f8e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d4b56990517b2ed server_cookie=445da7caaff06cc2 in_seq=0 out_seq=0 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3daeffd640 1 -- 172.21.15.94:0/519968260 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f73771170 con 0x556f7391d4c0 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3daeffd640 10 client.26336 handle_command_reply: tid=0 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 client.26336 shutdown 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 msgr2=0x556f738ab4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738ab4c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x556f73888070 tx=0x7f3dc4009bf0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 shutdown_connections 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x556f7391d4c0 0x556f7391f8e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f73892370 0x556f738abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f73888b40 0x556f738ab4c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 --2- 172.21.15.94:0/519968260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f738875d0 0x556f7388cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 >> 172.21.15.94:0/519968260 conn(0x556f73850310 msgr2=0x556f7388e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3daeffd640 0 client.26336 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:26:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.333+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 shutdown_connections 2024-09-19T04:26:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.334+0000 7f3dd8200740 1 -- 172.21.15.94:0/519968260 wait complete. 2024-09-19T04:26:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.334+0000 7f3dd8200740 20 client.26336 trim_cache size 0 max 16384 2024-09-19T04:26:03.337 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:26:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.334+0000 7f3dd5e27640 1 -- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3db80a35c0 msgr2=0x7f3db80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.334+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3db80a35c0 0x7f3db80a5a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x556f73884ac0 tx=0x7f3dc80b9040 comp rx=0 tx=0).stop 2024-09-19T04:26:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.334+0000 7f3dd5e27640 1 -- 172.21.15.94:0/3523807385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 msgr2=0x7f3dd01a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.334+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 0x7f3dd01a43d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3dc802aea0 tx=0x7f3dc8003ba0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.334+0000 7f3dd5e27640 1 -- 172.21.15.94:0/3523807385 shutdown_connections 2024-09-19T04:26:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.334+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3db80a35c0 0x7f3db80a5a80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.335+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd010d520 0x7f3dd019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.335+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd0108da0 0x7f3dd01a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.335+0000 7f3dd5e27640 1 --2- 172.21.15.94:0/3523807385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dd01083f0 0x7f3dd01a43d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.335+0000 7f3dd5e27640 1 -- 172.21.15.94:0/3523807385 >> 172.21.15.94:0/3523807385 conn(0x7f3dd0077af0 msgr2=0x7f3dd0078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.336+0000 7f3dd5e27640 1 -- 172.21.15.94:0/3523807385 shutdown_connections 2024-09-19T04:26:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:03.336+0000 7f3dd5e27640 1 -- 172.21.15.94:0/3523807385 wait complete. 2024-09-19T04:26:03.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:03 smithi094 ceph-mon[30972]: pgmap v341: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:26:03.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:03 smithi094 ceph-mon[30972]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:26:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:03 smithi104 ceph-mon[34145]: pgmap v341: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:26:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:03 smithi104 ceph-mon[34145]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:26:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:03 smithi112 ceph-mon[35051]: pgmap v341: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-09-19T04:26:03.793 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:03 smithi112 ceph-mon[35051]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:26:04.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:26:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:05.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.246+0000 7faea8506640 1 -- 172.21.15.94:0/4124486214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea0109110 msgr2=0x7faea0110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.246+0000 7faea8506640 1 --2- 172.21.15.94:0/4124486214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea0109110 0x7faea0110c90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fae98009be0 tx=0x7fae9802a6c0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.247+0000 7faea8506640 1 -- 172.21.15.94:0/4124486214 shutdown_connections 2024-09-19T04:26:05.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.247+0000 7faea8506640 1 --2- 172.21.15.94:0/4124486214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea0109110 0x7faea0110c90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.247+0000 7faea8506640 1 --2- 172.21.15.94:0/4124486214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea0104990 0x7faea01089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.247+0000 7faea8506640 1 --2- 172.21.15.94:0/4124486214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea0103fe0 0x7faea01043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.247+0000 7faea8506640 1 -- 172.21.15.94:0/4124486214 >> 172.21.15.94:0/4124486214 conn(0x7faea00ffa40 msgr2=0x7faea0101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:05.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.248+0000 7faea8506640 1 -- 172.21.15.94:0/4124486214 shutdown_connections 2024-09-19T04:26:05.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.248+0000 7faea8506640 1 -- 172.21.15.94:0/4124486214 wait complete. 2024-09-19T04:26:05.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.249+0000 7faea8506640 1 Processor -- start 2024-09-19T04:26:05.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.249+0000 7faea8506640 1 -- start start 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea8506640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea0103fe0 0x7faea0110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea8506640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea0104990 0x7faea010dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea8506640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea0109110 0x7faea010e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea62a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea0103fe0 0x7faea0110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea62a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea0103fe0 0x7faea0110a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33622/0 (socket says 172.21.15.94:33622) 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea62a2640 1 -- 172.21.15.94:0/2281464185 learned_addr learned my addr 172.21.15.94:0/2281464185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea5aa1640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea0104990 0x7faea010dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea6aa3640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea0109110 0x7faea010e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faea0113fd0 con 0x7faea0103fe0 2024-09-19T04:26:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faea0113e50 con 0x7faea0104990 2024-09-19T04:26:05.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.250+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faea0114150 con 0x7faea0109110 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.251+0000 7faea62a2640 1 -- 172.21.15.94:0/2281464185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea0109110 msgr2=0x7faea010e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.251+0000 7faea62a2640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea0109110 0x7faea010e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.251+0000 7faea62a2640 1 -- 172.21.15.94:0/2281464185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea0104990 msgr2=0x7faea010dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.251+0000 7faea62a2640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea0104990 0x7faea010dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.251+0000 7faea62a2640 1 -- 172.21.15.94:0/2281464185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea010e8b0 con 0x7faea0103fe0 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.251+0000 7faea5aa1640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea0104990 0x7faea010dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.251+0000 7faea62a2640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea0103fe0 0x7faea0110a30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00e9c0 tx=0x7fae9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.252+0000 7fae977fe640 1 -- 172.21.15.94:0/2281464185 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00ccc0 con 0x7faea0103fe0 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.252+0000 7fae977fe640 1 -- 172.21.15.94:0/2281464185 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae9c0054d0 con 0x7faea0103fe0 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.252+0000 7fae977fe640 1 -- 172.21.15.94:0/2281464185 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00ccc0 con 0x7faea0103fe0 2024-09-19T04:26:05.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.252+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea01aad10 con 0x7faea0103fe0 2024-09-19T04:26:05.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.252+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea01ab1a0 con 0x7faea0103fe0 2024-09-19T04:26:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.256+0000 7fae977fe640 1 -- 172.21.15.94:0/2281464185 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fae9c004d30 con 0x7faea0103fe0 2024-09-19T04:26:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.257+0000 7fae977fe640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.257+0000 7fae977fe640 1 -- 172.21.15.94:0/2281464185 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fae9c00cac0 con 0x7faea0103fe0 2024-09-19T04:26:05.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faeaa8df740 1 Processor -- start 2024-09-19T04:26:05.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faea5aa1640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faeaa8df740 1 -- start start 2024-09-19T04:26:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x557077767090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077767a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x557077774760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faeaa8df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570776cd2f0 con 0x5570777675d0 2024-09-19T04:26:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faeaa8df740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557077674cf0 con 0x557077768b40 2024-09-19T04:26:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faeaa8df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5570776c92b0 con 0x557077772370 2024-09-19T04:26:05.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.258+0000 7faea5aa1640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faea010f2a0 tx=0x7fae90005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:05.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea5aa1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077767a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea5aa1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077767a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33634/0 (socket says 172.21.15.94:33634) 2024-09-19T04:26:05.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea5aa1640 1 -- 172.21.15.94:0/1445712224 learned_addr learned my addr 172.21.15.94:0/1445712224 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea6aa3640 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x557077774760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea62a2640 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x557077767090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea6aa3640 1 -- 172.21.15.94:0/1445712224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 msgr2=0x557077767090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea6aa3640 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x557077767090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea6aa3640 1 -- 172.21.15.94:0/1445712224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 msgr2=0x557077767a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea6aa3640 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077767a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.259+0000 7faea6aa3640 1 -- 172.21.15.94:0/1445712224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570776593a0 con 0x557077772370 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.260+0000 7faea62a2640 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x557077767090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.260+0000 7faea6aa3640 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x557077774760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fae98009b30 tx=0x7fae98004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed918aaaa60b1a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.260+0000 7fae94ff9640 1 -- 172.21.15.94:0/1445712224 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98041070 con 0x557077772370 2024-09-19T04:26:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.260+0000 7fae94ff9640 1 -- 172.21.15.94:0/1445712224 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae98037040 con 0x557077772370 2024-09-19T04:26:05.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.260+0000 7fae94ff9640 1 -- 172.21.15.94:0/1445712224 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98036040 con 0x557077772370 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.261+0000 7faeaa8df740 1 -- 172.21.15.94:0/1445712224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 msgr2=0x557077774760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.261+0000 7faeaa8df740 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x557077774760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fae98009b30 tx=0x7fae98004a60 comp rx=0 tx=0).stop 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.261+0000 7faeaa8df740 1 -- 172.21.15.94:0/1445712224 shutdown_connections 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.261+0000 7faeaa8df740 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x557077774760 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.261+0000 7faeaa8df740 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077767a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.261+0000 7faeaa8df740 1 --2- 172.21.15.94:0/1445712224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x557077767090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.261+0000 7faeaa8df740 1 -- 172.21.15.94:0/1445712224 >> 172.21.15.94:0/1445712224 conn(0x557077730310 msgr2=0x557077763c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.261+0000 7faeaa8df740 1 -- 172.21.15.94:0/1445712224 shutdown_connections 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.261+0000 7faeaa8df740 1 -- 172.21.15.94:0/1445712224 wait complete. 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.262+0000 7faeaa8df740 1 -- start start 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.262+0000 7faeaa8df740 1 Processor -- start 2024-09-19T04:26:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.262+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077771df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.262+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x55707776f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.262+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x55707776f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea5aa1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x55707776f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea5aa1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x55707776f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:49730/0 (socket says 172.21.15.94:49730) 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea5aa1640 1 -- 172.21.15.94:0/461154606 learned_addr learned my addr 172.21.15.94:0/461154606 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea6aa3640 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x55707776f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea62a2640 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077771df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557077775010 con 0x5570777675d0 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557077688060 con 0x557077768b40 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557077775190 con 0x557077772370 2024-09-19T04:26:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faeaa8df740 10 client.0 _ll_register_callbacks cb 0x5570773db8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:05.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea6aa3640 1 -- 172.21.15.94:0/461154606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 msgr2=0x55707776f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea6aa3640 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x55707776f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea6aa3640 1 -- 172.21.15.94:0/461154606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 msgr2=0x557077771df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea6aa3640 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077771df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.263+0000 7faea6aa3640 1 -- 172.21.15.94:0/461154606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707743e3c0 con 0x557077772370 2024-09-19T04:26:05.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.264+0000 7faea5aa1640 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x55707776f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:05.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.264+0000 7faea6aa3640 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x55707776f7e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae98009b30 tx=0x7fae98005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64feacba0402cb9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:05.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.264+0000 7fae89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:05.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.264+0000 7fae89ffb640 1 -- 172.21.15.94:0/461154606 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9804d020 con 0x557077772370 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.264+0000 7fae89ffb640 1 -- 172.21.15.94:0/461154606 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fae98004040 con 0x557077772370 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.264+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae98037040 con 0x557077772370 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.264+0000 7fae89ffb640 1 -- 172.21.15.94:0/461154606 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fae98058050 con 0x557077772370 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.265+0000 7faeaa8df740 10 client.26320 fetch_fsmap learned FSMap version 45 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.265+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557077775010 con 0x557077772370 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.265+0000 7fae89ffb640 1 -- 172.21.15.94:0/461154606 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae98009800 con 0x557077772370 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.265+0000 7fae89ffb640 1 -- 172.21.15.94:0/461154606 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fae9803c040 con 0x557077772370 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.265+0000 7faeaa8df740 10 client.26320 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.265+0000 7faeaa8df740 10 client.26320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.265+0000 7faeaa8df740 20 client.26320 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.265+0000 7faeaa8df740 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557077781ca0 0x557077784090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.266+0000 7faeaa8df740 4 client.26320 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:05.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.266+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fae98037040 con 0x557077781ca0 2024-09-19T04:26:05.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.266+0000 7faea62a2640 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557077781ca0 0x557077784090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.266+0000 7fae89ffb640 10 client.26320 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:05.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.266+0000 7faea62a2640 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557077781ca0 0x557077784090 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782696e77a61d8e8 server_cookie=c14c4a81ea31e9b in_seq=0 out_seq=0 2024-09-19T04:26:05.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7fae89ffb640 1 -- 172.21.15.94:0/461154606 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fae98037040 con 0x557077781ca0 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7fae89ffb640 10 client.26320 handle_command_reply: tid=0 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7faeaa8df740 1 client.26320 shutdown 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 msgr2=0x55707776f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7faeaa8df740 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x55707776f7e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae98009b30 tx=0x7fae98005fd0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 shutdown_connections 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7faeaa8df740 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x557077781ca0 0x557077784090 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7fae89ffb640 0 client.26320 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7faeaa8df740 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x55707776f7e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7faeaa8df740 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x55707776f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7faeaa8df740 1 --2- 172.21.15.94:0/461154606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077771df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.267+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 >> 172.21.15.94:0/461154606 conn(0x557077730310 msgr2=0x557077769960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.268+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 shutdown_connections 2024-09-19T04:26:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.268+0000 7faeaa8df740 1 -- 172.21.15.94:0/461154606 wait complete. 2024-09-19T04:26:05.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.268+0000 7faeaa8df740 20 client.26320 trim_cache size 0 max 16384 2024-09-19T04:26:05.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.287+0000 7faeaa8df740 1 Processor -- start 2024-09-19T04:26:05.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.287+0000 7faeaa8df740 1 -- start start 2024-09-19T04:26:05.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.287+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570777675d0 0x55707776e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.287+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557077768b40 0x55707777e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faeaa8df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faea01b6800 con 0x557077768b40 2024-09-19T04:26:05.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faeaa8df740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faea01b40a0 con 0x5570777675d0 2024-09-19T04:26:05.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faeaa8df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faea01b1f10 con 0x557077772370 2024-09-19T04:26:05.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea62a2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570777675d0 0x55707776e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea62a2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570777675d0 0x55707776e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:49734/0 (socket says 172.21.15.94:49734) 2024-09-19T04:26:05.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea62a2640 1 -- 172.21.15.94:0/246362919 learned_addr learned my addr 172.21.15.94:0/246362919 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:05.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea5aa1640 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557077768b40 0x55707777e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea6aa3640 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea62a2640 1 -- 172.21.15.94:0/246362919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 msgr2=0x5570777897d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea62a2640 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777897d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea62a2640 1 -- 172.21.15.94:0/246362919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557077768b40 msgr2=0x55707777e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea62a2640 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557077768b40 0x55707777e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea62a2640 1 -- 172.21.15.94:0/246362919 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707776a120 con 0x5570777675d0 2024-09-19T04:26:05.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea5aa1640 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557077768b40 0x55707777e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:05.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.288+0000 7faea62a2640 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570777675d0 0x55707776e960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae9c098120 tx=0x7fae9c00efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6740afa9f2c2440 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:05.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.289+0000 7faea6aa3640 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777897d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:05.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.289+0000 7fae8bfff640 1 -- 172.21.15.94:0/246362919 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0a4930 con 0x5570777675d0 2024-09-19T04:26:05.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.289+0000 7fae8bfff640 1 -- 172.21.15.94:0/246362919 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae9c099fc0 con 0x5570777675d0 2024-09-19T04:26:05.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.289+0000 7fae8bfff640 1 -- 172.21.15.94:0/246362919 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c098330 con 0x5570777675d0 2024-09-19T04:26:05.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 -- 172.21.15.94:0/246362919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570777675d0 msgr2=0x55707776e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570777675d0 0x55707776e960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae9c098120 tx=0x7fae9c00efb0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 -- 172.21.15.94:0/246362919 shutdown_connections 2024-09-19T04:26:05.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777897d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557077768b40 0x55707777e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 --2- 172.21.15.94:0/246362919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570777675d0 0x55707776e960 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 -- 172.21.15.94:0/246362919 >> 172.21.15.94:0/246362919 conn(0x557077730310 msgr2=0x55707775ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:05.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 -- 172.21.15.94:0/246362919 shutdown_connections 2024-09-19T04:26:05.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 -- 172.21.15.94:0/246362919 wait complete. 2024-09-19T04:26:05.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.290+0000 7faeaa8df740 1 -- start start 2024-09-19T04:26:05.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faeaa8df740 1 Processor -- start 2024-09-19T04:26:05.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077775b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x5570777760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faeaa8df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777ea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faeaa8df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faea01b2560 con 0x5570777675d0 2024-09-19T04:26:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faeaa8df740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faea01b39d0 con 0x557077768b40 2024-09-19T04:26:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faeaa8df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faea01b2c30 con 0x557077772370 2024-09-19T04:26:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faeaa8df740 10 client.0 _ll_register_callbacks cb 0x7fae90009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faea62a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077775b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faea5aa1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x5570777760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faea5aa1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x5570777760a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:49738/0 (socket says 172.21.15.94:49738) 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faea6aa3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.291+0000 7faea5aa1640 1 -- 172.21.15.94:0/2064854665 learned_addr learned my addr 172.21.15.94:0/2064854665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7faea6aa3640 1 -- 172.21.15.94:0/2064854665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 msgr2=0x5570777760a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7faea6aa3640 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x5570777760a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7faea6aa3640 1 -- 172.21.15.94:0/2064854665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 msgr2=0x557077775b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7faea6aa3640 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077775b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7faea6aa3640 1 -- 172.21.15.94:0/2064854665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570777eadd0 con 0x557077772370 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7faea6aa3640 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777ea790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae980468b0 tx=0x7fae98009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c697969679e3e733 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7fae89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7fae89ffb640 1 -- 172.21.15.94:0/2064854665 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9804d030 con 0x557077772370 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7faeaa8df740 1 -- 172.21.15.94:0/2064854665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9c099fc0 con 0x557077772370 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.292+0000 7fae89ffb640 1 -- 172.21.15.94:0/2064854665 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fae98030e30 con 0x557077772370 2024-09-19T04:26:05.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7fae89ffb640 1 -- 172.21.15.94:0/2064854665 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fae98064050 con 0x557077772370 2024-09-19T04:26:05.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7faeaa8df740 10 client.26326 fetch_fsmap learned FSMap version 45 2024-09-19T04:26:05.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7faeaa8df740 1 -- 172.21.15.94:0/2064854665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5570777ec0b0 con 0x557077772370 2024-09-19T04:26:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7fae89ffb640 1 -- 172.21.15.94:0/2064854665 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae9c099fc0 con 0x557077772370 2024-09-19T04:26:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7fae89ffb640 1 -- 172.21.15.94:0/2064854665 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fae9803c070 con 0x557077772370 2024-09-19T04:26:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7faeaa8df740 10 client.26326 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:26:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7faeaa8df740 10 client.26326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7faeaa8df740 20 client.26326 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7faeaa8df740 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5570777ec830 0x5570777eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7faeaa8df740 4 client.26326 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7faeaa8df740 1 -- 172.21.15.94:0/2064854665 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570776da860 con 0x5570777ec830 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.293+0000 7faea62a2640 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5570777ec830 0x5570777eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.294+0000 7fae89ffb640 10 client.26326 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.294+0000 7faea62a2640 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5570777ec830 0x5570777eec50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b4868ef50cd4c8 server_cookie=3f505252e759093c in_seq=0 out_seq=0 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.294+0000 7fae89ffb640 1 -- 172.21.15.94:0/2064854665 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570776da860 con 0x5570777ec830 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.294+0000 7fae89ffb640 10 client.26326 handle_command_reply: tid=0 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.294+0000 7faeaa8df740 1 client.26326 shutdown 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.294+0000 7faeaa8df740 1 -- 172.21.15.94:0/2064854665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 msgr2=0x5570777ea790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.294+0000 7faeaa8df740 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777ea790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae980468b0 tx=0x7fae98009900 comp rx=0 tx=0).stop 2024-09-19T04:26:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7faeaa8df740 1 -- 172.21.15.94:0/2064854665 shutdown_connections 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7faeaa8df740 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5570777ec830 0x5570777eec50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7faeaa8df740 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557077772370 0x5570777ea790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7faeaa8df740 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557077768b40 0x5570777760a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7faeaa8df740 1 --2- 172.21.15.94:0/2064854665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570777675d0 0x557077775b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7faeaa8df740 1 -- 172.21.15.94:0/2064854665 >> 172.21.15.94:0/2064854665 conn(0x557077730310 msgr2=0x557077770900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7fae89ffb640 0 client.26326 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7faeaa8df740 1 -- 172.21.15.94:0/2064854665 shutdown_connections 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7faeaa8df740 1 -- 172.21.15.94:0/2064854665 wait complete. 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.295+0000 7faeaa8df740 20 client.26326 trim_cache size 0 max 16384 2024-09-19T04:26:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:26:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fae8c0a37b0 msgr2=0x7fae8c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faea010f2a0 tx=0x7fae90005f90 comp rx=0 tx=0).stop 2024-09-19T04:26:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea0103fe0 msgr2=0x7faea0110a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea0103fe0 0x7faea0110a30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00e9c0 tx=0x7fae9c00ee80 comp rx=0 tx=0).stop 2024-09-19T04:26:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 shutdown_connections 2024-09-19T04:26:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea0109110 0x7faea010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea0104990 0x7faea010dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 --2- 172.21.15.94:0/2281464185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea0103fe0 0x7faea0110a30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 >> 172.21.15.94:0/2281464185 conn(0x7faea00ffa40 msgr2=0x7faea0100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 shutdown_connections 2024-09-19T04:26:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:05.296+0000 7faea8506640 1 -- 172.21.15.94:0/2281464185 wait complete. 2024-09-19T04:26:05.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:05 smithi094 ceph-mon[30972]: pgmap v343: 97 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 37 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-09-19T04:26:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 590.0 s, 786.2 tps, lat 100.208 ms stddev 116.119 2024-09-19T04:26:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:05 smithi104 ceph-mon[34145]: pgmap v343: 97 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 37 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-09-19T04:26:05.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:05 smithi112 ceph-mon[35051]: pgmap v343: 97 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 37 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-09-19T04:26:07.032 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:26:07.032 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:26:07.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:07 smithi094 ceph-mon[30972]: pgmap v344: 97 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 37 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-09-19T04:26:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:07.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:07 smithi104 ceph-mon[34145]: pgmap v344: 97 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 37 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-09-19T04:26:07.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:07 smithi112 ceph-mon[35051]: pgmap v344: 97 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 37 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-09-19T04:26:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.009+0000 7ff7bbfff640 1 -- 172.21.15.94:0/3136880618 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc031e40 con 0x7ff7c4109450 2024-09-19T04:26:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.010+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3136880618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109450 msgr2=0x7ff7c4110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.010+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3136880618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109450 0x7ff7c4110250 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009ca0 tx=0x7ff7bc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.010+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3136880618 shutdown_connections 2024-09-19T04:26:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.010+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3136880618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c411b140 0x7ff7c411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.010+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3136880618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c4109830 0x7ff7c4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.010+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3136880618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109450 0x7ff7c4110250 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.010+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3136880618 >> 172.21.15.94:0/3136880618 conn(0x7ff7c406e140 msgr2=0x7ff7c406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.010+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3136880618 shutdown_connections 2024-09-19T04:26:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.010+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3136880618 wait complete. 2024-09-19T04:26:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.011+0000 7ff7cb48e640 1 Processor -- start 2024-09-19T04:26:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.011+0000 7ff7cb48e640 1 -- start start 2024-09-19T04:26:08.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.011+0000 7ff7cb48e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c4109450 0x7ff7c41a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.011+0000 7ff7cb48e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109830 0x7ff7c41a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7cb48e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c411b140 0x7ff7c41aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7cb48e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c4120c50 con 0x7ff7c4109830 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7cb48e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c4120ad0 con 0x7ff7c411b140 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7cb48e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c4120dd0 con 0x7ff7c4109450 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c8a29640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109830 0x7ff7c41a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c8a29640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109830 0x7ff7c41a76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33682/0 (socket says 172.21.15.94:33682) 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3783882717 learned_addr learned my addr 172.21.15.94:0/3783882717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c922a640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c4109450 0x7ff7c41a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3783882717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c4109450 msgr2=0x7ff7c41a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c4109450 0x7ff7c41a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3783882717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c411b140 msgr2=0x7ff7c41aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c411b140 0x7ff7c41aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3783882717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c41ac140 con 0x7ff7c4109830 2024-09-19T04:26:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.012+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109830 0x7ff7c41a76d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9f0 tx=0x7ff7c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.013+0000 7ff7ba7fc640 1 -- 172.21.15.94:0/3783882717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000cd80 con 0x7ff7c4109830 2024-09-19T04:26:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.013+0000 7ff7ba7fc640 1 -- 172.21.15.94:0/3783882717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c00108e0 con 0x7ff7c4109830 2024-09-19T04:26:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.013+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3783882717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c41ac430 con 0x7ff7c4109830 2024-09-19T04:26:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.013+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3783882717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c41b3d10 con 0x7ff7c4109830 2024-09-19T04:26:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.013+0000 7ff7ba7fc640 1 -- 172.21.15.94:0/3783882717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0018920 con 0x7ff7c4109830 2024-09-19T04:26:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.018+0000 7ff7ba7fc640 1 -- 172.21.15.94:0/3783882717 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff7c0010a90 con 0x7ff7c4109830 2024-09-19T04:26:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.020+0000 7ff7cd867740 1 Processor -- start 2024-09-19T04:26:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.020+0000 7ff7cd867740 1 -- start start 2024-09-19T04:26:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.020+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b33af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.020+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b345800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.020+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b348130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.020+0000 7ff7cd867740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55691b2a0280 con 0x55691b345d40 2024-09-19T04:26:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.020+0000 7ff7cd867740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55691b247b10 con 0x55691b33b480 2024-09-19T04:26:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.020+0000 7ff7cd867740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55691b29c240 con 0x55691b33ab60 2024-09-19T04:26:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c922a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b345800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c922a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b345800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:49754/0 (socket says 172.21.15.94:49754) 2024-09-19T04:26:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c922a640 1 -- 172.21.15.94:0/2062610774 learned_addr learned my addr 172.21.15.94:0/2062610774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b348130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c9a2b640 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b33af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c8a29640 1 -- 172.21.15.94:0/2062610774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 msgr2=0x55691b33af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b33af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c8a29640 1 -- 172.21.15.94:0/2062610774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 msgr2=0x55691b345800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b345800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c8a29640 1 -- 172.21.15.94:0/2062610774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691b2541f0 con 0x55691b345d40 2024-09-19T04:26:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7c922a640 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b345800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.021+0000 7ff7ba7fc640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7b00a3430 0x7ff7b00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.022+0000 7ff7ba7fc640 1 -- 172.21.15.94:0/3783882717 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7ff7c00d0220 con 0x7ff7c4109830 2024-09-19T04:26:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.022+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b348130 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00cd8a0 tx=0x7ff7c00d8910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a793b90f08830c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.022+0000 7ff7c9a2b640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7b00a3430 0x7ff7b00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.023+0000 7ff7af7fe640 1 -- 172.21.15.94:0/2062610774 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00986d0 con 0x55691b345d40 2024-09-19T04:26:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.023+0000 7ff7af7fe640 1 -- 172.21.15.94:0/2062610774 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c0010de0 con 0x55691b345d40 2024-09-19T04:26:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.023+0000 7ff7af7fe640 1 -- 172.21.15.94:0/2062610774 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c009bd90 con 0x55691b345d40 2024-09-19T04:26:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7cd867740 1 -- 172.21.15.94:0/2062610774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 msgr2=0x55691b348130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7c9a2b640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7b00a3430 0x7ff7b00a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009b90 tx=0x7ff7b40095d0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7cd867740 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b348130 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00cd8a0 tx=0x7ff7c00d8910 comp rx=0 tx=0).stop 2024-09-19T04:26:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7cd867740 1 -- 172.21.15.94:0/2062610774 shutdown_connections 2024-09-19T04:26:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7cd867740 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b348130 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7cd867740 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b345800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7cd867740 1 --2- 172.21.15.94:0/2062610774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b33af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7cd867740 1 -- 172.21.15.94:0/2062610774 >> 172.21.15.94:0/2062610774 conn(0x55691b3370f0 msgr2=0x55691b3374d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7cd867740 1 -- 172.21.15.94:0/2062610774 shutdown_connections 2024-09-19T04:26:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.024+0000 7ff7cd867740 1 -- 172.21.15.94:0/2062610774 wait complete. 2024-09-19T04:26:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.025+0000 7ff7cd867740 1 -- start start 2024-09-19T04:26:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.025+0000 7ff7cd867740 1 Processor -- start 2024-09-19T04:26:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.025+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b345780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b33b480 0x55691b342ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b345d40 0x55691b343260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c922a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b345780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7cd867740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55691b3489b0 con 0x55691b33b480 2024-09-19T04:26:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7cd867740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55691b25af10 con 0x55691b345d40 2024-09-19T04:26:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7cd867740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55691b348b30 con 0x55691b33ab60 2024-09-19T04:26:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c922a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b345780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44468/0 (socket says 172.21.15.94:44468) 2024-09-19T04:26:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c922a640 1 -- 172.21.15.94:0/1922349288 learned_addr learned my addr 172.21.15.94:0/1922349288 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7cd867740 10 client.0 _ll_register_callbacks cb 0x55691b223a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b33b480 0x55691b342ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c9a2b640 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b345d40 0x55691b343260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c922a640 1 -- 172.21.15.94:0/1922349288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b345d40 msgr2=0x55691b343260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c922a640 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b345d40 0x55691b343260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c922a640 1 -- 172.21.15.94:0/1922349288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b33b480 msgr2=0x55691b342ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c922a640 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b33b480 0x55691b342ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.026+0000 7ff7c922a640 1 -- 172.21.15.94:0/1922349288 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691b0116a0 con 0x55691b33ab60 2024-09-19T04:26:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.027+0000 7ff7c922a640 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b345780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009a30 tx=0x7ff7bc031f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe31921ed81df0d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.027+0000 7ff7acff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.027+0000 7ff7acff9640 1 -- 172.21.15.94:0/1922349288 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc031720 con 0x55691b33ab60 2024-09-19T04:26:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.027+0000 7ff7cd867740 1 -- 172.21.15.94:0/1922349288 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55691b2ad7f0 con 0x55691b33ab60 2024-09-19T04:26:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.027+0000 7ff7acff9640 1 -- 172.21.15.94:0/1922349288 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7ff7bc003970 con 0x55691b33ab60 2024-09-19T04:26:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.027+0000 7ff7acff9640 1 -- 172.21.15.94:0/1922349288 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff7bc003b70 con 0x55691b33ab60 2024-09-19T04:26:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.027+0000 7ff7cd867740 10 client.26338 fetch_fsmap learned FSMap version 45 2024-09-19T04:26:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.027+0000 7ff7cd867740 1 -- 172.21.15.94:0/1922349288 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55691b3544c0 con 0x55691b33ab60 2024-09-19T04:26:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.028+0000 7ff7acff9640 1 -- 172.21.15.94:0/1922349288 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7bc036040 con 0x55691b33ab60 2024-09-19T04:26:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.028+0000 7ff7acff9640 1 -- 172.21.15.94:0/1922349288 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7ff7bc0305c0 con 0x55691b33ab60 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.028+0000 7ff7cd867740 10 client.26338 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.028+0000 7ff7cd867740 10 client.26338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.028+0000 7ff7cd867740 20 client.26338 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.028+0000 7ff7cd867740 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55691b355d90 0x55691b358250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.028+0000 7ff7cd867740 4 client.26338 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.028+0000 7ff7cd867740 1 -- 172.21.15.94:0/1922349288 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55691b2ad7f0 con 0x55691b355d90 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.029+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55691b355d90 0x55691b358250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.029+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55691b355d90 0x55691b358250 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c5c569f3319edc server_cookie=ec495140d1c9844c in_seq=0 out_seq=0 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.029+0000 7ff7acff9640 10 client.26338 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.030+0000 7ff7acff9640 1 -- 172.21.15.94:0/1922349288 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55691b2ad7f0 con 0x55691b355d90 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.030+0000 7ff7acff9640 10 client.26338 handle_command_reply: tid=0 2024-09-19T04:26:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.030+0000 7ff7cd867740 1 client.26338 shutdown 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.030+0000 7ff7cd867740 1 -- 172.21.15.94:0/1922349288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 msgr2=0x55691b345780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.030+0000 7ff7cd867740 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b345780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009a30 tx=0x7ff7bc031f90 comp rx=0 tx=0).stop 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7cd867740 1 -- 172.21.15.94:0/1922349288 shutdown_connections 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7cd867740 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55691b355d90 0x55691b358250 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7cd867740 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b345d40 0x55691b343260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7cd867740 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b33b480 0x55691b342ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7cd867740 1 --2- 172.21.15.94:0/1922349288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b345780 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7cd867740 1 -- 172.21.15.94:0/1922349288 >> 172.21.15.94:0/1922349288 conn(0x55691b3370f0 msgr2=0x55691b33cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7acff9640 0 client.26338 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7cd867740 1 -- 172.21.15.94:0/1922349288 shutdown_connections 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7cd867740 1 -- 172.21.15.94:0/1922349288 wait complete. 2024-09-19T04:26:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.031+0000 7ff7cd867740 20 client.26338 trim_cache size 0 max 16384 2024-09-19T04:26:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.063+0000 7ff7cd867740 1 Processor -- start 2024-09-19T04:26:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.064+0000 7ff7cd867740 1 -- start start 2024-09-19T04:26:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.064+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33ab60 0x55691b35db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.064+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33b480 0x55691b35e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.064+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b33fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.064+0000 7ff7cd867740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41bf230 con 0x55691b345d40 2024-09-19T04:26:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.064+0000 7ff7cd867740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41bcad0 con 0x55691b33ab60 2024-09-19T04:26:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.064+0000 7ff7cd867740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41ba940 con 0x55691b33b480 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.064+0000 7ff7c9a2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b33fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c9a2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b33fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33702/0 (socket says 172.21.15.94:33702) 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c8a29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33b480 0x55691b35e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c9a2b640 1 -- 172.21.15.94:0/3429533081 learned_addr learned my addr 172.21.15.94:0/3429533081 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c922a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33ab60 0x55691b35db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3429533081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33ab60 msgr2=0x55691b35db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/3429533081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33ab60 0x55691b35db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3429533081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 msgr2=0x55691b33fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/3429533081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b33fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3429533081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691b3435d0 con 0x55691b33b480 2024-09-19T04:26:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.065+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/3429533081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33b480 0x55691b35e0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009380 tx=0x7ff7c00980d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcdf4f7a5e3c7e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.066+0000 7ff7aeffd640 1 -- 172.21.15.94:0/3429533081 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c009bf70 con 0x55691b33b480 2024-09-19T04:26:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.066+0000 7ff7aeffd640 1 -- 172.21.15.94:0/3429533081 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c009cb30 con 0x55691b33b480 2024-09-19T04:26:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.066+0000 7ff7aeffd640 1 -- 172.21.15.94:0/3429533081 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c009c200 con 0x55691b33b480 2024-09-19T04:26:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.067+0000 7ff7cd867740 1 -- 172.21.15.94:0/3429533081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33b480 msgr2=0x55691b35e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.067+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3429533081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33b480 0x55691b35e0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009380 tx=0x7ff7c00980d0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.067+0000 7ff7cd867740 1 -- 172.21.15.94:0/3429533081 shutdown_connections 2024-09-19T04:26:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.067+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3429533081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b33fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.067+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3429533081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33b480 0x55691b35e0a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.067+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3429533081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33ab60 0x55691b35db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.067+0000 7ff7cd867740 1 -- 172.21.15.94:0/3429533081 >> 172.21.15.94:0/3429533081 conn(0x55691b3370f0 msgr2=0x55691b3413a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.067+0000 7ff7cd867740 1 -- 172.21.15.94:0/3429533081 shutdown_connections 2024-09-19T04:26:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.067+0000 7ff7cd867740 1 -- 172.21.15.94:0/3429533081 wait complete. 2024-09-19T04:26:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.068+0000 7ff7cd867740 1 -- start start 2024-09-19T04:26:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.068+0000 7ff7cd867740 1 Processor -- start 2024-09-19T04:26:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b33fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b33ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7cd867740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b340480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7cd867740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41baf90 con 0x55691b345d40 2024-09-19T04:26:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7cd867740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41bc400 con 0x55691b33b480 2024-09-19T04:26:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7cd867740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41bb660 con 0x55691b33ab60 2024-09-19T04:26:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7c8a29640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b33ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7cd867740 10 client.0 _ll_register_callbacks cb 0x7ff7c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7c8a29640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b33ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:49790/0 (socket says 172.21.15.94:49790) 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3489894368 learned_addr learned my addr 172.21.15.94:0/3489894368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7c922a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b33fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7c9a2b640 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b340480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3489894368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 msgr2=0x55691b33fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b33fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3489894368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 msgr2=0x55691b340480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.069+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b340480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.070+0000 7ff7c8a29640 1 -- 172.21.15.94:0/3489894368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691b34c220 con 0x55691b33b480 2024-09-19T04:26:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.070+0000 7ff7c8a29640 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b33ff40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff7c009a6e0 tx=0x7ff7c00d0470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4886e5aa7e85f3c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.070+0000 7ff7acff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:26:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.070+0000 7ff7acff9640 1 -- 172.21.15.94:0/3489894368 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c009c200 con 0x55691b33b480 2024-09-19T04:26:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.070+0000 7ff7acff9640 1 -- 172.21.15.94:0/3489894368 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7ff7c0098aa0 con 0x55691b33b480 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.070+0000 7ff7cd867740 1 -- 172.21.15.94:0/3489894368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c009cb30 con 0x55691b33b480 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7acff9640 1 -- 172.21.15.94:0/3489894368 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff7c00986d0 con 0x55691b33b480 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7cd867740 10 client.26369 fetch_fsmap learned FSMap version 45 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7cd867740 1 -- 172.21.15.94:0/3489894368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55691b3c0da0 con 0x55691b33b480 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7acff9640 1 -- 172.21.15.94:0/3489894368 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c009c430 con 0x55691b33b480 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7acff9640 1 -- 172.21.15.94:0/3489894368 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7ff7c00abe80 con 0x55691b33b480 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7cd867740 10 client.26369 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7cd867740 10 client.26369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7cd867740 20 client.26369 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55691b3c1020 0x55691b3c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7cd867740 4 client.26369 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:26:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.071+0000 7ff7cd867740 1 -- 172.21.15.94:0/3489894368 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7ff7c009cb30 con 0x55691b3c1020 2024-09-19T04:26:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.072+0000 7ff7c922a640 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55691b3c1020 0x55691b3c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.072+0000 7ff7acff9640 10 client.26369 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.072+0000 7ff7c922a640 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55691b3c1020 0x55691b3c3410 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eedda04c7534e7a server_cookie=8b0cf9d7d59e698 in_seq=0 out_seq=0 2024-09-19T04:26:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.073+0000 7ff7acff9640 1 -- 172.21.15.94:0/3489894368 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7ff7c009cb30 con 0x55691b3c1020 2024-09-19T04:26:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.073+0000 7ff7acff9640 10 client.26369 handle_command_reply: tid=0 2024-09-19T04:26:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.073+0000 7ff7cd867740 1 client.26369 shutdown 2024-09-19T04:26:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.074+0000 7ff7cd867740 1 -- 172.21.15.94:0/3489894368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 msgr2=0x55691b33ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.074+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b33ff40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff7c009a6e0 tx=0x7ff7c00d0470 comp rx=0 tx=0).stop 2024-09-19T04:26:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.074+0000 7ff7cd867740 1 -- 172.21.15.94:0/3489894368 shutdown_connections 2024-09-19T04:26:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.074+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55691b3c1020 0x55691b3c3410 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.074+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55691b345d40 0x55691b340480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.074+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691b33b480 0x55691b33ff40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.074+0000 7ff7cd867740 1 --2- 172.21.15.94:0/3489894368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691b33ab60 0x55691b33fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.074+0000 7ff7cd867740 1 -- 172.21.15.94:0/3489894368 >> 172.21.15.94:0/3489894368 conn(0x55691b3370f0 msgr2=0x55691b335a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.074+0000 7ff7acff9640 0 client.26369 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.080+0000 7ff7cd867740 1 -- 172.21.15.94:0/3489894368 shutdown_connections 2024-09-19T04:26:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.080+0000 7ff7cd867740 1 -- 172.21.15.94:0/3489894368 wait complete. 2024-09-19T04:26:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.080+0000 7ff7cd867740 20 client.26369 trim_cache size 0 max 16384 2024-09-19T04:26:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:26:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:26:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "ed78b425-e7ce-4312-b731-60e43292f746", 2024-09-19T04:26:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:26:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:26:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.081+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7b00a3430 msgr2=0x7ff7b00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.081+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7b00a3430 0x7ff7b00a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009b90 tx=0x7ff7b40095d0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.081+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109830 msgr2=0x7ff7c41a76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.081+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109830 0x7ff7c41a76d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9f0 tx=0x7ff7c000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.082+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3783882717 shutdown_connections 2024-09-19T04:26:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.082+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7b00a3430 0x7ff7b00a58f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.082+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c411b140 0x7ff7c41aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.082+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c4109830 0x7ff7c41a76d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.082+0000 7ff7cb48e640 1 --2- 172.21.15.94:0/3783882717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c4109450 0x7ff7c41a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.082+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3783882717 >> 172.21.15.94:0/3783882717 conn(0x7ff7c406e140 msgr2=0x7ff7c411c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.082+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3783882717 shutdown_connections 2024-09-19T04:26:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:08.082+0000 7ff7cb48e640 1 -- 172.21.15.94:0/3783882717 wait complete. 2024-09-19T04:26:08.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:26:09.254 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: pgmap v345: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.336 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.336 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:09 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: pgmap v345: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-09-19T04:26:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:26:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:26:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:26:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:26:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:26:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:09 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: pgmap v345: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:09.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:09 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.958+0000 7f879abbb640 1 -- 172.21.15.94:0/906193904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8794100e20 msgr2=0x7f8794101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.958+0000 7f879abbb640 1 --2- 172.21.15.94:0/906193904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8794100e20 0x7f8794101200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f878c009cd0 tx=0x7f878c02c860 comp rx=0 tx=0).stop 2024-09-19T04:26:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.958+0000 7f879abbb640 1 -- 172.21.15.94:0/906193904 shutdown_connections 2024-09-19T04:26:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.958+0000 7f879abbb640 1 --2- 172.21.15.94:0/906193904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8794102ef0 0x7f879410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.958+0000 7f879abbb640 1 --2- 172.21.15.94:0/906193904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8794102570 0x7f87941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.958+0000 7f879abbb640 1 --2- 172.21.15.94:0/906193904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8794100e20 0x7f8794101200 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.958+0000 7f879abbb640 1 -- 172.21.15.94:0/906193904 >> 172.21.15.94:0/906193904 conn(0x7f87940fc7d0 msgr2=0x7f87940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:09.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.958+0000 7f879abbb640 1 -- 172.21.15.94:0/906193904 shutdown_connections 2024-09-19T04:26:09.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.958+0000 7f879abbb640 1 -- 172.21.15.94:0/906193904 wait complete. 2024-09-19T04:26:09.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.959+0000 7f879abbb640 1 Processor -- start 2024-09-19T04:26:09.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.960+0000 7f879abbb640 1 -- start start 2024-09-19T04:26:09.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.960+0000 7f879abbb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8794100e20 0x7f8794196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.960+0000 7f879abbb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8794102570 0x7f87941967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.961+0000 7f879abbb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8794102ef0 0x7f879419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.961+0000 7f879abbb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f879410fd30 con 0x7f8794102570 2024-09-19T04:26:09.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.961+0000 7f8793fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8794102570 0x7f87941967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.961+0000 7f879abbb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f879410fbb0 con 0x7f8794100e20 2024-09-19T04:26:09.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.961+0000 7f879abbb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f879410feb0 con 0x7f8794102ef0 2024-09-19T04:26:09.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.961+0000 7f8793fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8794102570 0x7f87941967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33728/0 (socket says 172.21.15.94:33728) 2024-09-19T04:26:09.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.961+0000 7f8793fff640 1 -- 172.21.15.94:0/2059739021 learned_addr learned my addr 172.21.15.94:0/2059739021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:09.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.961+0000 7f8798957640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8794100e20 0x7f8794196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.962+0000 7f8793fff640 1 -- 172.21.15.94:0/2059739021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8794102ef0 msgr2=0x7f879419ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.962+0000 7f8793fff640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8794102ef0 0x7f879419ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.962+0000 7f8793fff640 1 -- 172.21.15.94:0/2059739021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8794100e20 msgr2=0x7f8794196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.962+0000 7f8793fff640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8794100e20 0x7f8794196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.962+0000 7f8793fff640 1 -- 172.21.15.94:0/2059739021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879419b220 con 0x7f8794102570 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.962+0000 7f8793fff640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8794102570 0x7f87941967b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f878400c970 tx=0x7f878400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.963+0000 7f8791ffb640 1 -- 172.21.15.94:0/2059739021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784007bb0 con 0x7f8794102570 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.963+0000 7f8791ffb640 1 -- 172.21.15.94:0/2059739021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8784007d50 con 0x7f8794102570 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.963+0000 7f8791ffb640 1 -- 172.21.15.94:0/2059739021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878400f450 con 0x7f8794102570 2024-09-19T04:26:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.963+0000 7f879abbb640 1 -- 172.21.15.94:0/2059739021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879419b510 con 0x7f8794102570 2024-09-19T04:26:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.963+0000 7f879abbb640 1 -- 172.21.15.94:0/2059739021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87941a2df0 con 0x7f8794102570 2024-09-19T04:26:09.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.967+0000 7f879cf94740 1 Processor -- start 2024-09-19T04:26:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.968+0000 7f8791ffb640 1 -- 172.21.15.94:0/2059739021 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f8784004f00 con 0x7f8794102570 2024-09-19T04:26:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.969+0000 7f8791ffb640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f877c0a78c0 0x7f877c0a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.969+0000 7f8791ffb640 1 -- 172.21.15.94:0/2059739021 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f87840cf810 con 0x7f8794102570 2024-09-19T04:26:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.969+0000 7f879cf94740 1 -- start start 2024-09-19T04:26:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f879cf94740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 0x564fe83d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f879cf94740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f8799158640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f877c0a78c0 0x7f877c0a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f879cf94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f879cf94740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564fe833f2e0 con 0x564fe83d94a0 2024-09-19T04:26:09.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f879cf94740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564fe82e6ce0 con 0x564fe83d9dc0 2024-09-19T04:26:09.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f879cf94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564fe833b2a0 con 0x564fe83e4680 2024-09-19T04:26:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f8793fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f8793fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:49816/0 (socket says 172.21.15.94:49816) 2024-09-19T04:26:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.970+0000 7f8793fff640 1 -- 172.21.15.94:0/3605173953 learned_addr learned my addr 172.21.15.94:0/3605173953 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.971+0000 7f8798957640 1 --2- 172.21.15.94:0/3605173953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 0x564fe83d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.971+0000 7f8799158640 1 --2- 172.21.15.94:0/3605173953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.971+0000 7f8793fff640 1 -- 172.21.15.94:0/3605173953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 msgr2=0x564fe83e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.971+0000 7f8793fff640 1 --2- 172.21.15.94:0/3605173953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.971+0000 7f8793fff640 1 -- 172.21.15.94:0/3605173953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 msgr2=0x564fe83d9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.971+0000 7f8793fff640 1 --2- 172.21.15.94:0/3605173953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 0x564fe83d9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.971+0000 7f8793fff640 1 -- 172.21.15.94:0/3605173953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe82cb440 con 0x564fe83d9dc0 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.971+0000 7f8799158640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f877c0a78c0 0x7f877c0a9d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8788009e10 tx=0x7f87880127c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.971+0000 7f8793fff640 1 --2- 172.21.15.94:0/3605173953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e4140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f878400cfd0 tx=0x7f878409c1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6f8d1eeb7772ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.972+0000 7f877affd640 1 -- 172.21.15.94:0/3605173953 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878401d040 con 0x564fe83d9dc0 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.972+0000 7f877affd640 1 -- 172.21.15.94:0/3605173953 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87840a4690 con 0x564fe83d9dc0 2024-09-19T04:26:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.972+0000 7f877affd640 1 -- 172.21.15.94:0/3605173953 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87840051f0 con 0x564fe83d9dc0 2024-09-19T04:26:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.973+0000 7f879cf94740 1 -- 172.21.15.94:0/3605173953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 msgr2=0x564fe83e4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.973+0000 7f879cf94740 1 --2- 172.21.15.94:0/3605173953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e4140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f878400cfd0 tx=0x7f878409c1e0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.974+0000 7f879cf94740 1 -- 172.21.15.94:0/3605173953 shutdown_connections 2024-09-19T04:26:09.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.974+0000 7f879cf94740 1 --2- 172.21.15.94:0/3605173953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.974+0000 7f879cf94740 1 --2- 172.21.15.94:0/3605173953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e4140 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.974+0000 7f879cf94740 1 --2- 172.21.15.94:0/3605173953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 0x564fe83d9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.974+0000 7f879cf94740 1 -- 172.21.15.94:0/3605173953 >> 172.21.15.94:0/3605173953 conn(0x564fe83d5a30 msgr2=0x564fe83d5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:09.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.974+0000 7f879cf94740 1 -- 172.21.15.94:0/3605173953 shutdown_connections 2024-09-19T04:26:09.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.974+0000 7f879cf94740 1 -- 172.21.15.94:0/3605173953 wait complete. 2024-09-19T04:26:09.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.975+0000 7f879cf94740 1 -- start start 2024-09-19T04:26:09.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.975+0000 7f879cf94740 1 Processor -- start 2024-09-19T04:26:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.976+0000 7f879cf94740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 0x564fe83e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.976+0000 7f879cf94740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.976+0000 7f879cf94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.976+0000 7f879cf94740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564fe83e7250 con 0x564fe83d94a0 2024-09-19T04:26:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.976+0000 7f879cf94740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564fe82fa050 con 0x564fe83d9dc0 2024-09-19T04:26:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.976+0000 7f879cf94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564fe83e73d0 con 0x564fe83e4680 2024-09-19T04:26:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.976+0000 7f879cf94740 10 client.0 _ll_register_callbacks cb 0x564fe833a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.976+0000 7f8799158640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.976+0000 7f8799158640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44526/0 (socket says 172.21.15.94:44526) 2024-09-19T04:26:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.977+0000 7f8799158640 1 -- 172.21.15.94:0/126466629 learned_addr learned my addr 172.21.15.94:0/126466629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.977+0000 7f8793fff640 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.977+0000 7f8798957640 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 0x564fe83e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.977+0000 7f8799158640 1 -- 172.21.15.94:0/126466629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 msgr2=0x564fe83e1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.977+0000 7f8799158640 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.977+0000 7f8799158640 1 -- 172.21.15.94:0/126466629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 msgr2=0x564fe83e3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.977+0000 7f8799158640 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 0x564fe83e3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.977+0000 7f8799158640 1 -- 172.21.15.94:0/126466629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe80b03c0 con 0x564fe83e4680 2024-09-19T04:26:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.978+0000 7f8793fff640 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.979+0000 7f8799158640 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e1740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8788010820 tx=0x7f8788018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2387b930bd4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.979+0000 7f8773fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.979+0000 7f8773fff640 1 -- 172.21.15.94:0/126466629 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878800dac0 con 0x564fe83e4680 2024-09-19T04:26:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.979+0000 7f8773fff640 1 -- 172.21.15.94:0/126466629 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f878800dc60 con 0x564fe83e4680 2024-09-19T04:26:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.980+0000 7f879cf94740 1 -- 172.21.15.94:0/126466629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87840a4690 con 0x564fe83e4680 2024-09-19T04:26:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.980+0000 7f8773fff640 1 -- 172.21.15.94:0/126466629 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f878802e050 con 0x564fe83e4680 2024-09-19T04:26:09.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.980+0000 7f879cf94740 10 client.26353 fetch_fsmap learned FSMap version 46 2024-09-19T04:26:09.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.980+0000 7f879cf94740 1 -- 172.21.15.94:0/126466629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564fe83f2bf0 con 0x564fe83e4680 2024-09-19T04:26:09.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.981+0000 7f8773fff640 1 -- 172.21.15.94:0/126466629 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87840a4690 con 0x564fe83e4680 2024-09-19T04:26:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.981+0000 7f8773fff640 1 -- 172.21.15.94:0/126466629 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 46) ==== 8362+0+0 (secure 0 0 0) 0x7f8788004b10 con 0x564fe83e4680 2024-09-19T04:26:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.981+0000 7f879cf94740 10 client.26353 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T04:26:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.981+0000 7f879cf94740 10 client.26353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.981+0000 7f879cf94740 20 client.26353 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:09.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.981+0000 7f879cf94740 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564fe83f40b0 0x564fe83f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.982+0000 7f879cf94740 4 client.26353 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:09.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.982+0000 7f879cf94740 1 -- 172.21.15.94:0/126466629 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fe834c850 con 0x564fe83f40b0 2024-09-19T04:26:09.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.982+0000 7f8798957640 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564fe83f40b0 0x564fe83f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.983+0000 7f8798957640 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564fe83f40b0 0x564fe83f6510 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ecee0bac4d6855 server_cookie=d9e13922bb5add19 in_seq=0 out_seq=0 2024-09-19T04:26:09.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.983+0000 7f8773fff640 10 client.26353 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:09.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f8773fff640 1 -- 172.21.15.94:0/126466629 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fe834c850 con 0x564fe83f40b0 2024-09-19T04:26:09.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f8773fff640 10 client.26353 handle_command_reply: tid=0 2024-09-19T04:26:09.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f879cf94740 1 client.26353 shutdown 2024-09-19T04:26:09.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f879cf94740 1 -- 172.21.15.94:0/126466629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 msgr2=0x564fe83e1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f879cf94740 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e1740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8788010820 tx=0x7f8788018000 comp rx=0 tx=0).stop 2024-09-19T04:26:09.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f879cf94740 1 -- 172.21.15.94:0/126466629 shutdown_connections 2024-09-19T04:26:09.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f879cf94740 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564fe83f40b0 0x564fe83f6510 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f879cf94740 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83e4680 0x564fe83e1740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f879cf94740 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.984+0000 7f879cf94740 1 --2- 172.21.15.94:0/126466629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83d94a0 0x564fe83e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.985+0000 7f879cf94740 1 -- 172.21.15.94:0/126466629 >> 172.21.15.94:0/126466629 conn(0x564fe83d5a30 msgr2=0x564fe83db760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:09.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.985+0000 7f8773fff640 0 client.26353 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:09.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.985+0000 7f879cf94740 1 -- 172.21.15.94:0/126466629 shutdown_connections 2024-09-19T04:26:09.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.985+0000 7f879cf94740 1 -- 172.21.15.94:0/126466629 wait complete. 2024-09-19T04:26:09.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:09.985+0000 7f879cf94740 20 client.26353 trim_cache size 0 max 16384 2024-09-19T04:26:10.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.015+0000 7f879cf94740 1 Processor -- start 2024-09-19T04:26:10.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f879cf94740 1 -- start start 2024-09-19T04:26:10.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f879cf94740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d94a0 0x564fe83d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f879cf94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d9dc0 0x564fe83fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f879cf94740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 0x564fe83fc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f8798957640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d94a0 0x564fe83d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f8798957640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d94a0 0x564fe83d4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:49830/0 (socket says 172.21.15.94:49830) 2024-09-19T04:26:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f8793fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d9dc0 0x564fe83fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f8793fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d9dc0 0x564fe83fbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44534/0 (socket says 172.21.15.94:44534) 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f8798957640 1 -- 172.21.15.94:0/3925147133 learned_addr learned my addr 172.21.15.94:0/3925147133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f879cf94740 1 -- 172.21.15.94:0/3925147133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87941ae0a0 con 0x564fe83e4680 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f879cf94740 1 -- 172.21.15.94:0/3925147133 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87941ab910 con 0x564fe83d94a0 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.016+0000 7f879cf94740 1 -- 172.21.15.94:0/3925147133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87941a9780 con 0x564fe83d9dc0 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.017+0000 7f8799158640 1 --2- 172.21.15.94:0/3925147133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 0x564fe83fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.017+0000 7f8798957640 1 -- 172.21.15.94:0/3925147133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d9dc0 msgr2=0x564fe83fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.017+0000 7f8798957640 1 --2- 172.21.15.94:0/3925147133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d9dc0 0x564fe83fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.017+0000 7f8798957640 1 -- 172.21.15.94:0/3925147133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 msgr2=0x564fe83fc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.017+0000 7f8798957640 1 --2- 172.21.15.94:0/3925147133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 0x564fe83fc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.017+0000 7f8798957640 1 -- 172.21.15.94:0/3925147133 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe83d5860 con 0x564fe83d94a0 2024-09-19T04:26:10.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.017+0000 7f8798957640 1 --2- 172.21.15.94:0/3925147133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d94a0 0x564fe83d4b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f878c000f80 tx=0x7f878c009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6280a4ba4da645c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:10.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.018+0000 7f877a7fc640 1 -- 172.21.15.94:0/3925147133 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c039040 con 0x564fe83d94a0 2024-09-19T04:26:10.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.018+0000 7f877a7fc640 1 -- 172.21.15.94:0/3925147133 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f878c034c70 con 0x564fe83d94a0 2024-09-19T04:26:10.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.018+0000 7f877a7fc640 1 -- 172.21.15.94:0/3925147133 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c003d60 con 0x564fe83d94a0 2024-09-19T04:26:10.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.019+0000 7f879cf94740 1 -- 172.21.15.94:0/3925147133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d94a0 msgr2=0x564fe83d4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.019+0000 7f879cf94740 1 --2- 172.21.15.94:0/3925147133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d94a0 0x564fe83d4b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f878c000f80 tx=0x7f878c009360 comp rx=0 tx=0).stop 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.019+0000 7f879cf94740 1 -- 172.21.15.94:0/3925147133 shutdown_connections 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.019+0000 7f879cf94740 1 --2- 172.21.15.94:0/3925147133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 0x564fe83fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.019+0000 7f879cf94740 1 --2- 172.21.15.94:0/3925147133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d9dc0 0x564fe83fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.019+0000 7f879cf94740 1 --2- 172.21.15.94:0/3925147133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d94a0 0x564fe83d4b70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.019+0000 7f879cf94740 1 -- 172.21.15.94:0/3925147133 >> 172.21.15.94:0/3925147133 conn(0x564fe83d5a30 msgr2=0x564fe83e3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.019+0000 7f879cf94740 1 -- 172.21.15.94:0/3925147133 shutdown_connections 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.019+0000 7f879cf94740 1 -- 172.21.15.94:0/3925147133 wait complete. 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.020+0000 7f879cf94740 1 -- start start 2024-09-19T04:26:10.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.020+0000 7f879cf94740 1 Processor -- start 2024-09-19T04:26:10.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f879cf94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d94a0 0x564fe83ecc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:10.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f879cf94740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:10.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f879cf94740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 0x564fe83ea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:10.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f8798957640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d94a0 0x564fe83ecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:10.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f8798957640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d94a0 0x564fe83ecc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44538/0 (socket says 172.21.15.94:44538) 2024-09-19T04:26:10.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f8798957640 1 -- 172.21.15.94:0/1153891757 learned_addr learned my addr 172.21.15.94:0/1153891757 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:10.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87941a9dd0 con 0x564fe83e4680 2024-09-19T04:26:10.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87941ab240 con 0x564fe83d9dc0 2024-09-19T04:26:10.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87941aa4a0 con 0x564fe83d94a0 2024-09-19T04:26:10.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f8799158640 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 0x564fe83ea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:10.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.021+0000 7f879cf94740 10 client.0 _ll_register_callbacks cb 0x7f8788010730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:10.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.022+0000 7f8799158640 1 -- 172.21.15.94:0/1153891757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d94a0 msgr2=0x564fe83ecc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:10.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.022+0000 7f8799158640 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d94a0 0x564fe83ecc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.022+0000 7f8799158640 1 -- 172.21.15.94:0/1153891757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 msgr2=0x564fe83e9fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:26:10.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.022+0000 7f8799158640 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e9fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.022+0000 7f8799158640 1 -- 172.21.15.94:0/1153891757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe83eabd0 con 0x564fe83e4680 2024-09-19T04:26:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.022+0000 7f8799158640 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 0x564fe83ea4e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f878801e490 tx=0x7f878801ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d5b988b5bad3e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.023+0000 7f8773fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:26:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.023+0000 7f8773fff640 1 -- 172.21.15.94:0/1153891757 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788009b20 con 0x564fe83e4680 2024-09-19T04:26:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.023+0000 7f8773fff640 1 -- 172.21.15.94:0/1153891757 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f878800d7c0 con 0x564fe83e4680 2024-09-19T04:26:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.023+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878c034c70 con 0x564fe83e4680 2024-09-19T04:26:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.024+0000 7f8773fff640 1 -- 172.21.15.94:0/1153891757 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8788041050 con 0x564fe83e4680 2024-09-19T04:26:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.024+0000 7f879cf94740 10 client.16770 fetch_fsmap learned FSMap version 46 2024-09-19T04:26:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.024+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564fe845ec90 con 0x564fe83e4680 2024-09-19T04:26:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.024+0000 7f8773fff640 1 -- 172.21.15.94:0/1153891757 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8788004680 con 0x564fe83e4680 2024-09-19T04:26:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.024+0000 7f8773fff640 1 -- 172.21.15.94:0/1153891757 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 46) ==== 8362+0+0 (secure 0 0 0) 0x7f8788005000 con 0x564fe83e4680 2024-09-19T04:26:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.025+0000 7f879cf94740 10 client.16770 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T04:26:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.025+0000 7f879cf94740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.025+0000 7f879cf94740 20 client.16770 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.025+0000 7f879cf94740 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564fe845f380 0x564fe84617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.025+0000 7f879cf94740 4 client.16770 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:26:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.025+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f878c034c70 con 0x564fe845f380 2024-09-19T04:26:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.025+0000 7f8798957640 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564fe845f380 0x564fe84617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.026+0000 7f8798957640 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564fe845f380 0x564fe84617a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd5f2949b00e5f server_cookie=289e3fecb876752 in_seq=0 out_seq=0 2024-09-19T04:26:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.026+0000 7f8773fff640 10 client.16770 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.026+0000 7f8773fff640 1 -- 172.21.15.94:0/1153891757 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f878c034c70 con 0x564fe845f380 2024-09-19T04:26:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.026+0000 7f8773fff640 10 client.16770 handle_command_reply: tid=0 2024-09-19T04:26:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.026+0000 7f879cf94740 1 client.16770 shutdown 2024-09-19T04:26:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.028+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 msgr2=0x564fe83ea4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.028+0000 7f879cf94740 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 0x564fe83ea4e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f878801e490 tx=0x7f878801ebe0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.028+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 shutdown_connections 2024-09-19T04:26:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.028+0000 7f879cf94740 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564fe845f380 0x564fe84617a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.028+0000 7f879cf94740 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fe83e4680 0x564fe83ea4e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.028+0000 7f879cf94740 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fe83d9dc0 0x564fe83e9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.028+0000 7f879cf94740 1 --2- 172.21.15.94:0/1153891757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fe83d94a0 0x564fe83ecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.028+0000 7f8773fff640 0 client.16770 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.028+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 >> 172.21.15.94:0/1153891757 conn(0x564fe83d5a30 msgr2=0x564fe83d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.029+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 shutdown_connections 2024-09-19T04:26:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.029+0000 7f879cf94740 1 -- 172.21.15.94:0/1153891757 wait complete. 2024-09-19T04:26:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.029+0000 7f879cf94740 20 client.16770 trim_cache size 0 max 16384 2024-09-19T04:26:10.031 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout: "ed78b425-e7ce-4312-b731-60e43292f746": { 2024-09-19T04:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "ed78b425-e7ce-4312-b731-60e43292f746", 2024-09-19T04:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:26:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.031+0000 7f879abbb640 1 -- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f877c0a78c0 msgr2=0x7f877c0a9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.031+0000 7f879abbb640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f877c0a78c0 0x7f877c0a9d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8788009e10 tx=0x7f87880127c0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.031+0000 7f879abbb640 1 -- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8794102570 msgr2=0x7f87941967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.031+0000 7f879abbb640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8794102570 0x7f87941967b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f878400c970 tx=0x7f878400ce30 comp rx=0 tx=0).stop 2024-09-19T04:26:10.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.033+0000 7f879abbb640 1 -- 172.21.15.94:0/2059739021 shutdown_connections 2024-09-19T04:26:10.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.033+0000 7f879abbb640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f877c0a78c0 0x7f877c0a9d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.033+0000 7f879abbb640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8794102ef0 0x7f879419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.033+0000 7f879abbb640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8794102570 0x7f87941967b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.033+0000 7f879abbb640 1 --2- 172.21.15.94:0/2059739021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8794100e20 0x7f8794196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.033+0000 7f879abbb640 1 -- 172.21.15.94:0/2059739021 >> 172.21.15.94:0/2059739021 conn(0x7f87940fc7d0 msgr2=0x7f87940fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.033+0000 7f879abbb640 1 -- 172.21.15.94:0/2059739021 shutdown_connections 2024-09-19T04:26:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:10.033+0000 7f879abbb640 1 -- 172.21.15.94:0/2059739021 wait complete. 2024-09-19T04:26:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 595.0 s, 576.2 tps, lat 206.624 ms stddev 601.837 2024-09-19T04:26:11.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:11 smithi094 ceph-mon[30972]: pgmap v346: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 21 MiB/s wr, 1.34k op/s 2024-09-19T04:26:11.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:11 smithi094 ceph-mon[30972]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:26:11.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:11 smithi104 ceph-mon[34145]: pgmap v346: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 21 MiB/s wr, 1.34k op/s 2024-09-19T04:26:11.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:11 smithi104 ceph-mon[34145]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:26:11.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:11 smithi112 ceph-mon[35051]: pgmap v346: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 21 MiB/s wr, 1.34k op/s 2024-09-19T04:26:11.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:11 smithi112 ceph-mon[35051]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:26:12.391 INFO:tasks.cephfs.filesystem:scrub status for tag:ed78b425-e7ce-4312-b731-60e43292f746 - {'path': '/', 'tag': 'ed78b425-e7ce-4312-b731-60e43292f746', 'options': 'recursive,force'} 2024-09-19T04:26:13.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:13 smithi104 ceph-mon[34145]: pgmap v348: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 23 MiB/s wr, 1.45k op/s 2024-09-19T04:26:13.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:13 smithi112 ceph-mon[35051]: pgmap v348: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 23 MiB/s wr, 1.45k op/s 2024-09-19T04:26:13.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:13 smithi094 ceph-mon[30972]: pgmap v348: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 23 MiB/s wr, 1.45k op/s 2024-09-19T04:26:14.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:14 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:14.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:14 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:14.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:14 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 600.0 s, 875.4 tps, lat 105.693 ms stddev 144.270 2024-09-19T04:26:15.685 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:15 smithi112 ceph-mon[35051]: pgmap v349: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 19 MiB/s wr, 1.22k op/s 2024-09-19T04:26:15.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:15 smithi094 ceph-mon[30972]: pgmap v349: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 19 MiB/s wr, 1.22k op/s 2024-09-19T04:26:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:15 smithi104 ceph-mon[34145]: pgmap v349: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 19 MiB/s wr, 1.22k op/s 2024-09-19T04:26:17.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:17 smithi112 ceph-mon[35051]: pgmap v350: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 19 MiB/s wr, 1.22k op/s 2024-09-19T04:26:17.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:17 smithi094 ceph-mon[30972]: pgmap v350: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 19 MiB/s wr, 1.22k op/s 2024-09-19T04:26:17.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:17 smithi104 ceph-mon[34145]: pgmap v350: 97 pgs: 97 active+clean; 4.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 19 MiB/s wr, 1.22k op/s 2024-09-19T04:26:19.655 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:19 smithi104 ceph-mon[34145]: pgmap v351: 97 pgs: 97 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-09-19T04:26:19.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:19 smithi112 ceph-mon[35051]: pgmap v351: 97 pgs: 97 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-09-19T04:26:19.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:19 smithi094 ceph-mon[30972]: pgmap v351: 97 pgs: 97 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-09-19T04:26:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 605.0 s, 787.2 tps, lat 83.455 ms stddev 99.665 2024-09-19T04:26:21.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:21 smithi112 ceph-mon[35051]: pgmap v352: 97 pgs: 97 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-09-19T04:26:21.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:21 smithi094 ceph-mon[30972]: pgmap v352: 97 pgs: 97 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-09-19T04:26:21.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:21 smithi104 ceph-mon[34145]: pgmap v352: 97 pgs: 97 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-09-19T04:26:22.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:22 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:26:22.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:22 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:26:22.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:22 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:26:22.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:22 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:26:22.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:22 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:26:22.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:22 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:26:22.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:22 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:26:22.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:22 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:26:22.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:22 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:26:23.741 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:23 smithi094 ceph-mon[30972]: pgmap v353: 97 pgs: 97 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-19T04:26:23.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:23 smithi112 ceph-mon[35051]: pgmap v353: 97 pgs: 97 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-19T04:26:23.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:23 smithi104 ceph-mon[34145]: pgmap v353: 97 pgs: 97 active+clean; 4.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-19T04:26:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 610.0 s, 677.0 tps, lat 212.861 ms stddev 757.234 2024-09-19T04:26:25.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:25 smithi112 ceph-mon[35051]: pgmap v354: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 24 MiB/s wr, 1.66k op/s 2024-09-19T04:26:25.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:25 smithi094 ceph-mon[30972]: pgmap v354: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 24 MiB/s wr, 1.66k op/s 2024-09-19T04:26:25.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:25 smithi104 ceph-mon[34145]: pgmap v354: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 24 MiB/s wr, 1.66k op/s 2024-09-19T04:26:27.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:27 smithi112 ceph-mon[35051]: pgmap v355: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-19T04:26:27.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:27 smithi094 ceph-mon[30972]: pgmap v355: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-19T04:26:27.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:27 smithi104 ceph-mon[34145]: pgmap v355: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-19T04:26:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:28 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:26:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:28 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:26:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:28 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:26:28.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:28 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:26:28.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:28 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:26:28.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:28 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:26:28.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:28 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:26:28.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:28 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:26:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:28 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:26:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:28 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:26:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:28 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:26:28.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:28 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:26:29.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:29 smithi112 ceph-mon[35051]: pgmap v356: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 20 MiB/s wr, 1.53k op/s 2024-09-19T04:26:29.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:29 smithi094 ceph-mon[30972]: pgmap v356: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 20 MiB/s wr, 1.53k op/s 2024-09-19T04:26:29.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:29 smithi104 ceph-mon[34145]: pgmap v356: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 20 MiB/s wr, 1.53k op/s 2024-09-19T04:26:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 615.0 s, 1192.8 tps, lat 83.741 ms stddev 75.328 2024-09-19T04:26:31.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:31 smithi112 ceph-mon[35051]: pgmap v357: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-09-19T04:26:31.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:31 smithi094 ceph-mon[30972]: pgmap v357: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-09-19T04:26:31.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:31 smithi104 ceph-mon[34145]: pgmap v357: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-09-19T04:26:33.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:33 smithi112 ceph-mon[35051]: pgmap v358: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-19T04:26:33.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:33 smithi094 ceph-mon[30972]: pgmap v358: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-19T04:26:33.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:33 smithi104 ceph-mon[34145]: pgmap v358: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-19T04:26:34.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:34 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:34.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:34 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:34.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:34 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:26:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 620.0 s, 667.6 tps, lat 141.348 ms stddev 124.046 2024-09-19T04:26:35.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:35 smithi112 ceph-mon[35051]: pgmap v359: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 19 MiB/s wr, 1.64k op/s 2024-09-19T04:26:35.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:35 smithi094 ceph-mon[30972]: pgmap v359: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 19 MiB/s wr, 1.64k op/s 2024-09-19T04:26:35.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:35 smithi104 ceph-mon[34145]: pgmap v359: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 19 MiB/s wr, 1.64k op/s 2024-09-19T04:26:37.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:37 smithi112 ceph-mon[35051]: pgmap v360: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-09-19T04:26:37.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:37 smithi094 ceph-mon[30972]: pgmap v360: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-09-19T04:26:37.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:37 smithi104 ceph-mon[34145]: pgmap v360: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-09-19T04:26:39.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:39 smithi112 ceph-mon[35051]: pgmap v361: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 13 MiB/s wr, 1.25k op/s 2024-09-19T04:26:39.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:39 smithi094 ceph-mon[30972]: pgmap v361: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 13 MiB/s wr, 1.25k op/s 2024-09-19T04:26:39.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:39 smithi104 ceph-mon[34145]: pgmap v361: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 13 MiB/s wr, 1.25k op/s 2024-09-19T04:26:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 625.0 s, 330.8 tps, lat 313.027 ms stddev 82.342 2024-09-19T04:26:41.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:41 smithi112 ceph-mon[35051]: pgmap v362: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-19T04:26:41.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:41 smithi094 ceph-mon[30972]: pgmap v362: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-19T04:26:41.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:41 smithi104 ceph-mon[34145]: pgmap v362: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-19T04:26:42.393 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:26:42.791 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:43.734 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:43 smithi094 ceph-mon[30972]: pgmap v363: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 877 op/s 2024-09-19T04:26:43.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:43 smithi112 ceph-mon[35051]: pgmap v363: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 877 op/s 2024-09-19T04:26:43.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.851+0000 7f12194b3640 1 -- 172.21.15.94:0/2633103719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214106f20 msgr2=0x7f121410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:43.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.851+0000 7f12194b3640 1 --2- 172.21.15.94:0/2633103719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214106f20 0x7f121410eaa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f120c009be0 tx=0x7f120c02a6c0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.851+0000 7f12194b3640 1 -- 172.21.15.94:0/2633103719 shutdown_connections 2024-09-19T04:26:43.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.851+0000 7f12194b3640 1 --2- 172.21.15.94:0/2633103719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214106f20 0x7f121410eaa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.852+0000 7f12194b3640 1 --2- 172.21.15.94:0/2633103719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12141027a0 0x7f12141067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.852+0000 7f12194b3640 1 --2- 172.21.15.94:0/2633103719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1214101df0 0x7f12141021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.852+0000 7f12194b3640 1 -- 172.21.15.94:0/2633103719 >> 172.21.15.94:0/2633103719 conn(0x7f12140fd8f0 msgr2=0x7f12140ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:43.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.852+0000 7f12194b3640 1 -- 172.21.15.94:0/2633103719 shutdown_connections 2024-09-19T04:26:43.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.852+0000 7f12194b3640 1 -- 172.21.15.94:0/2633103719 wait complete. 2024-09-19T04:26:43.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:43 smithi104 ceph-mon[34145]: pgmap v363: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 877 op/s 2024-09-19T04:26:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.852+0000 7f12194b3640 1 Processor -- start 2024-09-19T04:26:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12194b3640 1 -- start start 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12194b3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1214101df0 0x7f121419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12194b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12141027a0 0x7f121419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12194b3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1214106f20 0x7f121419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12194b3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1214113ff0 con 0x7f1214106f20 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12194b3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1214113e70 con 0x7f1214101df0 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12194b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1214114170 con 0x7f12141027a0 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12127fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12141027a0 0x7f121419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12127fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12141027a0 0x7f121419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:43956/0 (socket says 172.21.15.94:43956) 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12127fc640 1 -- 172.21.15.94:0/3604976465 learned_addr learned my addr 172.21.15.94:0/3604976465 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1214101df0 0x7f121419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.853+0000 7f12137fe640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1214106f20 0x7f121419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f1212ffd640 1 -- 172.21.15.94:0/3604976465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12141027a0 msgr2=0x7f121419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12141027a0 0x7f121419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f1212ffd640 1 -- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1214106f20 msgr2=0x7f121419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1214106f20 0x7f121419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f1212ffd640 1 -- 172.21.15.94:0/3604976465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121419f360 con 0x7f1214101df0 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f12127fc640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12141027a0 0x7f121419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1214101df0 0x7f121419a3b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1204005fd0 tx=0x7f120400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f11fbfff640 1 -- 172.21.15.94:0/3604976465 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120400cd00 con 0x7f1214101df0 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f11fbfff640 1 -- 172.21.15.94:0/3604976465 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12040044d0 con 0x7f1214101df0 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f11fbfff640 1 -- 172.21.15.94:0/3604976465 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204010430 con 0x7f1214101df0 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f12194b3640 1 -- 172.21.15.94:0/3604976465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121419f650 con 0x7f1214101df0 2024-09-19T04:26:43.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.854+0000 7f12194b3640 1 -- 172.21.15.94:0/3604976465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141a6f30 con 0x7f1214101df0 2024-09-19T04:26:43.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.857+0000 7f11fbfff640 1 -- 172.21.15.94:0/3604976465 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1204004000 con 0x7f1214101df0 2024-09-19T04:26:43.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.858+0000 7f121b88c740 1 Processor -- start 2024-09-19T04:26:43.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.858+0000 7f121b88c740 1 -- start start 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f121b88c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a370880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f121b88c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a370dc0 0x563b1a37b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f121b88c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a37b680 0x563b1a37da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f11fbfff640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11fc0a3480 0x7f11fc0a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f121b88c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563b1a2d62e0 con 0x563b1a3704a0 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f121b88c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563b1a27dce0 con 0x563b1a37b680 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f121b88c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563b1a2d22a0 con 0x563b1a370dc0 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a370dc0 0x563b1a37b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a370dc0 0x563b1a37b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:43970/0 (socket says 172.21.15.94:43970) 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f11fbfff640 1 -- 172.21.15.94:0/3604976465 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f12040cfee0 con 0x7f1214101df0 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f12137fe640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11fc0a3480 0x7f11fc0a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f12127fc640 1 --2- 172.21.15.94:0/4029821889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a37b680 0x563b1a37da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.859+0000 7f1212ffd640 1 -- 172.21.15.94:0/4029821889 learned_addr learned my addr 172.21.15.94:0/4029821889 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f1212ffd640 1 -- 172.21.15.94:0/4029821889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a37b680 msgr2=0x563b1a37da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f1212ffd640 1 --2- 172.21.15.94:0/4029821889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a37b680 0x563b1a37da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f1212ffd640 1 -- 172.21.15.94:0/4029821889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 msgr2=0x563b1a370880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f1212ffd640 1 --2- 172.21.15.94:0/4029821889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a370880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f1212ffd640 1 -- 172.21.15.94:0/4029821889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1a262440 con 0x563b1a370dc0 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f12127fc640 1 --2- 172.21.15.94:0/4029821889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a37b680 0x563b1a37da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f1212ffd640 1 --2- 172.21.15.94:0/4029821889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a370dc0 0x563b1a37b140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f120400c6a0 tx=0x7f1204005790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d333ac4d7e1ee008 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f11f97fa640 1 -- 172.21.15.94:0/4029821889 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040b4020 con 0x563b1a370dc0 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f12137fe640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11fc0a3480 0x7f11fc0a5940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f120c009560 tx=0x7f120c005bd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f11f97fa640 1 -- 172.21.15.94:0/4029821889 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f120409c880 con 0x563b1a370dc0 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.860+0000 7f11f97fa640 1 -- 172.21.15.94:0/4029821889 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120409c180 con 0x563b1a370dc0 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.861+0000 7f121b88c740 1 -- 172.21.15.94:0/4029821889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a370dc0 msgr2=0x563b1a37b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.861+0000 7f121b88c740 1 --2- 172.21.15.94:0/4029821889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a370dc0 0x563b1a37b140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f120400c6a0 tx=0x7f1204005790 comp rx=0 tx=0).stop 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.861+0000 7f121b88c740 1 -- 172.21.15.94:0/4029821889 shutdown_connections 2024-09-19T04:26:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.861+0000 7f121b88c740 1 --2- 172.21.15.94:0/4029821889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a37b680 0x563b1a37da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.861+0000 7f121b88c740 1 --2- 172.21.15.94:0/4029821889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a370dc0 0x563b1a37b140 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.861+0000 7f121b88c740 1 --2- 172.21.15.94:0/4029821889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a370880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.861+0000 7f121b88c740 1 -- 172.21.15.94:0/4029821889 >> 172.21.15.94:0/4029821889 conn(0x563b1a36ca30 msgr2=0x563b1a36ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.861+0000 7f121b88c740 1 -- 172.21.15.94:0/4029821889 shutdown_connections 2024-09-19T04:26:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.861+0000 7f121b88c740 1 -- 172.21.15.94:0/4029821889 wait complete. 2024-09-19T04:26:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f121b88c740 1 -- start start 2024-09-19T04:26:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f121b88c740 1 Processor -- start 2024-09-19T04:26:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f121b88c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a37b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f121b88c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a3782c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f121b88c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 0x563b1a378840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f121b88c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563b1a37e250 con 0x563b1a3704a0 2024-09-19T04:26:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f121b88c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563b1a291050 con 0x563b1a370dc0 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f121b88c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563b1a37e3d0 con 0x563b1a37b680 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f121b88c740 10 client.0 _ll_register_callbacks cb 0x563b1a2d1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.862+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a37b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a37b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60606/0 (socket says 172.21.15.94:60606) 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f1212ffd640 1 -- 172.21.15.94:0/3777807192 learned_addr learned my addr 172.21.15.94:0/3777807192 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f12127fc640 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a3782c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f12137fe640 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 0x563b1a378840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f1212ffd640 1 -- 172.21.15.94:0/3777807192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 msgr2=0x563b1a378840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 0x563b1a378840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f1212ffd640 1 -- 172.21.15.94:0/3777807192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 msgr2=0x563b1a3782c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a3782c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f1212ffd640 1 -- 172.21.15.94:0/3777807192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1a0473c0 con 0x563b1a3704a0 2024-09-19T04:26:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f12127fc640 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a3782c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a37b010 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f12040cdbf0 tx=0x7f120409bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4a7e8eea4c868e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.863+0000 7f11f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.864+0000 7f11f6ff5640 1 -- 172.21.15.94:0/3777807192 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040b4070 con 0x563b1a3704a0 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.864+0000 7f11f6ff5640 1 -- 172.21.15.94:0/3777807192 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f12040a0170 con 0x563b1a3704a0 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.864+0000 7f121b88c740 1 -- 172.21.15.94:0/3777807192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120409c880 con 0x563b1a3704a0 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.864+0000 7f11f6ff5640 1 -- 172.21.15.94:0/3777807192 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f12040042f0 con 0x563b1a3704a0 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.864+0000 7f121b88c740 10 client.16779 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.864+0000 7f121b88c740 1 -- 172.21.15.94:0/3777807192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563b1a392450 con 0x563b1a3704a0 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.864+0000 7f11f6ff5640 1 -- 172.21.15.94:0/3777807192 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f120409c880 con 0x563b1a3704a0 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f11f6ff5640 1 -- 172.21.15.94:0/3777807192 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f12040850d0 con 0x563b1a3704a0 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f121b88c740 10 client.16779 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f121b88c740 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f121b88c740 20 client.16779 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f121b88c740 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563b1a3937a0 0x563b1a395b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f121b88c740 4 client.16779 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f121b88c740 1 -- 172.21.15.94:0/3777807192 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b1a2e3850 con 0x563b1a3937a0 2024-09-19T04:26:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f12127fc640 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563b1a3937a0 0x563b1a395b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f12127fc640 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563b1a3937a0 0x563b1a395b90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16fbfa1595cfa3f1 server_cookie=e945a6b0927241ef in_seq=0 out_seq=0 2024-09-19T04:26:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.865+0000 7f11f6ff5640 10 client.16779 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:43.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.866+0000 7f11f6ff5640 1 -- 172.21.15.94:0/3777807192 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b1a2e3850 con 0x563b1a3937a0 2024-09-19T04:26:43.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.866+0000 7f11f6ff5640 10 client.16779 handle_command_reply: tid=0 2024-09-19T04:26:43.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.866+0000 7f121b88c740 1 client.16779 shutdown 2024-09-19T04:26:43.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 -- 172.21.15.94:0/3777807192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 msgr2=0x563b1a37b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:43.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a37b010 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f12040cdbf0 tx=0x7f120409bdf0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 -- 172.21.15.94:0/3777807192 shutdown_connections 2024-09-19T04:26:43.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563b1a3937a0 0x563b1a395b90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f11f6ff5640 0 client.16779 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:43.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 0x563b1a378840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a3782c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 --2- 172.21.15.94:0/3777807192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a37b010 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 -- 172.21.15.94:0/3777807192 >> 172.21.15.94:0/3777807192 conn(0x563b1a36ca30 msgr2=0x563b1a36bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:43.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 -- 172.21.15.94:0/3777807192 shutdown_connections 2024-09-19T04:26:43.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 1 -- 172.21.15.94:0/3777807192 wait complete. 2024-09-19T04:26:43.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.867+0000 7f121b88c740 20 client.16779 trim_cache size 0 max 16384 2024-09-19T04:26:43.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f121b88c740 1 Processor -- start 2024-09-19T04:26:43.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f121b88c740 1 -- start start 2024-09-19T04:26:43.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f121b88c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a378290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f121b88c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a39b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f121b88c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 0x563b1a39b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f121b88c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f12141ae050 con 0x563b1a3704a0 2024-09-19T04:26:43.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f121b88c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f12141ab860 con 0x563b1a370dc0 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f121b88c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12141a9710 con 0x563b1a37b680 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a378290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a378290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60610/0 (socket says 172.21.15.94:60610) 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f1212ffd640 1 -- 172.21.15.94:0/3348705770 learned_addr learned my addr 172.21.15.94:0/3348705770 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f12127fc640 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a39b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f12137fe640 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 0x563b1a39b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f1212ffd640 1 -- 172.21.15.94:0/3348705770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 msgr2=0x563b1a39b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 0x563b1a39b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f1212ffd640 1 -- 172.21.15.94:0/3348705770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 msgr2=0x563b1a39b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a39b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f1212ffd640 1 -- 172.21.15.94:0/3348705770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1a370bf0 con 0x563b1a3704a0 2024-09-19T04:26:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.888+0000 7f12137fe640 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 0x563b1a39b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.889+0000 7f1212ffd640 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a378290 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f120409be80 tx=0x7f1204003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c23f01d2643f68d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.889+0000 7f11f8ff9640 1 -- 172.21.15.94:0/3348705770 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040b4050 con 0x563b1a3704a0 2024-09-19T04:26:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.889+0000 7f11f8ff9640 1 -- 172.21.15.94:0/3348705770 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f120409c180 con 0x563b1a3704a0 2024-09-19T04:26:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.889+0000 7f11f8ff9640 1 -- 172.21.15.94:0/3348705770 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204009510 con 0x563b1a3704a0 2024-09-19T04:26:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 -- 172.21.15.94:0/3348705770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 msgr2=0x563b1a378290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a378290 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f120409be80 tx=0x7f1204003a50 comp rx=0 tx=0).stop 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 -- 172.21.15.94:0/3348705770 shutdown_connections 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a37b680 0x563b1a39b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a39b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 --2- 172.21.15.94:0/3348705770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a3704a0 0x563b1a378290 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 -- 172.21.15.94:0/3348705770 >> 172.21.15.94:0/3348705770 conn(0x563b1a36ca30 msgr2=0x563b1a377b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 -- 172.21.15.94:0/3348705770 shutdown_connections 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 -- 172.21.15.94:0/3348705770 wait complete. 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 -- start start 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.890+0000 7f121b88c740 1 Processor -- start 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f121b88c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a3704a0 0x563b1a37eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f121b88c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a37f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f121b88c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a37b680 0x563b1a3fc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f121b88c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f12141a9d60 con 0x563b1a37b680 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f121b88c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f12141ab190 con 0x563b1a370dc0 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f121b88c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12141aa3f0 con 0x563b1a3704a0 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f121b88c740 10 client.0 _ll_register_callbacks cb 0x7f1208009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a3704a0 0x563b1a37eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a3704a0 0x563b1a37eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:43984/0 (socket says 172.21.15.94:43984) 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f1212ffd640 1 -- 172.21.15.94:0/1596836416 learned_addr learned my addr 172.21.15.94:0/1596836416 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f12137fe640 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a37b680 0x563b1a3fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f1212ffd640 1 -- 172.21.15.94:0/1596836416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 msgr2=0x563b1a37f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f1212ffd640 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a37f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f1212ffd640 1 -- 172.21.15.94:0/1596836416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a37b680 msgr2=0x563b1a3fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f1212ffd640 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a37b680 0x563b1a3fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f1212ffd640 1 -- 172.21.15.94:0/1596836416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1a3fc6c0 con 0x563b1a3704a0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.891+0000 7f12137fe640 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a37b680 0x563b1a3fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.892+0000 7f1212ffd640 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a3704a0 0x563b1a37eb80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1204098b50 tx=0x7f1204003ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0f912a23c8b9f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.892+0000 7f11f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.892+0000 7f11f6ff5640 1 -- 172.21.15.94:0/1596836416 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040b4070 con 0x563b1a3704a0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.892+0000 7f11f6ff5640 1 -- 172.21.15.94:0/1596836416 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f12040a4960 con 0x563b1a3704a0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.892+0000 7f121b88c740 1 -- 172.21.15.94:0/1596836416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b1a2ddd60 con 0x563b1a3704a0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.892+0000 7f11f6ff5640 1 -- 172.21.15.94:0/1596836416 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f12040bc250 con 0x563b1a3704a0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.892+0000 7f121b88c740 10 client.26377 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.892+0000 7f121b88c740 1 -- 172.21.15.94:0/1596836416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563b1a3fc6c0 con 0x563b1a3704a0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.892+0000 7f11f6ff5640 1 -- 172.21.15.94:0/1596836416 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563b1a2ddd60 con 0x563b1a3704a0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f11f6ff5640 1 -- 172.21.15.94:0/1596836416 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f120409c3e0 con 0x563b1a3704a0 2024-09-19T04:26:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f121b88c740 10 client.26377 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f121b88c740 10 client.26377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f121b88c740 20 client.26377 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f121b88c740 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563b1a3fdbf0 0x563b1a400000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f121b88c740 4 client.26377 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f121b88c740 1 -- 172.21.15.94:0/1596836416 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b1a25a210 con 0x563b1a3fdbf0 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f12127fc640 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563b1a3fdbf0 0x563b1a400000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f12127fc640 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563b1a3fdbf0 0x563b1a400000 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515e76fadc322f9e server_cookie=7ca85e1230d070f6 in_seq=0 out_seq=0 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.893+0000 7f11f6ff5640 10 client.26377 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f11f6ff5640 1 -- 172.21.15.94:0/1596836416 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b1a25a210 con 0x563b1a3fdbf0 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f11f6ff5640 10 client.26377 handle_command_reply: tid=0 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 client.26377 shutdown 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 -- 172.21.15.94:0/1596836416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a3704a0 msgr2=0x563b1a37eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a3704a0 0x563b1a37eb80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1204098b50 tx=0x7f1204003ac0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 -- 172.21.15.94:0/1596836416 shutdown_connections 2024-09-19T04:26:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563b1a3fdbf0 0x563b1a400000 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b1a37b680 0x563b1a3fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b1a370dc0 0x563b1a37f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 --2- 172.21.15.94:0/1596836416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1a3704a0 0x563b1a37eb80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f11f6ff5640 0 client.26377 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 -- 172.21.15.94:0/1596836416 >> 172.21.15.94:0/1596836416 conn(0x563b1a36ca30 msgr2=0x563b1a36b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 -- 172.21.15.94:0/1596836416 shutdown_connections 2024-09-19T04:26:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 1 -- 172.21.15.94:0/1596836416 wait complete. 2024-09-19T04:26:43.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f121b88c740 20 client.26377 trim_cache size 0 max 16384 2024-09-19T04:26:43.901 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:26:43.901 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:26:43.901 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:26:43.901 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f12194b3640 1 -- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11fc0a3480 msgr2=0x7f11fc0a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f12194b3640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11fc0a3480 0x7f11fc0a5940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f120c009560 tx=0x7f120c005bd0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f12194b3640 1 -- 172.21.15.94:0/3604976465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1214101df0 msgr2=0x7f121419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.894+0000 7f12194b3640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1214101df0 0x7f121419a3b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1204005fd0 tx=0x7f120400ee50 comp rx=0 tx=0).stop 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.895+0000 7f12194b3640 1 -- 172.21.15.94:0/3604976465 shutdown_connections 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.895+0000 7f12194b3640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f11fc0a3480 0x7f11fc0a5940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.895+0000 7f12194b3640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1214106f20 0x7f121419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.895+0000 7f12194b3640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12141027a0 0x7f121419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.895+0000 7f12194b3640 1 --2- 172.21.15.94:0/3604976465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1214101df0 0x7f121419a3b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.895+0000 7f12194b3640 1 -- 172.21.15.94:0/3604976465 >> 172.21.15.94:0/3604976465 conn(0x7f12140fd8f0 msgr2=0x7f12140fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.895+0000 7f12194b3640 1 -- 172.21.15.94:0/3604976465 shutdown_connections 2024-09-19T04:26:43.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:43.895+0000 7f12194b3640 1 -- 172.21.15.94:0/3604976465 wait complete. 2024-09-19T04:26:45.423 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:26:45.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:26:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 630.0 s, 310.6 tps, lat 322.294 ms stddev 86.764 2024-09-19T04:26:45.743 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:45 smithi094 ceph-mon[30972]: pgmap v364: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-09-19T04:26:45.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:45 smithi112 ceph-mon[35051]: pgmap v364: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-09-19T04:26:45.806 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:45.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:45 smithi104 ceph-mon[34145]: pgmap v364: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-09-19T04:26:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.360+0000 7f2791a25640 1 -- 172.21.15.94:0/2108978576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 msgr2=0x7f278c100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.360+0000 7f2791a25640 1 --2- 172.21.15.94:0/2108978576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 0x7f278c100ca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f277c009930 tx=0x7f277c02a440 comp rx=0 tx=0).stop 2024-09-19T04:26:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.360+0000 7f2791a25640 1 -- 172.21.15.94:0/2108978576 shutdown_connections 2024-09-19T04:26:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.360+0000 7f2791a25640 1 --2- 172.21.15.94:0/2108978576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f278c105ec0 0x7f278c106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.360+0000 7f2791a25640 1 --2- 172.21.15.94:0/2108978576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f278c1011e0 0x7f278c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.360+0000 7f2791a25640 1 --2- 172.21.15.94:0/2108978576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 0x7f278c100ca0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.360+0000 7f2791a25640 1 -- 172.21.15.94:0/2108978576 >> 172.21.15.94:0/2108978576 conn(0x7f278c0fc7d0 msgr2=0x7f278c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.360+0000 7f2791a25640 1 -- 172.21.15.94:0/2108978576 shutdown_connections 2024-09-19T04:26:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.361+0000 7f2791a25640 1 -- 172.21.15.94:0/2108978576 wait complete. 2024-09-19T04:26:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.362+0000 7f2791a25640 1 Processor -- start 2024-09-19T04:26:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.363+0000 7f2791a25640 1 -- start start 2024-09-19T04:26:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.363+0000 7f2791a25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 0x7f278c0738a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.364+0000 7f2791a25640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f278c1011e0 0x7f278c073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.364+0000 7f278affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 0x7f278c0738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.364+0000 7f278affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 0x7f278c0738a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44002/0 (socket says 172.21.15.94:44002) 2024-09-19T04:26:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.364+0000 7f278affd640 1 -- 172.21.15.94:0/330898041 learned_addr learned my addr 172.21.15.94:0/330898041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.364+0000 7f2791a25640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f278c105ec0 0x7f278c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.364+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f278c118510 con 0x7f278c105ec0 2024-09-19T04:26:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.364+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f278c118390 con 0x7f278c1011e0 2024-09-19T04:26:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.364+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f278c118690 con 0x7f278c1008c0 2024-09-19T04:26:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.364+0000 7f278a7fc640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f278c1011e0 0x7f278c073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.365+0000 7f278affd640 1 -- 172.21.15.94:0/330898041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f278c1011e0 msgr2=0x7f278c073de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.365+0000 7f278affd640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f278c1011e0 0x7f278c073de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.365+0000 7f278affd640 1 -- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f278c105ec0 msgr2=0x7f278c06ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.365+0000 7f278affd640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f278c105ec0 0x7f278c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.365+0000 7f278affd640 1 -- 172.21.15.94:0/330898041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c06d310 con 0x7f278c1008c0 2024-09-19T04:26:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.365+0000 7f278affd640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 0x7f278c0738a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f277c02a960 tx=0x7f277c030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.365+0000 7f2790a23640 1 -- 172.21.15.94:0/330898041 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c03f070 con 0x7f278c1008c0 2024-09-19T04:26:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.365+0000 7f2790a23640 1 -- 172.21.15.94:0/330898041 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f277c004460 con 0x7f278c1008c0 2024-09-19T04:26:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.365+0000 7f2790a23640 1 -- 172.21.15.94:0/330898041 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c035030 con 0x7f278c1008c0 2024-09-19T04:26:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.366+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c06d5a0 con 0x7f278c1008c0 2024-09-19T04:26:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.366+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c0715a0 con 0x7f278c1008c0 2024-09-19T04:26:46.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.367+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278c06a850 con 0x7f278c1008c0 2024-09-19T04:26:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.369+0000 7f2790a23640 1 -- 172.21.15.94:0/330898041 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f277c042070 con 0x7f278c1008c0 2024-09-19T04:26:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.370+0000 7f2790a23640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f27740a3580 0x7f27740a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:46.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.370+0000 7f278a7fc640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f27740a3580 0x7f27740a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:46.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.371+0000 7f2790a23640 1 -- 172.21.15.94:0/330898041 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f277c0f06a0 con 0x7f278c1008c0 2024-09-19T04:26:46.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.371+0000 7f278a7fc640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f27740a3580 0x7f27740a5a40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f278c06e0f0 tx=0x7f2780005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:46.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.374+0000 7f2790a23640 1 -- 172.21.15.94:0/330898041 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f277c0bd060 con 0x7f278c1008c0 2024-09-19T04:26:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.673+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f278c06d770 con 0x7f278c1008c0 2024-09-19T04:26:46.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.673+0000 7f2790a23640 1 -- 172.21.15.94:0/330898041 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11924 (secure 0 0 0) 0x7f277c0bc960 con 0x7f278c1008c0 2024-09-19T04:26:46.675 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:26:46.676 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":48,"btime":"2024-09-19T04:26:33:395724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:26:32.789785+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:26:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 48 2024-09-19T04:26:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f27740a3580 msgr2=0x7f27740a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f27740a3580 0x7f27740a5a40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f278c06e0f0 tx=0x7f2780005f90 comp rx=0 tx=0).stop 2024-09-19T04:26:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 msgr2=0x7f278c0738a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 0x7f278c0738a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f277c02a960 tx=0x7f277c030650 comp rx=0 tx=0).stop 2024-09-19T04:26:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 shutdown_connections 2024-09-19T04:26:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f27740a3580 0x7f27740a5a40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f278c105ec0 0x7f278c06ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f278c1011e0 0x7f278c073de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 --2- 172.21.15.94:0/330898041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c1008c0 0x7f278c0738a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 >> 172.21.15.94:0/330898041 conn(0x7f278c0fc7d0 msgr2=0x7f278c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.676+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 shutdown_connections 2024-09-19T04:26:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:46.677+0000 7f2791a25640 1 -- 172.21.15.94:0/330898041 wait complete. 2024-09-19T04:26:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:47 smithi094 ceph-mon[30972]: pgmap v365: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 515 op/s 2024-09-19T04:26:47.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:47 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/330898041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:47.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:26:47.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:47 smithi112 ceph-mon[35051]: pgmap v365: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 515 op/s 2024-09-19T04:26:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:47 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/330898041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:47 smithi104 ceph-mon[34145]: pgmap v365: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 515 op/s 2024-09-19T04:26:47.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:47 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/330898041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:48.087 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.604+0000 7ff3d760e640 1 -- 172.21.15.94:0/1732325847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3d01036e0 msgr2=0x7ff3d0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.604+0000 7ff3d760e640 1 --2- 172.21.15.94:0/1732325847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3d01036e0 0x7ff3d0107b90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009930 tx=0x7ff3c002a420 comp rx=0 tx=0).stop 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.604+0000 7ff3d760e640 1 -- 172.21.15.94:0/1732325847 shutdown_connections 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.604+0000 7ff3d760e640 1 --2- 172.21.15.94:0/1732325847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3d01082a0 0x7ff3d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.604+0000 7ff3d760e640 1 --2- 172.21.15.94:0/1732325847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3d01036e0 0x7ff3d0107b90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.604+0000 7ff3d760e640 1 --2- 172.21.15.94:0/1732325847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3d0102d30 0x7ff3d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.604+0000 7ff3d760e640 1 -- 172.21.15.94:0/1732325847 >> 172.21.15.94:0/1732325847 conn(0x7ff3d00fc7d0 msgr2=0x7ff3d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.604+0000 7ff3d760e640 1 -- 172.21.15.94:0/1732325847 shutdown_connections 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.604+0000 7ff3d760e640 1 -- 172.21.15.94:0/1732325847 wait complete. 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d760e640 1 Processor -- start 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d760e640 1 -- start start 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d760e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3d0102d30 0x7ff3d0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d760e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3d01036e0 0x7ff3d0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d760e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3d01082a0 0x7ff3d019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d760e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0111f10 con 0x7ff3d01082a0 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d760e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0111d90 con 0x7ff3d01036e0 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d760e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0112090 con 0x7ff3d0102d30 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d53aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3d0102d30 0x7ff3d0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d53aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3d0102d30 0x7ff3d0198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44030/0 (socket says 172.21.15.94:44030) 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d53aa640 1 -- 172.21.15.94:0/2032270403 learned_addr learned my addr 172.21.15.94:0/2032270403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d5bab640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3d01082a0 0x7ff3d019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.605+0000 7ff3d4ba9640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3d01036e0 0x7ff3d0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3d53aa640 1 -- 172.21.15.94:0/2032270403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3d01036e0 msgr2=0x7ff3d0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3d01036e0 0x7ff3d0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3d53aa640 1 -- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3d01082a0 msgr2=0x7ff3d019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3d01082a0 0x7ff3d019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3d53aa640 1 -- 172.21.15.94:0/2032270403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d019d4f0 con 0x7ff3d0102d30 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3d0102d30 0x7ff3d0198540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00e9f0 tx=0x7ff3cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3c67fc640 1 -- 172.21.15.94:0/2032270403 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00cd80 con 0x7ff3d0102d30 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3c67fc640 1 -- 172.21.15.94:0/2032270403 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3cc0108e0 con 0x7ff3d0102d30 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3c67fc640 1 -- 172.21.15.94:0/2032270403 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc018920 con 0x7ff3d0102d30 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3d760e640 1 -- 172.21.15.94:0/2032270403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d019d7e0 con 0x7ff3d0102d30 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.606+0000 7ff3d760e640 1 -- 172.21.15.94:0/2032270403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01008c0 con 0x7ff3d0102d30 2024-09-19T04:26:48.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.609+0000 7ff3c67fc640 1 -- 172.21.15.94:0/2032270403 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff3cc010a90 con 0x7ff3d0102d30 2024-09-19T04:26:48.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.610+0000 7ff3d99e7740 1 Processor -- start 2024-09-19T04:26:48.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.610+0000 7ff3d99e7740 1 -- start start 2024-09-19T04:26:48.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.610+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 0x559cff4db090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.610+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 0x559cff4dba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.610+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.610+0000 7ff3d99e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559cff4412f0 con 0x559cff4eeba0 2024-09-19T04:26:48.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.610+0000 7ff3d99e7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559cff3e8cf0 con 0x559cff4db5d0 2024-09-19T04:26:48.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.610+0000 7ff3d99e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559cff43d2b0 con 0x559cff4dcb50 2024-09-19T04:26:48.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.610+0000 7ff3c67fc640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff3bc0a3430 0x7ff3bc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d53aa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 0x559cff4dba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d53aa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 0x559cff4dba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56130/0 (socket says 172.21.15.94:56130) 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d53aa640 1 -- 172.21.15.94:0/4122198534 learned_addr learned my addr 172.21.15.94:0/4122198534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3c67fc640 1 -- 172.21.15.94:0/2032270403 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7ff3cc0cffe0 con 0x7ff3d0102d30 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d53aa640 1 -- 172.21.15.94:0/4122198534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 msgr2=0x559cff4db090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/4122198534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 0x559cff4db090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d53aa640 1 -- 172.21.15.94:0/4122198534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 msgr2=0x559cff4f0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/4122198534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d53aa640 1 -- 172.21.15.94:0/4122198534 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cff3cd3a0 con 0x559cff4db5d0 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d5bab640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff3bc0a3430 0x7ff3bc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/4122198534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 0x559cff4dba30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc09c2c0 tx=0x7ff3cc0917c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22be2a335ac5430 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.611+0000 7ff3bb7fe640 1 -- 172.21.15.94:0/4122198534 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc0a0630 con 0x559cff4db5d0 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.612+0000 7ff3bb7fe640 1 -- 172.21.15.94:0/4122198534 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3cc0a4ef0 con 0x559cff4db5d0 2024-09-19T04:26:48.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.612+0000 7ff3bb7fe640 1 -- 172.21.15.94:0/4122198534 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc09c7f0 con 0x559cff4db5d0 2024-09-19T04:26:48.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.612+0000 7ff3d99e7740 1 -- 172.21.15.94:0/4122198534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 msgr2=0x559cff4dba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/4122198534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 0x559cff4dba30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc09c2c0 tx=0x7ff3cc0917c0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d5bab640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff3bc0a3430 0x7ff3bc0a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8005fd0 tx=0x7ff3c8009c00 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 -- 172.21.15.94:0/4122198534 shutdown_connections 2024-09-19T04:26:48.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/4122198534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/4122198534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 0x559cff4dba30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/4122198534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 0x559cff4db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 -- 172.21.15.94:0/4122198534 >> 172.21.15.94:0/4122198534 conn(0x559cff4a4310 msgr2=0x559cff4d7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 -- 172.21.15.94:0/4122198534 shutdown_connections 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 -- 172.21.15.94:0/4122198534 wait complete. 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 -- start start 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.613+0000 7ff3d99e7740 1 Processor -- start 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 0x559cff4eb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 0x559cff4ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d99e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559cff4f1870 con 0x559cff4eeba0 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d99e7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559cff3fc060 con 0x559cff4db5d0 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d99e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559cff4f19f0 con 0x559cff4dcb50 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d99e7740 10 client.0 _ll_register_callbacks cb 0x559cff14f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d5bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d4ba9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 0x559cff4ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d5bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4ec3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60678/0 (socket says 172.21.15.94:60678) 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d5bab640 1 -- 172.21.15.94:0/264795659 learned_addr learned my addr 172.21.15.94:0/264795659 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d4ba9640 1 -- 172.21.15.94:0/264795659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 msgr2=0x559cff4eb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d4ba9640 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 0x559cff4eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d4ba9640 1 -- 172.21.15.94:0/264795659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 msgr2=0x559cff4ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.614+0000 7ff3d4ba9640 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3d4ba9640 1 -- 172.21.15.94:0/264795659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cff1b23c0 con 0x559cff4dcb50 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3d4ba9640 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 0x559cff4ebe60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff3c002a940 tx=0x7ff3c0032d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c6c69ccd181966 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/264795659 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0040070 con 0x559cff4dcb50 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3d99e7740 1 -- 172.21.15.94:0/264795659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3cc0a4ef0 con 0x559cff4dcb50 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/264795659 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7ff3c0004040 con 0x559cff4dcb50 2024-09-19T04:26:48.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/264795659 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff3c0035040 con 0x559cff4dcb50 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3d99e7740 10 client.26392 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3d99e7740 1 -- 172.21.15.94:0/264795659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x559cff4fd450 con 0x559cff4dcb50 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.615+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/264795659 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3cc0a4ef0 con 0x559cff4dcb50 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/264795659 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7ff3c0004770 con 0x559cff4dcb50 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3d99e7740 10 client.26392 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3d99e7740 10 client.26392 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3d99e7740 20 client.26392 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x559cff4fec50 0x559cff501110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3d99e7740 4 client.26392 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3d99e7740 1 -- 172.21.15.94:0/264795659 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cff44e860 con 0x559cff4fec50 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x559cff4fec50 0x559cff501110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3b8ff9640 10 client.26392 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:26:48.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.616+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x559cff4fec50 0x559cff501110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=886280f054766cd0 server_cookie=e6109586fd530c26 in_seq=0 out_seq=0 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.617+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/264795659 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cff44e860 con 0x559cff4fec50 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.617+0000 7ff3b8ff9640 10 client.26392 handle_command_reply: tid=0 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.617+0000 7ff3d99e7740 1 client.26392 shutdown 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.617+0000 7ff3d99e7740 1 -- 172.21.15.94:0/264795659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 msgr2=0x559cff4ebe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 0x559cff4ebe60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff3c002a940 tx=0x7ff3c0032d30 comp rx=0 tx=0).stop 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 1 -- 172.21.15.94:0/264795659 shutdown_connections 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x559cff4fec50 0x559cff501110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4dcb50 0x559cff4ebe60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/264795659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4db5d0 0x559cff4eb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 1 -- 172.21.15.94:0/264795659 >> 172.21.15.94:0/264795659 conn(0x559cff4a4310 msgr2=0x559cff4dd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3b8ff9640 0 client.26392 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:26:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 1 -- 172.21.15.94:0/264795659 shutdown_connections 2024-09-19T04:26:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 1 -- 172.21.15.94:0/264795659 wait complete. 2024-09-19T04:26:48.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.618+0000 7ff3d99e7740 20 client.26392 trim_cache size 0 max 16384 2024-09-19T04:26:48.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.635+0000 7ff3d99e7740 1 Processor -- start 2024-09-19T04:26:48.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.635+0000 7ff3d99e7740 1 -- start start 2024-09-19T04:26:48.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff506a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4dcb50 0x559cff506f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4eeba0 0x559cff4d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d53aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff506a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d53aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff506a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44046/0 (socket says 172.21.15.94:44046) 2024-09-19T04:26:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d4ba9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4dcb50 0x559cff506f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d99e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01ae100 con 0x559cff4dcb50 2024-09-19T04:26:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d4ba9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4dcb50 0x559cff506f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60684/0 (socket says 172.21.15.94:60684) 2024-09-19T04:26:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d99e7740 1 -- 172.21.15.94:0/1028506865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01ab910 con 0x559cff4eeba0 2024-09-19T04:26:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d4ba9640 1 -- 172.21.15.94:0/1028506865 learned_addr learned my addr 172.21.15.94:0/1028506865 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.636+0000 7ff3d99e7740 1 -- 172.21.15.94:0/1028506865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01a9780 con 0x559cff4db5d0 2024-09-19T04:26:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.637+0000 7ff3d53aa640 1 -- 172.21.15.94:0/1028506865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4eeba0 msgr2=0x559cff4d64e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:26:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.637+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/1028506865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4eeba0 0x559cff4d64e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.637+0000 7ff3d53aa640 1 -- 172.21.15.94:0/1028506865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4dcb50 msgr2=0x559cff506f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.637+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/1028506865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4dcb50 0x559cff506f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.637+0000 7ff3d53aa640 1 -- 172.21.15.94:0/1028506865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cff4dc000 con 0x559cff4db5d0 2024-09-19T04:26:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.637+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/1028506865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff506a20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0097b0 tx=0x7ff3cc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7455f3291c48046d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.637+0000 7ff3baffd640 1 -- 172.21.15.94:0/1028506865 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc0b5270 con 0x559cff4db5d0 2024-09-19T04:26:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.637+0000 7ff3baffd640 1 -- 172.21.15.94:0/1028506865 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3cc09a630 con 0x559cff4db5d0 2024-09-19T04:26:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.637+0000 7ff3baffd640 1 -- 172.21.15.94:0/1028506865 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc0a4a60 con 0x559cff4db5d0 2024-09-19T04:26:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.638+0000 7ff3d99e7740 1 -- 172.21.15.94:0/1028506865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 msgr2=0x559cff506a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.638+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/1028506865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff506a20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0097b0 tx=0x7ff3cc000bc0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.639+0000 7ff3d99e7740 1 -- 172.21.15.94:0/1028506865 shutdown_connections 2024-09-19T04:26:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.639+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/1028506865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4eeba0 0x559cff4d64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.639+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/1028506865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4dcb50 0x559cff506f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.639+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/1028506865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff506a20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.639+0000 7ff3d99e7740 1 -- 172.21.15.94:0/1028506865 >> 172.21.15.94:0/1028506865 conn(0x559cff4a4310 msgr2=0x559cff4eaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.641+0000 7ff3d99e7740 1 -- 172.21.15.94:0/1028506865 shutdown_connections 2024-09-19T04:26:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.641+0000 7ff3d99e7740 1 -- 172.21.15.94:0/1028506865 wait complete. 2024-09-19T04:26:48.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.642+0000 7ff3d99e7740 1 -- start start 2024-09-19T04:26:48.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.642+0000 7ff3d99e7740 1 Processor -- start 2024-09-19T04:26:48.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff4f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4dcb50 0x559cff4f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d99e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d53aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff4f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d53aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff4f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44054/0 (socket says 172.21.15.94:44054) 2024-09-19T04:26:48.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d53aa640 1 -- 172.21.15.94:0/38025175 learned_addr learned my addr 172.21.15.94:0/38025175 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:48.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01a9dd0 con 0x559cff4eeba0 2024-09-19T04:26:48.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d5bab640 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01ab240 con 0x559cff4dcb50 2024-09-19T04:26:48.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01aa4a0 con 0x559cff4db5d0 2024-09-19T04:26:48.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d99e7740 10 client.0 _ll_register_callbacks cb 0x7ff3c80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:48.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.643+0000 7ff3d4ba9640 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4dcb50 0x559cff4f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.644+0000 7ff3d53aa640 1 -- 172.21.15.94:0/38025175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4dcb50 msgr2=0x559cff4f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.644+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4dcb50 0x559cff4f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.644+0000 7ff3d53aa640 1 -- 172.21.15.94:0/38025175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 msgr2=0x559cff4f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.644+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.644+0000 7ff3d53aa640 1 -- 172.21.15.94:0/38025175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cff4f3480 con 0x559cff4db5d0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.644+0000 7ff3d4ba9640 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4dcb50 0x559cff4f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.644+0000 7ff3d53aa640 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff4f73e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0bc010 tx=0x7ff3cc0b5240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2b70d0834979068 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.644+0000 7ff3b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.644+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/38025175 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc09c590 con 0x559cff4db5d0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.645+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/38025175 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7ff3cc09c730 con 0x559cff4db5d0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.645+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3cc09a630 con 0x559cff4db5d0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.645+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/38025175 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff3cc0a4a60 con 0x559cff4db5d0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.645+0000 7ff3d99e7740 10 client.26398 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.645+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x559cff5699e0 con 0x559cff4db5d0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.645+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/38025175 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3cc09a630 con 0x559cff4db5d0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.646+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/38025175 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7ff3cc0bc0b0 con 0x559cff4db5d0 2024-09-19T04:26:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.646+0000 7ff3d99e7740 10 client.26398 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:48.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.646+0000 7ff3d99e7740 10 client.26398 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:26:48.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.646+0000 7ff3d99e7740 20 client.26398 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:48.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.646+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x559cff569bb0 0x559cff56bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.646+0000 7ff3d99e7740 4 client.26398 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:48.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.646+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cff448d70 con 0x559cff569bb0 2024-09-19T04:26:48.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.646+0000 7ff3d4ba9640 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x559cff569bb0 0x559cff56bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.647+0000 7ff3d4ba9640 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x559cff569bb0 0x559cff56bfa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=771afdba3da589d4 server_cookie=2dcf3ca3c9f025b2 in_seq=0 out_seq=0 2024-09-19T04:26:48.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.647+0000 7ff3b8ff9640 10 client.26398 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:26:48.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.647+0000 7ff3b8ff9640 1 -- 172.21.15.94:0/38025175 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cff448d70 con 0x559cff569bb0 2024-09-19T04:26:48.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.647+0000 7ff3b8ff9640 10 client.26398 handle_command_reply: tid=0 2024-09-19T04:26:48.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.647+0000 7ff3d99e7740 1 client.26398 shutdown 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.647+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 msgr2=0x559cff4f73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.647+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff4f73e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0bc010 tx=0x7ff3cc0b5240 comp rx=0 tx=0).stop 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 shutdown_connections 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x559cff569bb0 0x559cff56bfa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cff4eeba0 0x559cff4f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559cff4dcb50 0x559cff4f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3d99e7740 1 --2- 172.21.15.94:0/38025175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559cff4db5d0 0x559cff4f73e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 >> 172.21.15.94:0/38025175 conn(0x559cff4a4310 msgr2=0x559cff4ed180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3b8ff9640 0 client.26398 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 shutdown_connections 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3d99e7740 1 -- 172.21.15.94:0/38025175 wait complete. 2024-09-19T04:26:48.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.648+0000 7ff3d99e7740 20 client.26398 trim_cache size 0 max 16384 2024-09-19T04:26:48.652 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 -- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff3bc0a3430 msgr2=0x7ff3bc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff3bc0a3430 0x7ff3bc0a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8005fd0 tx=0x7ff3c8009c00 comp rx=0 tx=0).stop 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 -- 172.21.15.94:0/2032270403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3d0102d30 msgr2=0x7ff3d0198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3d0102d30 0x7ff3d0198540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00e9f0 tx=0x7ff3cc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 -- 172.21.15.94:0/2032270403 shutdown_connections 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff3bc0a3430 0x7ff3bc0a58f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3d01082a0 0x7ff3d019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3d01036e0 0x7ff3d0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 --2- 172.21.15.94:0/2032270403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3d0102d30 0x7ff3d0198540 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.649+0000 7ff3d760e640 1 -- 172.21.15.94:0/2032270403 >> 172.21.15.94:0/2032270403 conn(0x7ff3d00fc7d0 msgr2=0x7ff3d0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.650+0000 7ff3d760e640 1 -- 172.21.15.94:0/2032270403 shutdown_connections 2024-09-19T04:26:48.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:48.650+0000 7ff3d760e640 1 -- 172.21.15.94:0/2032270403 wait complete. 2024-09-19T04:26:49.469 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:26:49.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:49 smithi112 ceph-mon[35051]: pgmap v366: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 685 op/s 2024-09-19T04:26:49.823 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:49.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:49 smithi094 ceph-mon[30972]: pgmap v366: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 685 op/s 2024-09-19T04:26:49.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:49 smithi104 ceph-mon[34145]: pgmap v366: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 685 op/s 2024-09-19T04:26:50.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.366+0000 7ff30a23b640 1 -- 172.21.15.94:0/4238444878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102ef0 msgr2=0x7ff30410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.366+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4238444878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102ef0 0x7ff30410da40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009ca0 tx=0x7ff2fc02a950 comp rx=0 tx=0).stop 2024-09-19T04:26:50.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.367+0000 7ff30a23b640 1 -- 172.21.15.94:0/4238444878 shutdown_connections 2024-09-19T04:26:50.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.367+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4238444878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102ef0 0x7ff30410da40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.367+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4238444878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff304102570 0x7ff3041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.367+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4238444878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff304100e20 0x7ff304101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.367+0000 7ff30a23b640 1 -- 172.21.15.94:0/4238444878 >> 172.21.15.94:0/4238444878 conn(0x7ff3040fc7d0 msgr2=0x7ff3040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.368+0000 7ff30a23b640 1 -- 172.21.15.94:0/4238444878 shutdown_connections 2024-09-19T04:26:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.368+0000 7ff30a23b640 1 -- 172.21.15.94:0/4238444878 wait complete. 2024-09-19T04:26:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.368+0000 7ff30a23b640 1 Processor -- start 2024-09-19T04:26:50.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.369+0000 7ff30a23b640 1 -- start start 2024-09-19T04:26:50.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff30a23b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff304100e20 0x7ff30410d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff30a23b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102570 0x7ff30410a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff30a23b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff304102ef0 0x7ff30410aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff30a23b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff304114180 con 0x7ff304102570 2024-09-19T04:26:50.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff30a23b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff304114000 con 0x7ff304102ef0 2024-09-19T04:26:50.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff30a23b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff304114300 con 0x7ff304100e20 2024-09-19T04:26:50.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff302ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102570 0x7ff30410a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff302ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102570 0x7ff30410a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60712/0 (socket says 172.21.15.94:60712) 2024-09-19T04:26:50.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff302ffd640 1 -- 172.21.15.94:0/4136258010 learned_addr learned my addr 172.21.15.94:0/4136258010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:50.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff304100e20 0x7ff30410d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff303fff640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff304102ef0 0x7ff30410aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff3037fe640 1 -- 172.21.15.94:0/4136258010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff304102ef0 msgr2=0x7ff30410aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff3037fe640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff304102ef0 0x7ff30410aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff3037fe640 1 -- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102570 msgr2=0x7ff30410a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff3037fe640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102570 0x7ff30410a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff3037fe640 1 -- 172.21.15.94:0/4136258010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30410b3d0 con 0x7ff304100e20 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.370+0000 7ff302ffd640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102570 0x7ff30410a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.371+0000 7ff303fff640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff304102ef0 0x7ff30410aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.371+0000 7ff3037fe640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff304100e20 0x7ff30410d6b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40097d0 tx=0x7ff2f4009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.371+0000 7ff300ff9640 1 -- 172.21.15.94:0/4136258010 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4005020 con 0x7ff304100e20 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.371+0000 7ff30a23b640 1 -- 172.21.15.94:0/4136258010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3041a0660 con 0x7ff304100e20 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.371+0000 7ff300ff9640 1 -- 172.21.15.94:0/4136258010 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2f4005670 con 0x7ff304100e20 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.371+0000 7ff30a23b640 1 -- 172.21.15.94:0/4136258010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3041a0ba0 con 0x7ff304100e20 2024-09-19T04:26:50.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.371+0000 7ff300ff9640 1 -- 172.21.15.94:0/4136258010 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400ea60 con 0x7ff304100e20 2024-09-19T04:26:50.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.375+0000 7ff300ff9640 1 -- 172.21.15.94:0/4136258010 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff2f401b020 con 0x7ff304100e20 2024-09-19T04:26:50.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.376+0000 7ff30c614740 1 Processor -- start 2024-09-19T04:26:50.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.376+0000 7ff30c614740 1 -- start start 2024-09-19T04:26:50.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.376+0000 7ff30c614740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.376+0000 7ff30c614740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.377+0000 7ff303fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.377+0000 7ff303fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56192/0 (socket says 172.21.15.94:56192) 2024-09-19T04:26:50.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.377+0000 7ff30c614740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.377+0000 7ff303fff640 1 -- 172.21.15.94:0/580906573 learned_addr learned my addr 172.21.15.94:0/580906573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:50.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.377+0000 7ff30c614740 1 -- 172.21.15.94:0/580906573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5580a57462f0 con 0x5580a57eb370 2024-09-19T04:26:50.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.377+0000 7ff30c614740 1 -- 172.21.15.94:0/580906573 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5580a56edcf0 con 0x5580a57e1b40 2024-09-19T04:26:50.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.377+0000 7ff30c614740 1 -- 172.21.15.94:0/580906573 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5580a57422b0 con 0x5580a57e05d0 2024-09-19T04:26:50.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.377+0000 7ff303fff640 1 -- 172.21.15.94:0/580906573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 msgr2=0x5580a57e0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:50.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.377+0000 7ff302ffd640 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff3037fe640 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff300ff9640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff2ec0a3430 0x7ff2ec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff303fff640 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57e0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff303fff640 1 -- 172.21.15.94:0/580906573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 msgr2=0x5580a57ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff303fff640 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff303fff640 1 -- 172.21.15.94:0/580906573 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a56d23a0 con 0x5580a57e1b40 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff302ffd640 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff3037fe640 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff300ff9640 1 -- 172.21.15.94:0/4136258010 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7ff2f40cf510 con 0x7ff304100e20 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff303fff640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff2ec0a3430 0x7ff2ec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.378+0000 7ff303fff640 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e0090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009ca0 tx=0x7ff2fc0095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cff17685ab75f8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:50.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.379+0000 7ff2e9ffb640 1 -- 172.21.15.94:0/580906573 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc034040 con 0x5580a57e1b40 2024-09-19T04:26:50.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.379+0000 7ff2e9ffb640 1 -- 172.21.15.94:0/580906573 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2fc003d50 con 0x5580a57e1b40 2024-09-19T04:26:50.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.379+0000 7ff2e9ffb640 1 -- 172.21.15.94:0/580906573 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc030440 con 0x5580a57e1b40 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff30c614740 1 -- 172.21.15.94:0/580906573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 msgr2=0x5580a57e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff30c614740 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e0090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009ca0 tx=0x7ff2fc0095d0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff303fff640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff2ec0a3430 0x7ff2ec0a58f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc04a000 tx=0x7ff2fc030850 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff30c614740 1 -- 172.21.15.94:0/580906573 shutdown_connections 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff30c614740 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff30c614740 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff30c614740 1 --2- 172.21.15.94:0/580906573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e0090 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff30c614740 1 -- 172.21.15.94:0/580906573 >> 172.21.15.94:0/580906573 conn(0x5580a57a9310 msgr2=0x5580a57dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff30c614740 1 -- 172.21.15.94:0/580906573 shutdown_connections 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.380+0000 7ff30c614740 1 -- 172.21.15.94:0/580906573 wait complete. 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.381+0000 7ff30c614740 1 -- start start 2024-09-19T04:26:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.381+0000 7ff30c614740 1 Processor -- start 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.381+0000 7ff30c614740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57eab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.381+0000 7ff30c614740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.381+0000 7ff30c614740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57e8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.381+0000 7ff30c614740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5580a57ee010 con 0x5580a57eb370 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.381+0000 7ff30c614740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5580a5701060 con 0x5580a57e1b40 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.381+0000 7ff30c614740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5580a57ee190 con 0x5580a57e05d0 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.381+0000 7ff30c614740 10 client.0 _ll_register_callbacks cb 0x5580a54548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff302ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff302ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56208/0 (socket says 172.21.15.94:56208) 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff302ffd640 1 -- 172.21.15.94:0/3788393601 learned_addr learned my addr 172.21.15.94:0/3788393601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff3037fe640 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff302ffd640 1 -- 172.21.15.94:0/3788393601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 msgr2=0x5580a57eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff302ffd640 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57eab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff302ffd640 1 -- 172.21.15.94:0/3788393601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 msgr2=0x5580a57e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff302ffd640 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff302ffd640 1 -- 172.21.15.94:0/3788393601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a54b73c0 con 0x5580a57e1b40 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff3037fe640 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff302ffd640 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e7e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8013d30 tx=0x7ff2f80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0e1d53f31a4e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff2e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.382+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/3788393601 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f801b630 con 0x5580a57e1b40 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.383+0000 7ff30c614740 1 -- 172.21.15.94:0/3788393601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2fc003d50 con 0x5580a57e1b40 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.383+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/3788393601 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7ff2f801b7d0 con 0x5580a57e1b40 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.383+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/3788393601 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff2f801b9d0 con 0x5580a57e1b40 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.383+0000 7ff30c614740 10 client.26426 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:50.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.383+0000 7ff30c614740 1 -- 172.21.15.94:0/3788393601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5580a57f9930 con 0x5580a57e1b40 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.383+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/3788393601 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2fc003d50 con 0x5580a57e1b40 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.384+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/3788393601 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7ff2f800fc90 con 0x5580a57e1b40 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.384+0000 7ff30c614740 10 client.26426 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.384+0000 7ff30c614740 10 client.26426 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.384+0000 7ff30c614740 20 client.26426 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.384+0000 7ff30c614740 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5580a57faeb0 0x5580a57fd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.384+0000 7ff30c614740 4 client.26426 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.384+0000 7ff30c614740 1 -- 172.21.15.94:0/3788393601 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580a5753860 con 0x5580a57faeb0 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.384+0000 7ff3037fe640 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5580a57faeb0 0x5580a57fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.385+0000 7ff3037fe640 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5580a57faeb0 0x5580a57fd310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2159af9688ff104 server_cookie=f6f87a628a779770 in_seq=0 out_seq=0 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.385+0000 7ff2e2ffd640 10 client.26426 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:26:50.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/3788393601 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580a5753860 con 0x5580a57faeb0 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff2e2ffd640 10 client.26426 handle_command_reply: tid=0 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 client.26426 shutdown 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 -- 172.21.15.94:0/3788393601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 msgr2=0x5580a57e7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e7e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8013d30 tx=0x7ff2f80117b0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 -- 172.21.15.94:0/3788393601 shutdown_connections 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5580a57faeb0 0x5580a57fd310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57e7e30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 --2- 172.21.15.94:0/3788393601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 -- 172.21.15.94:0/3788393601 >> 172.21.15.94:0/3788393601 conn(0x5580a57a9310 msgr2=0x5580a57e2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff2e2ffd640 0 client.26426 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 -- 172.21.15.94:0/3788393601 shutdown_connections 2024-09-19T04:26:50.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 1 -- 172.21.15.94:0/3788393601 wait complete. 2024-09-19T04:26:50.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.386+0000 7ff30c614740 20 client.26426 trim_cache size 0 max 16384 2024-09-19T04:26:50.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff30c614740 1 Processor -- start 2024-09-19T04:26:50.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff30c614740 1 -- start start 2024-09-19T04:26:50.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff30c614740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e05d0 0x5580a5802be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff30c614740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57e1b40 0x5580a5803120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff30c614740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57eb370 0x5580a57e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff30c614740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041befa0 con 0x5580a57e1b40 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff302ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57e1b40 0x5580a5803120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff302ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57e1b40 0x5580a5803120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60730/0 (socket says 172.21.15.94:60730) 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff302ffd640 1 -- 172.21.15.94:0/1377616842 learned_addr learned my addr 172.21.15.94:0/1377616842 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff30c614740 1 -- 172.21.15.94:0/1377616842 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041bc7b0 con 0x5580a57e05d0 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.409+0000 7ff30c614740 1 -- 172.21.15.94:0/1377616842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041ba620 con 0x5580a57eb370 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.410+0000 7ff3037fe640 1 --2- 172.21.15.94:0/1377616842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e05d0 0x5580a5802be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.410+0000 7ff302ffd640 1 -- 172.21.15.94:0/1377616842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57eb370 msgr2=0x5580a57e50b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.410+0000 7ff302ffd640 1 --2- 172.21.15.94:0/1377616842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57eb370 0x5580a57e50b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.410+0000 7ff302ffd640 1 -- 172.21.15.94:0/1377616842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e05d0 msgr2=0x5580a5802be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.410+0000 7ff302ffd640 1 --2- 172.21.15.94:0/1377616842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e05d0 0x5580a5802be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.410+0000 7ff302ffd640 1 -- 172.21.15.94:0/1377616842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a57db310 con 0x5580a57e1b40 2024-09-19T04:26:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.410+0000 7ff302ffd640 1 --2- 172.21.15.94:0/1377616842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57e1b40 0x5580a5803120 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80224a0 tx=0x7ff2f8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f6e3f29803ba5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.411+0000 7ff2e97fa640 1 -- 172.21.15.94:0/1377616842 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8000e60 con 0x5580a57e1b40 2024-09-19T04:26:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.411+0000 7ff2e97fa640 1 -- 172.21.15.94:0/1377616842 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2f801bb70 con 0x5580a57e1b40 2024-09-19T04:26:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.411+0000 7ff2e97fa640 1 -- 172.21.15.94:0/1377616842 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8022a60 con 0x5580a57e1b40 2024-09-19T04:26:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.412+0000 7ff30c614740 1 -- 172.21.15.94:0/1377616842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57e1b40 msgr2=0x5580a5803120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.412+0000 7ff30c614740 1 --2- 172.21.15.94:0/1377616842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57e1b40 0x5580a5803120 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80224a0 tx=0x7ff2f8005c50 comp rx=0 tx=0).stop 2024-09-19T04:26:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.412+0000 7ff30c614740 1 -- 172.21.15.94:0/1377616842 shutdown_connections 2024-09-19T04:26:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.412+0000 7ff30c614740 1 --2- 172.21.15.94:0/1377616842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57eb370 0x5580a57e50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.412+0000 7ff30c614740 1 --2- 172.21.15.94:0/1377616842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57e1b40 0x5580a5803120 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.412+0000 7ff30c614740 1 --2- 172.21.15.94:0/1377616842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e05d0 0x5580a5802be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.412+0000 7ff30c614740 1 -- 172.21.15.94:0/1377616842 >> 172.21.15.94:0/1377616842 conn(0x5580a57a9310 msgr2=0x5580a57e9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:50.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.412+0000 7ff30c614740 1 -- 172.21.15.94:0/1377616842 shutdown_connections 2024-09-19T04:26:50.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.413+0000 7ff30c614740 1 -- 172.21.15.94:0/1377616842 wait complete. 2024-09-19T04:26:50.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.413+0000 7ff30c614740 1 -- start start 2024-09-19T04:26:50.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff30c614740 1 Processor -- start 2024-09-19T04:26:50.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff30c614740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff30c614740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57f3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff30c614740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57f0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff30c614740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041bac70 con 0x5580a57eb370 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff30c614740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041bc0e0 con 0x5580a57e1b40 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff30c614740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041bb340 con 0x5580a57e05d0 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff303fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44104/0 (socket says 172.21.15.94:44104) 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.414+0000 7ff3037fe640 1 -- 172.21.15.94:0/2538840377 learned_addr learned my addr 172.21.15.94:0/2538840377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff30c614740 10 client.0 _ll_register_callbacks cb 0x7ff2f8009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff303fff640 1 -- 172.21.15.94:0/2538840377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 msgr2=0x5580a57f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff303fff640 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff303fff640 1 -- 172.21.15.94:0/2538840377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 msgr2=0x5580a57f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:50.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff302ffd640 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff303fff640 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57f3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff303fff640 1 -- 172.21.15.94:0/2538840377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a57dd960 con 0x5580a57eb370 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff3037fe640 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff303fff640 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57f0420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc02ae70 tx=0x7ff2fc040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5d81db407d5a59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.415+0000 7ff2e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2538840377 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc059070 con 0x5580a57eb370 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2538840377 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7ff2fc04f840 con 0x5580a57eb370 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff30c614740 1 -- 172.21.15.94:0/2538840377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f801bb70 con 0x5580a57eb370 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2538840377 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff2fc034050 con 0x5580a57eb370 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff30c614740 10 client.16815 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff30c614740 1 -- 172.21.15.94:0/2538840377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5580a57dd1a0 con 0x5580a57eb370 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2538840377 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2f801bb70 con 0x5580a57eb370 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2538840377 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7ff2fc05e460 con 0x5580a57eb370 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff30c614740 10 client.16815 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff30c614740 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff30c614740 20 client.16815 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff30c614740 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5580a5877160 0x5580a5879550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff30c614740 4 client.16815 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.416+0000 7ff30c614740 1 -- 172.21.15.94:0/2538840377 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580a574dd70 con 0x5580a5877160 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.417+0000 7ff3037fe640 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5580a5877160 0x5580a5879550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.417+0000 7ff2e2ffd640 10 client.16815 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:26:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.417+0000 7ff3037fe640 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5580a5877160 0x5580a5879550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12064b73ee6e7fac server_cookie=89fd4a5b8046aa09 in_seq=0 out_seq=0 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.418+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2538840377 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580a574dd70 con 0x5580a5877160 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.418+0000 7ff2e2ffd640 10 client.16815 handle_command_reply: tid=0 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.418+0000 7ff30c614740 1 client.16815 shutdown 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.418+0000 7ff30c614740 1 -- 172.21.15.94:0/2538840377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 msgr2=0x5580a57f0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.418+0000 7ff30c614740 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57f0420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc02ae70 tx=0x7ff2fc040000 comp rx=0 tx=0).stop 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.418+0000 7ff30c614740 1 -- 172.21.15.94:0/2538840377 shutdown_connections 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.419+0000 7ff30c614740 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5580a5877160 0x5580a5879550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.419+0000 7ff30c614740 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580a57eb370 0x5580a57f0420 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.419+0000 7ff2e2ffd640 0 client.16815 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:26:50.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.419+0000 7ff30c614740 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580a57e1b40 0x5580a57f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.419+0000 7ff30c614740 1 --2- 172.21.15.94:0/2538840377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580a57e05d0 0x5580a57f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.419+0000 7ff30c614740 1 -- 172.21.15.94:0/2538840377 >> 172.21.15.94:0/2538840377 conn(0x5580a57a9310 msgr2=0x5580a57e9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:50.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.419+0000 7ff30c614740 1 -- 172.21.15.94:0/2538840377 shutdown_connections 2024-09-19T04:26:50.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.419+0000 7ff30c614740 1 -- 172.21.15.94:0/2538840377 wait complete. 2024-09-19T04:26:50.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30c614740 20 client.16815 trim_cache size 0 max 16384 2024-09-19T04:26:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:26:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 -- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff2ec0a3430 msgr2=0x7ff2ec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff2ec0a3430 0x7ff2ec0a58f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc04a000 tx=0x7ff2fc030850 comp rx=0 tx=0).stop 2024-09-19T04:26:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 -- 172.21.15.94:0/4136258010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff304100e20 msgr2=0x7ff30410d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff304100e20 0x7ff30410d6b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40097d0 tx=0x7ff2f4009c90 comp rx=0 tx=0).stop 2024-09-19T04:26:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 -- 172.21.15.94:0/4136258010 shutdown_connections 2024-09-19T04:26:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff2ec0a3430 0x7ff2ec0a58f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff304102ef0 0x7ff30410aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff304102570 0x7ff30410a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 --2- 172.21.15.94:0/4136258010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff304100e20 0x7ff30410d6b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.420+0000 7ff30a23b640 1 -- 172.21.15.94:0/4136258010 >> 172.21.15.94:0/4136258010 conn(0x7ff3040fc7d0 msgr2=0x7ff3041056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:50.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.421+0000 7ff30a23b640 1 -- 172.21.15.94:0/4136258010 shutdown_connections 2024-09-19T04:26:50.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:50.421+0000 7ff30a23b640 1 -- 172.21.15.94:0/4136258010 wait complete. 2024-09-19T04:26:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 635.0 s, 310.0 tps, lat 314.063 ms stddev 59.315 2024-09-19T04:26:51.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:51 smithi112 ceph-mon[35051]: pgmap v367: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.8 MiB/s wr, 566 op/s 2024-09-19T04:26:51.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:51 smithi094 ceph-mon[30972]: pgmap v367: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.8 MiB/s wr, 566 op/s 2024-09-19T04:26:51.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:51 smithi104 ceph-mon[34145]: pgmap v367: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.8 MiB/s wr, 566 op/s 2024-09-19T04:26:51.912 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:26:52.266 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:52.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.838+0000 7f893a974640 1 -- 172.21.15.94:0/1957313048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8934100e20 msgr2=0x7f8934101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.838+0000 7f893a974640 1 --2- 172.21.15.94:0/1957313048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8934100e20 0x7f8934101200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8924009890 tx=0x7f892402a410 comp rx=0 tx=0).stop 2024-09-19T04:26:52.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.838+0000 7f893a974640 1 -- 172.21.15.94:0/1957313048 shutdown_connections 2024-09-19T04:26:52.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.838+0000 7f893a974640 1 --2- 172.21.15.94:0/1957313048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8934102ef0 0x7f893410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.838+0000 7f893a974640 1 --2- 172.21.15.94:0/1957313048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102570 0x7f89341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.838+0000 7f893a974640 1 --2- 172.21.15.94:0/1957313048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8934100e20 0x7f8934101200 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.838+0000 7f893a974640 1 -- 172.21.15.94:0/1957313048 >> 172.21.15.94:0/1957313048 conn(0x7f89340fc7d0 msgr2=0x7f89340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.838+0000 7f893a974640 1 -- 172.21.15.94:0/1957313048 shutdown_connections 2024-09-19T04:26:52.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.838+0000 7f893a974640 1 -- 172.21.15.94:0/1957313048 wait complete. 2024-09-19T04:26:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.839+0000 7f893a974640 1 Processor -- start 2024-09-19T04:26:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.839+0000 7f893a974640 1 -- start start 2024-09-19T04:26:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.839+0000 7f893a974640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934100e20 0x7f8934198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f893a974640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8934102570 0x7f8934198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f893a974640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8934102ef0 0x7f893419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f8933fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934100e20 0x7f8934198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f893a974640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8934111f50 con 0x7f8934102570 2024-09-19T04:26:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f8933fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934100e20 0x7f8934198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:41982/0 (socket says 172.21.15.94:41982) 2024-09-19T04:26:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f8933fff640 1 -- 172.21.15.94:0/2011124243 learned_addr learned my addr 172.21.15.94:0/2011124243 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8934111dd0 con 0x7f8934102ef0 2024-09-19T04:26:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89341120d0 con 0x7f8934100e20 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f8938f11640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8934102ef0 0x7f893419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f89337fe640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8934102570 0x7f8934198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f8933fff640 1 -- 172.21.15.94:0/2011124243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8934102ef0 msgr2=0x7f893419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f8933fff640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8934102ef0 0x7f893419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f8933fff640 1 -- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8934102570 msgr2=0x7f8934198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f8933fff640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8934102570 0x7f8934198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.840+0000 7f8933fff640 1 -- 172.21.15.94:0/2011124243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893419d400 con 0x7f8934100e20 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.841+0000 7f89337fe640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8934102570 0x7f8934198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.841+0000 7f8933fff640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934100e20 0x7f8934198450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8924000c00 tx=0x7f8924030ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.841+0000 7f89317fa640 1 -- 172.21.15.94:0/2011124243 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924005460 con 0x7f8934100e20 2024-09-19T04:26:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.841+0000 7f89317fa640 1 -- 172.21.15.94:0/2011124243 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8924005600 con 0x7f8934100e20 2024-09-19T04:26:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.842+0000 7f89317fa640 1 -- 172.21.15.94:0/2011124243 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924004e10 con 0x7f8934100e20 2024-09-19T04:26:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.842+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893419d690 con 0x7f8934100e20 2024-09-19T04:26:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.842+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893419db20 con 0x7f8934100e20 2024-09-19T04:26:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.845+0000 7f89317fa640 1 -- 172.21.15.94:0/2011124243 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f8924039040 con 0x7f8934100e20 2024-09-19T04:26:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.846+0000 7f893cd4d740 1 Processor -- start 2024-09-19T04:26:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.846+0000 7f893cd4d740 1 -- start start 2024-09-19T04:26:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.846+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 0x55afb99049b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ff280 0x55afb98ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f8938f11640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 0x55afb99049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f8938f11640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 0x55afb99049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47624/0 (socket says 172.21.15.94:47624) 2024-09-19T04:26:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f8938f11640 1 -- 172.21.15.94:0/2571651048 learned_addr learned my addr 172.21.15.94:0/2571651048 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f893cd4d740 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb990b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f8933fff640 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ff280 0x55afb98ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f893cd4d740 1 -- 172.21.15.94:0/2571651048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55afb985b2f0 con 0x55afb99045d0 2024-09-19T04:26:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f893cd4d740 1 -- 172.21.15.94:0/2571651048 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55afb9802cf0 con 0x55afb98ff280 2024-09-19T04:26:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f89337fe640 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb990b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.847+0000 7f893cd4d740 1 -- 172.21.15.94:0/2571651048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55afb98572b0 con 0x55afb98ffc00 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.848+0000 7f8933fff640 1 -- 172.21.15.94:0/2571651048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 msgr2=0x55afb990b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.848+0000 7f8933fff640 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb990b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.848+0000 7f8933fff640 1 -- 172.21.15.94:0/2571651048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 msgr2=0x55afb99049b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.848+0000 7f8933fff640 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 0x55afb99049b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.848+0000 7f8933fff640 1 -- 172.21.15.94:0/2571651048 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb97e73a0 con 0x55afb98ff280 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.848+0000 7f8933fff640 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ff280 0x55afb98ff6c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f89240ee230 tx=0x7f89240f4ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=492e09b9b64a0dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.848+0000 7f891a7fc640 1 -- 172.21.15.94:0/2571651048 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89240f8040 con 0x55afb98ff280 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.848+0000 7f891a7fc640 1 -- 172.21.15.94:0/2571651048 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8924102070 con 0x55afb98ff280 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.848+0000 7f891a7fc640 1 -- 172.21.15.94:0/2571651048 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89240fdde0 con 0x55afb98ff280 2024-09-19T04:26:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.849+0000 7f89317fa640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f891c0a3580 0x7f891c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.849+0000 7f89317fa640 1 -- 172.21.15.94:0/2011124243 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f89240f0b80 con 0x7f8934100e20 2024-09-19T04:26:52.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.850+0000 7f893cd4d740 1 -- 172.21.15.94:0/2571651048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ff280 msgr2=0x55afb98ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.850+0000 7f8938f11640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f891c0a3580 0x7f891c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.850+0000 7f893cd4d740 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ff280 0x55afb98ff6c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f89240ee230 tx=0x7f89240f4ba0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.851+0000 7f893cd4d740 1 -- 172.21.15.94:0/2571651048 shutdown_connections 2024-09-19T04:26:52.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.851+0000 7f893cd4d740 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb990b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.851+0000 7f893cd4d740 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ff280 0x55afb98ff6c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.851+0000 7f893cd4d740 1 --2- 172.21.15.94:0/2571651048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 0x55afb99049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.851+0000 7f893cd4d740 1 -- 172.21.15.94:0/2571651048 >> 172.21.15.94:0/2571651048 conn(0x55afb98be310 msgr2=0x55afb98f16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.851+0000 7f8938f11640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f891c0a3580 0x7f891c0a5a40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f892c00e660 tx=0x7f892c010040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.851+0000 7f893cd4d740 1 -- 172.21.15.94:0/2571651048 shutdown_connections 2024-09-19T04:26:52.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.851+0000 7f893cd4d740 1 -- 172.21.15.94:0/2571651048 wait complete. 2024-09-19T04:26:52.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.851+0000 7f893cd4d740 1 -- start start 2024-09-19T04:26:52.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f893cd4d740 1 Processor -- start 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ff280 0x55afb990b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb9908410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 0x55afb9908950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f893cd4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55afb9905100 con 0x55afb98ff280 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f893cd4d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55afb9816060 con 0x55afb99045d0 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f893cd4d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55afb990b870 con 0x55afb98ffc00 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f89337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb9908410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f89337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb9908410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:42010/0 (socket says 172.21.15.94:42010) 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f89337fe640 1 -- 172.21.15.94:0/351334614 learned_addr learned my addr 172.21.15.94:0/351334614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f8933fff640 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ff280 0x55afb990b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f893cd4d740 10 client.0 _ll_register_callbacks cb 0x55afb95698c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.852+0000 7f8938f11640 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 0x55afb9908950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f89337fe640 1 -- 172.21.15.94:0/351334614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 msgr2=0x55afb9908950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f89337fe640 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 0x55afb9908950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f89337fe640 1 -- 172.21.15.94:0/351334614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ff280 msgr2=0x55afb990b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f89337fe640 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ff280 0x55afb990b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f89337fe640 1 -- 172.21.15.94:0/351334614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb95cc3c0 con 0x55afb98ffc00 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f89337fe640 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb9908410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8928013760 tx=0x7f8928013c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf93113322532516 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f89137fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f893cd4d740 1 -- 172.21.15.94:0/351334614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8924102070 con 0x55afb98ffc00 2024-09-19T04:26:52.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f89137fe640 1 -- 172.21.15.94:0/351334614 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928020060 con 0x55afb98ffc00 2024-09-19T04:26:52.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.853+0000 7f89137fe640 1 -- 172.21.15.94:0/351334614 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f8928011e30 con 0x55afb98ffc00 2024-09-19T04:26:52.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f89137fe640 1 -- 172.21.15.94:0/351334614 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f892801b3f0 con 0x55afb98ffc00 2024-09-19T04:26:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f893cd4d740 10 client.26422 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f893cd4d740 1 -- 172.21.15.94:0/351334614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=48}) -- 0x55afb9916f50 con 0x55afb98ffc00 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f89137fe640 1 -- 172.21.15.94:0/351334614 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f8928023050 con 0x55afb98ffc00 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f893cd4d740 10 client.26422 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f893cd4d740 10 client.26422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f893cd4d740 20 client.26422 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f893cd4d740 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55afb9918300 0x55afb991a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f893cd4d740 4 client.26422 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.854+0000 7f893cd4d740 1 -- 172.21.15.94:0/351334614 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8924102070 con 0x55afb9918300 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.855+0000 7f8933fff640 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55afb9918300 0x55afb991a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.855+0000 7f89137fe640 10 client.26422 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.855+0000 7f8933fff640 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55afb9918300 0x55afb991a6f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d6ca022175a5e7f server_cookie=92203690b94025f in_seq=0 out_seq=0 2024-09-19T04:26:52.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f89137fe640 1 -- 172.21.15.94:0/351334614 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8924102070 con 0x55afb9918300 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f89137fe640 10 client.26422 handle_command_reply: tid=0 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f893cd4d740 1 client.26422 shutdown 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f893cd4d740 1 -- 172.21.15.94:0/351334614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 msgr2=0x55afb9908410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f893cd4d740 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb9908410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8928013760 tx=0x7f8928013c20 comp rx=0 tx=0).stop 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f893cd4d740 1 -- 172.21.15.94:0/351334614 shutdown_connections 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f893cd4d740 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55afb9918300 0x55afb991a6f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f893cd4d740 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 0x55afb9908950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f89137fe640 0 client.26422 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f893cd4d740 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ffc00 0x55afb9908410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f893cd4d740 1 --2- 172.21.15.94:0/351334614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ff280 0x55afb990b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.856+0000 7f893cd4d740 1 -- 172.21.15.94:0/351334614 >> 172.21.15.94:0/351334614 conn(0x55afb98be310 msgr2=0x55afb98fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.857+0000 7f893cd4d740 1 -- 172.21.15.94:0/351334614 shutdown_connections 2024-09-19T04:26:52.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.857+0000 7f893cd4d740 1 -- 172.21.15.94:0/351334614 wait complete. 2024-09-19T04:26:52.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.857+0000 7f893cd4d740 20 client.26422 trim_cache size 0 max 16384 2024-09-19T04:26:52.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.880+0000 7f893cd4d740 1 Processor -- start 2024-09-19T04:26:52.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.884+0000 7f893cd4d740 1 -- start start 2024-09-19T04:26:52.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.888+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb9914880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.888+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ffc00 0x55afb98f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.888+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 0x55afb991ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.888+0000 7f893cd4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89341ae060 con 0x55afb98ffc00 2024-09-19T04:26:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.888+0000 7f893cd4d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89341ab870 con 0x55afb99045d0 2024-09-19T04:26:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.888+0000 7f893cd4d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89341a96e0 con 0x55afb98ff280 2024-09-19T04:26:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.890+0000 7f8938f11640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 0x55afb991ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.890+0000 7f89337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ffc00 0x55afb98f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.890+0000 7f89337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ffc00 0x55afb98f0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47632/0 (socket says 172.21.15.94:47632) 2024-09-19T04:26:52.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.891+0000 7f89337fe640 1 -- 172.21.15.94:0/1451925941 learned_addr learned my addr 172.21.15.94:0/1451925941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:52.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.891+0000 7f8933fff640 1 --2- 172.21.15.94:0/1451925941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb9914880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.894+0000 7f89337fe640 1 -- 172.21.15.94:0/1451925941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 msgr2=0x55afb9914880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.894+0000 7f89337fe640 1 --2- 172.21.15.94:0/1451925941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb9914880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.894+0000 7f89337fe640 1 -- 172.21.15.94:0/1451925941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 msgr2=0x55afb991ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.894+0000 7f89337fe640 1 --2- 172.21.15.94:0/1451925941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 0x55afb991ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.894+0000 7f89337fe640 1 -- 172.21.15.94:0/1451925941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb990b760 con 0x55afb98ffc00 2024-09-19T04:26:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.894+0000 7f89337fe640 1 --2- 172.21.15.94:0/1451925941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ffc00 0x55afb98f0bf0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f892801b720 tx=0x7f8928016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ace24729c3f7a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.895+0000 7f8919ffb640 1 -- 172.21.15.94:0/1451925941 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928011e40 con 0x55afb98ffc00 2024-09-19T04:26:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.895+0000 7f8919ffb640 1 -- 172.21.15.94:0/1451925941 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8928025420 con 0x55afb98ffc00 2024-09-19T04:26:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.895+0000 7f8919ffb640 1 -- 172.21.15.94:0/1451925941 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892802d660 con 0x55afb98ffc00 2024-09-19T04:26:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.896+0000 7f893cd4d740 1 -- 172.21.15.94:0/1451925941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ffc00 msgr2=0x55afb98f0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.896+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1451925941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ffc00 0x55afb98f0bf0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f892801b720 tx=0x7f8928016000 comp rx=0 tx=0).stop 2024-09-19T04:26:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.897+0000 7f893cd4d740 1 -- 172.21.15.94:0/1451925941 shutdown_connections 2024-09-19T04:26:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.897+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1451925941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb99045d0 0x55afb991ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.897+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1451925941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb98ffc00 0x55afb98f0bf0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.897+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1451925941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb9914880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.897+0000 7f893cd4d740 1 -- 172.21.15.94:0/1451925941 >> 172.21.15.94:0/1451925941 conn(0x55afb98be310 msgr2=0x55afb98f04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.897+0000 7f893cd4d740 1 -- 172.21.15.94:0/1451925941 shutdown_connections 2024-09-19T04:26:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.897+0000 7f893cd4d740 1 -- 172.21.15.94:0/1451925941 wait complete. 2024-09-19T04:26:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.897+0000 7f893cd4d740 1 -- start start 2024-09-19T04:26:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.898+0000 7f893cd4d740 1 Processor -- start 2024-09-19T04:26:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.898+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb997cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.898+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ffc00 0x55afb997d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.898+0000 7f893cd4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 0x55afb997d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.898+0000 7f893cd4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89341a9d30 con 0x55afb99045d0 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.898+0000 7f893cd4d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89341ab1a0 con 0x55afb98ffc00 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.898+0000 7f893cd4d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89341aa400 con 0x55afb98ff280 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.898+0000 7f893cd4d740 10 client.0 _ll_register_callbacks cb 0x7f8928009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8933fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb997cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8933fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb997cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:42028/0 (socket says 172.21.15.94:42028) 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8933fff640 1 -- 172.21.15.94:0/1925387854 learned_addr learned my addr 172.21.15.94:0/1925387854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f89337fe640 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ffc00 0x55afb997d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8938f11640 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 0x55afb997d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8933fff640 1 -- 172.21.15.94:0/1925387854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ffc00 msgr2=0x55afb997d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8933fff640 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ffc00 0x55afb997d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8933fff640 1 -- 172.21.15.94:0/1925387854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 msgr2=0x55afb997d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8933fff640 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 0x55afb997d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8933fff640 1 -- 172.21.15.94:0/1925387854 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb990f9c0 con 0x55afb98ff280 2024-09-19T04:26:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.899+0000 7f8933fff640 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb997cf40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f89240b9790 tx=0x7f8924005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd390630e0a9ac73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.900+0000 7f89137fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.900+0000 7f89337fe640 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ffc00 0x55afb997d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.900+0000 7f89137fe640 1 -- 172.21.15.94:0/1925387854 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89240bcb90 con 0x55afb98ff280 2024-09-19T04:26:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.900+0000 7f89137fe640 1 -- 172.21.15.94:0/1925387854 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f89240b9430 con 0x55afb98ff280 2024-09-19T04:26:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.900+0000 7f893cd4d740 1 -- 172.21.15.94:0/1925387854 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8928025420 con 0x55afb98ff280 2024-09-19T04:26:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.900+0000 7f89137fe640 1 -- 172.21.15.94:0/1925387854 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f89240bc720 con 0x55afb98ff280 2024-09-19T04:26:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.900+0000 7f893cd4d740 10 client.26428 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.900+0000 7f893cd4d740 1 -- 172.21.15.94:0/1925387854 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55afb9910ce0 con 0x55afb98ff280 2024-09-19T04:26:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.901+0000 7f89137fe640 1 -- 172.21.15.94:0/1925387854 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8928025420 con 0x55afb98ff280 2024-09-19T04:26:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.901+0000 7f89137fe640 1 -- 172.21.15.94:0/1925387854 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f89240f4320 con 0x55afb98ff280 2024-09-19T04:26:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.901+0000 7f893cd4d740 10 client.26428 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.901+0000 7f893cd4d740 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.901+0000 7f893cd4d740 20 client.26428 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.901+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55afb9918300 0x55afb98fda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.901+0000 7f893cd4d740 4 client.26428 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.901+0000 7f893cd4d740 1 -- 172.21.15.94:0/1925387854 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afb9868860 con 0x55afb9918300 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.902+0000 7f89337fe640 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55afb9918300 0x55afb98fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.902+0000 7f89137fe640 10 client.26428 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.902+0000 7f89337fe640 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55afb9918300 0x55afb98fda50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6277f984db5e9b0c server_cookie=1f70e3f1db471bdb in_seq=0 out_seq=0 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.902+0000 7f89137fe640 1 -- 172.21.15.94:0/1925387854 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afb9868860 con 0x55afb9918300 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f89137fe640 10 client.26428 handle_command_reply: tid=0 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 client.26428 shutdown 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 -- 172.21.15.94:0/1925387854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 msgr2=0x55afb997cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb997cf40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f89240b9790 tx=0x7f8924005f90 comp rx=0 tx=0).stop 2024-09-19T04:26:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 -- 172.21.15.94:0/1925387854 shutdown_connections 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55afb9918300 0x55afb98fda50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55afb99045d0 0x55afb997d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afb98ffc00 0x55afb997d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 --2- 172.21.15.94:0/1925387854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afb98ff280 0x55afb997cf40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 -- 172.21.15.94:0/1925387854 >> 172.21.15.94:0/1925387854 conn(0x55afb98be310 msgr2=0x55afb9909c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f89137fe640 0 client.26428 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 -- 172.21.15.94:0/1925387854 shutdown_connections 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 1 -- 172.21.15.94:0/1925387854 wait complete. 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.903+0000 7f893cd4d740 20 client.26428 trim_cache size 0 max 16384 2024-09-19T04:26:52.908 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:26:52.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.904+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f891c0a3580 msgr2=0x7f891c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.904+0000 7f893a974640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f891c0a3580 0x7f891c0a5a40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f892c00e660 tx=0x7f892c010040 comp rx=0 tx=0).stop 2024-09-19T04:26:52.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.904+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934100e20 msgr2=0x7f8934198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.904+0000 7f893a974640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934100e20 0x7f8934198450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8924000c00 tx=0x7f8924030ef0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.904+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 shutdown_connections 2024-09-19T04:26:52.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.904+0000 7f893a974640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f891c0a3580 0x7f891c0a5a40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.904+0000 7f893a974640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8934102ef0 0x7f893419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.904+0000 7f893a974640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8934102570 0x7f8934198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.905+0000 7f893a974640 1 --2- 172.21.15.94:0/2011124243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934100e20 0x7f8934198450 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.905+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 >> 172.21.15.94:0/2011124243 conn(0x7f89340fc7d0 msgr2=0x7f89340fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.905+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 shutdown_connections 2024-09-19T04:26:52.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:52.905+0000 7f893a974640 1 -- 172.21.15.94:0/2011124243 wait complete. 2024-09-19T04:26:53.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:53 smithi112 ceph-mon[35051]: pgmap v368: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 3.8 MiB/s wr, 476 op/s 2024-09-19T04:26:53.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:53 smithi094 ceph-mon[30972]: pgmap v368: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 3.8 MiB/s wr, 476 op/s 2024-09-19T04:26:53.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:53 smithi104 ceph-mon[34145]: pgmap v368: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 3.8 MiB/s wr, 476 op/s 2024-09-19T04:26:54.577 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:26:54.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:26:54.937 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 640.0 s, 319.0 tps, lat 320.409 ms stddev 89.262 2024-09-19T04:26:55.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:55 smithi112 ceph-mon[35051]: pgmap v369: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 5.7 MiB/s wr, 704 op/s 2024-09-19T04:26:55.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:55 smithi094 ceph-mon[30972]: pgmap v369: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 5.7 MiB/s wr, 704 op/s 2024-09-19T04:26:55.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:55 smithi104 ceph-mon[34145]: pgmap v369: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 5.7 MiB/s wr, 704 op/s 2024-09-19T04:26:56.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.857+0000 7f8cc0802640 1 -- 172.21.15.94:0/81518133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc1019e0 msgr2=0x7f8cbc101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.857+0000 7f8cc0802640 1 --2- 172.21.15.94:0/81518133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc1019e0 0x7f8cbc101e40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009930 tx=0x7f8ca402a420 comp rx=0 tx=0).stop 2024-09-19T04:26:56.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.857+0000 7f8cc0802640 1 -- 172.21.15.94:0/81518133 shutdown_connections 2024-09-19T04:26:56.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.857+0000 7f8cc0802640 1 --2- 172.21.15.94:0/81518133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cbc102380 0x7f8cbc10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.857+0000 7f8cc0802640 1 --2- 172.21.15.94:0/81518133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc1019e0 0x7f8cbc101e40 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.857+0000 7f8cc0802640 1 --2- 172.21.15.94:0/81518133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cbc107af0 0x7f8cbc107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.857+0000 7f8cc0802640 1 -- 172.21.15.94:0/81518133 >> 172.21.15.94:0/81518133 conn(0x7f8cbc0fd700 msgr2=0x7f8cbc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.857+0000 7f8cc0802640 1 -- 172.21.15.94:0/81518133 shutdown_connections 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.857+0000 7f8cc0802640 1 -- 172.21.15.94:0/81518133 wait complete. 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cc0802640 1 Processor -- start 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cc0802640 1 -- start start 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cc0802640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cbc1019e0 0x7f8cbc1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cc0802640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc102380 0x7f8cbc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cc0802640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cbc107af0 0x7f8cbc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cc0802640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111d60 con 0x7f8cbc102380 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cc0802640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111be0 con 0x7f8cbc107af0 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cc0802640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111ee0 con 0x7f8cbc1019e0 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc102380 0x7f8cbc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc102380 0x7f8cbc198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47644/0 (socket says 172.21.15.94:47644) 2024-09-19T04:26:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cb3fff640 1 -- 172.21.15.94:0/3471559011 learned_addr learned my addr 172.21.15.94:0/3471559011 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cbb59e640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cbc107af0 0x7f8cbc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cbc1019e0 0x7f8cbc1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cb3fff640 1 -- 172.21.15.94:0/3471559011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cbc1019e0 msgr2=0x7f8cbc1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cbc1019e0 0x7f8cbc1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cb3fff640 1 -- 172.21.15.94:0/3471559011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cbc107af0 msgr2=0x7f8cbc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cbc107af0 0x7f8cbc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.858+0000 7f8cb3fff640 1 -- 172.21.15.94:0/3471559011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc19d270 con 0x7f8cbc102380 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.859+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc102380 0x7f8cbc198800 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4004770 tx=0x7f8ca40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.859+0000 7f8cb8d99640 1 -- 172.21.15.94:0/3471559011 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca403f070 con 0x7f8cbc102380 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.859+0000 7f8cc0802640 1 -- 172.21.15.94:0/3471559011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc19d500 con 0x7f8cbc102380 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.859+0000 7f8cb8d99640 1 -- 172.21.15.94:0/3471559011 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ca403b460 con 0x7f8cbc102380 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.859+0000 7f8cb8d99640 1 -- 172.21.15.94:0/3471559011 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4035070 con 0x7f8cbc102380 2024-09-19T04:26:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.859+0000 7f8cc0802640 1 -- 172.21.15.94:0/3471559011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc1a4e40 con 0x7f8cbc102380 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.861+0000 7f8cb8d99640 1 -- 172.21.15.94:0/3471559011 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f8ca403b610 con 0x7f8cbc102380 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.861+0000 7f8cc33dc740 1 Processor -- start 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.861+0000 7f8cc33dc740 1 -- start start 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386360e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 0x56386361ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386361d360 0x56386361f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563863573280 con 0x56386360dc30 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cb8d99640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8ca80a3430 0x7f8ca80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56386351ab10 con 0x56386361d360 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56386356f240 con 0x56386360e550 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386360e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386360e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47652/0 (socket says 172.21.15.94:47652) 2024-09-19T04:26:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 -- 172.21.15.94:0/4290058112 learned_addr learned my addr 172.21.15.94:0/4290058112 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8ca80a3430 0x7f8ca80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/4290058112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 0x56386361ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cb8d99640 1 -- 172.21.15.94:0/3471559011 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f8ca40f0070 con 0x7f8cbc102380 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/4290058112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386361d360 0x56386361f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 -- 172.21.15.94:0/4290058112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 msgr2=0x56386361ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 --2- 172.21.15.94:0/4290058112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 0x56386361ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 -- 172.21.15.94:0/4290058112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386361d360 msgr2=0x56386361f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 --2- 172.21.15.94:0/4290058112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386361d360 0x56386361f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 -- 172.21.15.94:0/4290058112 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638635271f0 con 0x56386360dc30 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.862+0000 7f8cbb59e640 1 --2- 172.21.15.94:0/4290058112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386360e010 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4012d80 tx=0x7f8cb4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bd9869e3cd581a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.863+0000 7f8cbb59e640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8ca80a3430 0x7f8ca80a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4010c40 tx=0x7f8cb400c000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.863+0000 7f8cb17fa640 1 -- 172.21.15.94:0/4290058112 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4024070 con 0x56386360dc30 2024-09-19T04:26:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.863+0000 7f8cb17fa640 1 -- 172.21.15.94:0/4290058112 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cb40125a0 con 0x56386360dc30 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.863+0000 7f8cb17fa640 1 -- 172.21.15.94:0/4290058112 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4005280 con 0x56386360dc30 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.863+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4290058112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 msgr2=0x56386360e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.863+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4290058112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386360e010 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4012d80 tx=0x7f8cb4010740 comp rx=0 tx=0).stop 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4290058112 shutdown_connections 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4290058112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386361d360 0x56386361f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4290058112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 0x56386361ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4290058112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386360e010 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4290058112 >> 172.21.15.94:0/4290058112 conn(0x56386360a300 msgr2=0x56386360a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4290058112 shutdown_connections 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4290058112 wait complete. 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 -- start start 2024-09-19T04:26:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 Processor -- start 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386361a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 0x56386361a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 0x56386361ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56386361fed0 con 0x56386360dc30 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56386352df10 con 0x56386360e550 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563863620050 con 0x56386361d360 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cc33dc740 10 client.0 _ll_register_callbacks cb 0x5638634f6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cbad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386361a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cbad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386361a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47660/0 (socket says 172.21.15.94:47660) 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cbad9d640 1 -- 172.21.15.94:0/4001815999 learned_addr learned my addr 172.21.15.94:0/4001815999 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.864+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 0x56386361a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cbb59e640 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 0x56386361ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cbad9d640 1 -- 172.21.15.94:0/4001815999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 msgr2=0x56386361ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 0x56386361ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cbad9d640 1 -- 172.21.15.94:0/4001815999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 msgr2=0x56386361a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 0x56386361a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cbad9d640 1 -- 172.21.15.94:0/4001815999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638632e46a0 con 0x56386360dc30 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386361a2c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8cac011740 tx=0x7f8cac011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb49b42cdaa4391 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8ca27fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8ca27fc640 1 -- 172.21.15.94:0/4001815999 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00fdb0 con 0x56386360dc30 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8ca27fc640 1 -- 172.21.15.94:0/4001815999 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f8cac005020 con 0x56386360dc30 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4001815999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb40125a0 con 0x56386360dc30 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8ca27fc640 1 -- 172.21.15.94:0/4001815999 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8cac005220 con 0x56386360dc30 2024-09-19T04:26:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cc33dc740 10 client.16845 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4001815999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56386362bc20 con 0x56386360dc30 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8ca27fc640 1 -- 172.21.15.94:0/4001815999 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cb40125a0 con 0x56386360dc30 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8ca27fc640 1 -- 172.21.15.94:0/4001815999 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f8cac018030 con 0x56386360dc30 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cc33dc740 10 client.16845 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cc33dc740 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.865+0000 7f8cc33dc740 20 client.16845 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.866+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56386362d180 0x56386362f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.866+0000 7f8cc33dc740 4 client.16845 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.866+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4001815999 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638635807f0 con 0x56386362d180 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.866+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56386362d180 0x56386362f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.866+0000 7f8ca27fc640 10 client.16845 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.866+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56386362d180 0x56386362f5e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff2d08c5cea8d50 server_cookie=b9f36f59d25c09f2 in_seq=0 out_seq=0 2024-09-19T04:26:56.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8ca27fc640 1 -- 172.21.15.94:0/4001815999 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638635807f0 con 0x56386362d180 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8ca27fc640 10 client.16845 handle_command_reply: tid=0 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 client.16845 shutdown 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4001815999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 msgr2=0x56386361a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386361a2c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8cac011740 tx=0x7f8cac011c00 comp rx=0 tx=0).stop 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4001815999 shutdown_connections 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56386362d180 0x56386362f5e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 0x56386361ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 0x56386361a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/4001815999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x56386361a2c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8ca27fc640 0 client.16845 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4001815999 >> 172.21.15.94:0/4001815999 conn(0x56386360a300 msgr2=0x563863610060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4001815999 shutdown_connections 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 1 -- 172.21.15.94:0/4001815999 wait complete. 2024-09-19T04:26:56.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.867+0000 7f8cc33dc740 20 client.16845 trim_cache size 0 max 16384 2024-09-19T04:26:56.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cc33dc740 1 Processor -- start 2024-09-19T04:26:56.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cc33dc740 1 -- start start 2024-09-19T04:26:56.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360dc30 0x56386361c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 0x563863634eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386361d360 0x5638636353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1addb0 con 0x56386361d360 2024-09-19T04:26:56.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ab5c0 con 0x56386360dc30 2024-09-19T04:26:56.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1a9430 con 0x56386360e550 2024-09-19T04:26:56.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cbad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360dc30 0x56386361c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cbb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386361d360 0x5638636353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 0x563863634eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cbb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386361d360 0x5638636353f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47662/0 (socket says 172.21.15.94:47662) 2024-09-19T04:26:56.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cbad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360dc30 0x56386361c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50078/0 (socket says 172.21.15.94:50078) 2024-09-19T04:26:56.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cbb59e640 1 -- 172.21.15.94:0/3231174952 learned_addr learned my addr 172.21.15.94:0/3231174952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:56.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.886+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 0x563863634eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:42074/0 (socket says 172.21.15.94:42074) 2024-09-19T04:26:56.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.887+0000 7f8cbad9d640 1 -- 172.21.15.94:0/3231174952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 msgr2=0x563863634eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.887+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/3231174952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 0x563863634eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.887+0000 7f8cbad9d640 1 -- 172.21.15.94:0/3231174952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386361d360 msgr2=0x5638636353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.887+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/3231174952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386361d360 0x5638636353f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.887+0000 7f8cbad9d640 1 -- 172.21.15.94:0/3231174952 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386360e380 con 0x56386360dc30 2024-09-19T04:26:56.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.887+0000 7f8cbb59e640 1 --2- 172.21.15.94:0/3231174952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386361d360 0x5638636353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:56.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.887+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/3231174952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360dc30 0x56386361c120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8cac019570 tx=0x7f8cac0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8de7e194b11f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.887+0000 7f8cb0ff9640 1 -- 172.21.15.94:0/3231174952 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0055c0 con 0x56386360dc30 2024-09-19T04:26:56.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.888+0000 7f8cb0ff9640 1 -- 172.21.15.94:0/3231174952 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cac02e070 con 0x56386360dc30 2024-09-19T04:26:56.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.888+0000 7f8cb0ff9640 1 -- 172.21.15.94:0/3231174952 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00fe10 con 0x56386360dc30 2024-09-19T04:26:56.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 -- 172.21.15.94:0/3231174952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360dc30 msgr2=0x56386361c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/3231174952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360dc30 0x56386361c120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8cac019570 tx=0x7f8cac0058a0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 -- 172.21.15.94:0/3231174952 shutdown_connections 2024-09-19T04:26:56.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/3231174952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386361d360 0x5638636353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/3231174952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386360e550 0x563863634eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/3231174952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360dc30 0x56386361c120 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 -- 172.21.15.94:0/3231174952 >> 172.21.15.94:0/3231174952 conn(0x56386360a300 msgr2=0x56386361ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 -- 172.21.15.94:0/3231174952 shutdown_connections 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 -- 172.21.15.94:0/3231174952 wait complete. 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 -- start start 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.889+0000 7f8cc33dc740 1 Processor -- start 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x563863621080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 0x5638636215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cc33dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 0x563863621b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1a9a80 con 0x56386360dc30 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1aaef0 con 0x56386360e550 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cc33dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1aa150 con 0x56386361d360 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cc33dc740 10 client.0 _ll_register_callbacks cb 0x7f8cac00cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cbad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x563863621080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cbad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x563863621080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47676/0 (socket says 172.21.15.94:47676) 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cbad9d640 1 -- 172.21.15.94:0/1255839564 learned_addr learned my addr 172.21.15.94:0/1255839564 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 0x5638636215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.890+0000 7f8cbb59e640 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 0x563863621b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8cbad9d640 1 -- 172.21.15.94:0/1255839564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 msgr2=0x563863621b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 0x563863621b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8cbad9d640 1 -- 172.21.15.94:0/1255839564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 msgr2=0x5638636215c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 0x5638636215c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8cbad9d640 1 -- 172.21.15.94:0/1255839564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563863696ed0 con 0x56386360dc30 2024-09-19T04:26:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 0x5638636215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:56.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8cbad9d640 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x563863621080 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00fe40 tx=0x7f8cac00fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a26aff53b7796d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8ca27fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8ca27fc640 1 -- 172.21.15.94:0/1255839564 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac019c30 con 0x56386360dc30 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8ca27fc640 1 -- 172.21.15.94:0/1255839564 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f8cac019dd0 con 0x56386360dc30 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8cc33dc740 1 -- 172.21.15.94:0/1255839564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cac02e070 con 0x56386360dc30 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.891+0000 7f8ca27fc640 1 -- 172.21.15.94:0/1255839564 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8cac043050 con 0x56386360dc30 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.892+0000 7f8cc33dc740 10 client.16851 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.892+0000 7f8cc33dc740 1 -- 172.21.15.94:0/1255839564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5638636981f0 con 0x56386360dc30 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.892+0000 7f8ca27fc640 1 -- 172.21.15.94:0/1255839564 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cac037070 con 0x56386360dc30 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.892+0000 7f8ca27fc640 1 -- 172.21.15.94:0/1255839564 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f8cac018070 con 0x56386360dc30 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.892+0000 7f8cc33dc740 10 client.16851 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.892+0000 7f8cc33dc740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.892+0000 7f8cc33dc740 20 client.16851 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.892+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5638636987b0 0x56386369abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.892+0000 7f8cc33dc740 4 client.16851 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:26:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.893+0000 7f8cc33dc740 1 -- 172.21.15.94:0/1255839564 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f8cac02e070 con 0x5638636987b0 2024-09-19T04:26:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.893+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5638636987b0 0x56386369abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.894+0000 7f8ca27fc640 10 client.16851 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.894+0000 7f8cb3fff640 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5638636987b0 0x56386369abd0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb4fa5e866efc2a0 server_cookie=16dff75838502233 in_seq=0 out_seq=0 2024-09-19T04:26:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.894+0000 7f8ca27fc640 1 -- 172.21.15.94:0/1255839564 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f8cac02e070 con 0x5638636987b0 2024-09-19T04:26:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.894+0000 7f8ca27fc640 10 client.16851 handle_command_reply: tid=0 2024-09-19T04:26:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.894+0000 7f8cc33dc740 1 client.16851 shutdown 2024-09-19T04:26:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 -- 172.21.15.94:0/1255839564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 msgr2=0x563863621080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x563863621080 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00fe40 tx=0x7f8cac00fe70 comp rx=0 tx=0).stop 2024-09-19T04:26:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 -- 172.21.15.94:0/1255839564 shutdown_connections 2024-09-19T04:26:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5638636987b0 0x56386369abd0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56386361d360 0x563863621b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8ca27fc640 0 client.16851 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56386360e550 0x5638636215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 --2- 172.21.15.94:0/1255839564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56386360dc30 0x563863621080 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 -- 172.21.15.94:0/1255839564 >> 172.21.15.94:0/1255839564 conn(0x56386360a300 msgr2=0x563863609410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 -- 172.21.15.94:0/1255839564 shutdown_connections 2024-09-19T04:26:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 1 -- 172.21.15.94:0/1255839564 wait complete. 2024-09-19T04:26:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.895+0000 7f8cc33dc740 20 client.16851 trim_cache size 0 max 16384 2024-09-19T04:26:56.901 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:26:56.901 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:26:56.901 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "997d4d61-d3a3-437c-b603-744fe26fb10f", 2024-09-19T04:26:56.901 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:26:56.901 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 -- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8ca80a3430 msgr2=0x7f8ca80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8ca80a3430 0x7f8ca80a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4010c40 tx=0x7f8cb400c000 comp rx=0 tx=0).stop 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 -- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc102380 msgr2=0x7f8cbc198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc102380 0x7f8cbc198800 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4004770 tx=0x7f8ca40042f0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 -- 172.21.15.94:0/3471559011 shutdown_connections 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8ca80a3430 0x7f8ca80a58f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cbc107af0 0x7f8cbc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cbc102380 0x7f8cbc198800 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 --2- 172.21.15.94:0/3471559011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cbc1019e0 0x7f8cbc1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.896+0000 7f8cc0802640 1 -- 172.21.15.94:0/3471559011 >> 172.21.15.94:0/3471559011 conn(0x7f8cbc0fd700 msgr2=0x7f8cbc0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.897+0000 7f8cc0802640 1 -- 172.21.15.94:0/3471559011 shutdown_connections 2024-09-19T04:26:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:56.897+0000 7f8cc0802640 1 -- 172.21.15.94:0/3471559011 wait complete. 2024-09-19T04:26:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:57 smithi094 ceph-mon[30972]: pgmap v370: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 3.7 MiB/s wr, 470 op/s 2024-09-19T04:26:57.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:26:57.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:57 smithi112 ceph-mon[35051]: pgmap v370: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 3.7 MiB/s wr, 470 op/s 2024-09-19T04:26:57.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:57 smithi104 ceph-mon[34145]: pgmap v370: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 3.7 MiB/s wr, 470 op/s 2024-09-19T04:26:58.021 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.604+0000 7f758f3df640 1 -- 172.21.15.94:0/2969563727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588106f20 msgr2=0x7f758810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.604+0000 7f758f3df640 1 --2- 172.21.15.94:0/2969563727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588106f20 0x7f758810eaa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7580009be0 tx=0x7f758002a6c0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.604+0000 7f758f3df640 1 -- 172.21.15.94:0/2969563727 shutdown_connections 2024-09-19T04:26:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.604+0000 7f758f3df640 1 --2- 172.21.15.94:0/2969563727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588106f20 0x7f758810eaa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.604+0000 7f758f3df640 1 --2- 172.21.15.94:0/2969563727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75881027a0 0x7f75881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.604+0000 7f758f3df640 1 --2- 172.21.15.94:0/2969563727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7588101df0 0x7f75881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.604+0000 7f758f3df640 1 -- 172.21.15.94:0/2969563727 >> 172.21.15.94:0/2969563727 conn(0x7f75880fd8b0 msgr2=0x7f75880ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.604+0000 7f758f3df640 1 -- 172.21.15.94:0/2969563727 shutdown_connections 2024-09-19T04:26:58.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.604+0000 7f758f3df640 1 -- 172.21.15.94:0/2969563727 wait complete. 2024-09-19T04:26:58.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.605+0000 7f758f3df640 1 Processor -- start 2024-09-19T04:26:58.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.605+0000 7f758f3df640 1 -- start start 2024-09-19T04:26:58.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.605+0000 7f758f3df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588101df0 0x7f7588198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.605+0000 7f758f3df640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75881027a0 0x7f75881988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.605+0000 7f758f3df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7588106f20 0x7f758819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.605+0000 7f758f3df640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7588111e30 con 0x7f7588106f20 2024-09-19T04:26:58.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.605+0000 7f758f3df640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7588111cb0 con 0x7f75881027a0 2024-09-19T04:26:58.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.605+0000 7f758f3df640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7588111fb0 con 0x7f7588101df0 2024-09-19T04:26:58.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d97c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7588106f20 0x7f758819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d17b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588101df0 0x7f7588198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d17b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588101df0 0x7f7588198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:42120/0 (socket says 172.21.15.94:42120) 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d97c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7588106f20 0x7f758819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47698/0 (socket says 172.21.15.94:47698) 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d17b640 1 -- 172.21.15.94:0/3117507863 learned_addr learned my addr 172.21.15.94:0/3117507863 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758c97a640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75881027a0 0x7f75881988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d17b640 1 -- 172.21.15.94:0/3117507863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75881027a0 msgr2=0x7f75881988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d17b640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75881027a0 0x7f75881988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d17b640 1 -- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7588106f20 msgr2=0x7f758819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d17b640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7588106f20 0x7f758819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d17b640 1 -- 172.21.15.94:0/3117507863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758819d320 con 0x7f7588101df0 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d97c640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7588106f20 0x7f758819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:58.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758d17b640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588101df0 0x7f7588198370 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9f0 tx=0x7f758400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.606+0000 7f758c97a640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75881027a0 0x7f75881988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:26:58.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.607+0000 7f757e7fc640 1 -- 172.21.15.94:0/3117507863 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758400cd80 con 0x7f7588101df0 2024-09-19T04:26:58.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.607+0000 7f757e7fc640 1 -- 172.21.15.94:0/3117507863 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75840108e0 con 0x7f7588101df0 2024-09-19T04:26:58.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.607+0000 7f758f3df640 1 -- 172.21.15.94:0/3117507863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758819d610 con 0x7f7588101df0 2024-09-19T04:26:58.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.607+0000 7f757e7fc640 1 -- 172.21.15.94:0/3117507863 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75840188e0 con 0x7f7588101df0 2024-09-19T04:26:58.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.607+0000 7f758f3df640 1 -- 172.21.15.94:0/3117507863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7588074420 con 0x7f7588101df0 2024-09-19T04:26:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.611+0000 7f757e7fc640 1 -- 172.21.15.94:0/3117507863 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7584010a90 con 0x7f7588101df0 2024-09-19T04:26:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.611+0000 7f75917b8740 1 Processor -- start 2024-09-19T04:26:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.611+0000 7f75917b8740 1 -- start start 2024-09-19T04:26:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f75917b8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df98644a0 0x558df9864880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f75917b8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df9864dc0 0x558df986f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f75917b8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9871a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f75917b8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558df97ca2e0 con 0x558df98644a0 2024-09-19T04:26:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f75917b8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558df9771ce0 con 0x558df9864dc0 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f75917b8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558df97c62a0 con 0x558df986f680 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f758c97a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9871a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f758c97a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9871a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:42124/0 (socket says 172.21.15.94:42124) 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f758c97a640 1 -- 172.21.15.94:0/3856433368 learned_addr learned my addr 172.21.15.94:0/3856433368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f758d17b640 1 --2- 172.21.15.94:0/3856433368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df9864dc0 0x558df986f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f758d97c640 1 --2- 172.21.15.94:0/3856433368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df98644a0 0x558df9864880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f758c97a640 1 -- 172.21.15.94:0/3856433368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df9864dc0 msgr2=0x558df986f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f758c97a640 1 --2- 172.21.15.94:0/3856433368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df9864dc0 0x558df986f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f758c97a640 1 -- 172.21.15.94:0/3856433368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df98644a0 msgr2=0x558df9864880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.612+0000 7f758c97a640 1 --2- 172.21.15.94:0/3856433368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df98644a0 0x558df9864880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.613+0000 7f758c97a640 1 -- 172.21.15.94:0/3856433368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df9756440 con 0x558df986f680 2024-09-19T04:26:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.613+0000 7f757e7fc640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f75740a34b0 0x7f75740a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.613+0000 7f757e7fc640 1 -- 172.21.15.94:0/3117507863 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f75840d02a0 con 0x7f7588101df0 2024-09-19T04:26:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.613+0000 7f758c97a640 1 --2- 172.21.15.94:0/3856433368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9871a70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7578009be0 tx=0x7f757800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec566434866781 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.613+0000 7f758d97c640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f75740a34b0 0x7f75740a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.613+0000 7f75737fe640 1 -- 172.21.15.94:0/3856433368 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578005020 con 0x558df986f680 2024-09-19T04:26:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.613+0000 7f75737fe640 1 -- 172.21.15.94:0/3856433368 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75780047a0 con 0x558df986f680 2024-09-19T04:26:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.613+0000 7f75737fe640 1 -- 172.21.15.94:0/3856433368 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75780116a0 con 0x558df986f680 2024-09-19T04:26:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.614+0000 7f75917b8740 1 -- 172.21.15.94:0/3856433368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 msgr2=0x558df9871a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.614+0000 7f75917b8740 1 --2- 172.21.15.94:0/3856433368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9871a70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7578009be0 tx=0x7f757800f730 comp rx=0 tx=0).stop 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.614+0000 7f758d97c640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f75740a34b0 0x7f75740a5970 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7580004af0 tx=0x7f7580005e50 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.614+0000 7f75917b8740 1 -- 172.21.15.94:0/3856433368 shutdown_connections 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.614+0000 7f75917b8740 1 --2- 172.21.15.94:0/3856433368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9871a70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.614+0000 7f75917b8740 1 --2- 172.21.15.94:0/3856433368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df9864dc0 0x558df986f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.614+0000 7f75917b8740 1 --2- 172.21.15.94:0/3856433368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df98644a0 0x558df9864880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.614+0000 7f75917b8740 1 -- 172.21.15.94:0/3856433368 >> 172.21.15.94:0/3856433368 conn(0x558df9860a30 msgr2=0x558df9860e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.615+0000 7f75917b8740 1 -- 172.21.15.94:0/3856433368 shutdown_connections 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.615+0000 7f75917b8740 1 -- 172.21.15.94:0/3856433368 wait complete. 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.615+0000 7f75917b8740 1 -- start start 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.615+0000 7f75917b8740 1 Processor -- start 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f75917b8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df986eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f75917b8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df9864dc0 0x558df986c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f75917b8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df986c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f75917b8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558df9872250 con 0x558df9864dc0 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f75917b8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558df9785050 con 0x558df98644a0 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f758d97c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df986c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f758d97c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df986c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:42130/0 (socket says 172.21.15.94:42130) 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f758d97c640 1 -- 172.21.15.94:0/2710295932 learned_addr learned my addr 172.21.15.94:0/2710295932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f75917b8740 1 -- 172.21.15.94:0/2710295932 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558df98723d0 con 0x558df986f680 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.616+0000 7f758d17b640 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df986eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.617+0000 7f75917b8740 10 client.0 _ll_register_callbacks cb 0x558df97c5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.617+0000 7f758d97c640 1 -- 172.21.15.94:0/2710295932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 msgr2=0x558df986eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.617+0000 7f758d97c640 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df986eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.617+0000 7f758d97c640 1 -- 172.21.15.94:0/2710295932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df9864dc0 msgr2=0x558df986c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.617+0000 7f758d97c640 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df9864dc0 0x558df986c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.617+0000 7f758d97c640 1 -- 172.21.15.94:0/2710295932 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df953b3c0 con 0x558df986f680 2024-09-19T04:26:58.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.617+0000 7f758d97c640 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df986c800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7580006010 tx=0x7f7580009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25d298acb3011175 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.617+0000 7f7570ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.617+0000 7f7570ff9640 1 -- 172.21.15.94:0/2710295932 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580036040 con 0x558df986f680 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.618+0000 7f7570ff9640 1 -- 172.21.15.94:0/2710295932 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f7580045a70 con 0x558df986f680 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.618+0000 7f75917b8740 1 -- 172.21.15.94:0/2710295932 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75780047a0 con 0x558df986f680 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.618+0000 7f7570ff9640 1 -- 172.21.15.94:0/2710295932 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7580004040 con 0x558df986f680 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.618+0000 7f75917b8740 10 client.26455 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.618+0000 7f75917b8740 1 -- 172.21.15.94:0/2710295932 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558df987dc70 con 0x558df986f680 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.618+0000 7f7570ff9640 1 -- 172.21.15.94:0/2710295932 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7580054070 con 0x558df986f680 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.618+0000 7f7570ff9640 1 -- 172.21.15.94:0/2710295932 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f758004c030 con 0x558df986f680 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.619+0000 7f75917b8740 10 client.26455 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.619+0000 7f75917b8740 10 client.26455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.619+0000 7f75917b8740 20 client.26455 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.619+0000 7f75917b8740 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558df987f180 0x558df9881640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.619+0000 7f75917b8740 4 client.26455 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:58.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.619+0000 7f75917b8740 1 -- 172.21.15.94:0/2710295932 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f75780047a0 con 0x558df987f180 2024-09-19T04:26:58.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.619+0000 7f758d17b640 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558df987f180 0x558df9881640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.620+0000 7f7570ff9640 10 client.26455 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:58.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.621+0000 7f758d17b640 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558df987f180 0x558df9881640 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4e90abacbe356f server_cookie=9ff043062dc222fb in_seq=0 out_seq=0 2024-09-19T04:26:58.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f7570ff9640 1 -- 172.21.15.94:0/2710295932 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f75780047a0 con 0x558df987f180 2024-09-19T04:26:58.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f7570ff9640 10 client.26455 handle_command_reply: tid=0 2024-09-19T04:26:58.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f75917b8740 1 client.26455 shutdown 2024-09-19T04:26:58.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f75917b8740 1 -- 172.21.15.94:0/2710295932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 msgr2=0x558df986c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f75917b8740 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df986c800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7580006010 tx=0x7f7580009500 comp rx=0 tx=0).stop 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f75917b8740 1 -- 172.21.15.94:0/2710295932 shutdown_connections 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f75917b8740 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558df987f180 0x558df9881640 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f7570ff9640 0 client.26455 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f75917b8740 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df986c800 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f75917b8740 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df9864dc0 0x558df986c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f75917b8740 1 --2- 172.21.15.94:0/2710295932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df986eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.622+0000 7f75917b8740 1 -- 172.21.15.94:0/2710295932 >> 172.21.15.94:0/2710295932 conn(0x558df9860a30 msgr2=0x558df9866760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.623+0000 7f75917b8740 1 -- 172.21.15.94:0/2710295932 shutdown_connections 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.623+0000 7f75917b8740 1 -- 172.21.15.94:0/2710295932 wait complete. 2024-09-19T04:26:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.623+0000 7f75917b8740 20 client.26455 trim_cache size 0 max 16384 2024-09-19T04:26:58.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.652+0000 7f75917b8740 1 Processor -- start 2024-09-19T04:26:58.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.652+0000 7f75917b8740 1 -- start start 2024-09-19T04:26:58.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.652+0000 7f75917b8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df986c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.652+0000 7f75917b8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df9864dc0 0x558df986e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.652+0000 7f75917b8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df986f680 0x558df9886f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f75917b8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f75881ae150 con 0x558df986f680 2024-09-19T04:26:58.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f75917b8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75881ab960 con 0x558df98644a0 2024-09-19T04:26:58.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f75917b8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75881a97d0 con 0x558df9864dc0 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758d97c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df986f680 0x558df9886f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758d97c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df986f680 0x558df9886f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47734/0 (socket says 172.21.15.94:47734) 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758d97c640 1 -- 172.21.15.94:0/3979421877 learned_addr learned my addr 172.21.15.94:0/3979421877 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758c97a640 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df9864dc0 0x558df986e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758d17b640 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df986c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758d97c640 1 -- 172.21.15.94:0/3979421877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df9864dc0 msgr2=0x558df986e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758d97c640 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df9864dc0 0x558df986e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758d97c640 1 -- 172.21.15.94:0/3979421877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 msgr2=0x558df986c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758d97c640 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df986c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.653+0000 7f758d97c640 1 -- 172.21.15.94:0/3979421877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df986f360 con 0x558df986f680 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.654+0000 7f758d97c640 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df986f680 0x558df9886f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f758002abe0 tx=0x7f7580009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e41e712d87ac27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.654+0000 7f758d17b640 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df986c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:58.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.654+0000 7f7572ffd640 1 -- 172.21.15.94:0/3979421877 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580036040 con 0x558df986f680 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.654+0000 7f7572ffd640 1 -- 172.21.15.94:0/3979421877 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7580054030 con 0x558df986f680 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.654+0000 7f7572ffd640 1 -- 172.21.15.94:0/3979421877 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580031440 con 0x558df986f680 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.655+0000 7f75917b8740 1 -- 172.21.15.94:0/3979421877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df986f680 msgr2=0x558df9886f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.655+0000 7f75917b8740 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df986f680 0x558df9886f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f758002abe0 tx=0x7f7580009ae0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.655+0000 7f75917b8740 1 -- 172.21.15.94:0/3979421877 shutdown_connections 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.655+0000 7f75917b8740 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df986f680 0x558df9886f50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.655+0000 7f75917b8740 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df9864dc0 0x558df986e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.655+0000 7f75917b8740 1 --2- 172.21.15.94:0/3979421877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df986c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.655+0000 7f75917b8740 1 -- 172.21.15.94:0/3979421877 >> 172.21.15.94:0/3979421877 conn(0x558df9860a30 msgr2=0x558df986db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.655+0000 7f75917b8740 1 -- 172.21.15.94:0/3979421877 shutdown_connections 2024-09-19T04:26:58.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.655+0000 7f75917b8740 1 -- 172.21.15.94:0/3979421877 wait complete. 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f75917b8740 1 -- start start 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f75917b8740 1 Processor -- start 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f75917b8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df9875580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f75917b8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df9864dc0 0x558df9875ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f75917b8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9876000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f75917b8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f75881a9e20 con 0x558df9864dc0 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f75917b8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75881ab290 con 0x558df98644a0 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f75917b8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75881aa4f0 con 0x558df986f680 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f75917b8740 10 client.0 _ll_register_callbacks cb 0x7f7584005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:58.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f758d17b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df9875580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f758d97c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9876000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f758d17b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df9875580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:50124/0 (socket says 172.21.15.94:50124) 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f758d17b640 1 -- 172.21.15.94:0/1021571526 learned_addr learned my addr 172.21.15.94:0/1021571526 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.656+0000 7f758c97a640 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df9864dc0 0x558df9875ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f758d17b640 1 -- 172.21.15.94:0/1021571526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 msgr2=0x558df9876000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f758d17b640 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9876000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f758d17b640 1 -- 172.21.15.94:0/1021571526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df9864dc0 msgr2=0x558df9875ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f758d17b640 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df9864dc0 0x558df9875ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f758d17b640 1 -- 172.21.15.94:0/1021571526 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df98e9130 con 0x558df98644a0 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f758d17b640 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df9875580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7584098940 tx=0x7f758409bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c57fd0bd7f9ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f7570ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f7570ff9640 1 -- 172.21.15.94:0/1021571526 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758409cbe0 con 0x558df98644a0 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f7570ff9640 1 -- 172.21.15.94:0/1021571526 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8585+0+0 (secure 0 0 0) 0x7f75840cdbd0 con 0x558df98644a0 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f75917b8740 1 -- 172.21.15.94:0/1021571526 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7580054030 con 0x558df98644a0 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f7570ff9640 1 -- 172.21.15.94:0/1021571526 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f758409c4e0 con 0x558df98644a0 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f75917b8740 10 client.26477 fetch_fsmap learned FSMap version 48 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f75917b8740 1 -- 172.21.15.94:0/1021571526 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558df98ea460 con 0x558df98644a0 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.657+0000 7f7570ff9640 1 -- 172.21.15.94:0/1021571526 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7580054030 con 0x558df98644a0 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f7570ff9640 1 -- 172.21.15.94:0/1021571526 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f75840a4cc0 con 0x558df98644a0 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f75917b8740 10 client.26477 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f75917b8740 10 client.26477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f75917b8740 20 client.26477 populate_metadata read hostname 'smithi094' 2024-09-19T04:26:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f75917b8740 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558df987f180 0x558df98ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f75917b8740 4 client.26477 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:26:58.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f75917b8740 1 -- 172.21.15.94:0/1021571526 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558df97d7850 con 0x558df987f180 2024-09-19T04:26:58.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f758c97a640 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558df987f180 0x558df98ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f758c97a640 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558df987f180 0x558df98ecb70 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26bd1adaf7687f57 server_cookie=8aa0516f1c05b0d3 in_seq=0 out_seq=0 2024-09-19T04:26:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.658+0000 7f7570ff9640 10 client.26477 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f7570ff9640 1 -- 172.21.15.94:0/1021571526 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558df97d7850 con 0x558df987f180 2024-09-19T04:26:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f7570ff9640 10 client.26477 handle_command_reply: tid=0 2024-09-19T04:26:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 client.26477 shutdown 2024-09-19T04:26:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 -- 172.21.15.94:0/1021571526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 msgr2=0x558df9875580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df9875580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7584098940 tx=0x7f758409bf00 comp rx=0 tx=0).stop 2024-09-19T04:26:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 -- 172.21.15.94:0/1021571526 shutdown_connections 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558df987f180 0x558df98ecb70 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558df986f680 0x558df9876000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df9864dc0 0x558df9875ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f7570ff9640 0 client.26477 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 --2- 172.21.15.94:0/1021571526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558df98644a0 0x558df9875580 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 -- 172.21.15.94:0/1021571526 >> 172.21.15.94:0/1021571526 conn(0x558df9860a30 msgr2=0x558df986e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 -- 172.21.15.94:0/1021571526 shutdown_connections 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 1 -- 172.21.15.94:0/1021571526 wait complete. 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.659+0000 7f75917b8740 20 client.26477 trim_cache size 0 max 16384 2024-09-19T04:26:58.666 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:26:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:26:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:26:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "997d4d61-d3a3-437c-b603-744fe26fb10f": { 2024-09-19T04:26:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:26:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "997d4d61-d3a3-437c-b603-744fe26fb10f", 2024-09-19T04:26:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:26:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:26:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:26:58.667 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 -- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f75740a34b0 msgr2=0x7f75740a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f75740a34b0 0x7f75740a5970 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7580004af0 tx=0x7f7580005e50 comp rx=0 tx=0).stop 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 -- 172.21.15.94:0/3117507863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588101df0 msgr2=0x7f7588198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588101df0 0x7f7588198370 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9f0 tx=0x7f758400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 -- 172.21.15.94:0/3117507863 shutdown_connections 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f75740a34b0 0x7f75740a5970 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7588106f20 0x7f758819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75881027a0 0x7f75881988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 --2- 172.21.15.94:0/3117507863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7588101df0 0x7f7588198370 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 -- 172.21.15.94:0/3117507863 >> 172.21.15.94:0/3117507863 conn(0x7f75880fd8b0 msgr2=0x7f75880febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 -- 172.21.15.94:0/3117507863 shutdown_connections 2024-09-19T04:26:58.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:26:58.660+0000 7f758f3df640 1 -- 172.21.15.94:0/3117507863 wait complete. 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:58 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:26:58.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:26:58.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:26:58.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:26:58.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:26:58.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:26:58.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.842 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:58 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:26:58.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:26:58.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:26:58.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:26:58.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:26:58.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:26:58.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:58.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:58 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:59.740 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:26:59 smithi094 ceph-mon[30972]: pgmap v371: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 5.1 MiB/s wr, 640 op/s 2024-09-19T04:26:59.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:26:59 smithi112 ceph-mon[35051]: pgmap v371: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 5.1 MiB/s wr, 640 op/s 2024-09-19T04:26:59.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:26:59 smithi104 ceph-mon[34145]: pgmap v371: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 5.1 MiB/s wr, 640 op/s 2024-09-19T04:27:00.079 INFO:tasks.cephfs.filesystem:scrub status for tag:997d4d61-d3a3-437c-b603-744fe26fb10f - {'path': '/', 'tag': '997d4d61-d3a3-437c-b603-744fe26fb10f', 'options': 'recursive,force'} 2024-09-19T04:27:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 645.0 s, 320.6 tps, lat 311.165 ms stddev 62.484 2024-09-19T04:27:01.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:01 smithi112 ceph-mon[35051]: pgmap v372: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 4.4 MiB/s wr, 542 op/s 2024-09-19T04:27:01.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:01 smithi094 ceph-mon[30972]: pgmap v372: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 4.4 MiB/s wr, 542 op/s 2024-09-19T04:27:01.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:01 smithi104 ceph-mon[34145]: pgmap v372: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 4.4 MiB/s wr, 542 op/s 2024-09-19T04:27:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:03 smithi112 ceph-mon[35051]: pgmap v373: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.8 MiB/s wr, 469 op/s 2024-09-19T04:27:03.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:03 smithi094 ceph-mon[30972]: pgmap v373: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.8 MiB/s wr, 469 op/s 2024-09-19T04:27:03.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:03 smithi104 ceph-mon[34145]: pgmap v373: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.8 MiB/s wr, 469 op/s 2024-09-19T04:27:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 650.0 s, 360.6 tps, lat 280.623 ms stddev 52.835 2024-09-19T04:27:05.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:05 smithi112 ceph-mon[35051]: pgmap v374: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 6.2 MiB/s wr, 779 op/s 2024-09-19T04:27:05.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:05 smithi094 ceph-mon[30972]: pgmap v374: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 6.2 MiB/s wr, 779 op/s 2024-09-19T04:27:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:05 smithi104 ceph-mon[34145]: pgmap v374: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 6.2 MiB/s wr, 779 op/s 2024-09-19T04:27:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:07 smithi112 ceph-mon[35051]: pgmap v375: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 4.3 MiB/s wr, 551 op/s 2024-09-19T04:27:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:07 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:27:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:07 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:27:07.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:07 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:27:07.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:07 smithi094 ceph-mon[30972]: pgmap v375: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 4.3 MiB/s wr, 551 op/s 2024-09-19T04:27:07.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:07 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:27:07.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:07 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:27:07.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:07 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:27:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:07 smithi104 ceph-mon[34145]: pgmap v375: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 4.3 MiB/s wr, 551 op/s 2024-09-19T04:27:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:07 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:27:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:07 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:27:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:07 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:27:08.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:08 smithi112 ceph-mon[35051]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:27:08.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:08 smithi094 ceph-mon[30972]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:27:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:08 smithi104 ceph-mon[34145]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:27:09.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:09 smithi112 ceph-mon[35051]: pgmap v377: 97 pgs: 19 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 65 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 4.3 MiB/s wr, 599 op/s 2024-09-19T04:27:09.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:09 smithi094 ceph-mon[30972]: pgmap v377: 97 pgs: 19 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 65 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 4.3 MiB/s wr, 599 op/s 2024-09-19T04:27:09.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:09 smithi104 ceph-mon[34145]: pgmap v377: 97 pgs: 19 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 65 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 4.3 MiB/s wr, 599 op/s 2024-09-19T04:27:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 655.0 s, 338.0 tps, lat 294.173 ms stddev 72.353 2024-09-19T04:27:11.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:11 smithi112 ceph-mon[35051]: pgmap v378: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-09-19T04:27:11.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:11 smithi094 ceph-mon[30972]: pgmap v378: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-09-19T04:27:11.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:11 smithi104 ceph-mon[34145]: pgmap v378: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-09-19T04:27:13.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:13 smithi112 ceph-mon[35051]: pgmap v379: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-09-19T04:27:13.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:13 smithi094 ceph-mon[30972]: pgmap v379: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-09-19T04:27:13.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:13 smithi104 ceph-mon[34145]: pgmap v379: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-09-19T04:27:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 660.0 s, 362.4 tps, lat 277.591 ms stddev 48.138 2024-09-19T04:27:15.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:15 smithi112 ceph-mon[35051]: pgmap v380: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 4.9 MiB/s wr, 636 op/s 2024-09-19T04:27:15.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:15 smithi094 ceph-mon[30972]: pgmap v380: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 4.9 MiB/s wr, 636 op/s 2024-09-19T04:27:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:15 smithi104 ceph-mon[34145]: pgmap v380: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 4.9 MiB/s wr, 636 op/s 2024-09-19T04:27:17.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:16 smithi112 ceph-mon[35051]: pgmap v381: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 4.9 MiB/s wr, 636 op/s 2024-09-19T04:27:17.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:16 smithi112 ceph-mon[35051]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:27:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:16 smithi094 ceph-mon[30972]: pgmap v381: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 4.9 MiB/s wr, 636 op/s 2024-09-19T04:27:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:16 smithi094 ceph-mon[30972]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:27:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:16 smithi104 ceph-mon[34145]: pgmap v381: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 4.9 MiB/s wr, 636 op/s 2024-09-19T04:27:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:16 smithi104 ceph-mon[34145]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:27:19.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:18 smithi112 ceph-mon[35051]: pgmap v383: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 6.0 MiB/s wr, 723 op/s 2024-09-19T04:27:19.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:18 smithi094 ceph-mon[30972]: pgmap v383: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 6.0 MiB/s wr, 723 op/s 2024-09-19T04:27:19.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:18 smithi104 ceph-mon[34145]: pgmap v383: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 6.0 MiB/s wr, 723 op/s 2024-09-19T04:27:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 665.0 s, 380.4 tps, lat 261.116 ms stddev 34.455 2024-09-19T04:27:21.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:20 smithi112 ceph-mon[35051]: pgmap v384: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 5.2 MiB/s wr, 647 op/s 2024-09-19T04:27:21.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:20 smithi094 ceph-mon[30972]: pgmap v384: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 5.2 MiB/s wr, 647 op/s 2024-09-19T04:27:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:20 smithi104 ceph-mon[34145]: pgmap v384: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 5.2 MiB/s wr, 647 op/s 2024-09-19T04:27:23.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:22 smithi112 ceph-mon[35051]: pgmap v385: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 5.2 MiB/s wr, 647 op/s 2024-09-19T04:27:23.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:22 smithi094 ceph-mon[30972]: pgmap v385: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 5.2 MiB/s wr, 647 op/s 2024-09-19T04:27:23.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:22 smithi104 ceph-mon[34145]: pgmap v385: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 5.2 MiB/s wr, 647 op/s 2024-09-19T04:27:25.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:24 smithi112 ceph-mon[35051]: pgmap v386: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 5.2 MiB/s wr, 649 op/s 2024-09-19T04:27:25.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:24 smithi094 ceph-mon[30972]: pgmap v386: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 5.2 MiB/s wr, 649 op/s 2024-09-19T04:27:25.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:24 smithi104 ceph-mon[34145]: pgmap v386: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 5.2 MiB/s wr, 649 op/s 2024-09-19T04:27:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 670.0 s, 354.2 tps, lat 282.903 ms stddev 46.846 2024-09-19T04:27:27.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:26 smithi112 ceph-mon[35051]: pgmap v387: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 5.2 MiB/s wr, 649 op/s 2024-09-19T04:27:27.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:26 smithi094 ceph-mon[30972]: pgmap v387: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 5.2 MiB/s wr, 649 op/s 2024-09-19T04:27:27.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:26 smithi104 ceph-mon[34145]: pgmap v387: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 5.2 MiB/s wr, 649 op/s 2024-09-19T04:27:29.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:28 smithi112 ceph-mon[35051]: pgmap v388: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 5.0 MiB/s wr, 664 op/s 2024-09-19T04:27:29.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:28 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:27:29.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:28 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:29.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:28 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:29.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:28 smithi094 ceph-mon[30972]: pgmap v388: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 5.0 MiB/s wr, 664 op/s 2024-09-19T04:27:29.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:28 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:27:29.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:28 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:29.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:28 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:28 smithi104 ceph-mon[34145]: pgmap v388: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 5.0 MiB/s wr, 664 op/s 2024-09-19T04:27:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:28 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:27:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:28 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:28 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:29 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:29 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:29 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:29 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:29 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:27:30.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:29 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:27:30.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:29 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:27:30.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:29 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:29 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:29 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:29 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:29 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:27:30.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:29 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:27:30.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:29 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:29 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:29 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:29 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:29 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:29 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:27:30.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:29 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:27:30.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:29 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:27:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 675.0 s, 366.8 tps, lat 272.608 ms stddev 41.416 2024-09-19T04:27:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.919+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3167869537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 msgr2=0x7f4c4410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.919+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3167869537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 0x7f4c4410da40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009900 tx=0x7f4c3c02a480 comp rx=0 tx=0).stop 2024-09-19T04:27:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.919+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3167869537 shutdown_connections 2024-09-19T04:27:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.919+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3167869537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 0x7f4c4410da40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.919+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3167869537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c44102570 0x7f4c441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.919+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3167869537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c44100e20 0x7f4c44101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.919+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3167869537 >> 172.21.15.94:0/3167869537 conn(0x7f4c440fc7d0 msgr2=0x7f4c440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.920+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3167869537 shutdown_connections 2024-09-19T04:27:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.921+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3167869537 wait complete. 2024-09-19T04:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.921+0000 7f4c4aca9640 1 Processor -- start 2024-09-19T04:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.921+0000 7f4c4aca9640 1 -- start start 2024-09-19T04:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c4aca9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c44100e20 0x7f4c441992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c4aca9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c44102570 0x7f4c441997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c4aca9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 0x7f4c441a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c4aca9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c44110e30 con 0x7f4c44102ef0 2024-09-19T04:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c4aca9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c44110cb0 con 0x7f4c44102570 2024-09-19T04:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c4aca9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c44110fb0 con 0x7f4c44100e20 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c49246640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 0x7f4c441a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c3bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c44102570 0x7f4c441997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c49246640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 0x7f4c441a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47244/0 (socket says 172.21.15.94:47244) 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c49246640 1 -- 172.21.15.94:0/3576644946 learned_addr learned my addr 172.21.15.94:0/3576644946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c44100e20 0x7f4c441992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c3bfff640 1 -- 172.21.15.94:0/3576644946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c44100e20 msgr2=0x7f4c441992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c44100e20 0x7f4c441992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c3bfff640 1 -- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 msgr2=0x7f4c441a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 0x7f4c441a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c3bfff640 1 -- 172.21.15.94:0/3576644946 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c441a4860 con 0x7f4c44102570 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.922+0000 7f4c49246640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 0x7f4c441a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.923+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c44102570 0x7f4c441997e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400d6d0 tx=0x7f4c3400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.923+0000 7f4c39ffb640 1 -- 172.21.15.94:0/3576644946 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34017070 con 0x7f4c44102570 2024-09-19T04:27:30.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.923+0000 7f4c39ffb640 1 -- 172.21.15.94:0/3576644946 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c34009820 con 0x7f4c44102570 2024-09-19T04:27:30.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.923+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3576644946 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c441a4a30 con 0x7f4c44102570 2024-09-19T04:27:30.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.923+0000 7f4c39ffb640 1 -- 172.21.15.94:0/3576644946 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34004c40 con 0x7f4c44102570 2024-09-19T04:27:30.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.923+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3576644946 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c441a4f40 con 0x7f4c44102570 2024-09-19T04:27:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.925+0000 7f4c39ffb640 1 -- 172.21.15.94:0/3576644946 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4c340099d0 con 0x7f4c44102570 2024-09-19T04:27:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.925+0000 7f4c4d082740 1 Processor -- start 2024-09-19T04:27:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c4d082740 1 -- start start 2024-09-19T04:27:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a40834a0 0x5652a4083880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a408e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a408e680 0x5652a4090a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c4d082740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5652a3fe92e0 con 0x5652a4083dc0 2024-09-19T04:27:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c4d082740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5652a3f90ce0 con 0x5652a408e680 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c4d082740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5652a3fe52a0 con 0x5652a40834a0 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c48a45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a408e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c48a45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a408e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47252/0 (socket says 172.21.15.94:47252) 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c48a45640 1 -- 172.21.15.94:0/3960362672 learned_addr learned my addr 172.21.15.94:0/3960362672 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c39ffb640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4c2c0a33b0 0x7f4c2c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c49246640 1 --2- 172.21.15.94:0/3960362672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a40834a0 0x5652a4083880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c39ffb640 1 -- 172.21.15.94:0/3576644946 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f4c340cf790 con 0x7f4c44102570 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3960362672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a408e680 0x5652a4090a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c48a45640 1 -- 172.21.15.94:0/3960362672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a40834a0 msgr2=0x5652a4083880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c49246640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4c2c0a33b0 0x7f4c2c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3960362672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a40834a0 0x5652a4083880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c48a45640 1 -- 172.21.15.94:0/3960362672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a408e680 msgr2=0x5652a4090a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3960362672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a408e680 0x5652a4090a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.926+0000 7f4c48a45640 1 -- 172.21.15.94:0/3960362672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a3f75440 con 0x5652a4083dc0 2024-09-19T04:27:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.927+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3960362672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a408e140 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4c400039f0 tx=0x7f4c40003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adee96135de7307 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.927+0000 7f4c2affd640 1 -- 172.21.15.94:0/3960362672 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40004b20 con 0x5652a4083dc0 2024-09-19T04:27:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.927+0000 7f4c2affd640 1 -- 172.21.15.94:0/3960362672 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c40004cc0 con 0x5652a4083dc0 2024-09-19T04:27:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.927+0000 7f4c49246640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4c2c0a33b0 0x7f4c2c0a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c005c40 tx=0x7f4c3c02a480 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.927+0000 7f4c2affd640 1 -- 172.21.15.94:0/3960362672 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c400126b0 con 0x5652a4083dc0 2024-09-19T04:27:30.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.928+0000 7f4c4d082740 1 -- 172.21.15.94:0/3960362672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 msgr2=0x5652a408e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:30.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.928+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3960362672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a408e140 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4c400039f0 tx=0x7f4c40003eb0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.928+0000 7f4c4d082740 1 -- 172.21.15.94:0/3960362672 shutdown_connections 2024-09-19T04:27:30.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.928+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3960362672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a408e680 0x5652a4090a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.928+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3960362672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a408e140 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.928+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3960362672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a40834a0 0x5652a4083880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.928+0000 7f4c4d082740 1 -- 172.21.15.94:0/3960362672 >> 172.21.15.94:0/3960362672 conn(0x5652a407fa30 msgr2=0x5652a407fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.928+0000 7f4c4d082740 1 -- 172.21.15.94:0/3960362672 shutdown_connections 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.928+0000 7f4c4d082740 1 -- 172.21.15.94:0/3960362672 wait complete. 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.929+0000 7f4c4d082740 1 -- start start 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.929+0000 7f4c4d082740 1 Processor -- start 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.929+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a408de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.929+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.929+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a408b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.929+0000 7f4c4d082740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5652a40912f0 con 0x5652a40834a0 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.929+0000 7f4c4d082740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5652a3fa4050 con 0x5652a4083dc0 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.929+0000 7f4c4d082740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5652a4091470 con 0x5652a408e680 2024-09-19T04:27:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c3bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c48a45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a408de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c3bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:36866/0 (socket says 172.21.15.94:36866) 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c3bfff640 1 -- 172.21.15.94:0/3261870318 learned_addr learned my addr 172.21.15.94:0/3261870318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c49246640 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a408b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c4d082740 10 client.0 _ll_register_callbacks cb 0x5652a3fe4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c3bfff640 1 -- 172.21.15.94:0/3261870318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 msgr2=0x5652a408b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a408b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c3bfff640 1 -- 172.21.15.94:0/3261870318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 msgr2=0x5652a408de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a408de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c3bfff640 1 -- 172.21.15.94:0/3261870318 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a3d5a3c0 con 0x5652a4083dc0 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a408de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408b0b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4c3409ba70 tx=0x7f4c3409b4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec48b83f035143b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.930+0000 7f4c23fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c23fff640 1 -- 172.21.15.94:0/3261870318 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34009d40 con 0x5652a4083dc0 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c23fff640 1 -- 172.21.15.94:0/3261870318 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f4c340cd120 con 0x5652a4083dc0 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c4d082740 1 -- 172.21.15.94:0/3261870318 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c40004cc0 con 0x5652a4083dc0 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c23fff640 1 -- 172.21.15.94:0/3261870318 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4c340cd320 con 0x5652a4083dc0 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c4d082740 10 client.26489 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c4d082740 1 -- 172.21.15.94:0/3261870318 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5652a409cb90 con 0x5652a4083dc0 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c23fff640 1 -- 172.21.15.94:0/3261870318 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c3401e120 con 0x5652a4083dc0 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c23fff640 1 -- 172.21.15.94:0/3261870318 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f4c340ba2a0 con 0x5652a4083dc0 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c4d082740 10 client.26489 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c4d082740 10 client.26489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c4d082740 20 client.26489 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5652a409dfa0 0x5652a40a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c4d082740 4 client.26489 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:30.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.931+0000 7f4c4d082740 1 -- 172.21.15.94:0/3261870318 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c40004cc0 con 0x5652a409dfa0 2024-09-19T04:27:30.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.932+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5652a409dfa0 0x5652a40a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.932+0000 7f4c23fff640 10 client.26489 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.932+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5652a409dfa0 0x5652a40a03e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25a91e606d6ed353 server_cookie=8a36a187687560fc in_seq=0 out_seq=0 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c23fff640 1 -- 172.21.15.94:0/3261870318 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4c40004cc0 con 0x5652a409dfa0 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c23fff640 10 client.26489 handle_command_reply: tid=0 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 client.26489 shutdown 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 -- 172.21.15.94:0/3261870318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 msgr2=0x5652a408b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408b0b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4c3409ba70 tx=0x7f4c3409b4b0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 -- 172.21.15.94:0/3261870318 shutdown_connections 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5652a409dfa0 0x5652a40a03e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a408b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408b0b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3261870318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a408de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 -- 172.21.15.94:0/3261870318 >> 172.21.15.94:0/3261870318 conn(0x5652a407fa30 msgr2=0x5652a4085710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:30.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c23fff640 0 client.26489 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:30.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 -- 172.21.15.94:0/3261870318 shutdown_connections 2024-09-19T04:27:30.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 1 -- 172.21.15.94:0/3261870318 wait complete. 2024-09-19T04:27:30.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.933+0000 7f4c4d082740 20 client.26489 trim_cache size 0 max 16384 2024-09-19T04:27:30.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c4d082740 1 Processor -- start 2024-09-19T04:27:30.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c4d082740 1 -- start start 2024-09-19T04:27:30.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a40834a0 0x5652a40867f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a4089980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a40a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c4d082740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441adf70 con 0x5652a4083dc0 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c4d082740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441ab780 con 0x5652a40834a0 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c4d082740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441a9610 con 0x5652a408e680 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a4089980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c48a45640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a40834a0 0x5652a40867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a4089980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47274/0 (socket says 172.21.15.94:47274) 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c3bfff640 1 -- 172.21.15.94:0/1220595788 learned_addr learned my addr 172.21.15.94:0/1220595788 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.954+0000 7f4c49246640 1 --2- 172.21.15.94:0/1220595788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a40a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c3bfff640 1 -- 172.21.15.94:0/1220595788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 msgr2=0x5652a40a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:30.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/1220595788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a40a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c3bfff640 1 -- 172.21.15.94:0/1220595788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a40834a0 msgr2=0x5652a40867f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:30.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/1220595788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a40834a0 0x5652a40867f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c3bfff640 1 -- 172.21.15.94:0/1220595788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a408a060 con 0x5652a4083dc0 2024-09-19T04:27:30.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c48a45640 1 --2- 172.21.15.94:0/1220595788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a40834a0 0x5652a40867f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:30.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/1220595788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a4089980 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4c34099d90 tx=0x7f4c34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c911da12c1c3df95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c2a7fc640 1 -- 172.21.15.94:0/1220595788 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34097d80 con 0x5652a4083dc0 2024-09-19T04:27:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c2a7fc640 1 -- 172.21.15.94:0/1220595788 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c34016170 con 0x5652a4083dc0 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.955+0000 7f4c2a7fc640 1 -- 172.21.15.94:0/1220595788 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34097830 con 0x5652a4083dc0 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.956+0000 7f4c4d082740 1 -- 172.21.15.94:0/1220595788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 msgr2=0x5652a4089980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.956+0000 7f4c4d082740 1 --2- 172.21.15.94:0/1220595788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a4089980 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4c34099d90 tx=0x7f4c34000bc0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.956+0000 7f4c4d082740 1 -- 172.21.15.94:0/1220595788 shutdown_connections 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.956+0000 7f4c4d082740 1 --2- 172.21.15.94:0/1220595788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a40a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.956+0000 7f4c4d082740 1 --2- 172.21.15.94:0/1220595788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a4083dc0 0x5652a4089980 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.956+0000 7f4c4d082740 1 --2- 172.21.15.94:0/1220595788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a40834a0 0x5652a40867f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.956+0000 7f4c4d082740 1 -- 172.21.15.94:0/1220595788 >> 172.21.15.94:0/1220595788 conn(0x5652a407fa30 msgr2=0x5652a40860e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.956+0000 7f4c4d082740 1 -- 172.21.15.94:0/1220595788 shutdown_connections 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.956+0000 7f4c4d082740 1 -- 172.21.15.94:0/1220595788 wait complete. 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c4d082740 1 -- start start 2024-09-19T04:27:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c4d082740 1 Processor -- start 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a4097020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c4d082740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a408fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c4d082740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441a9c60 con 0x5652a40834a0 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c4d082740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441ab0b0 con 0x5652a4083dc0 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c4d082740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441aa310 con 0x5652a408e680 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c4d082740 10 client.0 _ll_register_callbacks cb 0x7f4c3401ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c48a45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a4097020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c48a45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a4097020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47280/0 (socket says 172.21.15.94:47280) 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c48a45640 1 -- 172.21.15.94:0/3996880624 learned_addr learned my addr 172.21.15.94:0/3996880624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.957+0000 7f4c49246640 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a408fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c48a45640 1 -- 172.21.15.94:0/3996880624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 msgr2=0x5652a408fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a408fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c48a45640 1 -- 172.21.15.94:0/3996880624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 msgr2=0x5652a408f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c48a45640 1 -- 172.21.15.94:0/3996880624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a4083bf0 con 0x5652a40834a0 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c48a45640 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a4097020 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4c40005b10 tx=0x7f4c40005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f88762e5202d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c23fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c23fff640 1 -- 172.21.15.94:0/3996880624 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40009ca0 con 0x5652a40834a0 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c23fff640 1 -- 172.21.15.94:0/3996880624 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f4c40012660 con 0x5652a40834a0 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c4d082740 1 -- 172.21.15.94:0/3996880624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c34016170 con 0x5652a40834a0 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c23fff640 1 -- 172.21.15.94:0/3996880624 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4c40004860 con 0x5652a40834a0 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c4d082740 10 client.16884 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.958+0000 7f4c4d082740 1 -- 172.21.15.94:0/3996880624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5652a4108d40 con 0x5652a40834a0 2024-09-19T04:27:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c23fff640 1 -- 172.21.15.94:0/3996880624 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c34016170 con 0x5652a40834a0 2024-09-19T04:27:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c23fff640 1 -- 172.21.15.94:0/3996880624 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f4c4000f040 con 0x5652a40834a0 2024-09-19T04:27:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c4d082740 10 client.16884 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c4d082740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c4d082740 20 client.16884 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5652a41092e0 0x5652a410b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c4d082740 4 client.16884 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:27:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c4d082740 1 -- 172.21.15.94:0/3996880624 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652a3ff6850 con 0x5652a41092e0 2024-09-19T04:27:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5652a41092e0 0x5652a410b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c23fff640 10 client.16884 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.959+0000 7f4c3bfff640 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5652a41092e0 0x5652a410b700 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61453572885cb69e server_cookie=1fa4a0b9e3d605e0 in_seq=0 out_seq=0 2024-09-19T04:27:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c23fff640 1 -- 172.21.15.94:0/3996880624 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5652a3ff6850 con 0x5652a41092e0 2024-09-19T04:27:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c23fff640 10 client.16884 handle_command_reply: tid=0 2024-09-19T04:27:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 client.16884 shutdown 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 -- 172.21.15.94:0/3996880624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 msgr2=0x5652a4097020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a4097020 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4c40005b10 tx=0x7f4c40005bd0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 -- 172.21.15.94:0/3996880624 shutdown_connections 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5652a41092e0 0x5652a410b700 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652a408e680 0x5652a408fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652a4083dc0 0x5652a408f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 --2- 172.21.15.94:0/3996880624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652a40834a0 0x5652a4097020 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 -- 172.21.15.94:0/3996880624 >> 172.21.15.94:0/3996880624 conn(0x5652a407fa30 msgr2=0x5652a408a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c23fff640 0 client.16884 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 -- 172.21.15.94:0/3996880624 shutdown_connections 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 1 -- 172.21.15.94:0/3996880624 wait complete. 2024-09-19T04:27:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.960+0000 7f4c4d082740 20 client.16884 trim_cache size 0 max 16384 2024-09-19T04:27:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:27:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:27:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:27:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:27:30.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4c2c0a33b0 msgr2=0x7f4c2c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:30.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4c2c0a33b0 0x7f4c2c0a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c005c40 tx=0x7f4c3c02a480 comp rx=0 tx=0).stop 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3576644946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c44102570 msgr2=0x7f4c441997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c44102570 0x7f4c441997e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400d6d0 tx=0x7f4c3400db90 comp rx=0 tx=0).stop 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3576644946 shutdown_connections 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f4c2c0a33b0 0x7f4c2c0a5870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c44102ef0 0x7f4c441a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c44102570 0x7f4c441997e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 --2- 172.21.15.94:0/3576644946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c44100e20 0x7f4c441992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3576644946 >> 172.21.15.94:0/3576644946 conn(0x7f4c440fc7d0 msgr2=0x7f4c440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3576644946 shutdown_connections 2024-09-19T04:27:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:30.961+0000 7f4c4aca9640 1 -- 172.21.15.94:0/3576644946 wait complete. 2024-09-19T04:27:31.555 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:31 smithi094 ceph-mon[30972]: pgmap v389: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 4.8 MiB/s wr, 619 op/s 2024-09-19T04:27:31.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:31 smithi104 ceph-mon[34145]: pgmap v389: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 4.8 MiB/s wr, 619 op/s 2024-09-19T04:27:31.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:31 smithi112 ceph-mon[35051]: pgmap v389: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 4.8 MiB/s wr, 619 op/s 2024-09-19T04:27:32.554 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:27:32.555 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:27:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.445+0000 7f8622b0f640 1 -- 172.21.15.94:0/751539444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f861c1027a0 msgr2=0x7f861c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.445+0000 7f8622b0f640 1 --2- 172.21.15.94:0/751539444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f861c1027a0 0x7f861c1067f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8608009900 tx=0x7f860802a410 comp rx=0 tx=0).stop 2024-09-19T04:27:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.445+0000 7f8622b0f640 1 -- 172.21.15.94:0/751539444 shutdown_connections 2024-09-19T04:27:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.445+0000 7f8622b0f640 1 --2- 172.21.15.94:0/751539444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f861c106f20 0x7f861c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.445+0000 7f8622b0f640 1 --2- 172.21.15.94:0/751539444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f861c1027a0 0x7f861c1067f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.445+0000 7f8622b0f640 1 --2- 172.21.15.94:0/751539444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f861c101df0 0x7f861c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.445+0000 7f8622b0f640 1 -- 172.21.15.94:0/751539444 >> 172.21.15.94:0/751539444 conn(0x7f861c0fd8f0 msgr2=0x7f861c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.445+0000 7f8622b0f640 1 -- 172.21.15.94:0/751539444 shutdown_connections 2024-09-19T04:27:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.445+0000 7f8622b0f640 1 -- 172.21.15.94:0/751539444 wait complete. 2024-09-19T04:27:33.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f8622b0f640 1 Processor -- start 2024-09-19T04:27:33.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f8622b0f640 1 -- start start 2024-09-19T04:27:33.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f8622b0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f861c101df0 0x7f861c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f8622b0f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f861c1027a0 0x7f861c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f8622b0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f861c106f20 0x7f861c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f8622b0f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f861c111e80 con 0x7f861c106f20 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f8622b0f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f861c111d00 con 0x7f861c1027a0 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f8622b0f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f861c112000 con 0x7f861c101df0 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f86210ac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f861c106f20 0x7f861c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f861bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f861c1027a0 0x7f861c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f861bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f861c1027a0 0x7f861c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33344/0 (socket says 172.21.15.94:33344) 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f861bfff640 1 -- 172.21.15.94:0/2083683652 learned_addr learned my addr 172.21.15.94:0/2083683652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.446+0000 7f86208ab640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f861c101df0 0x7f861c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.447+0000 7f86208ab640 1 -- 172.21.15.94:0/2083683652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f861c1027a0 msgr2=0x7f861c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.447+0000 7f86208ab640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f861c1027a0 0x7f861c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.447+0000 7f86208ab640 1 -- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f861c106f20 msgr2=0x7f861c19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.447+0000 7f86208ab640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f861c106f20 0x7f861c19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.447+0000 7f86208ab640 1 -- 172.21.15.94:0/2083683652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c19f510 con 0x7f861c101df0 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.447+0000 7f86210ac640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f861c106f20 0x7f861c19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.447+0000 7f86208ab640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f861c101df0 0x7f861c19a560 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f861000e9f0 tx=0x7f861000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.447+0000 7f8619ffb640 1 -- 172.21.15.94:0/2083683652 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861000ccc0 con 0x7f861c101df0 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.447+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c19f800 con 0x7f861c101df0 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.448+0000 7f8619ffb640 1 -- 172.21.15.94:0/2083683652 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86100108e0 con 0x7f861c101df0 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.448+0000 7f8619ffb640 1 -- 172.21.15.94:0/2083683652 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610018880 con 0x7f861c101df0 2024-09-19T04:27:33.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.448+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c1a70e0 con 0x7f861c101df0 2024-09-19T04:27:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.449+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c1021d0 con 0x7f861c101df0 2024-09-19T04:27:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.451+0000 7f8619ffb640 1 -- 172.21.15.94:0/2083683652 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f8610010a90 con 0x7f861c101df0 2024-09-19T04:27:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.451+0000 7f8619ffb640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f860c0a33b0 0x7f860c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.452+0000 7f8619ffb640 1 -- 172.21.15.94:0/2083683652 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f86100d04f0 con 0x7f861c101df0 2024-09-19T04:27:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.452+0000 7f861bfff640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f860c0a33b0 0x7f860c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:33.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.452+0000 7f861bfff640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f860c0a33b0 0x7f860c0a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f861c076480 tx=0x7f8608005a20 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:33.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.457+0000 7f8619ffb640 1 -- 172.21.15.94:0/2083683652 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f861009c840 con 0x7f861c101df0 2024-09-19T04:27:33.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:33 smithi094 ceph-mon[30972]: pgmap v390: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 4.1 MiB/s wr, 541 op/s 2024-09-19T04:27:33.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:33 smithi104 ceph-mon[34145]: pgmap v390: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 4.1 MiB/s wr, 541 op/s 2024-09-19T04:27:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.731+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f861c1a7310 con 0x7f861c101df0 2024-09-19T04:27:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.732+0000 7f8619ffb640 1 -- 172.21.15.94:0/2083683652 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11924 (secure 0 0 0) 0x7f86100a0b40 con 0x7f861c101df0 2024-09-19T04:27:33.734 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:27:33.734 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":48,"btime":"2024-09-19T04:26:33:395724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:26:32.789785+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:27:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 48 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f860c0a33b0 msgr2=0x7f860c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f860c0a33b0 0x7f860c0a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f861c076480 tx=0x7f8608005a20 comp rx=0 tx=0).stop 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f861c101df0 msgr2=0x7f861c19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f861c101df0 0x7f861c19a560 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f861000e9f0 tx=0x7f861000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 shutdown_connections 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f860c0a33b0 0x7f860c0a5870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f861c106f20 0x7f861c19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f861c1027a0 0x7f861c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 --2- 172.21.15.94:0/2083683652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f861c101df0 0x7f861c19a560 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 >> 172.21.15.94:0/2083683652 conn(0x7f861c0fd8f0 msgr2=0x7f861c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 shutdown_connections 2024-09-19T04:27:33.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:33.735+0000 7f8622b0f640 1 -- 172.21.15.94:0/2083683652 wait complete. 2024-09-19T04:27:33.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:33 smithi112 ceph-mon[35051]: pgmap v390: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 4.1 MiB/s wr, 541 op/s 2024-09-19T04:27:34.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:34 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2083683652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:27:34.421 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:27:34.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:34 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2083683652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:27:34.716 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:34.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:34 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2083683652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:27:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.234+0000 7f9801c38640 1 -- 172.21.15.94:0/103564601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc106f10 msgr2=0x7f97fc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:35.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.234+0000 7f9801c38640 1 --2- 172.21.15.94:0/103564601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc106f10 0x7f97fc10ea90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009ca0 tx=0x7f97f402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.235+0000 7f9801c38640 1 -- 172.21.15.94:0/103564601 shutdown_connections 2024-09-19T04:27:35.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.235+0000 7f9801c38640 1 --2- 172.21.15.94:0/103564601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc106f10 0x7f97fc10ea90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.235+0000 7f9801c38640 1 --2- 172.21.15.94:0/103564601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97fc102790 0x7f97fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.235+0000 7f9801c38640 1 --2- 172.21.15.94:0/103564601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc101de0 0x7f97fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.235+0000 7f9801c38640 1 -- 172.21.15.94:0/103564601 >> 172.21.15.94:0/103564601 conn(0x7f97fc0fd8c0 msgr2=0x7f97fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:35.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.235+0000 7f9801c38640 1 -- 172.21.15.94:0/103564601 shutdown_connections 2024-09-19T04:27:35.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.235+0000 7f9801c38640 1 -- 172.21.15.94:0/103564601 wait complete. 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.236+0000 7f9801c38640 1 Processor -- start 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.236+0000 7f9801c38640 1 -- start start 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.236+0000 7f9801c38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc101de0 0x7f97fc073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f9801c38640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97fc102790 0x7f97fc06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc101de0 0x7f97fc073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc101de0 0x7f97fc073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48392/0 (socket says 172.21.15.94:48392) 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fb7fe640 1 -- 172.21.15.94:0/2422641910 learned_addr learned my addr 172.21.15.94:0/2422641910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f9801c38640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc106f10 0x7f97fc06cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc111e20 con 0x7f97fc106f10 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc111ca0 con 0x7f97fc102790 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc111fa0 con 0x7f97fc101de0 2024-09-19T04:27:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97faffd640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97fc102790 0x7f97fc06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fbfff640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc106f10 0x7f97fc06cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fb7fe640 1 -- 172.21.15.94:0/2422641910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97fc102790 msgr2=0x7f97fc06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97fc102790 0x7f97fc06caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fb7fe640 1 -- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc106f10 msgr2=0x7f97fc06cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc106f10 0x7f97fc06cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fb7fe640 1 -- 172.21.15.94:0/2422641910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc06d5d0 con 0x7f97fc101de0 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.237+0000 7f97fbfff640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc106f10 0x7f97fc06cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.238+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc101de0 0x7f97fc073a10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00e9f0 tx=0x7f97ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.238+0000 7f97f8ff9640 1 -- 172.21.15.94:0/2422641910 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec00cd80 con 0x7f97fc101de0 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.238+0000 7f97f8ff9640 1 -- 172.21.15.94:0/2422641910 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97ec0108e0 con 0x7f97fc101de0 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.238+0000 7f97f8ff9640 1 -- 172.21.15.94:0/2422641910 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec018920 con 0x7f97fc101de0 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.238+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc06d8c0 con 0x7f97fc101de0 2024-09-19T04:27:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.238+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc071860 con 0x7f97fc101de0 2024-09-19T04:27:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.241+0000 7f97f8ff9640 1 -- 172.21.15.94:0/2422641910 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f97ec010a90 con 0x7f97fc101de0 2024-09-19T04:27:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.241+0000 7f9804011740 1 Processor -- start 2024-09-19T04:27:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.241+0000 7f9804011740 1 -- start start 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.241+0000 7f9804011740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a93090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.241+0000 7f9804011740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92a93a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.241+0000 7f9804011740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a9e370 0x557e92aa0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.241+0000 7f9804011740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e929f92f0 con 0x557e92a94b40 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.241+0000 7f9804011740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557e929a0cf0 con 0x557e92a935d0 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.241+0000 7f9804011740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e929f52b0 con 0x557e92a9e370 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97fb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92a93a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97fb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92a93a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33366/0 (socket says 172.21.15.94:33366) 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97fb7fe640 1 -- 172.21.15.94:0/3366250981 learned_addr learned my addr 172.21.15.94:0/3366250981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97faffd640 1 --2- 172.21.15.94:0/3366250981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a9e370 0x557e92aa0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97fbfff640 1 --2- 172.21.15.94:0/3366250981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a93090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97f8ff9640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f97e40a3430 0x7f97e40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97faffd640 1 -- 172.21.15.94:0/3366250981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 msgr2=0x557e92a93a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97faffd640 1 --2- 172.21.15.94:0/3366250981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92a93a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97faffd640 1 -- 172.21.15.94:0/3366250981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 msgr2=0x557e92a93090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97faffd640 1 --2- 172.21.15.94:0/3366250981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a93090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97faffd640 1 -- 172.21.15.94:0/3366250981 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e929853a0 con 0x557e92a9e370 2024-09-19T04:27:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97f8ff9640 1 -- 172.21.15.94:0/2422641910 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f97ec0d00e0 con 0x7f97fc101de0 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.242+0000 7f97faffd640 1 --2- 172.21.15.94:0/3366250981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a9e370 0x557e92aa0760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f97fc06e2f0 tx=0x7f97f0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789158673a424856 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.243+0000 7f97fbfff640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f97e40a3430 0x7f97e40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.243+0000 7f97e1ffb640 1 -- 172.21.15.94:0/3366250981 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0004050 con 0x557e92a9e370 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.243+0000 7f97e1ffb640 1 -- 172.21.15.94:0/3366250981 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97f00054d0 con 0x557e92a9e370 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.243+0000 7f97e1ffb640 1 -- 172.21.15.94:0/3366250981 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0004c00 con 0x557e92a9e370 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f9804011740 1 -- 172.21.15.94:0/3366250981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a9e370 msgr2=0x557e92aa0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f9804011740 1 --2- 172.21.15.94:0/3366250981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a9e370 0x557e92aa0760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f97fc06e2f0 tx=0x7f97f0011810 comp rx=0 tx=0).stop 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f97fbfff640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f97e40a3430 0x7f97e40a58f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009ca0 tx=0x7f97f40095c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f9804011740 1 -- 172.21.15.94:0/3366250981 shutdown_connections 2024-09-19T04:27:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f9804011740 1 --2- 172.21.15.94:0/3366250981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a9e370 0x557e92aa0760 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f9804011740 1 --2- 172.21.15.94:0/3366250981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92a93a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f9804011740 1 --2- 172.21.15.94:0/3366250981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a93090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f9804011740 1 -- 172.21.15.94:0/3366250981 >> 172.21.15.94:0/3366250981 conn(0x557e92a5c310 msgr2=0x557e92a8fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f9804011740 1 -- 172.21.15.94:0/3366250981 shutdown_connections 2024-09-19T04:27:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.244+0000 7f9804011740 1 -- 172.21.15.94:0/3366250981 wait complete. 2024-09-19T04:27:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.245+0000 7f9804011740 1 -- start start 2024-09-19T04:27:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.245+0000 7f9804011740 1 Processor -- start 2024-09-19T04:27:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.245+0000 7f9804011740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a935d0 0x557e92a9dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.245+0000 7f9804011740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a9af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f9804011740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92a9b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f9804011740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e92aa1070 con 0x557e92a94b40 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f9804011740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557e929b4060 con 0x557e92a9e370 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f9804011740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e92aa11f0 con 0x557e92a935d0 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97faffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a9af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a935d0 0x557e92a9dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f9804011740 10 client.0 _ll_register_callbacks cb 0x557e927078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97faffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a9af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45342/0 (socket says 172.21.15.94:45342) 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97faffd640 1 -- 172.21.15.94:0/2837334087 learned_addr learned my addr 172.21.15.94:0/2837334087 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97fbfff640 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92a9b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a935d0 0x557e92a9dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48408/0 (socket says 172.21.15.94:48408) 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97fbfff640 1 -- 172.21.15.94:0/2837334087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a935d0 msgr2=0x557e92a9dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97fbfff640 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a935d0 0x557e92a9dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97fbfff640 1 -- 172.21.15.94:0/2837334087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 msgr2=0x557e92a9af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97fbfff640 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a9af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97fbfff640 1 -- 172.21.15.94:0/2837334087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9276a3c0 con 0x557e92a9e370 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a935d0 0x557e92a9dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.246+0000 7f97faffd640 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a9af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.247+0000 7f97fbfff640 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92a9b4a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f97f40316d0 tx=0x7f97f402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d7611e7cf4c3b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.247+0000 7f97daffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:27:35.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.247+0000 7f97daffd640 1 -- 172.21.15.94:0/2837334087 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4004240 con 0x557e92a9e370 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.247+0000 7f97daffd640 1 -- 172.21.15.94:0/2837334087 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f97f40043e0 con 0x557e92a9e370 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.247+0000 7f9804011740 1 -- 172.21.15.94:0/2837334087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f00054d0 con 0x557e92a9e370 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.247+0000 7f97daffd640 1 -- 172.21.15.94:0/2837334087 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f97f40045e0 con 0x557e92a9e370 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.247+0000 7f9804011740 10 client.26513 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.247+0000 7f9804011740 1 -- 172.21.15.94:0/2837334087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557e92aaca60 con 0x557e92a9e370 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.247+0000 7f97daffd640 1 -- 172.21.15.94:0/2837334087 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97f00054d0 con 0x557e92a9e370 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.248+0000 7f97daffd640 1 -- 172.21.15.94:0/2837334087 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f97f4059470 con 0x557e92a9e370 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.248+0000 7f9804011740 10 client.26513 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.248+0000 7f9804011740 10 client.26513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.248+0000 7f9804011740 20 client.26513 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:35.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.248+0000 7f9804011740 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e92aade70 0x557e92ab0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.248+0000 7f9804011740 4 client.26513 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:35.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.248+0000 7f9804011740 1 -- 172.21.15.94:0/2837334087 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e92a06860 con 0x557e92aade70 2024-09-19T04:27:35.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.248+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e92aade70 0x557e92ab0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.249+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e92aade70 0x557e92ab0310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46342e49fec68e00 server_cookie=adf96d786d210dfa in_seq=0 out_seq=0 2024-09-19T04:27:35.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.249+0000 7f97daffd640 10 client.26513 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:27:35.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.249+0000 7f97daffd640 1 -- 172.21.15.94:0/2837334087 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e92a06860 con 0x557e92aade70 2024-09-19T04:27:35.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.249+0000 7f97daffd640 10 client.26513 handle_command_reply: tid=0 2024-09-19T04:27:35.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.249+0000 7f9804011740 1 client.26513 shutdown 2024-09-19T04:27:35.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 -- 172.21.15.94:0/2837334087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 msgr2=0x557e92a9b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:35.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92a9b4a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f97f40316d0 tx=0x7f97f402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 -- 172.21.15.94:0/2837334087 shutdown_connections 2024-09-19T04:27:35.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e92aade70 0x557e92ab0310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92a9b4a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f97daffd640 0 client.26513 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:27:35.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a94b40 0x557e92a9af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 --2- 172.21.15.94:0/2837334087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a935d0 0x557e92a9dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 -- 172.21.15.94:0/2837334087 >> 172.21.15.94:0/2837334087 conn(0x557e92a5c310 msgr2=0x557e92a95930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:35.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 -- 172.21.15.94:0/2837334087 shutdown_connections 2024-09-19T04:27:35.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.250+0000 7f9804011740 1 -- 172.21.15.94:0/2837334087 wait complete. 2024-09-19T04:27:35.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.251+0000 7f9804011740 20 client.26513 trim_cache size 0 max 16384 2024-09-19T04:27:35.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.268+0000 7f9804011740 1 Processor -- start 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.268+0000 7f9804011740 1 -- start start 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f9804011740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a935d0 0x557e92aaa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f9804011740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 0x557e92a99f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f9804011740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92ab5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f9804011740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1bac70 con 0x557e92a935d0 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f9804011740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1b8480 con 0x557e92a9e370 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f9804011740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1b62f0 con 0x557e92a94b40 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a935d0 0x557e92aaa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a935d0 0x557e92aaa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45348/0 (socket says 172.21.15.94:45348) 2024-09-19T04:27:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fb7fe640 1 -- 172.21.15.94:0/1488953893 learned_addr learned my addr 172.21.15.94:0/1488953893 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97faffd640 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 0x557e92a99f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fbfff640 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92ab5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fb7fe640 1 -- 172.21.15.94:0/1488953893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 msgr2=0x557e92a99f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 0x557e92a99f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fb7fe640 1 -- 172.21.15.94:0/1488953893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 msgr2=0x557e92ab5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92ab5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fb7fe640 1 -- 172.21.15.94:0/1488953893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e92a94300 con 0x557e92a935d0 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.269+0000 7f97fbfff640 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92ab5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.270+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a935d0 0x557e92aaa310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00c6a0 tx=0x7f97ec0d25a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd580febcb9d2da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.270+0000 7f97e17fa640 1 -- 172.21.15.94:0/1488953893 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec09be60 con 0x557e92a935d0 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.270+0000 7f97e17fa640 1 -- 172.21.15.94:0/1488953893 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97ec010de0 con 0x557e92a935d0 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.270+0000 7f97e17fa640 1 -- 172.21.15.94:0/1488953893 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec09bc80 con 0x557e92a935d0 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 -- 172.21.15.94:0/1488953893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a935d0 msgr2=0x557e92aaa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a935d0 0x557e92aaa310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00c6a0 tx=0x7f97ec0d25a0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 -- 172.21.15.94:0/1488953893 shutdown_connections 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a9e370 0x557e92ab5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 0x557e92a99f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 --2- 172.21.15.94:0/1488953893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a935d0 0x557e92aaa310 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 -- 172.21.15.94:0/1488953893 >> 172.21.15.94:0/1488953893 conn(0x557e92a5c310 msgr2=0x557e92a99830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 -- 172.21.15.94:0/1488953893 shutdown_connections 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 -- 172.21.15.94:0/1488953893 wait complete. 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.271+0000 7f9804011740 1 -- start start 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f9804011740 1 Processor -- start 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f9804011740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92aa19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f9804011740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 0x557e92aa1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f9804011740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a9e370 0x557e92b16be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f9804011740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1b6940 con 0x557e92a9e370 2024-09-19T04:27:35.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f9804011740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1b7db0 con 0x557e92a935d0 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f9804011740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1b7010 con 0x557e92a94b40 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f9804011740 10 client.0 _ll_register_callbacks cb 0x7f97f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f97fb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92aa19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f97fb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92aa19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33392/0 (socket says 172.21.15.94:33392) 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f97fb7fe640 1 -- 172.21.15.94:0/1068933203 learned_addr learned my addr 172.21.15.94:0/1068933203 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f97fbfff640 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a9e370 0x557e92b16be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.272+0000 7f97faffd640 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 0x557e92aa1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97fb7fe640 1 -- 172.21.15.94:0/1068933203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 msgr2=0x557e92aa1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 0x557e92aa1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97fb7fe640 1 -- 172.21.15.94:0/1068933203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a9e370 msgr2=0x557e92b16be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a9e370 0x557e92b16be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97fb7fe640 1 -- 172.21.15.94:0/1068933203 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e92b17120 con 0x557e92a935d0 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97faffd640 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 0x557e92aa1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97fbfff640 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a9e370 0x557e92b16be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97fb7fe640 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92aa19a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0a4a60 tx=0x7f97ec09bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33743abdd9a8ed00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97daffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97daffd640 1 -- 172.21.15.94:0/1068933203 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0a06d0 con 0x557e92a935d0 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f97daffd640 1 -- 172.21.15.94:0/1068933203 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f97ec0cda10 con 0x557e92a935d0 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.273+0000 7f9804011740 1 -- 172.21.15.94:0/1068933203 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e92a00d70 con 0x557e92a935d0 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f97daffd640 1 -- 172.21.15.94:0/1068933203 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f97ec0cdc10 con 0x557e92a935d0 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f9804011740 10 client.26519 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f9804011740 1 -- 172.21.15.94:0/1068933203 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557e92b18360 con 0x557e92a935d0 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f97daffd640 1 -- 172.21.15.94:0/1068933203 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x557e92a00d70 con 0x557e92a935d0 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f97daffd640 1 -- 172.21.15.94:0/1068933203 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f97ec09b770 con 0x557e92a935d0 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f9804011740 10 client.26519 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f9804011740 10 client.26519 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f9804011740 20 client.26519 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f9804011740 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e92aade70 0x557e92b1aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f9804011740 4 client.26519 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f9804011740 1 -- 172.21.15.94:0/1068933203 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e9297d170 con 0x557e92aade70 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.274+0000 7f97faffd640 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e92aade70 0x557e92b1aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f97daffd640 10 client.26519 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:27:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f97faffd640 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e92aade70 0x557e92b1aaf0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fd5d013dd4a2d27 server_cookie=c571ac293df819de in_seq=0 out_seq=0 2024-09-19T04:27:35.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f97daffd640 1 -- 172.21.15.94:0/1068933203 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e9297d170 con 0x557e92aade70 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f97daffd640 10 client.26519 handle_command_reply: tid=0 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f9804011740 1 client.26519 shutdown 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f9804011740 1 -- 172.21.15.94:0/1068933203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 msgr2=0x557e92aa19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f9804011740 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92aa19a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0a4a60 tx=0x7f97ec09bc80 comp rx=0 tx=0).stop 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f9804011740 1 -- 172.21.15.94:0/1068933203 shutdown_connections 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f9804011740 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e92aade70 0x557e92b1aaf0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f9804011740 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e92a9e370 0x557e92b16be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f9804011740 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e92a94b40 0x557e92aa1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f9804011740 1 --2- 172.21.15.94:0/1068933203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e92a935d0 0x557e92aa19a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f9804011740 1 -- 172.21.15.94:0/1068933203 >> 172.21.15.94:0/1068933203 conn(0x557e92a5c310 msgr2=0x557e92a9a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:35.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.275+0000 7f97daffd640 0 client.26519 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:27:35.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9804011740 1 -- 172.21.15.94:0/1068933203 shutdown_connections 2024-09-19T04:27:35.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9804011740 1 -- 172.21.15.94:0/1068933203 wait complete. 2024-09-19T04:27:35.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9804011740 20 client.26519 trim_cache size 0 max 16384 2024-09-19T04:27:35.284 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:27:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f97e40a3430 msgr2=0x7f97e40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f97e40a3430 0x7f97e40a58f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009ca0 tx=0x7f97f40095c0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc101de0 msgr2=0x7f97fc073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc101de0 0x7f97fc073a10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00e9f0 tx=0x7f97ec00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 shutdown_connections 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f97e40a3430 0x7f97e40a58f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc106f10 0x7f97fc06cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97fc102790 0x7f97fc06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 --2- 172.21.15.94:0/2422641910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc101de0 0x7f97fc073a10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.276+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 >> 172.21.15.94:0/2422641910 conn(0x7f97fc0fd8c0 msgr2=0x7f97fc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.277+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 shutdown_connections 2024-09-19T04:27:35.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:35.277+0000 7f9801c38640 1 -- 172.21.15.94:0/2422641910 wait complete. 2024-09-19T04:27:35.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:35 smithi094 ceph-mon[30972]: pgmap v391: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.2 MiB/s wr, 799 op/s 2024-09-19T04:27:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 680.0 s, 374.4 tps, lat 267.965 ms stddev 41.086 2024-09-19T04:27:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:35 smithi104 ceph-mon[34145]: pgmap v391: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.2 MiB/s wr, 799 op/s 2024-09-19T04:27:35.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:35 smithi112 ceph-mon[35051]: pgmap v391: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.2 MiB/s wr, 799 op/s 2024-09-19T04:27:36.106 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:27:36.436 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:36.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.988+0000 7f6940375640 1 -- 172.21.15.94:0/2498879784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6938102790 msgr2=0x7f69381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.988+0000 7f6940375640 1 --2- 172.21.15.94:0/2498879784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6938102790 0x7f69381067e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6928009860 tx=0x7f692802a340 comp rx=0 tx=0).stop 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.988+0000 7f6940375640 1 -- 172.21.15.94:0/2498879784 shutdown_connections 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.988+0000 7f6940375640 1 --2- 172.21.15.94:0/2498879784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938106f10 0x7f693810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.988+0000 7f6940375640 1 --2- 172.21.15.94:0/2498879784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6938102790 0x7f69381067e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.988+0000 7f6940375640 1 --2- 172.21.15.94:0/2498879784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6938101de0 0x7f69381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.988+0000 7f6940375640 1 -- 172.21.15.94:0/2498879784 >> 172.21.15.94:0/2498879784 conn(0x7f69380fd8e0 msgr2=0x7f69380ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.988+0000 7f6940375640 1 -- 172.21.15.94:0/2498879784 shutdown_connections 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.988+0000 7f6940375640 1 -- 172.21.15.94:0/2498879784 wait complete. 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f6940375640 1 Processor -- start 2024-09-19T04:27:36.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f6940375640 1 -- start start 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f6940375640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6938101de0 0x7f69380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f6940375640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938102790 0x7f6938074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f6940375640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6938106f10 0x7f6938074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f693e111640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6938101de0 0x7f69380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f693e912640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6938106f10 0x7f6938074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f693d910640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938102790 0x7f6938074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f693e111640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6938101de0 0x7f69380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48458/0 (socket says 172.21.15.94:48458) 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f693d910640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938102790 0x7f6938074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33414/0 (socket says 172.21.15.94:33414) 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f693e912640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6938106f10 0x7f6938074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45364/0 (socket says 172.21.15.94:45364) 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6938111e30 con 0x7f6938106f10 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f693e111640 1 -- 172.21.15.94:0/2108603904 learned_addr learned my addr 172.21.15.94:0/2108603904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:36.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6938111cb0 con 0x7f6938102790 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.989+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6938111fb0 con 0x7f6938101de0 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f693e111640 1 -- 172.21.15.94:0/2108603904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938102790 msgr2=0x7f6938074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f693e111640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938102790 0x7f6938074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f693e111640 1 -- 172.21.15.94:0/2108603904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6938106f10 msgr2=0x7f6938074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f693e111640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6938106f10 0x7f6938074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f693e111640 1 -- 172.21.15.94:0/2108603904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69381a6890 con 0x7f6938101de0 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f693d910640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938102790 0x7f6938074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f693e111640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6938101de0 0x7f69380744f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f693400e9c0 tx=0x7f693400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f692f7fe640 1 -- 172.21.15.94:0/2108603904 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693400ccc0 con 0x7f6938101de0 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f692f7fe640 1 -- 172.21.15.94:0/2108603904 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69340054d0 con 0x7f6938101de0 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69381a6b80 con 0x7f6938101de0 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f692f7fe640 1 -- 172.21.15.94:0/2108603904 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934010430 con 0x7f6938101de0 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.990+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69381a70e0 con 0x7f6938101de0 2024-09-19T04:27:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.993+0000 7f692f7fe640 1 -- 172.21.15.94:0/2108603904 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f69340046c0 con 0x7f6938101de0 2024-09-19T04:27:36.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.993+0000 7f694274e740 1 Processor -- start 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.993+0000 7f694274e740 1 -- start start 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f692f7fe640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f69240a3480 0x7f69240a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f694274e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703934f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f694274e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f693e111640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703934f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f693d910640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f693e111640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703934f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33422/0 (socket says 172.21.15.94:33422) 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f693e912640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f69240a3480 0x7f69240a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f692f7fe640 1 -- 172.21.15.94:0/2108603904 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f69340cff20 con 0x7f6938101de0 2024-09-19T04:27:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f693d910640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45372/0 (socket says 172.21.15.94:45372) 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f693e111640 1 -- 172.21.15.94:0/2836187779 learned_addr learned my addr 172.21.15.94:0/2836187779 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f694274e740 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703935a370 0x55703935c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f694274e740 1 -- 172.21.15.94:0/2836187779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570392b52f0 con 0x55703934f5d0 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f694274e740 1 -- 172.21.15.94:0/2836187779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55703925ccf0 con 0x557039350b40 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.994+0000 7f694274e740 1 -- 172.21.15.94:0/2836187779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5570392b12b0 con 0x55703935a370 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.995+0000 7f693e912640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f69240a3480 0x7f69240a5940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6930005fd0 tx=0x7f6930009500 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.995+0000 7f693e912640 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703935a370 0x55703935c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.995+0000 7f693e111640 1 -- 172.21.15.94:0/2836187779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703935a370 msgr2=0x55703935c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.995+0000 7f693e111640 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703935a370 0x55703935c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.995+0000 7f693e111640 1 -- 172.21.15.94:0/2836187779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 msgr2=0x55703934fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.995+0000 7f693e111640 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.995+0000 7f693e111640 1 -- 172.21.15.94:0/2836187779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570392413a0 con 0x557039350b40 2024-09-19T04:27:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.995+0000 7f693d910640 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.995+0000 7f693e111640 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703934f090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6934098620 tx=0x7f693409a970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf34ce7feafa478 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.996+0000 7f692cdea640 1 -- 172.21.15.94:0/2836187779 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693409a0a0 con 0x557039350b40 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.996+0000 7f692cdea640 1 -- 172.21.15.94:0/2836187779 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6934005680 con 0x557039350b40 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.996+0000 7f692cdea640 1 -- 172.21.15.94:0/2836187779 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693409bf90 con 0x557039350b40 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.996+0000 7f694274e740 1 -- 172.21.15.94:0/2836187779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 msgr2=0x55703934f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703934f090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6934098620 tx=0x7f693409a970 comp rx=0 tx=0).stop 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 -- 172.21.15.94:0/2836187779 shutdown_connections 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703935a370 0x55703935c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 --2- 172.21.15.94:0/2836187779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703934f090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 -- 172.21.15.94:0/2836187779 >> 172.21.15.94:0/2836187779 conn(0x557039318310 msgr2=0x55703934bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 -- 172.21.15.94:0/2836187779 shutdown_connections 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 -- 172.21.15.94:0/2836187779 wait complete. 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 -- start start 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.997+0000 7f694274e740 1 Processor -- start 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f694274e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x557039359d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f694274e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557039350b40 0x557039357040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f694274e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 0x557039357650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f694274e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55703935d010 con 0x557039350b40 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f694274e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557039270060 con 0x55703935a370 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f694274e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55703935d190 con 0x55703934f5d0 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f694274e740 10 client.0 _ll_register_callbacks cb 0x557038fc38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693e111640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x557039359d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693d910640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557039350b40 0x557039357040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693e111640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x557039359d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48478/0 (socket says 172.21.15.94:48478) 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693e111640 1 -- 172.21.15.94:0/526674990 learned_addr learned my addr 172.21.15.94:0/526674990 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693e912640 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 0x557039357650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693e111640 1 -- 172.21.15.94:0/526674990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 msgr2=0x557039357650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693e111640 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 0x557039357650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693e111640 1 -- 172.21.15.94:0/526674990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557039350b40 msgr2=0x557039357040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693e111640 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557039350b40 0x557039357040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693e111640 1 -- 172.21.15.94:0/526674990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570390263c0 con 0x55703934f5d0 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.998+0000 7f693d910640 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557039350b40 0x557039357040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.999+0000 7f693e111640 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x557039359d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6934099dd0 tx=0x7f693400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980b669fc8d1f1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.999+0000 7f6921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:27:37.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.999+0000 7f6921ffb640 1 -- 172.21.15.94:0/526674990 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340b4020 con 0x55703934f5d0 2024-09-19T04:27:37.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.999+0000 7f694274e740 1 -- 172.21.15.94:0/526674990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6934005680 con 0x55703934f5d0 2024-09-19T04:27:37.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.999+0000 7f6921ffb640 1 -- 172.21.15.94:0/526674990 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f69340a49a0 con 0x55703934f5d0 2024-09-19T04:27:37.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.999+0000 7f6921ffb640 1 -- 172.21.15.94:0/526674990 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f69340a3b30 con 0x55703934f5d0 2024-09-19T04:27:37.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.999+0000 7f694274e740 10 client.26515 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:37.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.999+0000 7f694274e740 1 -- 172.21.15.94:0/526674990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55703935d060 con 0x55703934f5d0 2024-09-19T04:27:37.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:36.999+0000 7f6921ffb640 1 -- 172.21.15.94:0/526674990 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6934005680 con 0x55703934f5d0 2024-09-19T04:27:37.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.000+0000 7f6921ffb640 1 -- 172.21.15.94:0/526674990 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f6934098830 con 0x55703934f5d0 2024-09-19T04:27:37.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.000+0000 7f694274e740 10 client.26515 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:37.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.000+0000 7f694274e740 10 client.26515 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:27:37.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.000+0000 7f694274e740 20 client.26515 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:37.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.000+0000 7f694274e740 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x557039369d20 0x55703936c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.000+0000 7f694274e740 4 client.26515 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:37.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.000+0000 7f694274e740 1 -- 172.21.15.94:0/526674990 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570392c2860 con 0x557039369d20 2024-09-19T04:27:37.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.000+0000 7f693d910640 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x557039369d20 0x55703936c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.001+0000 7f6921ffb640 10 client.26515 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:27:37.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.001+0000 7f693d910640 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x557039369d20 0x55703936c150 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9fecb23c59a2eaa server_cookie=a47cd586ed5c424a in_seq=0 out_seq=0 2024-09-19T04:27:37.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.001+0000 7f6921ffb640 1 -- 172.21.15.94:0/526674990 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570392c2860 con 0x557039369d20 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.001+0000 7f6921ffb640 10 client.26515 handle_command_reply: tid=0 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 client.26515 shutdown 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 -- 172.21.15.94:0/526674990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 msgr2=0x557039359d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x557039359d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6934099dd0 tx=0x7f693400ce60 comp rx=0 tx=0).stop 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 -- 172.21.15.94:0/526674990 shutdown_connections 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x557039369d20 0x55703936c150 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 0x557039357650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557039350b40 0x557039357040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 --2- 172.21.15.94:0/526674990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x557039359d90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 -- 172.21.15.94:0/526674990 >> 172.21.15.94:0/526674990 conn(0x557039318310 msgr2=0x5570393519c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f6921ffb640 0 client.26515 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 -- 172.21.15.94:0/526674990 shutdown_connections 2024-09-19T04:27:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 1 -- 172.21.15.94:0/526674990 wait complete. 2024-09-19T04:27:37.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.002+0000 7f694274e740 20 client.26515 trim_cache size 0 max 16384 2024-09-19T04:27:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f694274e740 1 Processor -- start 2024-09-19T04:27:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f694274e740 1 -- start start 2024-09-19T04:27:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f694274e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f694274e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557039350b40 0x55703934b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f694274e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 0x557039352120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f694274e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69381ae110 con 0x55703934f5d0 2024-09-19T04:27:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f694274e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f69381ab920 con 0x55703935a370 2024-09-19T04:27:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f694274e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69381a9790 con 0x557039350b40 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f693e111640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f693e111640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45394/0 (socket says 172.21.15.94:45394) 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f693e111640 1 -- 172.21.15.94:0/2380935476 learned_addr learned my addr 172.21.15.94:0/2380935476 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f693d910640 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557039350b40 0x55703934b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f693e912640 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 0x557039352120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f693e111640 1 -- 172.21.15.94:0/2380935476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557039350b40 msgr2=0x55703934b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.020+0000 7f693e111640 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557039350b40 0x55703934b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.021+0000 7f693e111640 1 -- 172.21.15.94:0/2380935476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 msgr2=0x557039352120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.021+0000 7f693e111640 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 0x557039352120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.021+0000 7f693e111640 1 -- 172.21.15.94:0/2380935476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557039355590 con 0x55703934f5d0 2024-09-19T04:27:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.021+0000 7f693e912640 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 0x557039352120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.021+0000 7f693e111640 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934ac10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f69340cdc30 tx=0x7f69340a4d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296f0d98ce0fa561 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.021+0000 7f6923fff640 1 -- 172.21.15.94:0/2380935476 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340b4070 con 0x55703934f5d0 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.021+0000 7f6923fff640 1 -- 172.21.15.94:0/2380935476 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69340a3970 con 0x55703934f5d0 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.021+0000 7f6923fff640 1 -- 172.21.15.94:0/2380935476 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340a45a0 con 0x55703934f5d0 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.022+0000 7f694274e740 1 -- 172.21.15.94:0/2380935476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 msgr2=0x55703934ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.022+0000 7f694274e740 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934ac10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f69340cdc30 tx=0x7f69340a4d30 comp rx=0 tx=0).stop 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.022+0000 7f694274e740 1 -- 172.21.15.94:0/2380935476 shutdown_connections 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.022+0000 7f694274e740 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55703935a370 0x557039352120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.022+0000 7f694274e740 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557039350b40 0x55703934b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.022+0000 7f694274e740 1 --2- 172.21.15.94:0/2380935476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703934f5d0 0x55703934ac10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.022+0000 7f694274e740 1 -- 172.21.15.94:0/2380935476 >> 172.21.15.94:0/2380935476 conn(0x557039318310 msgr2=0x55703934a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.022+0000 7f694274e740 1 -- 172.21.15.94:0/2380935476 shutdown_connections 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.022+0000 7f694274e740 1 -- 172.21.15.94:0/2380935476 wait complete. 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f694274e740 1 -- start start 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f694274e740 1 Processor -- start 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f694274e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x55703935db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f694274e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703935e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f694274e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703935a370 0x5570393cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f694274e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69381a9de0 con 0x55703935a370 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f694274e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f69381ab250 con 0x557039350b40 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f694274e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69381aa4b0 con 0x55703934f5d0 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f694274e740 10 client.0 _ll_register_callbacks cb 0x7f69300093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f693e912640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703935a370 0x5570393cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f693e111640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x55703935db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f693e111640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x55703935db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48500/0 (socket says 172.21.15.94:48500) 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.023+0000 7f693e111640 1 -- 172.21.15.94:0/2532586635 learned_addr learned my addr 172.21.15.94:0/2532586635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f693d910640 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703935e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f693e111640 1 -- 172.21.15.94:0/2532586635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 msgr2=0x55703935e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f693e111640 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703935e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f693e111640 1 -- 172.21.15.94:0/2532586635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703935a370 msgr2=0x5570393cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f693e111640 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703935a370 0x5570393cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f693e111640 1 -- 172.21.15.94:0/2532586635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570393cd060 con 0x55703934f5d0 2024-09-19T04:27:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f693d910640 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703935e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f693e111640 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x55703935db60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f693409c150 tx=0x7f6934005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9502d9e78856d7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f6921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f6921ffb640 1 -- 172.21.15.94:0/2532586635 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340b4030 con 0x55703934f5d0 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f6921ffb640 1 -- 172.21.15.94:0/2532586635 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f69340850d0 con 0x55703934f5d0 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f694274e740 1 -- 172.21.15.94:0/2532586635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5570392bcd70 con 0x55703934f5d0 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f6921ffb640 1 -- 172.21.15.94:0/2532586635 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f69340a4790 con 0x55703934f5d0 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f694274e740 10 client.26521 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.024+0000 7f694274e740 1 -- 172.21.15.94:0/2532586635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5570393cd060 con 0x55703934f5d0 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.025+0000 7f6921ffb640 1 -- 172.21.15.94:0/2532586635 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5570392bcd70 con 0x55703934f5d0 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.025+0000 7f6921ffb640 1 -- 172.21.15.94:0/2532586635 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f6934081020 con 0x55703934f5d0 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.025+0000 7f694274e740 10 client.26521 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:37.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.025+0000 7f694274e740 10 client.26521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:27:37.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.025+0000 7f694274e740 20 client.26521 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:37.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.025+0000 7f694274e740 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5570393d4680 0x5570393d6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.025+0000 7f694274e740 4 client.26521 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:27:37.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.025+0000 7f694274e740 1 -- 172.21.15.94:0/2532586635 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557039239170 con 0x5570393d4680 2024-09-19T04:27:37.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.025+0000 7f693d910640 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5570393d4680 0x5570393d6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f693d910640 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5570393d4680 0x5570393d6a70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75eee4cfd6265557 server_cookie=9f9539227d619ab5 in_seq=0 out_seq=0 2024-09-19T04:27:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f6921ffb640 10 client.26521 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:27:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f6921ffb640 1 -- 172.21.15.94:0/2532586635 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557039239170 con 0x5570393d4680 2024-09-19T04:27:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f6921ffb640 10 client.26521 handle_command_reply: tid=0 2024-09-19T04:27:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 client.26521 shutdown 2024-09-19T04:27:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 -- 172.21.15.94:0/2532586635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 msgr2=0x55703935db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x55703935db60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f693409c150 tx=0x7f6934005880 comp rx=0 tx=0).stop 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 -- 172.21.15.94:0/2532586635 shutdown_connections 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5570393d4680 0x5570393d6a70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703935a370 0x5570393cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557039350b40 0x55703935e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f6921ffb640 0 client.26521 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 --2- 172.21.15.94:0/2532586635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55703934f5d0 0x55703935db60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 -- 172.21.15.94:0/2532586635 >> 172.21.15.94:0/2532586635 conn(0x557039318310 msgr2=0x5570393590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.026+0000 7f694274e740 1 -- 172.21.15.94:0/2532586635 shutdown_connections 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f694274e740 1 -- 172.21.15.94:0/2532586635 wait complete. 2024-09-19T04:27:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f694274e740 20 client.26521 trim_cache size 0 max 16384 2024-09-19T04:27:37.034 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:27:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f69240a3480 msgr2=0x7f69240a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f69240a3480 0x7f69240a5940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6930005fd0 tx=0x7f6930009500 comp rx=0 tx=0).stop 2024-09-19T04:27:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6938101de0 msgr2=0x7f69380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6938101de0 0x7f69380744f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f693400e9c0 tx=0x7f693400ee80 comp rx=0 tx=0).stop 2024-09-19T04:27:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 shutdown_connections 2024-09-19T04:27:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f69240a3480 0x7f69240a5940 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6938106f10 0x7f6938074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938102790 0x7f6938074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 --2- 172.21.15.94:0/2108603904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6938101de0 0x7f69380744f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.027+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 >> 172.21.15.94:0/2108603904 conn(0x7f69380fd8e0 msgr2=0x7f69380fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.028+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 shutdown_connections 2024-09-19T04:27:37.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:37.028+0000 7f6940375640 1 -- 172.21.15.94:0/2108603904 wait complete. 2024-09-19T04:27:37.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:37 smithi094 ceph-mon[30972]: pgmap v392: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 4.1 MiB/s wr, 527 op/s 2024-09-19T04:27:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:37 smithi104 ceph-mon[34145]: pgmap v392: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 4.1 MiB/s wr, 527 op/s 2024-09-19T04:27:37.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:37 smithi112 ceph-mon[35051]: pgmap v392: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 4.1 MiB/s wr, 527 op/s 2024-09-19T04:27:39.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:27:39.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:39 smithi094 ceph-mon[30972]: pgmap v393: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 5.5 MiB/s wr, 690 op/s 2024-09-19T04:27:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:39 smithi104 ceph-mon[34145]: pgmap v393: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 5.5 MiB/s wr, 690 op/s 2024-09-19T04:27:39.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:39 smithi112 ceph-mon[35051]: pgmap v393: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 5.5 MiB/s wr, 690 op/s 2024-09-19T04:27:39.861 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:40.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.427+0000 7f7365754640 1 -- 172.21.15.94:0/1688339095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7360100480 msgr2=0x7f7360100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:40.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.427+0000 7f7365754640 1 --2- 172.21.15.94:0/1688339095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7360100480 0x7f7360100860 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7350009900 tx=0x7f735002a500 comp rx=0 tx=0).stop 2024-09-19T04:27:40.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.427+0000 7f7365754640 1 -- 172.21.15.94:0/1688339095 shutdown_connections 2024-09-19T04:27:40.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.427+0000 7f7365754640 1 --2- 172.21.15.94:0/1688339095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73601055b0 0x7f736010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.427+0000 7f7365754640 1 --2- 172.21.15.94:0/1688339095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7360100e30 0x7f7360104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.427+0000 7f7365754640 1 --2- 172.21.15.94:0/1688339095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7360100480 0x7f7360100860 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.427+0000 7f7365754640 1 -- 172.21.15.94:0/1688339095 >> 172.21.15.94:0/1688339095 conn(0x7f7360077210 msgr2=0x7f73600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:40.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.427+0000 7f7365754640 1 -- 172.21.15.94:0/1688339095 shutdown_connections 2024-09-19T04:27:40.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.428+0000 7f7365754640 1 -- 172.21.15.94:0/1688339095 wait complete. 2024-09-19T04:27:40.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.428+0000 7f7365754640 1 Processor -- start 2024-09-19T04:27:40.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f7365754640 1 -- start start 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f7365754640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7360100480 0x7f7360196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f7365754640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7360100e30 0x7f7360196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f7365754640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73601055b0 0x7f736019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f7365754640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f736010fc10 con 0x7f7360100e30 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f7365754640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f736010fa90 con 0x7f73601055b0 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f7365754640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f736010fd90 con 0x7f7360100480 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f735f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73601055b0 0x7f736019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f735effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7360100480 0x7f7360196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f735f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73601055b0 0x7f736019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33468/0 (socket says 172.21.15.94:33468) 2024-09-19T04:27:40.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.429+0000 7f735f7fe640 1 -- 172.21.15.94:0/2734758798 learned_addr learned my addr 172.21.15.94:0/2734758798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:40.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.430+0000 7f735effd640 1 -- 172.21.15.94:0/2734758798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73601055b0 msgr2=0x7f736019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:40.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.430+0000 7f735effd640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73601055b0 0x7f736019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.430+0000 7f735effd640 1 -- 172.21.15.94:0/2734758798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7360100e30 msgr2=0x7f7360196690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:27:40.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.430+0000 7f735effd640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7360100e30 0x7f7360196690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.430+0000 7f735effd640 1 -- 172.21.15.94:0/2734758798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736019b100 con 0x7f7360100480 2024-09-19T04:27:40.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.430+0000 7f735effd640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7360100480 0x7f7360196150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f735002aa20 tx=0x7f7350031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:40.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.430+0000 7f7347fff640 1 -- 172.21.15.94:0/2734758798 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735003f030 con 0x7f7360100480 2024-09-19T04:27:40.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.430+0000 7f7347fff640 1 -- 172.21.15.94:0/2734758798 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7350004490 con 0x7f7360100480 2024-09-19T04:27:40.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.430+0000 7f7347fff640 1 -- 172.21.15.94:0/2734758798 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350035070 con 0x7f7360100480 2024-09-19T04:27:40.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.431+0000 7f7365754640 1 -- 172.21.15.94:0/2734758798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736019b390 con 0x7f7360100480 2024-09-19T04:27:40.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.431+0000 7f7365754640 1 -- 172.21.15.94:0/2734758798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736019b760 con 0x7f7360100480 2024-09-19T04:27:40.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.433+0000 7f7347fff640 1 -- 172.21.15.94:0/2734758798 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7350042030 con 0x7f7360100480 2024-09-19T04:27:40.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.434+0000 7f7347fff640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f73480a33b0 0x7f73480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.434+0000 7f735e7fc640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f73480a33b0 0x7f73480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.434+0000 7f7347fff640 1 -- 172.21.15.94:0/2734758798 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f73500f0030 con 0x7f7360100480 2024-09-19T04:27:40.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.435+0000 7f735e7fc640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f73480a33b0 0x7f73480a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73600ff860 tx=0x7f7354005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:40.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.436+0000 7f7367b2d740 1 Processor -- start 2024-09-19T04:27:40.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.436+0000 7f7367b2d740 1 -- start start 2024-09-19T04:27:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b8492d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b8492d5d0 0x556b8492da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f735effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b8492d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b84940b80 0x556b84942f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f735effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b8492d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48528/0 (socket says 172.21.15.94:48528) 2024-09-19T04:27:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f735e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b8492d5d0 0x556b8492da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f735e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b8492d5d0 0x556b8492da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33480/0 (socket says 172.21.15.94:33480) 2024-09-19T04:27:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f735effd640 1 -- 172.21.15.94:0/2917502352 learned_addr learned my addr 172.21.15.94:0/2917502352 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f7367b2d740 1 -- 172.21.15.94:0/2917502352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b848932f0 con 0x556b84940b80 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f7367b2d740 1 -- 172.21.15.94:0/2917502352 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556b8483acf0 con 0x556b8492d5d0 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f7367b2d740 1 -- 172.21.15.94:0/2917502352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556b8488f2b0 con 0x556b8492eb40 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.437+0000 7f735f7fe640 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b84940b80 0x556b84942f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f735e7fc640 1 -- 172.21.15.94:0/2917502352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 msgr2=0x556b8492d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f735e7fc640 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b8492d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f735e7fc640 1 -- 172.21.15.94:0/2917502352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b84940b80 msgr2=0x556b84942f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f735e7fc640 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b84940b80 0x556b84942f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f735e7fc640 1 -- 172.21.15.94:0/2917502352 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8481f3a0 con 0x556b8492d5d0 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f735f7fe640 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b84940b80 0x556b84942f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f735effd640 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b8492d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f735e7fc640 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b8492d5d0 0x556b8492da10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f735400ed30 tx=0x7f7354011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f5b431203cd7f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:40.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f73457fa640 1 -- 172.21.15.94:0/2917502352 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354020070 con 0x556b8492d5d0 2024-09-19T04:27:40.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f73457fa640 1 -- 172.21.15.94:0/2917502352 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7354010e40 con 0x556b8492d5d0 2024-09-19T04:27:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.438+0000 7f73457fa640 1 -- 172.21.15.94:0/2917502352 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735401c420 con 0x556b8492d5d0 2024-09-19T04:27:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.439+0000 7f7367b2d740 1 -- 172.21.15.94:0/2917502352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b8492d5d0 msgr2=0x556b8492da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.439+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b8492d5d0 0x556b8492da10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f735400ed30 tx=0x7f7354011000 comp rx=0 tx=0).stop 2024-09-19T04:27:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.440+0000 7f7367b2d740 1 -- 172.21.15.94:0/2917502352 shutdown_connections 2024-09-19T04:27:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.440+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b84940b80 0x556b84942f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.440+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b8492d5d0 0x556b8492da10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.440+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2917502352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b8492d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.440+0000 7f7367b2d740 1 -- 172.21.15.94:0/2917502352 >> 172.21.15.94:0/2917502352 conn(0x556b848f6310 msgr2=0x556b84929c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.440+0000 7f7367b2d740 1 -- 172.21.15.94:0/2917502352 shutdown_connections 2024-09-19T04:27:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.440+0000 7f7367b2d740 1 -- 172.21.15.94:0/2917502352 wait complete. 2024-09-19T04:27:40.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.440+0000 7f7367b2d740 1 -- start start 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f7367b2d740 1 Processor -- start 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b8493d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b8493de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b8493e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f7367b2d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b84943880 con 0x556b8492eb40 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f7367b2d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556b8484e060 con 0x556b84940b80 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f7367b2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556b84943a00 con 0x556b8492d5d0 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f7367b2d740 10 client.0 _ll_register_callbacks cb 0x556b845a18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f735f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b8493e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f735e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b8493de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.441+0000 7f735effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b8493d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b8493de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45428/0 (socket says 172.21.15.94:45428) 2024-09-19T04:27:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735e7fc640 1 -- 172.21.15.94:0/2955085030 learned_addr learned my addr 172.21.15.94:0/2955085030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735effd640 1 -- 172.21.15.94:0/2955085030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 msgr2=0x556b8493e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735effd640 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b8493e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735effd640 1 -- 172.21.15.94:0/2955085030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 msgr2=0x556b8493de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735effd640 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b8493de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735effd640 1 -- 172.21.15.94:0/2955085030 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b846043c0 con 0x556b8492d5d0 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735e7fc640 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b8493de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735f7fe640 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b8493e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f735effd640 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b8493d900 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f73500b8730 tx=0x7f73500bca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c591251ae1f4a44d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.442+0000 7f7342ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7342ff5640 1 -- 172.21.15.94:0/2955085030 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500d5060 con 0x556b8492d5d0 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7367b2d740 1 -- 172.21.15.94:0/2955085030 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b848a0860 con 0x556b8492d5d0 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7342ff5640 1 -- 172.21.15.94:0/2955085030 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f73500ed9c0 con 0x556b8492d5d0 2024-09-19T04:27:40.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7342ff5640 1 -- 172.21.15.94:0/2955085030 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7350049050 con 0x556b8492d5d0 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7367b2d740 10 client.26533 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7367b2d740 1 -- 172.21.15.94:0/2955085030 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556b84943930 con 0x556b8492d5d0 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7342ff5640 1 -- 172.21.15.94:0/2955085030 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556b848a0860 con 0x556b8492d5d0 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7342ff5640 1 -- 172.21.15.94:0/2955085030 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f73500c4ab0 con 0x556b8492d5d0 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7367b2d740 10 client.26533 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7367b2d740 10 client.26533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7367b2d740 20 client.26533 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556b849507f0 0x556b84952c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7367b2d740 4 client.26533 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.443+0000 7f7367b2d740 1 -- 172.21.15.94:0/2955085030 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b8489ad70 con 0x556b849507f0 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.444+0000 7f735e7fc640 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556b849507f0 0x556b84952c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.444+0000 7f7342ff5640 10 client.26533 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.444+0000 7f735e7fc640 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556b849507f0 0x556b84952c50 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480bd062df0fc01 server_cookie=9bc3ecb648c63b93 in_seq=0 out_seq=0 2024-09-19T04:27:40.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.445+0000 7f7342ff5640 1 -- 172.21.15.94:0/2955085030 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b8489ad70 con 0x556b849507f0 2024-09-19T04:27:40.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.445+0000 7f7342ff5640 10 client.26533 handle_command_reply: tid=0 2024-09-19T04:27:40.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.445+0000 7f7367b2d740 1 client.26533 shutdown 2024-09-19T04:27:40.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.445+0000 7f7367b2d740 1 -- 172.21.15.94:0/2955085030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 msgr2=0x556b8493d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:40.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.446+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b8493d900 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f73500b8730 tx=0x7f73500bca90 comp rx=0 tx=0).stop 2024-09-19T04:27:40.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.446+0000 7f7367b2d740 1 -- 172.21.15.94:0/2955085030 shutdown_connections 2024-09-19T04:27:40.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.446+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556b849507f0 0x556b84952c50 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.446+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b8493e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.446+0000 7f7342ff5640 0 client.26533 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:40.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.446+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b8493de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.446+0000 7f7367b2d740 1 --2- 172.21.15.94:0/2955085030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b8493d900 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.447+0000 7f7367b2d740 1 -- 172.21.15.94:0/2955085030 >> 172.21.15.94:0/2955085030 conn(0x556b848f6310 msgr2=0x556b8492f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:40.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.447+0000 7f7367b2d740 1 -- 172.21.15.94:0/2955085030 shutdown_connections 2024-09-19T04:27:40.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.447+0000 7f7367b2d740 1 -- 172.21.15.94:0/2955085030 wait complete. 2024-09-19T04:27:40.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.447+0000 7f7367b2d740 20 client.26533 trim_cache size 0 max 16384 2024-09-19T04:27:40.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f7367b2d740 1 Processor -- start 2024-09-19T04:27:40.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f7367b2d740 1 -- start start 2024-09-19T04:27:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b84958520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b84958a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b8493d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f7367b2d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73601adfd0 con 0x556b8492eb40 2024-09-19T04:27:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f7367b2d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f73601ab7e0 con 0x556b84940b80 2024-09-19T04:27:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f7367b2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73601a9650 con 0x556b8492d5d0 2024-09-19T04:27:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f735effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b84958520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f735effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b84958520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48546/0 (socket says 172.21.15.94:48546) 2024-09-19T04:27:40.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f735e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b84958a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.467+0000 7f735effd640 1 -- 172.21.15.94:0/3139587712 learned_addr learned my addr 172.21.15.94:0/3139587712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:40.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.468+0000 7f735f7fe640 1 --2- 172.21.15.94:0/3139587712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b8493d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.468+0000 7f735effd640 1 -- 172.21.15.94:0/3139587712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 msgr2=0x556b8493d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.468+0000 7f735effd640 1 --2- 172.21.15.94:0/3139587712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b8493d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.468+0000 7f735effd640 1 -- 172.21.15.94:0/3139587712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 msgr2=0x556b84958a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.468+0000 7f735effd640 1 --2- 172.21.15.94:0/3139587712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b84958a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.468+0000 7f735effd640 1 -- 172.21.15.94:0/3139587712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b84943760 con 0x556b8492d5d0 2024-09-19T04:27:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.468+0000 7f735e7fc640 1 --2- 172.21.15.94:0/3139587712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b84958a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.468+0000 7f735effd640 1 --2- 172.21.15.94:0/3139587712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b84958520 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f73500096c0 tx=0x7f7350005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d694c19226115872 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.468+0000 7f7344ff9640 1 -- 172.21.15.94:0/3139587712 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500c4ab0 con 0x556b8492d5d0 2024-09-19T04:27:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.469+0000 7f7344ff9640 1 -- 172.21.15.94:0/3139587712 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73500c4c50 con 0x556b8492d5d0 2024-09-19T04:27:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.469+0000 7f7344ff9640 1 -- 172.21.15.94:0/3139587712 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500d52c0 con 0x556b8492d5d0 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.469+0000 7f7367b2d740 1 -- 172.21.15.94:0/3139587712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 msgr2=0x556b84958520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.469+0000 7f7367b2d740 1 --2- 172.21.15.94:0/3139587712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b84958520 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f73500096c0 tx=0x7f7350005c00 comp rx=0 tx=0).stop 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.470+0000 7f7367b2d740 1 -- 172.21.15.94:0/3139587712 shutdown_connections 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.470+0000 7f7367b2d740 1 --2- 172.21.15.94:0/3139587712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b8493d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.470+0000 7f7367b2d740 1 --2- 172.21.15.94:0/3139587712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492eb40 0x556b84958a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.470+0000 7f7367b2d740 1 --2- 172.21.15.94:0/3139587712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492d5d0 0x556b84958520 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.470+0000 7f7367b2d740 1 -- 172.21.15.94:0/3139587712 >> 172.21.15.94:0/3139587712 conn(0x556b848f6310 msgr2=0x556b8493c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.470+0000 7f7367b2d740 1 -- 172.21.15.94:0/3139587712 shutdown_connections 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.470+0000 7f7367b2d740 1 -- 172.21.15.94:0/3139587712 wait complete. 2024-09-19T04:27:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.470+0000 7f7367b2d740 1 -- start start 2024-09-19T04:27:40.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.471+0000 7f7367b2d740 1 Processor -- start 2024-09-19T04:27:40.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.471+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492d5d0 0x556b84949090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.471+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b84944410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.471+0000 7f7367b2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b84944950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f7367b2d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73601a9ca0 con 0x556b8492d5d0 2024-09-19T04:27:40.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b84944950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b84944950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:33504/0 (socket says 172.21.15.94:33504) 2024-09-19T04:27:40.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735f7fe640 1 -- 172.21.15.94:0/1403260917 learned_addr learned my addr 172.21.15.94:0/1403260917 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:40.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735effd640 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492d5d0 0x556b84949090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f73601ab110 con 0x556b84940b80 2024-09-19T04:27:40.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73601aa370 con 0x556b8492eb40 2024-09-19T04:27:40.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735e7fc640 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b84944410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f7367b2d740 10 client.0 _ll_register_callbacks cb 0x7f73500bbea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735f7fe640 1 -- 172.21.15.94:0/1403260917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 msgr2=0x556b84944410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735f7fe640 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b84944410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735f7fe640 1 -- 172.21.15.94:0/1403260917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492d5d0 msgr2=0x556b84949090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735f7fe640 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492d5d0 0x556b84949090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.472+0000 7f735f7fe640 1 -- 172.21.15.94:0/1403260917 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8492a960 con 0x556b84940b80 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.473+0000 7f735f7fe640 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b84944950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f735800deb0 tx=0x7f735801a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700aea0a78ecc1b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.473+0000 7f7342ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.473+0000 7f7342ff5640 1 -- 172.21.15.94:0/1403260917 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7358004d20 con 0x556b84940b80 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.473+0000 7f7342ff5640 1 -- 172.21.15.94:0/1403260917 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f7358018650 con 0x556b84940b80 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.473+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73500c4c50 con 0x556b84940b80 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.473+0000 7f7342ff5640 1 -- 172.21.15.94:0/1403260917 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7358018850 con 0x556b84940b80 2024-09-19T04:27:40.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.473+0000 7f7367b2d740 10 client.26555 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556b849bb010 con 0x556b84940b80 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f7342ff5640 1 -- 172.21.15.94:0/1403260917 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73500c4c50 con 0x556b84940b80 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f7342ff5640 1 -- 172.21.15.94:0/1403260917 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f7358021070 con 0x556b84940b80 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f7367b2d740 10 client.26555 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f7367b2d740 10 client.26555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f7367b2d740 20 client.26555 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f7367b2d740 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556b849507f0 0x556b849bd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f7367b2d740 4 client.26555 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b84817170 con 0x556b849507f0 2024-09-19T04:27:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.474+0000 7f735effd640 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556b849507f0 0x556b849bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f735effd640 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556b849507f0 0x556b849bd800 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b9b497271793091 server_cookie=9ac917a5253cf0a8 in_seq=0 out_seq=0 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7342ff5640 10 client.26555 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7342ff5640 1 -- 172.21.15.94:0/1403260917 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b84817170 con 0x556b849507f0 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7342ff5640 10 client.26555 handle_command_reply: tid=0 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7367b2d740 1 client.26555 shutdown 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 msgr2=0x556b84944950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7367b2d740 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b84944950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f735800deb0 tx=0x7f735801a7d0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 shutdown_connections 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7367b2d740 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x556b849507f0 0x556b849bd800 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7367b2d740 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b84940b80 0x556b84944950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7367b2d740 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b8492eb40 0x556b84944410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7367b2d740 1 --2- 172.21.15.94:0/1403260917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8492d5d0 0x556b84949090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7342ff5640 0 client.26555 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.475+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 >> 172.21.15.94:0/1403260917 conn(0x556b848f6310 msgr2=0x556b849284e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:40.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.476+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 shutdown_connections 2024-09-19T04:27:40.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.476+0000 7f7367b2d740 1 -- 172.21.15.94:0/1403260917 wait complete. 2024-09-19T04:27:40.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.476+0000 7f7367b2d740 20 client.26555 trim_cache size 0 max 16384 2024-09-19T04:27:40.481 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.476+0000 7f7365754640 1 -- 172.21.15.94:0/2734758798 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f73480a33b0 msgr2=0x7f73480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.476+0000 7f7365754640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f73480a33b0 0x7f73480a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73600ff860 tx=0x7f7354005f90 comp rx=0 tx=0).stop 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.476+0000 7f7365754640 1 -- 172.21.15.94:0/2734758798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7360100480 msgr2=0x7f7360196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.476+0000 7f7365754640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7360100480 0x7f7360196150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f735002aa20 tx=0x7f7350031a20 comp rx=0 tx=0).stop 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.477+0000 7f7365754640 1 -- 172.21.15.94:0/2734758798 shutdown_connections 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.477+0000 7f7365754640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f73480a33b0 0x7f73480a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.477+0000 7f7365754640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73601055b0 0x7f736019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.477+0000 7f7365754640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7360100e30 0x7f7360196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.477+0000 7f7365754640 1 --2- 172.21.15.94:0/2734758798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7360100480 0x7f7360196150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:40.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.477+0000 7f7365754640 1 -- 172.21.15.94:0/2734758798 >> 172.21.15.94:0/2734758798 conn(0x7f7360077210 msgr2=0x7f7360103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:40.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.477+0000 7f7365754640 1 -- 172.21.15.94:0/2734758798 shutdown_connections 2024-09-19T04:27:40.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:40.477+0000 7f7365754640 1 -- 172.21.15.94:0/2734758798 wait complete. 2024-09-19T04:27:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 685.0 s, 347.4 tps, lat 286.828 ms stddev 87.845 2024-09-19T04:27:41.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:41 smithi094 ceph-mon[30972]: pgmap v394: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 5.4 MiB/s wr, 664 op/s 2024-09-19T04:27:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:41 smithi104 ceph-mon[34145]: pgmap v394: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 5.4 MiB/s wr, 664 op/s 2024-09-19T04:27:41.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:41 smithi112 ceph-mon[35051]: pgmap v394: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 5.4 MiB/s wr, 664 op/s 2024-09-19T04:27:42.232 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:27:42.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:27:42.567 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:43.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:43 smithi094 ceph-mon[30972]: pgmap v395: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 4.0 MiB/s wr, 514 op/s 2024-09-19T04:27:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:43 smithi104 ceph-mon[34145]: pgmap v395: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 4.0 MiB/s wr, 514 op/s 2024-09-19T04:27:43.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.766+0000 7fa9b480a640 1 -- 172.21.15.94:0/2643041195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b0100930 msgr2=0x7fa9b0100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.766+0000 7fa9b480a640 1 --2- 172.21.15.94:0/2643041195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b0100930 0x7fa9b0100d10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009900 tx=0x7fa9a002a410 comp rx=0 tx=0).stop 2024-09-19T04:27:43.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.766+0000 7fa9b480a640 1 -- 172.21.15.94:0/2643041195 shutdown_connections 2024-09-19T04:27:43.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.766+0000 7fa9b480a640 1 --2- 172.21.15.94:0/2643041195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9b010e4e0 0x7fa9b01108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.766+0000 7fa9b480a640 1 --2- 172.21.15.94:0/2643041195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9b01012e0 0x7fa9b010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.766+0000 7fa9b480a640 1 --2- 172.21.15.94:0/2643041195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b0100930 0x7fa9b0100d10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.766+0000 7fa9b480a640 1 -- 172.21.15.94:0/2643041195 >> 172.21.15.94:0/2643041195 conn(0x7fa9b00fc7a0 msgr2=0x7fa9b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.766+0000 7fa9b480a640 1 -- 172.21.15.94:0/2643041195 shutdown_connections 2024-09-19T04:27:43.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.766+0000 7fa9b480a640 1 -- 172.21.15.94:0/2643041195 wait complete. 2024-09-19T04:27:43.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.767+0000 7fa9b480a640 1 Processor -- start 2024-09-19T04:27:43.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.767+0000 7fa9b480a640 1 -- start start 2024-09-19T04:27:43.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9b480a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9b0100930 0x7fa9b019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9b480a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b01012e0 0x7fa9b019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9b480a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9b010e4e0 0x7fa9b019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9b480a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0114020 con 0x7fa9b01012e0 2024-09-19T04:27:43.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9b480a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0113ea0 con 0x7fa9b0100930 2024-09-19T04:27:43.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9b480a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01141a0 con 0x7fa9b010e4e0 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b01012e0 0x7fa9b019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b01012e0 0x7fa9b019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48918/0 (socket says 172.21.15.94:48918) 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9a7fff640 1 -- 172.21.15.94:0/930898494 learned_addr learned my addr 172.21.15.94:0/930898494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9aed9d640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9b010e4e0 0x7fa9b019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9b0100930 0x7fa9b019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9a7fff640 1 -- 172.21.15.94:0/930898494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9b010e4e0 msgr2=0x7fa9b019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9b010e4e0 0x7fa9b019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9a7fff640 1 -- 172.21.15.94:0/930898494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9b0100930 msgr2=0x7fa9b019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9b0100930 0x7fa9b019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9a7fff640 1 -- 172.21.15.94:0/930898494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b019f530 con 0x7fa9b01012e0 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.768+0000 7fa9aed9d640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9b010e4e0 0x7fa9b019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:43.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.769+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b01012e0 0x7fa9b019aac0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa99800e9f0 tx=0x7fa99800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.769+0000 7fa9a77fe640 1 -- 172.21.15.94:0/930898494 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99800cd80 con 0x7fa9b01012e0 2024-09-19T04:27:43.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.769+0000 7fa9b480a640 1 -- 172.21.15.94:0/930898494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b019f820 con 0x7fa9b01012e0 2024-09-19T04:27:43.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.769+0000 7fa9a77fe640 1 -- 172.21.15.94:0/930898494 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9980108e0 con 0x7fa9b01012e0 2024-09-19T04:27:43.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.769+0000 7fa9a77fe640 1 -- 172.21.15.94:0/930898494 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998018920 con 0x7fa9b01012e0 2024-09-19T04:27:43.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.769+0000 7fa9b480a640 1 -- 172.21.15.94:0/930898494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b01a7100 con 0x7fa9b01012e0 2024-09-19T04:27:43.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.773+0000 7fa9b6be3740 1 Processor -- start 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.773+0000 7fa9b6be3740 1 -- start start 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.773+0000 7fa9a77fe640 1 -- 172.21.15.94:0/930898494 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa998010430 con 0x7fa9b01012e0 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f65f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f70800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 0x5618f9f73130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5618f9ecb280 con 0x5618f9f70d40 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5618f9e72b10 con 0x5618f9f66480 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5618f9ec7240 con 0x5618f9f65b60 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f65f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9ae59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f70800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f65f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:56606/0 (socket says 172.21.15.94:56606) 2024-09-19T04:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9ae59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f70800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59430/0 (socket says 172.21.15.94:59430) 2024-09-19T04:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9aed9d640 1 -- 172.21.15.94:0/3192574656 learned_addr learned my addr 172.21.15.94:0/3192574656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/3192574656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 0x5618f9f73130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9a77fe640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa99c0a33b0 0x7fa99c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9ae59c640 1 -- 172.21.15.94:0/3192574656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 msgr2=0x5618f9f65f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/3192574656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f65f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9ae59c640 1 -- 172.21.15.94:0/3192574656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 msgr2=0x5618f9f73130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/3192574656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 0x5618f9f73130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9ae59c640 1 -- 172.21.15.94:0/3192574656 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f9e7f1f0 con 0x5618f9f66480 2024-09-19T04:27:43.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.774+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/3192574656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 0x5618f9f73130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:27:43.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.775+0000 7fa9a77fe640 1 -- 172.21.15.94:0/930898494 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7fa9980cff70 con 0x7fa9b01012e0 2024-09-19T04:27:43.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.775+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/3192574656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f70800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009230 tx=0x7fa9a0032b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32e5ad8d3e344ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.775+0000 7fa9a4ff9640 1 -- 172.21.15.94:0/3192574656 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0040070 con 0x5618f9f66480 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.775+0000 7fa9a4ff9640 1 -- 172.21.15.94:0/3192574656 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a0004030 con 0x5618f9f66480 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.775+0000 7fa9a4ff9640 1 -- 172.21.15.94:0/3192574656 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0035040 con 0x5618f9f66480 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.775+0000 7fa9aed9d640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa99c0a33b0 0x7fa99c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.776+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3192574656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 msgr2=0x5618f9f70800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.776+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3192574656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f70800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009230 tx=0x7fa9a0032b00 comp rx=0 tx=0).stop 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.776+0000 7fa9aed9d640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa99c0a33b0 0x7fa99c0a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009fd0 tx=0x7fa9a8004ab0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.776+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3192574656 shutdown_connections 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.776+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3192574656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 0x5618f9f73130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.776+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3192574656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f70800 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.776+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3192574656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f65f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.776+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3192574656 >> 172.21.15.94:0/3192574656 conn(0x5618f9f620f0 msgr2=0x5618f9f624d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.776+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3192574656 shutdown_connections 2024-09-19T04:27:43.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.777+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3192574656 wait complete. 2024-09-19T04:27:43.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.777+0000 7fa9b6be3740 1 -- start start 2024-09-19T04:27:43.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.777+0000 7fa9b6be3740 1 Processor -- start 2024-09-19T04:27:43.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.777+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f65b60 0x5618f9f6db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.777+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f66480 0x5618f9f6e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.777+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 0x5618f9f6e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5618f9f73910 con 0x5618f9f70d40 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5618f9e85f10 con 0x5618f9f65b60 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5618f9f73a90 con 0x5618f9f66480 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 0x5618f9f6e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9b6be3740 10 client.0 _ll_register_callbacks cb 0x5618f9e4ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9ae59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f65b60 0x5618f9f6db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f66480 0x5618f9f6e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f66480 0x5618f9f6e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:56610/0 (socket says 172.21.15.94:56610) 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9a7fff640 1 -- 172.21.15.94:0/3248333286 learned_addr learned my addr 172.21.15.94:0/3248333286 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9a7fff640 1 -- 172.21.15.94:0/3248333286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f65b60 msgr2=0x5618f9f6db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f65b60 0x5618f9f6db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9a7fff640 1 -- 172.21.15.94:0/3248333286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 msgr2=0x5618f9f6e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 0x5618f9f6e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9a7fff640 1 -- 172.21.15.94:0/3248333286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f9c3c6a0 con 0x5618f9f66480 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.778+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f65b60 0x5618f9f6db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.779+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f66480 0x5618f9f6e0b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa998018530 tx=0x7fa998010720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e00b9628b133d928 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.779+0000 7fa995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.779+0000 7fa995ffb640 1 -- 172.21.15.94:0/3248333286 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99809c2c0 con 0x5618f9f66480 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.779+0000 7fa995ffb640 1 -- 172.21.15.94:0/3248333286 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7fa998098930 con 0x5618f9f66480 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.779+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3248333286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9a0004030 con 0x5618f9f66480 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.779+0000 7fa995ffb640 1 -- 172.21.15.94:0/3248333286 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa9980a05c0 con 0x5618f9f66480 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.779+0000 7fa9b6be3740 10 client.26551 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.779+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3248333286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5618f9f7f400 con 0x5618f9f66480 2024-09-19T04:27:43.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.779+0000 7fa995ffb640 1 -- 172.21.15.94:0/3248333286 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a0004030 con 0x5618f9f66480 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa995ffb640 1 -- 172.21.15.94:0/3248333286 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fa9980980a0 con 0x5618f9f66480 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa9b6be3740 10 client.26551 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa9b6be3740 10 client.26551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa9b6be3740 20 client.26551 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5618f9f808b0 0x5618f9f82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa9b6be3740 4 client.26551 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3248333286 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618f9ed87f0 con 0x5618f9f808b0 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5618f9f808b0 0x5618f9f82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa995ffb640 10 client.26551 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.780+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5618f9f808b0 0x5618f9f82cd0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a365da46da25e0ac server_cookie=cf1e887e7eaf9502 in_seq=0 out_seq=0 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.781+0000 7fa995ffb640 1 -- 172.21.15.94:0/3248333286 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618f9ed87f0 con 0x5618f9f808b0 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.781+0000 7fa995ffb640 10 client.26551 handle_command_reply: tid=0 2024-09-19T04:27:43.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.781+0000 7fa9b6be3740 1 client.26551 shutdown 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.781+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3248333286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f66480 msgr2=0x5618f9f6e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.781+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f66480 0x5618f9f6e0b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa998018530 tx=0x7fa998010720 comp rx=0 tx=0).stop 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.781+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3248333286 shutdown_connections 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.782+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5618f9f808b0 0x5618f9f82cd0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.782+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f70d40 0x5618f9f6e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.782+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f66480 0x5618f9f6e0b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.782+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3248333286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f65b60 0x5618f9f6db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.782+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3248333286 >> 172.21.15.94:0/3248333286 conn(0x5618f9f620f0 msgr2=0x5618f9f67dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.782+0000 7fa995ffb640 0 client.26551 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.782+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3248333286 shutdown_connections 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.782+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3248333286 wait complete. 2024-09-19T04:27:43.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.782+0000 7fa9b6be3740 20 client.26551 trim_cache size 0 max 16384 2024-09-19T04:27:43.789 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:43 smithi112 ceph-mon[35051]: pgmap v395: 97 pgs: 97 active+clean; 3.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 4.0 MiB/s wr, 514 op/s 2024-09-19T04:27:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9b6be3740 1 Processor -- start 2024-09-19T04:27:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9b6be3740 1 -- start start 2024-09-19T04:27:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f65b60 0x5618f9f6f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f70d40 0x5618f9f88b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa990011110 con 0x5618f9f65b60 2024-09-19T04:27:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa99000e920 con 0x5618f9f66480 2024-09-19T04:27:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa99000c790 con 0x5618f9f70d40 2024-09-19T04:27:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f65b60 0x5618f9f6f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f65b60 0x5618f9f6f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48950/0 (socket says 172.21.15.94:48950) 2024-09-19T04:27:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9ae59c640 1 -- 172.21.15.94:0/1832281312 learned_addr learned my addr 172.21.15.94:0/1832281312 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.806+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa9aed9d640 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f70d40 0x5618f9f88b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa9a7fff640 1 -- 172.21.15.94:0/1832281312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f70d40 msgr2=0x5618f9f88b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f70d40 0x5618f9f88b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa9a7fff640 1 -- 172.21.15.94:0/1832281312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f65b60 msgr2=0x5618f9f6f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f65b60 0x5618f9f6f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa9a7fff640 1 -- 172.21.15.94:0/1832281312 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f9f61f20 con 0x5618f9f66480 2024-09-19T04:27:43.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa9aed9d640 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f70d40 0x5618f9f88b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:43.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f885c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cd730 tx=0x7fa9980b5250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17054ec35ed72847 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa997fff640 1 -- 172.21.15.94:0/1832281312 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99809c2c0 con 0x5618f9f66480 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa997fff640 1 -- 172.21.15.94:0/1832281312 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9980bae50 con 0x5618f9f66480 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.807+0000 7fa997fff640 1 -- 172.21.15.94:0/1832281312 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9980bc310 con 0x5618f9f66480 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.808+0000 7fa9b6be3740 1 -- 172.21.15.94:0/1832281312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 msgr2=0x5618f9f885c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.808+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f885c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cd730 tx=0x7fa9980b5250 comp rx=0 tx=0).stop 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.808+0000 7fa9b6be3740 1 -- 172.21.15.94:0/1832281312 shutdown_connections 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.808+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f70d40 0x5618f9f88b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.808+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f66480 0x5618f9f885c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.808+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/1832281312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f65b60 0x5618f9f6f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.808+0000 7fa9b6be3740 1 -- 172.21.15.94:0/1832281312 >> 172.21.15.94:0/1832281312 conn(0x5618f9f620f0 msgr2=0x5618f9f6f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.808+0000 7fa9b6be3740 1 -- 172.21.15.94:0/1832281312 shutdown_connections 2024-09-19T04:27:43.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.808+0000 7fa9b6be3740 1 -- 172.21.15.94:0/1832281312 wait complete. 2024-09-19T04:27:43.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9b6be3740 1 -- start start 2024-09-19T04:27:43.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9b6be3740 1 Processor -- start 2024-09-19T04:27:43.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f78d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f66480 0x5618f9f79260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9b6be3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f70d40 0x5618f9f73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa99000cde0 con 0x5618f9f66480 2024-09-19T04:27:43.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa99000e250 con 0x5618f9f70d40 2024-09-19T04:27:43.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9b6be3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa99000d4b0 con 0x5618f9f65b60 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9b6be3740 10 client.0 _ll_register_callbacks cb 0x7fa99800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f78d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f66480 0x5618f9f79260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f78d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:56626/0 (socket says 172.21.15.94:56626) 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9ae59c640 1 -- 172.21.15.94:0/3900820180 learned_addr learned my addr 172.21.15.94:0/3900820180 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.809+0000 7fa9aed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f70d40 0x5618f9f73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa9ae59c640 1 -- 172.21.15.94:0/3900820180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f70d40 msgr2=0x5618f9f73cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f70d40 0x5618f9f73cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa9ae59c640 1 -- 172.21.15.94:0/3900820180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f66480 msgr2=0x5618f9f79260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f66480 0x5618f9f79260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa9ae59c640 1 -- 172.21.15.94:0/3900820180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f9f74210 con 0x5618f9f65b60 2024-09-19T04:27:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa9aed9d640 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f70d40 0x5618f9f73cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f66480 0x5618f9f79260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa9ae59c640 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f78d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0000c00 tx=0x7fa9a002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1722d5498371150 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:27:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa995ffb640 1 -- 172.21.15.94:0/3900820180 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0040040 con 0x5618f9f65b60 2024-09-19T04:27:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa995ffb640 1 -- 172.21.15.94:0/3900820180 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7fa9a0004040 con 0x5618f9f65b60 2024-09-19T04:27:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.810+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3900820180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9980bae50 con 0x5618f9f65b60 2024-09-19T04:27:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa995ffb640 1 -- 172.21.15.94:0/3900820180 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa9a006e050 con 0x5618f9f65b60 2024-09-19T04:27:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa9b6be3740 10 client.26557 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3900820180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5618f9ff4640 con 0x5618f9f65b60 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa995ffb640 1 -- 172.21.15.94:0/3900820180 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9980bae50 con 0x5618f9f65b60 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa995ffb640 1 -- 172.21.15.94:0/3900820180 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fa9a00046c0 con 0x5618f9f65b60 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa9b6be3740 10 client.26557 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa9b6be3740 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa9b6be3740 20 client.26557 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5618f9ff49d0 0x5618f9ff6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa9b6be3740 4 client.26557 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3900820180 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5618f9ed2d00 con 0x5618f9ff49d0 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.811+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5618f9ff49d0 0x5618f9ff6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.812+0000 7fa995ffb640 10 client.26557 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.812+0000 7fa9a7fff640 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5618f9ff49d0 0x5618f9ff6df0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23fdccadc71a9ade server_cookie=571aad2fd456ffd2 in_seq=0 out_seq=0 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa995ffb640 1 -- 172.21.15.94:0/3900820180 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5618f9ed2d00 con 0x5618f9ff49d0 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa995ffb640 10 client.26557 handle_command_reply: tid=0 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 client.26557 shutdown 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3900820180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 msgr2=0x5618f9f78d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f78d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0000c00 tx=0x7fa9a002a410 comp rx=0 tx=0).stop 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3900820180 shutdown_connections 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5618f9ff49d0 0x5618f9ff6df0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618f9f70d40 0x5618f9f73cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618f9f66480 0x5618f9f79260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa995ffb640 0 client.26557 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 --2- 172.21.15.94:0/3900820180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618f9f65b60 0x5618f9f78d20 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0000c00 tx=0x7fa9a002a410 comp rx=0 tx=0).stop 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3900820180 >> 172.21.15.94:0/3900820180 conn(0x5618f9f620f0 msgr2=0x5618f9f61270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.813+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3900820180 shutdown_connections 2024-09-19T04:27:43.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.814+0000 7fa9b6be3740 1 -- 172.21.15.94:0/3900820180 wait complete. 2024-09-19T04:27:43.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.814+0000 7fa9b6be3740 20 client.26557 trim_cache size 0 max 16384 2024-09-19T04:27:43.820 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:27:43.820 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:27:43.820 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "72938a9f-4cdc-424d-8456-2a7c459d68e6", 2024-09-19T04:27:43.820 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:27:43.820 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:27:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.814+0000 7fa9b480a640 1 -- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa99c0a33b0 msgr2=0x7fa99c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.814+0000 7fa9b480a640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa99c0a33b0 0x7fa99c0a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009fd0 tx=0x7fa9a8004ab0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.814+0000 7fa9b480a640 1 -- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b01012e0 msgr2=0x7fa9b019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.814+0000 7fa9b480a640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b01012e0 0x7fa9b019aac0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa99800e9f0 tx=0x7fa99800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.815+0000 7fa9b480a640 1 -- 172.21.15.94:0/930898494 shutdown_connections 2024-09-19T04:27:43.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.815+0000 7fa9b480a640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa99c0a33b0 0x7fa99c0a5870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.815+0000 7fa9b480a640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9b010e4e0 0x7fa9b019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.815+0000 7fa9b480a640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b01012e0 0x7fa9b019aac0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.815+0000 7fa9b480a640 1 --2- 172.21.15.94:0/930898494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9b0100930 0x7fa9b019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.815+0000 7fa9b480a640 1 -- 172.21.15.94:0/930898494 >> 172.21.15.94:0/930898494 conn(0x7fa9b00fc7a0 msgr2=0x7fa9b010f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.815+0000 7fa9b480a640 1 -- 172.21.15.94:0/930898494 shutdown_connections 2024-09-19T04:27:43.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:43.815+0000 7fa9b480a640 1 -- 172.21.15.94:0/930898494 wait complete. 2024-09-19T04:27:44.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:27:44.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:27:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:27:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:27:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:27:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:27:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:27:44.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:44 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:27:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:27:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:27:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:27:44.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:27:44.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:27:44.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:44 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:27:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:44 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:44.912 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:45.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.527+0000 7f9c3022f640 1 -- 172.21.15.94:0/2780147377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28102d30 msgr2=0x7f9c28103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.527+0000 7f9c3022f640 1 --2- 172.21.15.94:0/2780147377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28102d30 0x7f9c28103110 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009930 tx=0x7f9c2402a420 comp rx=0 tx=0).stop 2024-09-19T04:27:45.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.527+0000 7f9c3022f640 1 -- 172.21.15.94:0/2780147377 shutdown_connections 2024-09-19T04:27:45.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.527+0000 7f9c3022f640 1 --2- 172.21.15.94:0/2780147377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c281082a0 0x7f9c2810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.527+0000 7f9c3022f640 1 --2- 172.21.15.94:0/2780147377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c281036e0 0x7f9c28107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.527+0000 7f9c3022f640 1 --2- 172.21.15.94:0/2780147377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28102d30 0x7f9c28103110 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.527+0000 7f9c3022f640 1 -- 172.21.15.94:0/2780147377 >> 172.21.15.94:0/2780147377 conn(0x7f9c280fc7d0 msgr2=0x7f9c280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.527+0000 7f9c3022f640 1 -- 172.21.15.94:0/2780147377 shutdown_connections 2024-09-19T04:27:45.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.527+0000 7f9c3022f640 1 -- 172.21.15.94:0/2780147377 wait complete. 2024-09-19T04:27:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c3022f640 1 Processor -- start 2024-09-19T04:27:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c3022f640 1 -- start start 2024-09-19T04:27:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c3022f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28102d30 0x7f9c28198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c3022f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c281036e0 0x7f9c28198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c3022f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c281082a0 0x7f9c2819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c3022f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28111f90 con 0x7f9c281036e0 2024-09-19T04:27:45.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c3022f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28111e10 con 0x7f9c28102d30 2024-09-19T04:27:45.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c3022f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28112110 con 0x7f9c281082a0 2024-09-19T04:27:45.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c2d7ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c281036e0 0x7f9c28198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c2e7cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c281082a0 0x7f9c2819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c2e7cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c281082a0 0x7f9c2819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:56650/0 (socket says 172.21.15.94:56650) 2024-09-19T04:27:45.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.528+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/4277160980 learned_addr learned my addr 172.21.15.94:0/4277160980 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:45.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28102d30 0x7f9c28198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/4277160980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28102d30 msgr2=0x7f9c28198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28102d30 0x7f9c28198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/4277160980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c281036e0 msgr2=0x7f9c28198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c281036e0 0x7f9c28198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/4277160980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2819d4b0 con 0x7f9c281082a0 2024-09-19T04:27:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c281082a0 0x7f9c2819cdd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000eb10 tx=0x7f9c2000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c1effd640 1 -- 172.21.15.94:0/4277160980 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20004b20 con 0x7f9c281082a0 2024-09-19T04:27:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c1effd640 1 -- 172.21.15.94:0/4277160980 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c20004cc0 con 0x7f9c281082a0 2024-09-19T04:27:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c3022f640 1 -- 172.21.15.94:0/4277160980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2819d7a0 con 0x7f9c281082a0 2024-09-19T04:27:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.529+0000 7f9c1effd640 1 -- 172.21.15.94:0/4277160980 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20005570 con 0x7f9c281082a0 2024-09-19T04:27:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.530+0000 7f9c3022f640 1 -- 172.21.15.94:0/4277160980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c281008c0 con 0x7f9c281082a0 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.532+0000 7f9c1effd640 1 -- 172.21.15.94:0/4277160980 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9c20005730 con 0x7f9c281082a0 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.533+0000 7f9c1effd640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9c140a3430 0x7f9c140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.533+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9c140a3430 0x7f9c140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.533+0000 7f9c1effd640 1 -- 172.21.15.94:0/4277160980 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f9c200d41c0 con 0x7f9c281082a0 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c32608740 1 Processor -- start 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c32608740 1 -- start start 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9c140a3430 0x7f9c140a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c24006010 tx=0x7f9c24000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c32608740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601ea4a0 0x5624601ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c32608740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601eadc0 0x5624601f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c32608740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601f5680 0x5624601f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c32608740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5624601502e0 con 0x5624601f5680 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c32608740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5624600f7ce0 con 0x5624601ea4a0 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c32608740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56246014c2a0 con 0x5624601eadc0 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c2dfcb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601ea4a0 0x5624601ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c2dfcb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601ea4a0 0x5624601ea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59480/0 (socket says 172.21.15.94:59480) 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.534+0000 7f9c2dfcb640 1 -- 172.21.15.94:0/3183961069 learned_addr learned my addr 172.21.15.94:0/3183961069 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.535+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/3183961069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601f5680 0x5624601f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.535+0000 7f9c2dfcb640 1 -- 172.21.15.94:0/3183961069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601eadc0 msgr2=0x5624601f5140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.535+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/3183961069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601eadc0 0x5624601f5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.535+0000 7f9c2dfcb640 1 -- 172.21.15.94:0/3183961069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601f5680 msgr2=0x5624601f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.535+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/3183961069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601f5680 0x5624601f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.535+0000 7f9c2dfcb640 1 -- 172.21.15.94:0/3183961069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624600dc440 con 0x5624601ea4a0 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.535+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/3183961069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601ea4a0 0x5624601ea880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c24033000 tx=0x7f9c2403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=170134d5c463211c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.535+0000 7f9c13fff640 1 -- 172.21.15.94:0/3183961069 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24035050 con 0x5624601ea4a0 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.536+0000 7f9c13fff640 1 -- 172.21.15.94:0/3183961069 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c2404d070 con 0x5624601ea4a0 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.536+0000 7f9c13fff640 1 -- 172.21.15.94:0/3183961069 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24003c90 con 0x5624601ea4a0 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.536+0000 7f9c32608740 1 -- 172.21.15.94:0/3183961069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601ea4a0 msgr2=0x5624601ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.536+0000 7f9c32608740 1 --2- 172.21.15.94:0/3183961069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601ea4a0 0x5624601ea880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c24033000 tx=0x7f9c2403c000 comp rx=0 tx=0).stop 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.537+0000 7f9c32608740 1 -- 172.21.15.94:0/3183961069 shutdown_connections 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.537+0000 7f9c32608740 1 --2- 172.21.15.94:0/3183961069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601f5680 0x5624601f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.537+0000 7f9c32608740 1 --2- 172.21.15.94:0/3183961069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601eadc0 0x5624601f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.537+0000 7f9c32608740 1 --2- 172.21.15.94:0/3183961069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601ea4a0 0x5624601ea880 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.537+0000 7f9c32608740 1 -- 172.21.15.94:0/3183961069 >> 172.21.15.94:0/3183961069 conn(0x5624601e6a30 msgr2=0x5624601e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.537+0000 7f9c32608740 1 -- 172.21.15.94:0/3183961069 shutdown_connections 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.537+0000 7f9c32608740 1 -- 172.21.15.94:0/3183961069 wait complete. 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.537+0000 7f9c32608740 1 -- start start 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.538+0000 7f9c32608740 1 Processor -- start 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.538+0000 7f9c32608740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.538+0000 7f9c32608740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601f2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.538+0000 7f9c32608740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624601f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c32608740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562460200a60 con 0x5624601eadc0 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c32608740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56246010b050 con 0x5624601f5680 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c32608740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562460200be0 con 0x5624601ea4a0 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2d7ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2d7ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48998/0 (socket says 172.21.15.94:48998) 2024-09-19T04:27:45.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2d7ca640 1 -- 172.21.15.94:0/2381452351 learned_addr learned my addr 172.21.15.94:0/2381452351 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:45.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624601f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c32608740 10 client.0 _ll_register_callbacks cb 0x56246014b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:45.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/2381452351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 msgr2=0x5624601f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/2381452351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 msgr2=0x5624601f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/2381452351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245fec13c0 con 0x5624601f5680 2024-09-19T04:27:45.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.539+0000 7f9c2d7ca640 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.540+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624601f27f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9c200d1980 tx=0x7f9c2009fe70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee418530e61c59b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.540+0000 7f9c117fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.540+0000 7f9c117fa640 1 -- 172.21.15.94:0/2381452351 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200a8ef0 con 0x5624601f5680 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.540+0000 7f9c117fa640 1 -- 172.21.15.94:0/2381452351 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f9c2009f960 con 0x5624601f5680 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.540+0000 7f9c32608740 1 -- 172.21.15.94:0/2381452351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c2404d070 con 0x5624601f5680 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.540+0000 7f9c117fa640 1 -- 172.21.15.94:0/2381452351 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f9c200a48c0 con 0x5624601f5680 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.540+0000 7f9c32608740 10 client.26585 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.540+0000 7f9c32608740 1 -- 172.21.15.94:0/2381452351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56246020c460 con 0x5624601f5680 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.540+0000 7f9c117fa640 1 -- 172.21.15.94:0/2381452351 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c200a9180 con 0x5624601f5680 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.541+0000 7f9c117fa640 1 -- 172.21.15.94:0/2381452351 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f9c2009f3d0 con 0x5624601f5680 2024-09-19T04:27:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.541+0000 7f9c32608740 10 client.26585 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.541+0000 7f9c32608740 10 client.26585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.541+0000 7f9c32608740 20 client.26585 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.541+0000 7f9c32608740 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56246020d970 0x56246020fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.541+0000 7f9c32608740 4 client.26585 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.541+0000 7f9c32608740 1 -- 172.21.15.94:0/2381452351 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9c2404d070 con 0x56246020d970 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.541+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56246020d970 0x56246020fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.542+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56246020d970 0x56246020fdd0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163d523ccecf5519 server_cookie=9ca6e688b0392ad5 in_seq=0 out_seq=0 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.542+0000 7f9c117fa640 10 client.26585 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c117fa640 1 -- 172.21.15.94:0/2381452351 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9c2404d070 con 0x56246020d970 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c117fa640 10 client.26585 handle_command_reply: tid=0 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 client.26585 shutdown 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 -- 172.21.15.94:0/2381452351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 msgr2=0x5624601f27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624601f27f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9c200d1980 tx=0x7f9c2009fe70 comp rx=0 tx=0).stop 2024-09-19T04:27:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 -- 172.21.15.94:0/2381452351 shutdown_connections 2024-09-19T04:27:45.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56246020d970 0x56246020fdd0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c117fa640 0 client.26585 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:45.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624601f27f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 --2- 172.21.15.94:0/2381452351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 -- 172.21.15.94:0/2381452351 >> 172.21.15.94:0/2381452351 conn(0x5624601e6a30 msgr2=0x5624601e5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 -- 172.21.15.94:0/2381452351 shutdown_connections 2024-09-19T04:27:45.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.543+0000 7f9c32608740 1 -- 172.21.15.94:0/2381452351 wait complete. 2024-09-19T04:27:45.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.544+0000 7f9c32608740 20 client.26585 trim_cache size 0 max 16384 2024-09-19T04:27:45.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.569+0000 7f9c32608740 1 Processor -- start 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.569+0000 7f9c32608740 1 -- start start 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.569+0000 7f9c32608740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601ea4a0 0x562460209cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.569+0000 7f9c32608740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601eadc0 0x5624601f4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.569+0000 7f9c2dfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601ea4a0 0x562460209cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.569+0000 7f9c32608740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624602156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.569+0000 7f9c2dfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601ea4a0 0x562460209cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49008/0 (socket says 172.21.15.94:49008) 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.569+0000 7f9c2dfcb640 1 -- 172.21.15.94:0/935964484 learned_addr learned my addr 172.21.15.94:0/935964484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c2d7ca640 1 --2- 172.21.15.94:0/935964484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601eadc0 0x5624601f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c32608740 1 -- 172.21.15.94:0/935964484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281ae060 con 0x5624601ea4a0 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c32608740 1 -- 172.21.15.94:0/935964484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281ab870 con 0x5624601f5680 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c32608740 1 -- 172.21.15.94:0/935964484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281a96e0 con 0x5624601eadc0 2024-09-19T04:27:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/935964484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624602156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/935964484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601eadc0 msgr2=0x5624601f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/935964484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601eadc0 0x5624601f4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/935964484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601ea4a0 msgr2=0x562460209cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/935964484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601ea4a0 0x562460209cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.570+0000 7f9c2e7cc640 1 -- 172.21.15.94:0/935964484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624601f0880 con 0x5624601f5680 2024-09-19T04:27:45.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.571+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/935964484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624602156e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c2009f7a0 tx=0x7f9c200a0690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38129304b69acb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.571+0000 7f9c137fe640 1 -- 172.21.15.94:0/935964484 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200a9180 con 0x5624601f5680 2024-09-19T04:27:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.571+0000 7f9c137fe640 1 -- 172.21.15.94:0/935964484 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c200a0c10 con 0x5624601f5680 2024-09-19T04:27:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.571+0000 7f9c137fe640 1 -- 172.21.15.94:0/935964484 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200a8cf0 con 0x5624601f5680 2024-09-19T04:27:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.572+0000 7f9c32608740 1 -- 172.21.15.94:0/935964484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 msgr2=0x5624602156e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.572+0000 7f9c32608740 1 --2- 172.21.15.94:0/935964484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624602156e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c2009f7a0 tx=0x7f9c200a0690 comp rx=0 tx=0).stop 2024-09-19T04:27:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.572+0000 7f9c32608740 1 -- 172.21.15.94:0/935964484 shutdown_connections 2024-09-19T04:27:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.572+0000 7f9c32608740 1 --2- 172.21.15.94:0/935964484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624602156e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.572+0000 7f9c32608740 1 --2- 172.21.15.94:0/935964484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601eadc0 0x5624601f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.572+0000 7f9c32608740 1 --2- 172.21.15.94:0/935964484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601ea4a0 0x562460209cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.572+0000 7f9c32608740 1 -- 172.21.15.94:0/935964484 >> 172.21.15.94:0/935964484 conn(0x5624601e6a30 msgr2=0x5624601f3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.572+0000 7f9c32608740 1 -- 172.21.15.94:0/935964484 shutdown_connections 2024-09-19T04:27:45.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.572+0000 7f9c32608740 1 -- 172.21.15.94:0/935964484 wait complete. 2024-09-19T04:27:45.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.573+0000 7f9c32608740 1 -- start start 2024-09-19T04:27:45.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.573+0000 7f9c32608740 1 Processor -- start 2024-09-19T04:27:45.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.573+0000 7f9c32608740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c32608740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c32608740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624601efef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c32608740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281a9d30 con 0x5624601eadc0 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c32608740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281ab1a0 con 0x5624601f5680 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c32608740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281aa400 con 0x5624601ea4a0 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2dfcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2dfcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:56680/0 (socket says 172.21.15.94:56680) 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2dfcb640 1 -- 172.21.15.94:0/3737314548 learned_addr learned my addr 172.21.15.94:0/3737314548 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2e7cc640 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624601efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2d7ca640 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c32608740 10 client.0 _ll_register_callbacks cb 0x7f9c20004ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2dfcb640 1 -- 172.21.15.94:0/3737314548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 msgr2=0x5624601efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624601efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2dfcb640 1 -- 172.21.15.94:0/3737314548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 msgr2=0x5624601ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2dfcb640 1 -- 172.21.15.94:0/3737314548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624602777e0 con 0x5624601ea4a0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c2dfcb640 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601ef470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c2402aa10 tx=0x7f9c24005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca1c4790e4d56a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.574+0000 7f9c117fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.575+0000 7f9c117fa640 1 -- 172.21.15.94:0/3737314548 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24035040 con 0x5624601ea4a0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.575+0000 7f9c32608740 1 -- 172.21.15.94:0/3737314548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c200a0c10 con 0x5624601ea4a0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.575+0000 7f9c117fa640 1 -- 172.21.15.94:0/3737314548 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8585+0+0 (secure 0 0 0) 0x7f9c240038e0 con 0x5624601ea4a0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.575+0000 7f9c117fa640 1 -- 172.21.15.94:0/3737314548 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f9c2406c050 con 0x5624601ea4a0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.575+0000 7f9c32608740 10 client.26569 fetch_fsmap learned FSMap version 48 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.575+0000 7f9c32608740 1 -- 172.21.15.94:0/3737314548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562460278a90 con 0x5624601ea4a0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.575+0000 7f9c117fa640 1 -- 172.21.15.94:0/3737314548 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c2404d040 con 0x5624601ea4a0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.575+0000 7f9c117fa640 1 -- 172.21.15.94:0/3737314548 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f9c24004ca0 con 0x5624601ea4a0 2024-09-19T04:27:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.576+0000 7f9c32608740 10 client.26569 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:27:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.576+0000 7f9c32608740 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.576+0000 7f9c32608740 20 client.26569 populate_metadata read hostname 'smithi094' 2024-09-19T04:27:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.576+0000 7f9c32608740 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56246020d970 0x56246027ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.576+0000 7f9c32608740 4 client.26569 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:27:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.576+0000 7f9c32608740 1 -- 172.21.15.94:0/3737314548 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9c200a0c10 con 0x56246020d970 2024-09-19T04:27:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.576+0000 7f9c2d7ca640 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56246020d970 0x56246027ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.576+0000 7f9c117fa640 10 client.26569 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.576+0000 7f9c2d7ca640 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56246020d970 0x56246027ade0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a639f0dfb57295 server_cookie=a12d4a25440b1036 in_seq=0 out_seq=0 2024-09-19T04:27:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c117fa640 1 -- 172.21.15.94:0/3737314548 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f9c200a0c10 con 0x56246020d970 2024-09-19T04:27:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c117fa640 10 client.26569 handle_command_reply: tid=0 2024-09-19T04:27:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 client.26569 shutdown 2024-09-19T04:27:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 -- 172.21.15.94:0/3737314548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 msgr2=0x5624601ef470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601ef470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c2402aa10 tx=0x7f9c24005a40 comp rx=0 tx=0).stop 2024-09-19T04:27:45.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 -- 172.21.15.94:0/3737314548 shutdown_connections 2024-09-19T04:27:45.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56246020d970 0x56246027ade0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624601f5680 0x5624601efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624601eadc0 0x5624601ef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c117fa640 0 client.26569 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:27:45.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 --2- 172.21.15.94:0/3737314548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624601ea4a0 0x5624601ef470 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 -- 172.21.15.94:0/3737314548 >> 172.21.15.94:0/3737314548 conn(0x5624601e6a30 msgr2=0x5624601ed870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 -- 172.21.15.94:0/3737314548 shutdown_connections 2024-09-19T04:27:45.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 1 -- 172.21.15.94:0/3737314548 wait complete. 2024-09-19T04:27:45.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.577+0000 7f9c32608740 20 client.26569 trim_cache size 0 max 16384 2024-09-19T04:27:45.585 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:27:45.586 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:27:45.586 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:27:45.586 INFO:teuthology.orchestra.run.smithi094.stdout: "72938a9f-4cdc-424d-8456-2a7c459d68e6": { 2024-09-19T04:27:45.586 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:27:45.586 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "72938a9f-4cdc-424d-8456-2a7c459d68e6", 2024-09-19T04:27:45.586 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:27:45.586 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:27:45.586 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:27:45.587 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:27:45.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 -- 172.21.15.94:0/4277160980 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9c140a3430 msgr2=0x7f9c140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9c140a3430 0x7f9c140a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c24006010 tx=0x7f9c24000bc0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 -- 172.21.15.94:0/4277160980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c281082a0 msgr2=0x7f9c2819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c281082a0 0x7f9c2819cdd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000eb10 tx=0x7f9c2000c2d0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 -- 172.21.15.94:0/4277160980 shutdown_connections 2024-09-19T04:27:45.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9c140a3430 0x7f9c140a58f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c281082a0 0x7f9c2819cdd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c281036e0 0x7f9c28198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 --2- 172.21.15.94:0/4277160980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28102d30 0x7f9c28198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.578+0000 7f9c3022f640 1 -- 172.21.15.94:0/4277160980 >> 172.21.15.94:0/4277160980 conn(0x7f9c280fc7d0 msgr2=0x7f9c28105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.588 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:45 smithi094 ceph-mon[30972]: pgmap v396: 97 pgs: 97 active+clean; 4.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 784 op/s 2024-09-19T04:27:45.588 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:45 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:45.588 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:45 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:45.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.579+0000 7f9c3022f640 1 -- 172.21.15.94:0/4277160980 shutdown_connections 2024-09-19T04:27:45.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:27:45.579+0000 7f9c3022f640 1 -- 172.21.15.94:0/4277160980 wait complete. 2024-09-19T04:27:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 690.0 s, 346.8 tps, lat 288.149 ms stddev 109.198 2024-09-19T04:27:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:45 smithi104 ceph-mon[34145]: pgmap v396: 97 pgs: 97 active+clean; 4.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 784 op/s 2024-09-19T04:27:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:45 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:45 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:45.685 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:45 smithi112 ceph-mon[35051]: pgmap v396: 97 pgs: 97 active+clean; 4.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 784 op/s 2024-09-19T04:27:45.685 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:45 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:45.685 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:45 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:46.264 INFO:tasks.cephfs.filesystem:scrub status for tag:72938a9f-4cdc-424d-8456-2a7c459d68e6 - {'path': '/', 'tag': '72938a9f-4cdc-424d-8456-2a7c459d68e6', 'options': 'recursive,force'} 2024-09-19T04:27:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:47 smithi094 ceph-mon[30972]: pgmap v397: 97 pgs: 97 active+clean; 4.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 4.1 MiB/s wr, 526 op/s 2024-09-19T04:27:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:47 smithi104 ceph-mon[34145]: pgmap v397: 97 pgs: 97 active+clean; 4.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 4.1 MiB/s wr, 526 op/s 2024-09-19T04:27:47.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:47 smithi112 ceph-mon[35051]: pgmap v397: 97 pgs: 97 active+clean; 4.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 4.1 MiB/s wr, 526 op/s 2024-09-19T04:27:49.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:49 smithi094 ceph-mon[30972]: pgmap v398: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 4.9 MiB/s wr, 651 op/s 2024-09-19T04:27:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:49 smithi104 ceph-mon[34145]: pgmap v398: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 4.9 MiB/s wr, 651 op/s 2024-09-19T04:27:49.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:49 smithi112 ceph-mon[35051]: pgmap v398: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 4.9 MiB/s wr, 651 op/s 2024-09-19T04:27:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 695.0 s, 393.0 tps, lat 254.575 ms stddev 37.259 2024-09-19T04:27:51.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:51 smithi094 ceph-mon[30972]: pgmap v399: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 5.0 MiB/s wr, 664 op/s 2024-09-19T04:27:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:51 smithi104 ceph-mon[34145]: pgmap v399: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 5.0 MiB/s wr, 664 op/s 2024-09-19T04:27:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:51 smithi112 ceph-mon[35051]: pgmap v399: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 5.0 MiB/s wr, 664 op/s 2024-09-19T04:27:52.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:52 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:27:52.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:52 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:27:52.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:52 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:27:52.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:52 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:27:52.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:52 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:27:52.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:52 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:27:52.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:52 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:27:52.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:52 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:27:52.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:52 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:27:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:53 smithi104 ceph-mon[34145]: pgmap v400: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 4.4 MiB/s wr, 570 op/s 2024-09-19T04:27:53.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:53 smithi112 ceph-mon[35051]: pgmap v400: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 4.4 MiB/s wr, 570 op/s 2024-09-19T04:27:53.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:53 smithi094 ceph-mon[30972]: pgmap v400: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 4.4 MiB/s wr, 570 op/s 2024-09-19T04:27:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 700.0 s, 396.6 tps, lat 253.052 ms stddev 38.082 2024-09-19T04:27:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:55 smithi104 ceph-mon[34145]: pgmap v401: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 5.8 MiB/s wr, 757 op/s 2024-09-19T04:27:55.642 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:55 smithi094 ceph-mon[30972]: pgmap v401: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 5.8 MiB/s wr, 757 op/s 2024-09-19T04:27:55.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:55 smithi112 ceph-mon[35051]: pgmap v401: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 5.8 MiB/s wr, 757 op/s 2024-09-19T04:27:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:57 smithi104 ceph-mon[34145]: pgmap v402: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.7 MiB/s wr, 487 op/s 2024-09-19T04:27:57.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:57 smithi112 ceph-mon[35051]: pgmap v402: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.7 MiB/s wr, 487 op/s 2024-09-19T04:27:57.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:57 smithi094 ceph-mon[30972]: pgmap v402: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.7 MiB/s wr, 487 op/s 2024-09-19T04:27:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:27:59 smithi104 ceph-mon[34145]: pgmap v403: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.5 MiB/s wr, 596 op/s 2024-09-19T04:27:59.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:27:59 smithi112 ceph-mon[35051]: pgmap v403: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.5 MiB/s wr, 596 op/s 2024-09-19T04:27:59.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:27:59 smithi094 ceph-mon[30972]: pgmap v403: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.5 MiB/s wr, 596 op/s 2024-09-19T04:28:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 705.0 s, 356.0 tps, lat 278.727 ms stddev 40.050 2024-09-19T04:28:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:01 smithi104 ceph-mon[34145]: pgmap v404: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 4.7 MiB/s wr, 613 op/s 2024-09-19T04:28:01.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:01 smithi112 ceph-mon[35051]: pgmap v404: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 4.7 MiB/s wr, 613 op/s 2024-09-19T04:28:01.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:01 smithi094 ceph-mon[30972]: pgmap v404: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 4.7 MiB/s wr, 613 op/s 2024-09-19T04:28:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:03 smithi104 ceph-mon[34145]: pgmap v405: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 3.2 MiB/s wr, 437 op/s 2024-09-19T04:28:03.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:03 smithi112 ceph-mon[35051]: pgmap v405: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 3.2 MiB/s wr, 437 op/s 2024-09-19T04:28:03.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:03 smithi094 ceph-mon[30972]: pgmap v405: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 3.2 MiB/s wr, 437 op/s 2024-09-19T04:28:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 710.0 s, 384.0 tps, lat 262.938 ms stddev 45.162 2024-09-19T04:28:05.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:05 smithi112 ceph-mon[35051]: pgmap v406: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 5.4 MiB/s wr, 733 op/s 2024-09-19T04:28:05.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:05 smithi094 ceph-mon[30972]: pgmap v406: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 5.4 MiB/s wr, 733 op/s 2024-09-19T04:28:05.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:05 smithi104 ceph-mon[34145]: pgmap v406: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 5.4 MiB/s wr, 733 op/s 2024-09-19T04:28:07.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:07 smithi112 ceph-mon[35051]: pgmap v407: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 4.1 MiB/s wr, 547 op/s 2024-09-19T04:28:07.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:07 smithi094 ceph-mon[30972]: pgmap v407: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 4.1 MiB/s wr, 547 op/s 2024-09-19T04:28:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:07 smithi104 ceph-mon[34145]: pgmap v407: 97 pgs: 97 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 4.1 MiB/s wr, 547 op/s 2024-09-19T04:28:08.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:08 smithi094 ceph-mon[30972]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:28:08.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:08 smithi112 ceph-mon[35051]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:28:08.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:08 smithi104 ceph-mon[34145]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:28:09.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:09 smithi094 ceph-mon[30972]: pgmap v409: 97 pgs: 6 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 75 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.4 MiB/s wr, 609 op/s 2024-09-19T04:28:09.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:09 smithi112 ceph-mon[35051]: pgmap v409: 97 pgs: 6 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 75 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.4 MiB/s wr, 609 op/s 2024-09-19T04:28:09.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:09 smithi104 ceph-mon[34145]: pgmap v409: 97 pgs: 6 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 75 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.4 MiB/s wr, 609 op/s 2024-09-19T04:28:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 715.0 s, 354.4 tps, lat 282.106 ms stddev 69.164 2024-09-19T04:28:11.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:11 smithi112 ceph-mon[35051]: pgmap v410: 97 pgs: 10 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 62 active+clean; 3.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 4.9 MiB/s wr, 652 op/s 2024-09-19T04:28:11.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:11 smithi094 ceph-mon[30972]: pgmap v410: 97 pgs: 10 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 62 active+clean; 3.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 4.9 MiB/s wr, 652 op/s 2024-09-19T04:28:11.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:11 smithi104 ceph-mon[34145]: pgmap v410: 97 pgs: 10 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 62 active+clean; 3.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 4.9 MiB/s wr, 652 op/s 2024-09-19T04:28:13.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:13 smithi112 ceph-mon[35051]: pgmap v411: 97 pgs: 10 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 62 active+clean; 3.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 4.9 MiB/s wr, 652 op/s 2024-09-19T04:28:13.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:13 smithi094 ceph-mon[30972]: pgmap v411: 97 pgs: 10 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 62 active+clean; 3.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 4.9 MiB/s wr, 652 op/s 2024-09-19T04:28:13.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:13 smithi104 ceph-mon[34145]: pgmap v411: 97 pgs: 10 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 62 active+clean; 3.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 4.9 MiB/s wr, 652 op/s 2024-09-19T04:28:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 720.0 s, 392.4 tps, lat 253.850 ms stddev 32.970 2024-09-19T04:28:15.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:15 smithi112 ceph-mon[35051]: pgmap v412: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.6 MiB/s wr, 601 op/s 2024-09-19T04:28:15.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:15 smithi094 ceph-mon[30972]: pgmap v412: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.6 MiB/s wr, 601 op/s 2024-09-19T04:28:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:15 smithi104 ceph-mon[34145]: pgmap v412: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.6 MiB/s wr, 601 op/s 2024-09-19T04:28:16.266 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:28:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:16.935 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:16 smithi094 ceph-mon[30972]: pgmap v413: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.6 MiB/s wr, 601 op/s 2024-09-19T04:28:16.936 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:16 smithi094 ceph-mon[30972]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:28:17.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:16 smithi112 ceph-mon[35051]: pgmap v413: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.6 MiB/s wr, 601 op/s 2024-09-19T04:28:17.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:16 smithi112 ceph-mon[35051]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:28:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:16 smithi104 ceph-mon[34145]: pgmap v413: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.6 MiB/s wr, 601 op/s 2024-09-19T04:28:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:16 smithi104 ceph-mon[34145]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:28:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.198+0000 7f5dbc08d640 1 -- 172.21.15.94:0/3624010703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 msgr2=0x7f5db410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.198+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/3624010703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 0x7f5db410d960 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009cd0 tx=0x7f5dac02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.198+0000 7f5dbc08d640 1 -- 172.21.15.94:0/3624010703 shutdown_connections 2024-09-19T04:28:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.198+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/3624010703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 0x7f5db410d960 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.198+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/3624010703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5db4100a90 0x7f5db4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.198+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/3624010703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db4106a90 0x7f5db4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.198+0000 7f5dbc08d640 1 -- 172.21.15.94:0/3624010703 >> 172.21.15.94:0/3624010703 conn(0x7f5db40fc7d0 msgr2=0x7f5db40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.198+0000 7f5dbc08d640 1 -- 172.21.15.94:0/3624010703 shutdown_connections 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.198+0000 7f5dbc08d640 1 -- 172.21.15.94:0/3624010703 wait complete. 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.199+0000 7f5dbc08d640 1 Processor -- start 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.199+0000 7f5dbc08d640 1 -- start start 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.199+0000 7f5dbc08d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5db4100a90 0x7f5db41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.199+0000 7f5dbc08d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 0x7f5db4199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5dbc08d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db4106a90 0x7f5db41a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9e29640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5db4100a90 0x7f5db41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9e29640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5db4100a90 0x7f5db41991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:34456/0 (socket says 172.21.15.94:34456) 2024-09-19T04:28:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9628640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 0x7f5db4199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5dbc08d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4110d90 con 0x7f5db4101430 2024-09-19T04:28:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5dbc08d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4110c10 con 0x7f5db4100a90 2024-09-19T04:28:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5dbc08d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4110f10 con 0x7f5db4106a90 2024-09-19T04:28:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9628640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 0x7f5db4199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45640/0 (socket says 172.21.15.94:45640) 2024-09-19T04:28:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9628640 1 -- 172.21.15.94:0/1169270569 learned_addr learned my addr 172.21.15.94:0/1169270569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db4106a90 0x7f5db41a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9e29640 1 -- 172.21.15.94:0/1169270569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db4106a90 msgr2=0x7f5db41a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9e29640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db4106a90 0x7f5db41a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9e29640 1 -- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 msgr2=0x7f5db4199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9e29640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 0x7f5db4199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9e29640 1 -- 172.21.15.94:0/1169270569 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db4196fb0 con 0x7f5db4100a90 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db4106a90 0x7f5db41a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9628640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 0x7f5db4199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.200+0000 7f5db9e29640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5db4100a90 0x7f5db41991c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e9f0 tx=0x7f5db000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.201+0000 7f5daaffd640 1 -- 172.21.15.94:0/1169270569 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000ccc0 con 0x7f5db4100a90 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.201+0000 7f5dbc08d640 1 -- 172.21.15.94:0/1169270569 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db41972a0 con 0x7f5db4100a90 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.201+0000 7f5daaffd640 1 -- 172.21.15.94:0/1169270569 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5db0005020 con 0x7f5db4100a90 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.201+0000 7f5dbc08d640 1 -- 172.21.15.94:0/1169270569 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db41977e0 con 0x7f5db4100a90 2024-09-19T04:28:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.201+0000 7f5daaffd640 1 -- 172.21.15.94:0/1169270569 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0004a00 con 0x7f5db4100a90 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.203+0000 7f5daaffd640 1 -- 172.21.15.94:0/1169270569 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5db0004ba0 con 0x7f5db4100a90 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.204+0000 7f5dbe466740 1 Processor -- start 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.204+0000 7f5dbe466740 1 -- start start 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.204+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f91fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.204+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f9209140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.204+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f920ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.204+0000 7f5dbe466740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5605f91642e0 con 0x5605f91fe4a0 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.204+0000 7f5dbe466740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5605f910bce0 con 0x5605f91fedc0 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.204+0000 7f5dbe466740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605f91602a0 con 0x5605f9209680 2024-09-19T04:28:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5db9e29640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f9209140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5db9628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f920ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5dba62a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f91fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5db9628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f920ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:51836/0 (socket says 172.21.15.94:51836) 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5db9628640 1 -- 172.21.15.94:0/1390235394 learned_addr learned my addr 172.21.15.94:0/1390235394 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5daaffd640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5da00a3430 0x7f5da00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5dba62a640 1 -- 172.21.15.94:0/1390235394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 msgr2=0x5605f920ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1390235394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f920ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5dba62a640 1 -- 172.21.15.94:0/1390235394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 msgr2=0x5605f9209140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1390235394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f9209140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5dba62a640 1 -- 172.21.15.94:0/1390235394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f90f0440 con 0x5605f91fe4a0 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5db9628640 1 --2- 172.21.15.94:0/1390235394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f920ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5daaffd640 1 -- 172.21.15.94:0/1169270569 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f5db00cfc80 con 0x7f5db4100a90 2024-09-19T04:28:17.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5da00a3430 0x7f5da00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.205+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1390235394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f91fe880 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009ca0 tx=0x7f5dac009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67583b5c05e2eed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.206+0000 7f5d9ffff640 1 -- 172.21.15.94:0/1390235394 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac003a70 con 0x5605f91fe4a0 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.206+0000 7f5d9ffff640 1 -- 172.21.15.94:0/1390235394 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dac036040 con 0x5605f91fe4a0 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.206+0000 7f5d9ffff640 1 -- 172.21.15.94:0/1390235394 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0305a0 con 0x5605f91fe4a0 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 -- 172.21.15.94:0/1390235394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 msgr2=0x5605f91fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5da00a3430 0x7f5da00a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5dac047000 tx=0x7f5dac04c000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1390235394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f91fe880 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009ca0 tx=0x7f5dac009a20 comp rx=0 tx=0).stop 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 -- 172.21.15.94:0/1390235394 shutdown_connections 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1390235394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f920ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1390235394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f9209140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1390235394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f91fe880 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 -- 172.21.15.94:0/1390235394 >> 172.21.15.94:0/1390235394 conn(0x5605f91faa30 msgr2=0x5605f91fae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 -- 172.21.15.94:0/1390235394 shutdown_connections 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 -- 172.21.15.94:0/1390235394 wait complete. 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 -- start start 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.207+0000 7f5dbe466740 1 Processor -- start 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fe4a0 0x5605f9208f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f9206200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f9206740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dbe466740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5605f920c250 con 0x5605f9209680 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dbe466740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5605f911f050 con 0x5605f91fedc0 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dbe466740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605f920c3d0 con 0x5605f91fe4a0 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dbe466740 10 client.0 _ll_register_callbacks cb 0x5605f915f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dba62a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f9206740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dba62a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f9206740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45654/0 (socket says 172.21.15.94:45654) 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dba62a640 1 -- 172.21.15.94:0/189147671 learned_addr learned my addr 172.21.15.94:0/189147671 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5db9628640 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f9206200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5db9e29640 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fe4a0 0x5605f9208f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.208+0000 7f5dba62a640 1 -- 172.21.15.94:0/189147671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fe4a0 msgr2=0x5605f9208f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5dba62a640 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fe4a0 0x5605f9208f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5dba62a640 1 -- 172.21.15.94:0/189147671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 msgr2=0x5605f9206200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5dba62a640 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f9206200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5dba62a640 1 -- 172.21.15.94:0/189147671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f8ed53c0 con 0x5605f9209680 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5db9e29640 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fe4a0 0x5605f9208f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5db9628640 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f9206200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5dba62a640 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f9206740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5dac02aed0 tx=0x7f5dac030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451ee122c824d49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5d9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:28:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/189147671 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0305a0 con 0x5605f9209680 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5dbe466740 1 -- 172.21.15.94:0/189147671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dac036040 con 0x5605f9209680 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.209+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/189147671 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f5dac05f410 con 0x5605f9209680 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/189147671 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5dac05f610 con 0x5605f9209680 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5dbe466740 10 client.16983 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5dbe466740 1 -- 172.21.15.94:0/189147671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5605f9217ba0 con 0x5605f9209680 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/189147671 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dac036040 con 0x5605f9209680 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/189147671 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f5dac030070 con 0x5605f9209680 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5dbe466740 10 client.16983 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5dbe466740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5dbe466740 20 client.16983 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5dbe466740 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5605f92191d0 0x5605f921b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5dbe466740 4 client.16983 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5dbe466740 1 -- 172.21.15.94:0/189147671 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605f9171850 con 0x5605f92191d0 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.210+0000 7f5db9e29640 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5605f92191d0 0x5605f921b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.211+0000 7f5d9d7fa640 10 client.16983 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.211+0000 7f5db9e29640 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5605f92191d0 0x5605f921b630 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8291b4264d84687 server_cookie=aa4fa8c17e648852 in_seq=0 out_seq=0 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/189147671 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605f9171850 con 0x5605f92191d0 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5d9d7fa640 10 client.16983 handle_command_reply: tid=0 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 client.16983 shutdown 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 -- 172.21.15.94:0/189147671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 msgr2=0x5605f9206740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f9206740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5dac02aed0 tx=0x7f5dac030fc0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 -- 172.21.15.94:0/189147671 shutdown_connections 2024-09-19T04:28:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5605f92191d0 0x5605f921b630 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f9206740 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f9206200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 --2- 172.21.15.94:0/189147671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fe4a0 0x5605f9208f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 -- 172.21.15.94:0/189147671 >> 172.21.15.94:0/189147671 conn(0x5605f91faa30 msgr2=0x5605f9200760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5d9d7fa640 0 client.16983 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 -- 172.21.15.94:0/189147671 shutdown_connections 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.212+0000 7f5dbe466740 1 -- 172.21.15.94:0/189147671 wait complete. 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.213+0000 7f5dbe466740 20 client.16983 trim_cache size 0 max 16384 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.234+0000 7f5dbe466740 1 Processor -- start 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.234+0000 7f5dbe466740 1 -- start start 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f92154b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f91fae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f9220f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5dbe466740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41b24b0 con 0x5605f91fe4a0 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5dbe466740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41afcc0 con 0x5605f91fedc0 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5dbe466740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41adb30 con 0x5605f9209680 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9e29640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f92154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9e29640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f92154b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45668/0 (socket says 172.21.15.94:45668) 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9e29640 1 -- 172.21.15.94:0/3173507729 learned_addr learned my addr 172.21.15.94:0/3173507729 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5dba62a640 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f9220f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9628640 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f91fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9e29640 1 -- 172.21.15.94:0/3173507729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 msgr2=0x5605f9220f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9e29640 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f9220f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9e29640 1 -- 172.21.15.94:0/3173507729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 msgr2=0x5605f91fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9e29640 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f91fae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9e29640 1 -- 172.21.15.94:0/3173507729 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f92092c0 con 0x5605f91fe4a0 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9628640 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f91fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.235+0000 7f5db9e29640 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f92154b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e9c0 tx=0x7f5db00bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d8e8d9d1ddf3b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.236+0000 7f5d9f7fe640 1 -- 172.21.15.94:0/3173507729 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db009b880 con 0x5605f91fe4a0 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.236+0000 7f5d9f7fe640 1 -- 172.21.15.94:0/3173507729 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5db009c620 con 0x5605f91fe4a0 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.236+0000 7f5d9f7fe640 1 -- 172.21.15.94:0/3173507729 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db00055f0 con 0x5605f91fe4a0 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.236+0000 7f5dbe466740 1 -- 172.21.15.94:0/3173507729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 msgr2=0x5605f92154b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.236+0000 7f5dbe466740 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f92154b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e9c0 tx=0x7f5db00bc040 comp rx=0 tx=0).stop 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 -- 172.21.15.94:0/3173507729 shutdown_connections 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f9209680 0x5605f9220f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fedc0 0x5605f91fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 --2- 172.21.15.94:0/3173507729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f91fe4a0 0x5605f92154b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 -- 172.21.15.94:0/3173507729 >> 172.21.15.94:0/3173507729 conn(0x5605f91faa30 msgr2=0x5605f9204ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 -- 172.21.15.94:0/3173507729 shutdown_connections 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 -- 172.21.15.94:0/3173507729 wait complete. 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 -- start start 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 Processor -- start 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fe4a0 0x5605f920e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fedc0 0x5605f920eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f920f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41ae180 con 0x5605f9209680 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41af5f0 con 0x5605f91fe4a0 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41ae850 con 0x5605f91fedc0 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.237+0000 7f5dbe466740 10 client.0 _ll_register_callbacks cb 0x7f5dac04a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5db9e29640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fe4a0 0x5605f920e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5dba62a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f920f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5db9e29640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fe4a0 0x5605f920e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:34472/0 (socket says 172.21.15.94:34472) 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5db9e29640 1 -- 172.21.15.94:0/1490328791 learned_addr learned my addr 172.21.15.94:0/1490328791 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5db9628640 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fedc0 0x5605f920eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5dba62a640 1 -- 172.21.15.94:0/1490328791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fedc0 msgr2=0x5605f920eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fedc0 0x5605f920eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5dba62a640 1 -- 172.21.15.94:0/1490328791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fe4a0 msgr2=0x5605f920e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fe4a0 0x5605f920e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5dba62a640 1 -- 172.21.15.94:0/1490328791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f9281ed0 con 0x5605f9209680 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5db9e29640 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fe4a0 0x5605f920e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5dba62a640 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f920f070 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0049d0 tx=0x7f5dac009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a5756964eb5e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5d9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:28:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.238+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/1490328791 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0305a0 con 0x5605f9209680 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/1490328791 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f5dac047cc0 con 0x5605f9209680 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5dbe466740 1 -- 172.21.15.94:0/1490328791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db009c620 con 0x5605f9209680 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/1490328791 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5dac05f7b0 con 0x5605f9209680 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5dbe466740 10 client.16989 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5dbe466740 1 -- 172.21.15.94:0/1490328791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5605f92830b0 con 0x5605f9209680 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/1490328791 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dac036050 con 0x5605f9209680 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/1490328791 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f5dac030040 con 0x5605f9209680 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5dbe466740 10 client.16989 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5dbe466740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5dbe466740 20 client.16989 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5605f92836f0 0x5605f9285b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5dbe466740 4 client.16989 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.239+0000 7f5dbe466740 1 -- 172.21.15.94:0/1490328791 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5db009c620 con 0x5605f92836f0 2024-09-19T04:28:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.240+0000 7f5db9e29640 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5605f92836f0 0x5605f9285b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.240+0000 7f5db9e29640 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5605f92836f0 0x5605f9285b10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b5c2ffa95363dd server_cookie=98c963d8b8514ffe in_seq=0 out_seq=0 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.240+0000 7f5d9d7fa640 10 client.16989 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.240+0000 7f5d9d7fa640 1 -- 172.21.15.94:0/1490328791 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f5db009c620 con 0x5605f92836f0 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.240+0000 7f5d9d7fa640 10 client.16989 handle_command_reply: tid=0 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.240+0000 7f5dbe466740 1 client.16989 shutdown 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 -- 172.21.15.94:0/1490328791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 msgr2=0x5605f920f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f920f070 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0049d0 tx=0x7f5dac009a20 comp rx=0 tx=0).stop 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 -- 172.21.15.94:0/1490328791 shutdown_connections 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5605f92836f0 0x5605f9285b10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605f9209680 0x5605f920f070 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f91fedc0 0x5605f920eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 --2- 172.21.15.94:0/1490328791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605f91fe4a0 0x5605f920e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 -- 172.21.15.94:0/1490328791 >> 172.21.15.94:0/1490328791 conn(0x5605f91faa30 msgr2=0x5605f9208280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5d9d7fa640 0 client.16989 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 -- 172.21.15.94:0/1490328791 shutdown_connections 2024-09-19T04:28:17.264 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:28:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:28:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:28:17.264 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 1 -- 172.21.15.94:0/1490328791 wait complete. 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.241+0000 7f5dbe466740 20 client.16989 trim_cache size 0 max 16384 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 -- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5da00a3430 msgr2=0x7f5da00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5da00a3430 0x7f5da00a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5dac047000 tx=0x7f5dac04c000 comp rx=0 tx=0).stop 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 -- 172.21.15.94:0/1169270569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5db4100a90 msgr2=0x7f5db41991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5db4100a90 0x7f5db41991c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e9f0 tx=0x7f5db000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 -- 172.21.15.94:0/1169270569 shutdown_connections 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5da00a3430 0x7f5da00a58f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db4106a90 0x7f5db41a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5db4101430 0x7f5db4199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 --2- 172.21.15.94:0/1169270569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5db4100a90 0x7f5db41991c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 -- 172.21.15.94:0/1169270569 >> 172.21.15.94:0/1169270569 conn(0x7f5db40fc7d0 msgr2=0x7f5db40fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 -- 172.21.15.94:0/1169270569 shutdown_connections 2024-09-19T04:28:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:17.242+0000 7f5dbc08d640 1 -- 172.21.15.94:0/1169270569 wait complete. 2024-09-19T04:28:18.146 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:28:18.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:28:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:18.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:18 smithi094 ceph-mon[30972]: pgmap v415: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 5.1 MiB/s wr, 654 op/s 2024-09-19T04:28:19.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:18 smithi112 ceph-mon[35051]: pgmap v415: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 5.1 MiB/s wr, 654 op/s 2024-09-19T04:28:19.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.095+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/1349731537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc88101270 msgr2=0x7fbc8810d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:19.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.095+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/1349731537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc88101270 0x7fbc8810d7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009c10 tx=0x7fbc8002a6e0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.095+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/1349731537 shutdown_connections 2024-09-19T04:28:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.095+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/1349731537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc88101270 0x7fbc8810d7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.095+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/1349731537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc881008d0 0x7fbc88100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.095+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/1349731537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc881068d0 0x7fbc88106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.095+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/1349731537 >> 172.21.15.94:0/1349731537 conn(0x7fbc880fc610 msgr2=0x7fbc880fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.096+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/1349731537 shutdown_connections 2024-09-19T04:28:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.096+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/1349731537 wait complete. 2024-09-19T04:28:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.096+0000 7fbc8f2f9640 1 Processor -- start 2024-09-19T04:28:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.096+0000 7fbc8f2f9640 1 -- start start 2024-09-19T04:28:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8f2f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc881008d0 0x7fbc88195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8f2f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc88101270 0x7fbc88196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8f2f9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc881068d0 0x7fbc8819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8e2f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc881008d0 0x7fbc88195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8e2f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc881008d0 0x7fbc88195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:34478/0 (socket says 172.21.15.94:34478) 2024-09-19T04:28:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8e2f7640 1 -- 172.21.15.94:0/672608858 learned_addr learned my addr 172.21.15.94:0/672608858 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8daf6640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc88101270 0x7fbc88196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8eaf8640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc881068d0 0x7fbc8819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8810fa70 con 0x7fbc881068d0 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.097+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8810f8f0 con 0x7fbc881008d0 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.098+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8810fbf0 con 0x7fbc88101270 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.098+0000 7fbc8e2f7640 1 -- 172.21.15.94:0/672608858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc88101270 msgr2=0x7fbc88196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.098+0000 7fbc8e2f7640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc88101270 0x7fbc88196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.098+0000 7fbc8e2f7640 1 -- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc881068d0 msgr2=0x7fbc8819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.098+0000 7fbc8e2f7640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc881068d0 0x7fbc8819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.098+0000 7fbc8e2f7640 1 -- 172.21.15.94:0/672608858 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8819ae80 con 0x7fbc881008d0 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.098+0000 7fbc8e2f7640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc881008d0 0x7fbc88195ed0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e9c0 tx=0x7fbc8400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.098+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/672608858 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400ccc0 con 0x7fbc881008d0 2024-09-19T04:28:19.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.099+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/672608858 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc840054d0 con 0x7fbc881008d0 2024-09-19T04:28:19.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.099+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/672608858 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84010430 con 0x7fbc881008d0 2024-09-19T04:28:19.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.099+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8819b170 con 0x7fbc881008d0 2024-09-19T04:28:19.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.099+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc881a2a50 con 0x7fbc881008d0 2024-09-19T04:28:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.101+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc88100d30 con 0x7fbc881008d0 2024-09-19T04:28:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.101+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/672608858 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fbc84004ef0 con 0x7fbc881008d0 2024-09-19T04:28:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.102+0000 7fbc7f7fe640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbc740a5640 0x7fbc740a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.102+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/672608858 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7fbc840d01a0 con 0x7fbc881008d0 2024-09-19T04:28:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.102+0000 7fbc8daf6640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbc740a5640 0x7fbc740a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.102+0000 7fbc8daf6640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbc740a5640 0x7fbc740a7b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc880fec10 tx=0x7fbc7800a2d0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:19.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:18 smithi104 ceph-mon[34145]: pgmap v415: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 5.1 MiB/s wr, 654 op/s 2024-09-19T04:28:19.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.106+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/672608858 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbc8409cb60 con 0x7fbc881008d0 2024-09-19T04:28:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.398+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc88108ea0 con 0x7fbc881008d0 2024-09-19T04:28:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.399+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/672608858 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11924 (secure 0 0 0) 0x7fbc8409c460 con 0x7fbc881008d0 2024-09-19T04:28:19.401 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:28:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 48 2024-09-19T04:28:19.402 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":48,"btime":"2024-09-19T04:26:33:395724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:26:32.789785+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:28:19.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbc740a5640 msgr2=0x7fbc740a7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:19.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbc740a5640 0x7fbc740a7b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc880fec10 tx=0x7fbc7800a2d0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc881008d0 msgr2=0x7fbc88195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:19.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc881008d0 0x7fbc88195ed0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e9c0 tx=0x7fbc8400edc0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 shutdown_connections 2024-09-19T04:28:19.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fbc740a5640 0x7fbc740a7b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc881068d0 0x7fbc8819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc88101270 0x7fbc88196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 --2- 172.21.15.94:0/672608858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc881008d0 0x7fbc88195ed0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:19.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 >> 172.21.15.94:0/672608858 conn(0x7fbc880fc610 msgr2=0x7fbc880fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:19.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 shutdown_connections 2024-09-19T04:28:19.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:19.402+0000 7fbc8f2f9640 1 -- 172.21.15.94:0/672608858 wait complete. 2024-09-19T04:28:20.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:19 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/672608858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:28:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:19 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/672608858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:28:20.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:19 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/672608858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:28:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 725.0 s, 352.2 tps, lat 284.008 ms stddev 75.723 2024-09-19T04:28:21.005 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:20 smithi104 ceph-mon[34145]: pgmap v416: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 4.7 MiB/s wr, 621 op/s 2024-09-19T04:28:21.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:20 smithi112 ceph-mon[35051]: pgmap v416: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 4.7 MiB/s wr, 621 op/s 2024-09-19T04:28:21.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:20 smithi094 ceph-mon[30972]: pgmap v416: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 4.7 MiB/s wr, 621 op/s 2024-09-19T04:28:22.465 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:28:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:22.973 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:22 smithi094 ceph-mon[30972]: pgmap v417: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 4.7 MiB/s wr, 621 op/s 2024-09-19T04:28:23.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:22 smithi112 ceph-mon[35051]: pgmap v417: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 4.7 MiB/s wr, 621 op/s 2024-09-19T04:28:23.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:22 smithi104 ceph-mon[34145]: pgmap v417: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 4.7 MiB/s wr, 621 op/s 2024-09-19T04:28:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.454+0000 7ff93a242640 1 -- 172.21.15.94:0/1093342373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 msgr2=0x7ff934103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:23.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.454+0000 7ff93a242640 1 --2- 172.21.15.94:0/1093342373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 0x7ff934103110 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff924009930 tx=0x7ff92402a420 comp rx=0 tx=0).stop 2024-09-19T04:28:23.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.454+0000 7ff93a242640 1 -- 172.21.15.94:0/1093342373 shutdown_connections 2024-09-19T04:28:23.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.454+0000 7ff93a242640 1 --2- 172.21.15.94:0/1093342373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9341082a0 0x7ff93410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.454+0000 7ff93a242640 1 --2- 172.21.15.94:0/1093342373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9341036e0 0x7ff934107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.454+0000 7ff93a242640 1 --2- 172.21.15.94:0/1093342373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 0x7ff934103110 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.454+0000 7ff93a242640 1 -- 172.21.15.94:0/1093342373 >> 172.21.15.94:0/1093342373 conn(0x7ff9340fc7d0 msgr2=0x7ff9340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:23.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.454+0000 7ff93a242640 1 -- 172.21.15.94:0/1093342373 shutdown_connections 2024-09-19T04:28:23.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.454+0000 7ff93a242640 1 -- 172.21.15.94:0/1093342373 wait complete. 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.455+0000 7ff93a242640 1 Processor -- start 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.455+0000 7ff93a242640 1 -- start start 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff93a242640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 0x7ff934100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff93a242640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9341036e0 0x7ff934100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff93a242640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9341082a0 0x7ff934101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff93a242640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff934111f50 con 0x7ff934102d30 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff93a242640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff934111dd0 con 0x7ff9341036e0 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff93a242640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341120d0 con 0x7ff9341082a0 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 0x7ff934100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff932ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9341036e0 0x7ff934100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 0x7ff934100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54572/0 (socket says 172.21.15.94:54572) 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff9337fe640 1 -- 172.21.15.94:0/1166815923 learned_addr learned my addr 172.21.15.94:0/1166815923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff933fff640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9341082a0 0x7ff934101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff932ffd640 1 -- 172.21.15.94:0/1166815923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9341082a0 msgr2=0x7ff934101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff932ffd640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9341082a0 0x7ff934101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff932ffd640 1 -- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 msgr2=0x7ff934100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff932ffd640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 0x7ff934100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.456+0000 7ff932ffd640 1 -- 172.21.15.94:0/1166815923 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9341a69b0 con 0x7ff9341036e0 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.457+0000 7ff9337fe640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 0x7ff934100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.457+0000 7ff932ffd640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9341036e0 0x7ff934100ed0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff9340697d0 tx=0x7ff92800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.457+0000 7ff930ff9640 1 -- 172.21.15.94:0/1166815923 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92800cd00 con 0x7ff9341036e0 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.457+0000 7ff93a242640 1 -- 172.21.15.94:0/1166815923 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9341a6ca0 con 0x7ff9341036e0 2024-09-19T04:28:23.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.457+0000 7ff930ff9640 1 -- 172.21.15.94:0/1166815923 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9280035f0 con 0x7ff9341036e0 2024-09-19T04:28:23.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.457+0000 7ff930ff9640 1 -- 172.21.15.94:0/1166815923 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928010690 con 0x7ff9341036e0 2024-09-19T04:28:23.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.457+0000 7ff93a242640 1 -- 172.21.15.94:0/1166815923 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9341a71d0 con 0x7ff9341036e0 2024-09-19T04:28:23.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.460+0000 7ff930ff9640 1 -- 172.21.15.94:0/1166815923 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff9280037a0 con 0x7ff9341036e0 2024-09-19T04:28:23.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.460+0000 7ff93c61b740 1 Processor -- start 2024-09-19T04:28:23.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.460+0000 7ff93c61b740 1 -- start start 2024-09-19T04:28:23.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e395090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3955d0 0x55a00e395a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3a0370 0x55a00e3a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff93c61b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a00e2fb2f0 con 0x55a00e396b40 2024-09-19T04:28:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff93c61b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a00e2a2cf0 con 0x55a00e3a0370 2024-09-19T04:28:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff93c61b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a00e2f72b0 con 0x55a00e3955d0 2024-09-19T04:28:23.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3955d0 0x55a00e395a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff933fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e395090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3955d0 0x55a00e395a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54332/0 (socket says 172.21.15.94:54332) 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff933fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e395090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54576/0 (socket says 172.21.15.94:54576) 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff9337fe640 1 -- 172.21.15.94:0/1651012464 learned_addr learned my addr 172.21.15.94:0/1651012464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff932ffd640 1 --2- 172.21.15.94:0/1651012464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3a0370 0x55a00e3a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff9337fe640 1 -- 172.21.15.94:0/1651012464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3a0370 msgr2=0x55a00e3a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff9337fe640 1 --2- 172.21.15.94:0/1651012464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3a0370 0x55a00e3a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff9337fe640 1 -- 172.21.15.94:0/1651012464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 msgr2=0x55a00e395090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff930ff9640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff91c0a3430 0x7ff91c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff9337fe640 1 --2- 172.21.15.94:0/1651012464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e395090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff9337fe640 1 -- 172.21.15.94:0/1651012464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a00e2873a0 con 0x55a00e3955d0 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.461+0000 7ff933fff640 1 --2- 172.21.15.94:0/1651012464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e395090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.462+0000 7ff930ff9640 1 -- 172.21.15.94:0/1166815923 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7ff9280cfef0 con 0x7ff9341036e0 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.462+0000 7ff9337fe640 1 --2- 172.21.15.94:0/1651012464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3955d0 0x55a00e395a10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff924009230 tx=0x7ff924032eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c66ca698f4b81e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.462+0000 7ff919ffb640 1 -- 172.21.15.94:0/1651012464 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924035040 con 0x55a00e3955d0 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.462+0000 7ff932ffd640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff91c0a3430 0x7ff91c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.462+0000 7ff919ffb640 1 -- 172.21.15.94:0/1651012464 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9240317a0 con 0x55a00e3955d0 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.462+0000 7ff919ffb640 1 -- 172.21.15.94:0/1651012464 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924004a00 con 0x55a00e3955d0 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 -- 172.21.15.94:0/1651012464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3955d0 msgr2=0x55a00e395a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff932ffd640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff91c0a3430 0x7ff91c0a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9280cd6b0 tx=0x7ff928003e30 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 --2- 172.21.15.94:0/1651012464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3955d0 0x55a00e395a10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff924009230 tx=0x7ff924032eb0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 -- 172.21.15.94:0/1651012464 shutdown_connections 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 --2- 172.21.15.94:0/1651012464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3a0370 0x55a00e3a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 --2- 172.21.15.94:0/1651012464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3955d0 0x55a00e395a10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 --2- 172.21.15.94:0/1651012464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e395090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 -- 172.21.15.94:0/1651012464 >> 172.21.15.94:0/1651012464 conn(0x55a00e35e310 msgr2=0x55a00e391c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:23.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 -- 172.21.15.94:0/1651012464 shutdown_connections 2024-09-19T04:28:23.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 -- 172.21.15.94:0/1651012464 wait complete. 2024-09-19T04:28:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.463+0000 7ff93c61b740 1 -- start start 2024-09-19T04:28:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff93c61b740 1 Processor -- start 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e3955d0 0x55a00e39fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e396b40 0x55a00e39d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e39d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff93c61b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a00e3a3010 con 0x55a00e3955d0 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff93c61b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a00e2b6060 con 0x55a00e396b40 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff93c61b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a00e3a3190 con 0x55a00e3a0370 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff93c61b740 10 client.0 _ll_register_callbacks cb 0x55a00e0098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff933fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e3955d0 0x55a00e39fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff933fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e3955d0 0x55a00e39fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54580/0 (socket says 172.21.15.94:54580) 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff933fff640 1 -- 172.21.15.94:0/542631485 learned_addr learned my addr 172.21.15.94:0/542631485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff9337fe640 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e396b40 0x55a00e39d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff933fff640 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e39d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff933fff640 1 -- 172.21.15.94:0/542631485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 msgr2=0x55a00e39d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff933fff640 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e39d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff933fff640 1 -- 172.21.15.94:0/542631485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e396b40 msgr2=0x55a00e39d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff933fff640 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e396b40 0x55a00e39d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.464+0000 7ff933fff640 1 -- 172.21.15.94:0/542631485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00e06c3c0 con 0x55a00e3955d0 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.465+0000 7ff933fff640 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e3955d0 0x55a00e39fd50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00f810 tx=0x7ff92c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea36f4edd34f119 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.465+0000 7ff912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.465+0000 7ff912ffd640 1 -- 172.21.15.94:0/542631485 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c015030 con 0x55a00e3955d0 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.465+0000 7ff912ffd640 1 -- 172.21.15.94:0/542631485 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7ff92c004b20 con 0x55a00e3955d0 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.465+0000 7ff93c61b740 1 -- 172.21.15.94:0/542631485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9240317a0 con 0x55a00e3955d0 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.465+0000 7ff912ffd640 1 -- 172.21.15.94:0/542631485 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff92c00fe50 con 0x55a00e3955d0 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.465+0000 7ff93c61b740 10 client.17007 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.465+0000 7ff93c61b740 1 -- 172.21.15.94:0/542631485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a00e3aea90 con 0x55a00e3955d0 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.466+0000 7ff912ffd640 1 -- 172.21.15.94:0/542631485 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9240317a0 con 0x55a00e3955d0 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.466+0000 7ff912ffd640 1 -- 172.21.15.94:0/542631485 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7ff92c00ec70 con 0x55a00e3955d0 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.466+0000 7ff93c61b740 10 client.17007 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.466+0000 7ff93c61b740 10 client.17007 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.466+0000 7ff93c61b740 20 client.17007 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.466+0000 7ff93c61b740 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a00e3afef0 0x55a00e3b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.466+0000 7ff93c61b740 4 client.17007 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.466+0000 7ff93c61b740 1 -- 172.21.15.94:0/542631485 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a00e308860 con 0x55a00e3afef0 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.466+0000 7ff9337fe640 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a00e3afef0 0x55a00e3b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.467+0000 7ff912ffd640 10 client.17007 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.467+0000 7ff9337fe640 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a00e3afef0 0x55a00e3b2350 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f868f9e7063d42b7 server_cookie=271c6f445b28a112 in_seq=0 out_seq=0 2024-09-19T04:28:23.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.468+0000 7ff912ffd640 1 -- 172.21.15.94:0/542631485 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a00e308860 con 0x55a00e3afef0 2024-09-19T04:28:23.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.468+0000 7ff912ffd640 10 client.17007 handle_command_reply: tid=0 2024-09-19T04:28:23.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.468+0000 7ff93c61b740 1 client.17007 shutdown 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.468+0000 7ff93c61b740 1 -- 172.21.15.94:0/542631485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e3955d0 msgr2=0x55a00e39fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.468+0000 7ff93c61b740 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e3955d0 0x55a00e39fd50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00f810 tx=0x7ff92c009500 comp rx=0 tx=0).stop 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff93c61b740 1 -- 172.21.15.94:0/542631485 shutdown_connections 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff93c61b740 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a00e3afef0 0x55a00e3b2350 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff93c61b740 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e39d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff93c61b740 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e396b40 0x55a00e39d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff93c61b740 1 --2- 172.21.15.94:0/542631485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e3955d0 0x55a00e39fd50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff912ffd640 0 client.17007 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff93c61b740 1 -- 172.21.15.94:0/542631485 >> 172.21.15.94:0/542631485 conn(0x55a00e35e310 msgr2=0x55a00e397960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff93c61b740 1 -- 172.21.15.94:0/542631485 shutdown_connections 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff93c61b740 1 -- 172.21.15.94:0/542631485 wait complete. 2024-09-19T04:28:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.469+0000 7ff93c61b740 20 client.17007 trim_cache size 0 max 16384 2024-09-19T04:28:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.488+0000 7ff93c61b740 1 Processor -- start 2024-09-19T04:28:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.488+0000 7ff93c61b740 1 -- start start 2024-09-19T04:28:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.488+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e396300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.488+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e3ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.488+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e391700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.488+0000 7ff93c61b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341b24c0 con 0x55a00e396b40 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.488+0000 7ff93c61b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341afcd0 con 0x55a00e3955d0 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.488+0000 7ff93c61b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341adb40 con 0x55a00e3a0370 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e3ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e3ac2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54590/0 (socket says 172.21.15.94:54590) 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9337fe640 1 -- 172.21.15.94:0/3318686229 learned_addr learned my addr 172.21.15.94:0/3318686229 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff933fff640 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e396300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff933fff640 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e391700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9337fe640 1 -- 172.21.15.94:0/3318686229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 msgr2=0x55a00e391700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9337fe640 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e391700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9337fe640 1 -- 172.21.15.94:0/3318686229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 msgr2=0x55a00e396300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9337fe640 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e396300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9337fe640 1 -- 172.21.15.94:0/3318686229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a00e3a2f50 con 0x55a00e396b40 2024-09-19T04:28:23.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff933fff640 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e396300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:23.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9337fe640 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e3ac2b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff924009a50 tx=0x7ff92402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1522d045eca45d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:23.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.489+0000 7ff9197fa640 1 -- 172.21.15.94:0/3318686229 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924035040 con 0x55a00e396b40 2024-09-19T04:28:23.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff9197fa640 1 -- 172.21.15.94:0/3318686229 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff924004040 con 0x55a00e396b40 2024-09-19T04:28:23.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff9197fa640 1 -- 172.21.15.94:0/3318686229 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240038e0 con 0x55a00e396b40 2024-09-19T04:28:23.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff93c61b740 1 -- 172.21.15.94:0/3318686229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 msgr2=0x55a00e3ac2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:23.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff93c61b740 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e3ac2b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff924009a50 tx=0x7ff92402a420 comp rx=0 tx=0).stop 2024-09-19T04:28:23.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff93c61b740 1 -- 172.21.15.94:0/3318686229 shutdown_connections 2024-09-19T04:28:23.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff93c61b740 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e391700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff93c61b740 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e3ac2b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff93c61b740 1 --2- 172.21.15.94:0/3318686229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e396300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff93c61b740 1 -- 172.21.15.94:0/3318686229 >> 172.21.15.94:0/3318686229 conn(0x55a00e35e310 msgr2=0x55a00e38ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff93c61b740 1 -- 172.21.15.94:0/3318686229 shutdown_connections 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.490+0000 7ff93c61b740 1 -- 172.21.15.94:0/3318686229 wait complete. 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff93c61b740 1 -- start start 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff93c61b740 1 Processor -- start 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e3a88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e399b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff93c61b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e39a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff93c61b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341ae190 con 0x55a00e396b40 2024-09-19T04:28:23.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff93c61b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341af600 con 0x55a00e3955d0 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff93c61b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341ae860 con 0x55a00e3a0370 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff93c61b740 10 client.0 _ll_register_callbacks cb 0x7ff92c009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff933fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e3a88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff933fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e3a88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51062/0 (socket says 172.21.15.94:51062) 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff933fff640 1 -- 172.21.15.94:0/2299162627 learned_addr learned my addr 172.21.15.94:0/2299162627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.491+0000 7ff933fff640 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e39a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff9337fe640 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e399b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff933fff640 1 -- 172.21.15.94:0/2299162627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 msgr2=0x55a00e39a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff933fff640 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e39a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff933fff640 1 -- 172.21.15.94:0/2299162627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 msgr2=0x55a00e399b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff933fff640 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e399b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff933fff640 1 -- 172.21.15.94:0/2299162627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00e39a610 con 0x55a00e3955d0 2024-09-19T04:28:23.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff933fff640 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e3a88f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff92c004260 tx=0x7ff92c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b97f9219e6a75ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:23.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:28:23.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff912ffd640 1 -- 172.21.15.94:0/2299162627 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c003d80 con 0x55a00e3955d0 2024-09-19T04:28:23.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff93c61b740 1 -- 172.21.15.94:0/2299162627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff924004040 con 0x55a00e3955d0 2024-09-19T04:28:23.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff912ffd640 1 -- 172.21.15.94:0/2299162627 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7ff92c00e890 con 0x55a00e3955d0 2024-09-19T04:28:23.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff912ffd640 1 -- 172.21.15.94:0/2299162627 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff92c00fe50 con 0x55a00e3955d0 2024-09-19T04:28:23.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff93c61b740 10 client.26630 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:23.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.492+0000 7ff93c61b740 1 -- 172.21.15.94:0/2299162627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a00e41a960 con 0x55a00e3955d0 2024-09-19T04:28:23.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff912ffd640 1 -- 172.21.15.94:0/2299162627 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff924004040 con 0x55a00e3955d0 2024-09-19T04:28:23.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff912ffd640 1 -- 172.21.15.94:0/2299162627 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7ff92c02b990 con 0x55a00e3955d0 2024-09-19T04:28:23.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff93c61b740 10 client.26630 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:23.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff93c61b740 10 client.26630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:28:23.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff93c61b740 20 client.26630 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:23.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff93c61b740 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a00e3afef0 0x55a00e41d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:23.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff93c61b740 4 client.26630 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:28:23.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff93c61b740 1 -- 172.21.15.94:0/2299162627 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a00e302d70 con 0x55a00e3afef0 2024-09-19T04:28:23.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff9337fe640 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a00e3afef0 0x55a00e41d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:23.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.493+0000 7ff912ffd640 10 client.26630 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:28:23.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff9337fe640 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a00e3afef0 0x55a00e41d0f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d67f652931c8cf server_cookie=89bd20641840e1bc in_seq=0 out_seq=0 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff912ffd640 1 -- 172.21.15.94:0/2299162627 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a00e302d70 con 0x55a00e3afef0 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff912ffd640 10 client.26630 handle_command_reply: tid=0 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 client.26630 shutdown 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 -- 172.21.15.94:0/2299162627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 msgr2=0x55a00e3a88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e3a88f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff92c004260 tx=0x7ff92c000f40 comp rx=0 tx=0).stop 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 -- 172.21.15.94:0/2299162627 shutdown_connections 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a00e3afef0 0x55a00e41d0f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a00e3a0370 0x55a00e39a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a00e396b40 0x55a00e399b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff912ffd640 0 client.26630 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 --2- 172.21.15.94:0/2299162627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00e3955d0 0x55a00e3a88f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 -- 172.21.15.94:0/2299162627 >> 172.21.15.94:0/2299162627 conn(0x55a00e35e310 msgr2=0x55a00e391700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:23.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 -- 172.21.15.94:0/2299162627 shutdown_connections 2024-09-19T04:28:23.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 1 -- 172.21.15.94:0/2299162627 wait complete. 2024-09-19T04:28:23.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.494+0000 7ff93c61b740 20 client.26630 trim_cache size 0 max 16384 2024-09-19T04:28:23.502 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:28:23.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 -- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff91c0a3430 msgr2=0x7ff91c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:23.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff91c0a3430 0x7ff91c0a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9280cd6b0 tx=0x7ff928003e30 comp rx=0 tx=0).stop 2024-09-19T04:28:23.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 -- 172.21.15.94:0/1166815923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9341036e0 msgr2=0x7ff934100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:23.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9341036e0 0x7ff934100ed0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff9340697d0 tx=0x7ff92800ee50 comp rx=0 tx=0).stop 2024-09-19T04:28:23.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 -- 172.21.15.94:0/1166815923 shutdown_connections 2024-09-19T04:28:23.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff91c0a3430 0x7ff91c0a58f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9341082a0 0x7ff934101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9341036e0 0x7ff934100ed0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 --2- 172.21.15.94:0/1166815923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff934102d30 0x7ff934100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:23.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.495+0000 7ff93a242640 1 -- 172.21.15.94:0/1166815923 >> 172.21.15.94:0/1166815923 conn(0x7ff9340fc7d0 msgr2=0x7ff934105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:23.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.496+0000 7ff93a242640 1 -- 172.21.15.94:0/1166815923 shutdown_connections 2024-09-19T04:28:23.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:23.496+0000 7ff93a242640 1 -- 172.21.15.94:0/1166815923 wait complete. 2024-09-19T04:28:24.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:28:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:24.942 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:24 smithi094 ceph-mon[30972]: pgmap v418: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.5 MiB/s wr, 599 op/s 2024-09-19T04:28:25.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:24 smithi112 ceph-mon[35051]: pgmap v418: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.5 MiB/s wr, 599 op/s 2024-09-19T04:28:25.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:24 smithi104 ceph-mon[34145]: pgmap v418: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.5 MiB/s wr, 599 op/s 2024-09-19T04:28:25.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.195+0000 7f591f54e640 1 -- 172.21.15.94:0/3061459122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5918102ef0 msgr2=0x7f591810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.195+0000 7f591f54e640 1 --2- 172.21.15.94:0/3061459122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5918102ef0 0x7f591810da40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5910009ca0 tx=0x7f591002a950 comp rx=0 tx=0).stop 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.195+0000 7f591f54e640 1 -- 172.21.15.94:0/3061459122 shutdown_connections 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.195+0000 7f591f54e640 1 --2- 172.21.15.94:0/3061459122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5918102ef0 0x7f591810da40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.195+0000 7f591f54e640 1 --2- 172.21.15.94:0/3061459122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5918102570 0x7f59181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.195+0000 7f591f54e640 1 --2- 172.21.15.94:0/3061459122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5918100e20 0x7f5918101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.195+0000 7f591f54e640 1 -- 172.21.15.94:0/3061459122 >> 172.21.15.94:0/3061459122 conn(0x7f59180fc7d0 msgr2=0x7f59180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.195+0000 7f591f54e640 1 -- 172.21.15.94:0/3061459122 shutdown_connections 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.195+0000 7f591f54e640 1 -- 172.21.15.94:0/3061459122 wait complete. 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591f54e640 1 Processor -- start 2024-09-19T04:28:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591f54e640 1 -- start start 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591f54e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5918100e20 0x7f591810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591f54e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5918102570 0x7f591810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591f54e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5918102ef0 0x7f591810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591f54e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5918111f50 con 0x7f5918102570 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591f54e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5918111dd0 con 0x7f5918100e20 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591f54e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59181120d0 con 0x7f5918102ef0 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591d2ea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5918100e20 0x7f591810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591daeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5918102ef0 0x7f591810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591daeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5918102ef0 0x7f591810ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54372/0 (socket says 172.21.15.94:54372) 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.196+0000 7f591daeb640 1 -- 172.21.15.94:0/4292642801 learned_addr learned my addr 172.21.15.94:0/4292642801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f591cae9640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5918102570 0x7f591810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f591d2ea640 1 -- 172.21.15.94:0/4292642801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5918102ef0 msgr2=0x7f591810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f591d2ea640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5918102ef0 0x7f591810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f591d2ea640 1 -- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5918102570 msgr2=0x7f591810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f591d2ea640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5918102570 0x7f591810ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f591d2ea640 1 -- 172.21.15.94:0/4292642801 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59181a68d0 con 0x7f5918100e20 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f591daeb640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5918102ef0 0x7f591810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f591d2ea640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5918100e20 0x7f591810e4d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9f0 tx=0x7f591400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f590e7fc640 1 -- 172.21.15.94:0/4292642801 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591400ccc0 con 0x7f5918100e20 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f590e7fc640 1 -- 172.21.15.94:0/4292642801 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59140108e0 con 0x7f5918100e20 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f590e7fc640 1 -- 172.21.15.94:0/4292642801 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914018880 con 0x7f5918100e20 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.197+0000 7f591f54e640 1 -- 172.21.15.94:0/4292642801 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59181a6bc0 con 0x7f5918100e20 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.198+0000 7f591f54e640 1 -- 172.21.15.94:0/4292642801 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181a7100 con 0x7f5918100e20 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.200+0000 7f590e7fc640 1 -- 172.21.15.94:0/4292642801 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5914010a90 con 0x7f5918100e20 2024-09-19T04:28:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f5921927740 1 Processor -- start 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f5921927740 1 -- start start 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f5921927740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d8ab40 0x563fd0d89090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f5921927740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0d89a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f5921927740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d94370 0x563fd0d96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f5921927740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563fd0cef2f0 con 0x563fd0d94370 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f5921927740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563fd0c96cf0 con 0x563fd0d895d0 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f5921927740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563fd0ceb2b0 con 0x563fd0d8ab40 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591d2ea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0d89a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591daeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d8ab40 0x563fd0d89090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591d2ea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0d89a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51108/0 (socket says 172.21.15.94:51108) 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591d2ea640 1 -- 172.21.15.94:0/844763570 learned_addr learned my addr 172.21.15.94:0/844763570 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591daeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d8ab40 0x563fd0d89090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54378/0 (socket says 172.21.15.94:54378) 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591cae9640 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d94370 0x563fd0d96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f590e7fc640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59040a33b0 0x7f59040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591d2ea640 1 -- 172.21.15.94:0/844763570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d8ab40 msgr2=0x563fd0d89090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591d2ea640 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d8ab40 0x563fd0d89090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591d2ea640 1 -- 172.21.15.94:0/844763570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d94370 msgr2=0x563fd0d96760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.201+0000 7f591d2ea640 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d94370 0x563fd0d96760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.202+0000 7f591d2ea640 1 -- 172.21.15.94:0/844763570 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd0c7b3a0 con 0x563fd0d895d0 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.202+0000 7f591cae9640 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d94370 0x563fd0d96760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.202+0000 7f591daeb640 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d8ab40 0x563fd0d89090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.202+0000 7f591daeb640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59040a33b0 0x7f59040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.202+0000 7f591d2ea640 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0d89a10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f59140cd730 tx=0x7f59140d87d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb6c7dc2b1e16eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.202+0000 7f590e7fc640 1 -- 172.21.15.94:0/4292642801 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f59140d00b0 con 0x7f5918100e20 2024-09-19T04:28:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.202+0000 7f59037fe640 1 -- 172.21.15.94:0/844763570 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140986a0 con 0x563fd0d895d0 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.202+0000 7f59037fe640 1 -- 172.21.15.94:0/844763570 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5914010e00 con 0x563fd0d895d0 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.202+0000 7f59037fe640 1 -- 172.21.15.94:0/844763570 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591409bd60 con 0x563fd0d895d0 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f5921927740 1 -- 172.21.15.94:0/844763570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 msgr2=0x563fd0d89a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f5921927740 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0d89a10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f59140cd730 tx=0x7f59140d87d0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f591daeb640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59040a33b0 0x7f59040a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5910005fe0 tx=0x7f591003c040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f5921927740 1 -- 172.21.15.94:0/844763570 shutdown_connections 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f5921927740 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d94370 0x563fd0d96760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f5921927740 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0d89a10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f5921927740 1 --2- 172.21.15.94:0/844763570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d8ab40 0x563fd0d89090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f5921927740 1 -- 172.21.15.94:0/844763570 >> 172.21.15.94:0/844763570 conn(0x563fd0d52310 msgr2=0x563fd0d85c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f5921927740 1 -- 172.21.15.94:0/844763570 shutdown_connections 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.203+0000 7f5921927740 1 -- 172.21.15.94:0/844763570 wait complete. 2024-09-19T04:28:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f5921927740 1 -- start start 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f5921927740 1 Processor -- start 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f5921927740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 0x563fd0d93e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f5921927740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0d910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f5921927740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0d91850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f5921927740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563fd0d97010 con 0x563fd0d895d0 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f5921927740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563fd0caa060 con 0x563fd0d8ab40 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f5921927740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563fd0d97190 con 0x563fd0d94370 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f5921927740 10 client.0 _ll_register_callbacks cb 0x563fd09fd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f591cae9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0d910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f591cae9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0d910c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51114/0 (socket says 172.21.15.94:51114) 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f591cae9640 1 -- 172.21.15.94:0/531995149 learned_addr learned my addr 172.21.15.94:0/531995149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.204+0000 7f591d2ea640 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 0x563fd0d93e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f591daeb640 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0d91850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f591cae9640 1 -- 172.21.15.94:0/531995149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 msgr2=0x563fd0d91850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f591cae9640 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0d91850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f591cae9640 1 -- 172.21.15.94:0/531995149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 msgr2=0x563fd0d93e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f591cae9640 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 0x563fd0d93e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f591cae9640 1 -- 172.21.15.94:0/531995149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd0a603c0 con 0x563fd0d8ab40 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f591cae9640 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0d910c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5908012700 tx=0x7f5908012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9750e3ca222721f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f5900ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f5900ff9640 1 -- 172.21.15.94:0/531995149 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908004050 con 0x563fd0d8ab40 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.205+0000 7f5921927740 1 -- 172.21.15.94:0/531995149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563fd0cfc860 con 0x563fd0d8ab40 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.206+0000 7f5900ff9640 1 -- 172.21.15.94:0/531995149 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f59080041f0 con 0x563fd0d8ab40 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.206+0000 7f5900ff9640 1 -- 172.21.15.94:0/531995149 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f59080043f0 con 0x563fd0d8ab40 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.206+0000 7f5921927740 10 client.26642 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:25.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.206+0000 7f5921927740 1 -- 172.21.15.94:0/531995149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563fd0da2c40 con 0x563fd0d8ab40 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.206+0000 7f5900ff9640 1 -- 172.21.15.94:0/531995149 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563fd0cfc860 con 0x563fd0d8ab40 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.206+0000 7f5900ff9640 1 -- 172.21.15.94:0/531995149 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f5908003b10 con 0x563fd0d8ab40 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.206+0000 7f5921927740 10 client.26642 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.206+0000 7f5921927740 10 client.26642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.206+0000 7f5921927740 20 client.26642 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.207+0000 7f5921927740 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563fd0da4480 0x563fd0da6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.207+0000 7f5921927740 4 client.26642 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.207+0000 7f5921927740 1 -- 172.21.15.94:0/531995149 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fd0cf6d70 con 0x563fd0da4480 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.207+0000 7f591d2ea640 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563fd0da4480 0x563fd0da6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.207+0000 7f5900ff9640 10 client.26642 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.207+0000 7f591d2ea640 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563fd0da4480 0x563fd0da6920 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd199421e0359ab3 server_cookie=1aa1e2e989e970f7 in_seq=0 out_seq=0 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.208+0000 7f5900ff9640 1 -- 172.21.15.94:0/531995149 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fd0cf6d70 con 0x563fd0da4480 2024-09-19T04:28:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5900ff9640 10 client.26642 handle_command_reply: tid=0 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 client.26642 shutdown 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 -- 172.21.15.94:0/531995149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 msgr2=0x563fd0d910c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0d910c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5908012700 tx=0x7f5908012bc0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 -- 172.21.15.94:0/531995149 shutdown_connections 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563fd0da4480 0x563fd0da6920 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0d91850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0d910c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 --2- 172.21.15.94:0/531995149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 0x563fd0d93e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 -- 172.21.15.94:0/531995149 >> 172.21.15.94:0/531995149 conn(0x563fd0d52310 msgr2=0x563fd0d8b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5900ff9640 0 client.26642 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.209+0000 7f5921927740 1 -- 172.21.15.94:0/531995149 shutdown_connections 2024-09-19T04:28:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.210+0000 7f5921927740 1 -- 172.21.15.94:0/531995149 wait complete. 2024-09-19T04:28:25.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.210+0000 7f5921927740 20 client.26642 trim_cache size 0 max 16384 2024-09-19T04:28:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f5921927740 1 Processor -- start 2024-09-19T04:28:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f5921927740 1 -- start start 2024-09-19T04:28:25.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f5921927740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 0x563fd0d8a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f5921927740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0dac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f5921927740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0dac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f5921927740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b2500 con 0x563fd0d895d0 2024-09-19T04:28:25.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f5921927740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59181afd10 con 0x563fd0d8ab40 2024-09-19T04:28:25.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f5921927740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59181adb80 con 0x563fd0d94370 2024-09-19T04:28:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f591d2ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 0x563fd0d8a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f591cae9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0dac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f591cae9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0dac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51118/0 (socket says 172.21.15.94:51118) 2024-09-19T04:28:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.229+0000 7f591cae9640 1 -- 172.21.15.94:0/356249957 learned_addr learned my addr 172.21.15.94:0/356249957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f591daeb640 1 --2- 172.21.15.94:0/356249957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0dac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f591d2ea640 1 -- 172.21.15.94:0/356249957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 msgr2=0x563fd0dac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f591d2ea640 1 --2- 172.21.15.94:0/356249957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0dac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f591d2ea640 1 -- 172.21.15.94:0/356249957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 msgr2=0x563fd0dac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f591d2ea640 1 --2- 172.21.15.94:0/356249957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0dac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f591d2ea640 1 -- 172.21.15.94:0/356249957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd0d91bc0 con 0x563fd0d895d0 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f591cae9640 1 --2- 172.21.15.94:0/356249957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0dac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f591d2ea640 1 --2- 172.21.15.94:0/356249957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 0x563fd0d8a3a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9c0 tx=0x7f5914009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df788a0747958f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f5902ffd640 1 -- 172.21.15.94:0/356249957 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591409bf40 con 0x563fd0d895d0 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f5902ffd640 1 -- 172.21.15.94:0/356249957 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59140d3d50 con 0x563fd0d895d0 2024-09-19T04:28:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.230+0000 7f5902ffd640 1 -- 172.21.15.94:0/356249957 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591409c1d0 con 0x563fd0d895d0 2024-09-19T04:28:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.231+0000 7f5921927740 1 -- 172.21.15.94:0/356249957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 msgr2=0x563fd0d8a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.231+0000 7f5921927740 1 --2- 172.21.15.94:0/356249957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 0x563fd0d8a3a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9c0 tx=0x7f5914009700 comp rx=0 tx=0).stop 2024-09-19T04:28:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.231+0000 7f5921927740 1 -- 172.21.15.94:0/356249957 shutdown_connections 2024-09-19T04:28:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.231+0000 7f5921927740 1 --2- 172.21.15.94:0/356249957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0dac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.231+0000 7f5921927740 1 --2- 172.21.15.94:0/356249957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d8ab40 0x563fd0dac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.231+0000 7f5921927740 1 --2- 172.21.15.94:0/356249957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d895d0 0x563fd0d8a3a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.231+0000 7f5921927740 1 -- 172.21.15.94:0/356249957 >> 172.21.15.94:0/356249957 conn(0x563fd0d52310 msgr2=0x563fd0d80a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.231+0000 7f5921927740 1 -- 172.21.15.94:0/356249957 shutdown_connections 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.231+0000 7f5921927740 1 -- 172.21.15.94:0/356249957 wait complete. 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f5921927740 1 -- start start 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f5921927740 1 Processor -- start 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f5921927740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0e07560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f5921927740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d8ab40 0x563fd0e07aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f5921927740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0d979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f5921927740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59181ae1d0 con 0x563fd0d8ab40 2024-09-19T04:28:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f5921927740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59181af640 con 0x563fd0d895d0 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f5921927740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59181ae8a0 con 0x563fd0d94370 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f5921927740 10 client.0 _ll_register_callbacks cb 0x7f5908009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f591cae9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d8ab40 0x563fd0e07aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f591cae9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d8ab40 0x563fd0e07aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54650/0 (socket says 172.21.15.94:54650) 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f591cae9640 1 -- 172.21.15.94:0/2343810267 learned_addr learned my addr 172.21.15.94:0/2343810267 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f591daeb640 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0d979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.232+0000 7f591d2ea640 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0e07560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f591cae9640 1 -- 172.21.15.94:0/2343810267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 msgr2=0x563fd0d979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f591cae9640 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0d979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f591cae9640 1 -- 172.21.15.94:0/2343810267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 msgr2=0x563fd0e07560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f591cae9640 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0e07560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f591cae9640 1 -- 172.21.15.94:0/2343810267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd0d981f0 con 0x563fd0d8ab40 2024-09-19T04:28:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f591cae9640 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d8ab40 0x563fd0e07aa0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5908010520 tx=0x7f59080107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946a5d4d547f817 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f5900ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f5900ff9640 1 -- 172.21.15.94:0/2343810267 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908028070 con 0x563fd0d8ab40 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f5900ff9640 1 -- 172.21.15.94:0/2343810267 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f5908010a40 con 0x563fd0d8ab40 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f5921927740 1 -- 172.21.15.94:0/2343810267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563fd0c73170 con 0x563fd0d8ab40 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.233+0000 7f5900ff9640 1 -- 172.21.15.94:0/2343810267 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5908004210 con 0x563fd0d8ab40 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5921927740 10 client.17031 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5921927740 1 -- 172.21.15.94:0/2343810267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563fd0e0ed30 con 0x563fd0d8ab40 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5900ff9640 1 -- 172.21.15.94:0/2343810267 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5908032070 con 0x563fd0d8ab40 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5900ff9640 1 -- 172.21.15.94:0/2343810267 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f59080051d0 con 0x563fd0d8ab40 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5921927740 10 client.17031 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5921927740 10 client.17031 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5921927740 20 client.17031 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5921927740 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563fd0e0f530 0x563fd0e11950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5921927740 4 client.17031 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:28:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f5921927740 1 -- 172.21.15.94:0/2343810267 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fd0c73170 con 0x563fd0e0f530 2024-09-19T04:28:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.234+0000 7f591d2ea640 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563fd0e0f530 0x563fd0e11950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f591d2ea640 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563fd0e0f530 0x563fd0e11950 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55dd61f59b205023 server_cookie=611431efe565a98d in_seq=0 out_seq=0 2024-09-19T04:28:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5900ff9640 10 client.17031 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:28:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5900ff9640 1 -- 172.21.15.94:0/2343810267 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fd0c73170 con 0x563fd0e0f530 2024-09-19T04:28:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5900ff9640 10 client.17031 handle_command_reply: tid=0 2024-09-19T04:28:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5921927740 1 client.17031 shutdown 2024-09-19T04:28:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5921927740 1 -- 172.21.15.94:0/2343810267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d8ab40 msgr2=0x563fd0e07aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5921927740 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d8ab40 0x563fd0e07aa0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5908010520 tx=0x7f59080107d0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5921927740 1 -- 172.21.15.94:0/2343810267 shutdown_connections 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5921927740 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x563fd0e0f530 0x563fd0e11950 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5921927740 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fd0d94370 0x563fd0d979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5921927740 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563fd0d8ab40 0x563fd0e07aa0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5921927740 1 --2- 172.21.15.94:0/2343810267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563fd0d895d0 0x563fd0e07560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.235+0000 7f5921927740 1 -- 172.21.15.94:0/2343810267 >> 172.21.15.94:0/2343810267 conn(0x563fd0d52310 msgr2=0x563fd0d8f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.236+0000 7f5900ff9640 0 client.17031 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.236+0000 7f5921927740 1 -- 172.21.15.94:0/2343810267 shutdown_connections 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.236+0000 7f5921927740 1 -- 172.21.15.94:0/2343810267 wait complete. 2024-09-19T04:28:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.236+0000 7f5921927740 20 client.17031 trim_cache size 0 max 16384 2024-09-19T04:28:25.243 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:28:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 -- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59040a33b0 msgr2=0x7f59040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59040a33b0 0x7f59040a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5910005fe0 tx=0x7f591003c040 comp rx=0 tx=0).stop 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 -- 172.21.15.94:0/4292642801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5918100e20 msgr2=0x7f591810e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5918100e20 0x7f591810e4d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9f0 tx=0x7f591400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 -- 172.21.15.94:0/4292642801 shutdown_connections 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f59040a33b0 0x7f59040a5870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5918102ef0 0x7f591810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5918102570 0x7f591810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 --2- 172.21.15.94:0/4292642801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5918100e20 0x7f591810e4d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 -- 172.21.15.94:0/4292642801 >> 172.21.15.94:0/4292642801 conn(0x7f59180fc7d0 msgr2=0x7f59180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 -- 172.21.15.94:0/4292642801 shutdown_connections 2024-09-19T04:28:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:25.237+0000 7f591f54e640 1 -- 172.21.15.94:0/4292642801 wait complete. 2024-09-19T04:28:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 730.0 s, 392.4 tps, lat 255.874 ms stddev 28.387 2024-09-19T04:28:26.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:28:26.340 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:26.753 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:26 smithi094 ceph-mon[30972]: pgmap v419: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.5 MiB/s wr, 599 op/s 2024-09-19T04:28:26.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.875+0000 7f2a94cb0640 1 -- 172.21.15.94:0/2046538367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102ef0 msgr2=0x7f2a9010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.875+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/2046538367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102ef0 0x7f2a9010da40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009c10 tx=0x7f2a8802a770 comp rx=0 tx=0).stop 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.876+0000 7f2a94cb0640 1 -- 172.21.15.94:0/2046538367 shutdown_connections 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.876+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/2046538367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102ef0 0x7f2a9010da40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.876+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/2046538367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a90102570 0x7f2a901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.876+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/2046538367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90100e20 0x7f2a90101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.876+0000 7f2a94cb0640 1 -- 172.21.15.94:0/2046538367 >> 172.21.15.94:0/2046538367 conn(0x7f2a900fc7d0 msgr2=0x7f2a900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.876+0000 7f2a94cb0640 1 -- 172.21.15.94:0/2046538367 shutdown_connections 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.876+0000 7f2a94cb0640 1 -- 172.21.15.94:0/2046538367 wait complete. 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.876+0000 7f2a94cb0640 1 Processor -- start 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.876+0000 7f2a94cb0640 1 -- start start 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a94cb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a90100e20 0x7f2a90198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a94cb0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102570 0x7f2a90198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a94cb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90102ef0 0x7f2a9019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a94cb0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a90111f30 con 0x7f2a90100e20 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a94cb0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a90111db0 con 0x7f2a90102570 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a94cb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901120b0 con 0x7f2a90102ef0 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a90100e20 0x7f2a90198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102570 0x7f2a90198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a90100e20 0x7f2a90198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54660/0 (socket says 172.21.15.94:54660) 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102570 0x7f2a90198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51142/0 (socket says 172.21.15.94:51142) 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8e59c640 1 -- 172.21.15.94:0/3490689445 learned_addr learned my addr 172.21.15.94:0/3490689445 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90102ef0 0x7f2a9019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/3490689445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90102ef0 msgr2=0x7f2a9019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:26.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90102ef0 0x7f2a9019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/3490689445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a90100e20 msgr2=0x7f2a90198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:26.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a90100e20 0x7f2a90198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/3490689445 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9019d3c0 con 0x7f2a90102570 2024-09-19T04:28:26.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102570 0x7f2a90198950 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009bf0 tx=0x7f2a8400b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:26.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.877+0000 7f2a7f7fe640 1 -- 172.21.15.94:0/3490689445 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8401e070 con 0x7f2a90102570 2024-09-19T04:28:26.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.878+0000 7f2a94cb0640 1 -- 172.21.15.94:0/3490689445 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9019d6b0 con 0x7f2a90102570 2024-09-19T04:28:26.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.878+0000 7f2a7f7fe640 1 -- 172.21.15.94:0/3490689445 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a84009ce0 con 0x7f2a90102570 2024-09-19T04:28:26.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.878+0000 7f2a7f7fe640 1 -- 172.21.15.94:0/3490689445 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84018610 con 0x7f2a90102570 2024-09-19T04:28:26.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.878+0000 7f2a94cb0640 1 -- 172.21.15.94:0/3490689445 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9010f930 con 0x7f2a90102570 2024-09-19T04:28:26.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a7f7fe640 1 -- 172.21.15.94:0/3490689445 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2a84021070 con 0x7f2a90102570 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a97089740 1 Processor -- start 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a97089740 1 -- start start 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a97089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a97089740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a25a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a97089740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 0x55c143a32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a97089740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c14398b2f0 con 0x55c143a255d0 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a97089740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c143932cf0 con 0x55c143a30370 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a97089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c1439872b0 con 0x55c143a26b40 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.881+0000 7f2a7f7fe640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2a780a3430 0x7f2a780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a25a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a25a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54672/0 (socket says 172.21.15.94:54672) 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8e59c640 1 -- 172.21.15.94:0/4155816192 learned_addr learned my addr 172.21.15.94:0/4155816192 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 0x55c143a32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/4155816192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2a780a3430 0x7f2a780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a7f7fe640 1 -- 172.21.15.94:0/3490689445 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f2a840d3360 con 0x7f2a90102570 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/4155816192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 msgr2=0x55c143a25090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/4155816192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a25090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/4155816192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 msgr2=0x55c143a25a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/4155816192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a25a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/4155816192 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1439173a0 con 0x55c143a30370 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/4155816192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2a780a3430 0x7f2a780a58f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a8802ac90 tx=0x7f2a88005fd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:26.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/4155816192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 0x55c143a32760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a840d0a10 tx=0x7f2a8409efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=199072ce4e813711 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a7cff9640 1 -- 172.21.15.94:0/4155816192 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84028040 con 0x55c143a30370 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.882+0000 7f2a7cff9640 1 -- 172.21.15.94:0/4155816192 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a840751c0 con 0x55c143a30370 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.883+0000 7f2a7cff9640 1 -- 172.21.15.94:0/4155816192 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a840d0c90 con 0x55c143a30370 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.883+0000 7f2a97089740 1 -- 172.21.15.94:0/4155816192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 msgr2=0x55c143a32760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.883+0000 7f2a97089740 1 --2- 172.21.15.94:0/4155816192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 0x55c143a32760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a840d0a10 tx=0x7f2a8409efb0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.883+0000 7f2a97089740 1 -- 172.21.15.94:0/4155816192 shutdown_connections 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.883+0000 7f2a97089740 1 --2- 172.21.15.94:0/4155816192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 0x55c143a32760 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.883+0000 7f2a97089740 1 --2- 172.21.15.94:0/4155816192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.883+0000 7f2a97089740 1 --2- 172.21.15.94:0/4155816192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a25090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.883+0000 7f2a97089740 1 -- 172.21.15.94:0/4155816192 >> 172.21.15.94:0/4155816192 conn(0x55c1439ee310 msgr2=0x55c143a21c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.884+0000 7f2a97089740 1 -- 172.21.15.94:0/4155816192 shutdown_connections 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.884+0000 7f2a97089740 1 -- 172.21.15.94:0/4155816192 wait complete. 2024-09-19T04:28:26.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.884+0000 7f2a97089740 1 -- start start 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.884+0000 7f2a97089740 1 Processor -- start 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a97089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a255d0 0x55c143a2fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a97089740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a26b40 0x55c143a2d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a97089740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a30370 0x55c143a2d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a97089740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c143a33070 con 0x55c143a30370 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a97089740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c143946060 con 0x55c143a26b40 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a97089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c143a331f0 con 0x55c143a255d0 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a30370 0x55c143a2d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a30370 0x55c143a2d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54684/0 (socket says 172.21.15.94:54684) 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8ed9d640 1 -- 172.21.15.94:0/918774099 learned_addr learned my addr 172.21.15.94:0/918774099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a26b40 0x55c143a2d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a255d0 0x55c143a2fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a97089740 10 client.0 _ll_register_callbacks cb 0x55c1436998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8ed9d640 1 -- 172.21.15.94:0/918774099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a255d0 msgr2=0x55c143a2fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a255d0 0x55c143a2fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8ed9d640 1 -- 172.21.15.94:0/918774099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a26b40 msgr2=0x55c143a2d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a26b40 0x55c143a2d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8ed9d640 1 -- 172.21.15.94:0/918774099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1436fc3c0 con 0x55c143a30370 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a30370 0x55c143a2d870 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009c10 tx=0x7f2a88009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74337aedeecb123c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.885+0000 7f2a75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a75ffb640 1 -- 172.21.15.94:0/918774099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88036070 con 0x55c143a30370 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a97089740 1 -- 172.21.15.94:0/918774099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a840751c0 con 0x55c143a30370 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a75ffb640 1 -- 172.21.15.94:0/918774099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f2a88004690 con 0x55c143a30370 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a75ffb640 1 -- 172.21.15.94:0/918774099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2a880564a0 con 0x55c143a30370 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a97089740 10 client.17043 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:26.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a97089740 1 -- 172.21.15.94:0/918774099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c143a473d0 con 0x55c143a30370 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a75ffb640 1 -- 172.21.15.94:0/918774099 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a840751c0 con 0x55c143a30370 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a75ffb640 1 -- 172.21.15.94:0/918774099 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f2a8803c030 con 0x55c143a30370 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a97089740 10 client.17043 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a97089740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a97089740 20 client.17043 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a97089740 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c143a488e0 0x55c143a4ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a97089740 4 client.17043 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.886+0000 7f2a97089740 1 -- 172.21.15.94:0/918774099 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c143998860 con 0x55c143a488e0 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.887+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c143a488e0 0x55c143a4ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.887+0000 7f2a75ffb640 10 client.17043 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.887+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c143a488e0 0x55c143a4ad80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52130b9b7873483c server_cookie=17df950c4fe6d541 in_seq=0 out_seq=0 2024-09-19T04:28:26.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a75ffb640 1 -- 172.21.15.94:0/918774099 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c143998860 con 0x55c143a488e0 2024-09-19T04:28:26.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a75ffb640 10 client.17043 handle_command_reply: tid=0 2024-09-19T04:28:26.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a97089740 1 client.17043 shutdown 2024-09-19T04:28:26.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a97089740 1 -- 172.21.15.94:0/918774099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a30370 msgr2=0x55c143a2d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:26.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a97089740 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a30370 0x55c143a2d870 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009c10 tx=0x7f2a88009500 comp rx=0 tx=0).stop 2024-09-19T04:28:26.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a97089740 1 -- 172.21.15.94:0/918774099 shutdown_connections 2024-09-19T04:28:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a97089740 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c143a488e0 0x55c143a4ad80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a97089740 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a30370 0x55c143a2d870 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a97089740 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a26b40 0x55c143a2d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a97089740 1 --2- 172.21.15.94:0/918774099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a255d0 0x55c143a2fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a97089740 1 -- 172.21.15.94:0/918774099 >> 172.21.15.94:0/918774099 conn(0x55c1439ee310 msgr2=0x55c143a279c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.888+0000 7f2a75ffb640 0 client.17043 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.889+0000 7f2a97089740 1 -- 172.21.15.94:0/918774099 shutdown_connections 2024-09-19T04:28:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.889+0000 7f2a97089740 1 -- 172.21.15.94:0/918774099 wait complete. 2024-09-19T04:28:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.889+0000 7f2a97089740 20 client.17043 trim_cache size 0 max 16384 2024-09-19T04:28:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a97089740 1 Processor -- start 2024-09-19T04:28:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a97089740 1 -- start start 2024-09-19T04:28:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a97089740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a4ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a97089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a21700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a97089740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 0x55c143a21c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a97089740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901ae050 con 0x55c143a255d0 2024-09-19T04:28:26.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a8dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a21700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a97089740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901ab860 con 0x55c143a30370 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a97089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901a96d0 con 0x55c143a26b40 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a8dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a21700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54462/0 (socket says 172.21.15.94:54462) 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a8e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a4ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/3266047428 learned_addr learned my addr 172.21.15.94:0/3266047428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.905+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/3266047428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 0x55c143a21c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a8e59c640 1 -- 172.21.15.94:0/3266047428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 msgr2=0x55c143a21700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/3266047428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a21700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a8e59c640 1 -- 172.21.15.94:0/3266047428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 msgr2=0x55c143a21c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/3266047428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 0x55c143a21c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a8e59c640 1 -- 172.21.15.94:0/3266047428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c143a22180 con 0x55c143a255d0 2024-09-19T04:28:26.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/3266047428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a21700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:26.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/3266047428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a4ad00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55c143a48cc0 tx=0x7f2a8000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286ec1c2dd2b367e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:26.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a77fff640 1 -- 172.21.15.94:0/3266047428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80011440 con 0x55c143a255d0 2024-09-19T04:28:26.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a77fff640 1 -- 172.21.15.94:0/3266047428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a80011a90 con 0x55c143a255d0 2024-09-19T04:28:26.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.906+0000 7f2a77fff640 1 -- 172.21.15.94:0/3266047428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8001da60 con 0x55c143a255d0 2024-09-19T04:28:26.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.907+0000 7f2a97089740 1 -- 172.21.15.94:0/3266047428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 msgr2=0x55c143a4ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:26.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.907+0000 7f2a97089740 1 --2- 172.21.15.94:0/3266047428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a4ad00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55c143a48cc0 tx=0x7f2a8000b480 comp rx=0 tx=0).stop 2024-09-19T04:28:26.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.907+0000 7f2a97089740 1 -- 172.21.15.94:0/3266047428 shutdown_connections 2024-09-19T04:28:26.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.907+0000 7f2a97089740 1 --2- 172.21.15.94:0/3266047428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a30370 0x55c143a21c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.907+0000 7f2a97089740 1 --2- 172.21.15.94:0/3266047428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a26b40 0x55c143a21700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.907+0000 7f2a97089740 1 --2- 172.21.15.94:0/3266047428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a255d0 0x55c143a4ad00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.907+0000 7f2a97089740 1 -- 172.21.15.94:0/3266047428 >> 172.21.15.94:0/3266047428 conn(0x55c1439ee310 msgr2=0x55c143a1ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.907+0000 7f2a97089740 1 -- 172.21.15.94:0/3266047428 shutdown_connections 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.907+0000 7f2a97089740 1 -- 172.21.15.94:0/3266047428 wait complete. 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a97089740 1 -- start start 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a97089740 1 Processor -- start 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a97089740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a255d0 0x55c143a414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a97089740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a26b40 0x55c143a29ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a97089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a30370 0x55c143a2a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a97089740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901a9d20 con 0x55c143a26b40 2024-09-19T04:28:26.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a97089740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901ab190 con 0x55c143a255d0 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a97089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901aa3f0 con 0x55c143a30370 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a97089740 10 client.0 _ll_register_callbacks cb 0x7f2a84005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a8dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a26b40 0x55c143a29ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a8dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a26b40 0x55c143a29ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54710/0 (socket says 172.21.15.94:54710) 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/152333309 learned_addr learned my addr 172.21.15.94:0/152333309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a30370 0x55c143a2a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.908+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a255d0 0x55c143a414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/152333309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a30370 msgr2=0x55c143a2a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a30370 0x55c143a2a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/152333309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a255d0 msgr2=0x55c143a414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a255d0 0x55c143a414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a8dd9b640 1 -- 172.21.15.94:0/152333309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c143a2acb0 con 0x55c143a26b40 2024-09-19T04:28:26.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a8dd9b640 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a26b40 0x55c143a29ee0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2a8409f590 tx=0x7f2a8409f0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff2b11accca6201 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a8ed9d640 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a30370 0x55c143a2a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a255d0 0x55c143a414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a75ffb640 1 -- 172.21.15.94:0/152333309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a840bc020 con 0x55c143a26b40 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a75ffb640 1 -- 172.21.15.94:0/152333309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f2a84010400 con 0x55c143a26b40 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a97089740 1 -- 172.21.15.94:0/152333309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a80011a90 con 0x55c143a26b40 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a75ffb640 1 -- 172.21.15.94:0/152333309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2a840b7040 con 0x55c143a26b40 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a97089740 10 client.17049 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.909+0000 7f2a97089740 1 -- 172.21.15.94:0/152333309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c143ab36f0 con 0x55c143a26b40 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.910+0000 7f2a75ffb640 1 -- 172.21.15.94:0/152333309 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a80011a90 con 0x55c143a26b40 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.910+0000 7f2a75ffb640 1 -- 172.21.15.94:0/152333309 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f2a8407b070 con 0x55c143a26b40 2024-09-19T04:28:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.910+0000 7f2a97089740 10 client.17049 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:26.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.910+0000 7f2a97089740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:26.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.910+0000 7f2a97089740 20 client.17049 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:26.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.910+0000 7f2a97089740 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c143ab39d0 0x55c143ab5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:26.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.910+0000 7f2a97089740 4 client.17049 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:28:26.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.910+0000 7f2a97089740 1 -- 172.21.15.94:0/152333309 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c143992d70 con 0x55c143ab39d0 2024-09-19T04:28:26.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.910+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c143ab39d0 0x55c143ab5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:26.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a8e59c640 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c143ab39d0 0x55c143ab5dc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a15f2ef7da8eecc2 server_cookie=72ff05af3da2316f in_seq=0 out_seq=0 2024-09-19T04:28:26.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a75ffb640 10 client.17049 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:26.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a75ffb640 1 -- 172.21.15.94:0/152333309 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c143992d70 con 0x55c143ab39d0 2024-09-19T04:28:26.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a75ffb640 10 client.17049 handle_command_reply: tid=0 2024-09-19T04:28:26.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 client.17049 shutdown 2024-09-19T04:28:26.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 -- 172.21.15.94:0/152333309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a26b40 msgr2=0x55c143a29ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:26.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a26b40 0x55c143a29ee0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2a8409f590 tx=0x7f2a8409f0e0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 -- 172.21.15.94:0/152333309 shutdown_connections 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55c143ab39d0 0x55c143ab5dc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c143a30370 0x55c143a2a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a75ffb640 0 client.17049 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c143a26b40 0x55c143a29ee0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 --2- 172.21.15.94:0/152333309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c143a255d0 0x55c143a414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 -- 172.21.15.94:0/152333309 >> 172.21.15.94:0/152333309 conn(0x55c1439ee310 msgr2=0x55c143a21700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 -- 172.21.15.94:0/152333309 shutdown_connections 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 1 -- 172.21.15.94:0/152333309 wait complete. 2024-09-19T04:28:26.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.911+0000 7f2a97089740 20 client.17049 trim_cache size 0 max 16384 2024-09-19T04:28:26.919 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:28:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 -- 172.21.15.94:0/3490689445 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2a780a3430 msgr2=0x7f2a780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2a780a3430 0x7f2a780a58f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a8802ac90 tx=0x7f2a88005fd0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 -- 172.21.15.94:0/3490689445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102570 msgr2=0x7f2a90198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102570 0x7f2a90198950 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009bf0 tx=0x7f2a8400b690 comp rx=0 tx=0).stop 2024-09-19T04:28:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 -- 172.21.15.94:0/3490689445 shutdown_connections 2024-09-19T04:28:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2a780a3430 0x7f2a780a58f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90102ef0 0x7f2a9019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a90102570 0x7f2a90198950 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 --2- 172.21.15.94:0/3490689445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a90100e20 0x7f2a90198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 -- 172.21.15.94:0/3490689445 >> 172.21.15.94:0/3490689445 conn(0x7f2a900fc7d0 msgr2=0x7f2a901048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 -- 172.21.15.94:0/3490689445 shutdown_connections 2024-09-19T04:28:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:26.912+0000 7f2a94cb0640 1 -- 172.21.15.94:0/3490689445 wait complete. 2024-09-19T04:28:27.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:26 smithi112 ceph-mon[35051]: pgmap v419: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.5 MiB/s wr, 599 op/s 2024-09-19T04:28:27.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:26 smithi104 ceph-mon[34145]: pgmap v419: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.5 MiB/s wr, 599 op/s 2024-09-19T04:28:28.603 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:28:28.603 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:28:28.942 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:29.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:28 smithi112 ceph-mon[35051]: pgmap v420: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 4.3 MiB/s wr, 583 op/s 2024-09-19T04:28:29.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:28 smithi094 ceph-mon[30972]: pgmap v420: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 4.3 MiB/s wr, 583 op/s 2024-09-19T04:28:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:28 smithi104 ceph-mon[34145]: pgmap v420: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 4.3 MiB/s wr, 583 op/s 2024-09-19T04:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.523+0000 7f4640394640 1 -- 172.21.15.94:0/2030272916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638101420 msgr2=0x7f463810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.523+0000 7f4640394640 1 --2- 172.21.15.94:0/2030272916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638101420 0x7f463810d950 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4630009cd0 tx=0x7f463002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.523+0000 7f4640394640 1 -- 172.21.15.94:0/2030272916 shutdown_connections 2024-09-19T04:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.523+0000 7f4640394640 1 --2- 172.21.15.94:0/2030272916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638101420 0x7f463810d950 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.523+0000 7f4640394640 1 --2- 172.21.15.94:0/2030272916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4638100a80 0x7f4638100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.523+0000 7f4640394640 1 --2- 172.21.15.94:0/2030272916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4638106a80 0x7f4638106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.523+0000 7f4640394640 1 -- 172.21.15.94:0/2030272916 >> 172.21.15.94:0/2030272916 conn(0x7f46380fc7a0 msgr2=0x7f46380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.524+0000 7f4640394640 1 -- 172.21.15.94:0/2030272916 shutdown_connections 2024-09-19T04:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.524+0000 7f4640394640 1 -- 172.21.15.94:0/2030272916 wait complete. 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.524+0000 7f4640394640 1 Processor -- start 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.524+0000 7f4640394640 1 -- start start 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f4640394640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4638100a80 0x7f4638198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f4640394640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4638101420 0x7f4638198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f4640394640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638106a80 0x7f463819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f4640394640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4638110d80 con 0x7f4638106a80 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f4640394640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4638110c00 con 0x7f4638100a80 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f4640394640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4638110f00 con 0x7f4638101420 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e931640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638106a80 0x7f463819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e931640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638106a80 0x7f463819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54738/0 (socket says 172.21.15.94:54738) 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e130640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4638100a80 0x7f4638198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e931640 1 -- 172.21.15.94:0/2849322091 learned_addr learned my addr 172.21.15.94:0/2849322091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463d92f640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4638101420 0x7f4638198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e130640 1 -- 172.21.15.94:0/2849322091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4638101420 msgr2=0x7f4638198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e130640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4638101420 0x7f4638198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e130640 1 -- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638106a80 msgr2=0x7f463819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e130640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638106a80 0x7f463819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e130640 1 -- 172.21.15.94:0/2849322091 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463819d2e0 con 0x7f4638100a80 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.525+0000 7f463e931640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638106a80 0x7f463819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.526+0000 7f463e130640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4638100a80 0x7f4638198330 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f463400e9f0 tx=0x7f463400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.526+0000 7f462f7fe640 1 -- 172.21.15.94:0/2849322091 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463400ccc0 con 0x7f4638100a80 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.526+0000 7f462f7fe640 1 -- 172.21.15.94:0/2849322091 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46340108e0 con 0x7f4638100a80 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.526+0000 7f4640394640 1 -- 172.21.15.94:0/2849322091 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463819d5d0 con 0x7f4638100a80 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.526+0000 7f462f7fe640 1 -- 172.21.15.94:0/2849322091 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634018880 con 0x7f4638100a80 2024-09-19T04:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.526+0000 7f4640394640 1 -- 172.21.15.94:0/2849322091 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4638102580 con 0x7f4638100a80 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.529+0000 7f462f7fe640 1 -- 172.21.15.94:0/2849322091 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4634010a90 con 0x7f4638100a80 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.529+0000 7f464276d740 1 Processor -- start 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.529+0000 7f464276d740 1 -- start start 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.529+0000 7f464276d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c340ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.529+0000 7f464276d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 0x55d0c3419800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.529+0000 7f464276d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c341c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.529+0000 7f464276d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3374280 con 0x55d0c3419d40 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.529+0000 7f464276d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d0c331bb10 con 0x55d0c340f480 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.529+0000 7f464276d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3370240 con 0x55d0c340eb60 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e931640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c340ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e130640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 0x55d0c3419800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e931640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c340ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54492/0 (socket says 172.21.15.94:54492) 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e931640 1 -- 172.21.15.94:0/2503803415 learned_addr learned my addr 172.21.15.94:0/2503803415 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463d92f640 1 --2- 172.21.15.94:0/2503803415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c341c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e931640 1 -- 172.21.15.94:0/2503803415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 msgr2=0x55d0c3419800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e931640 1 --2- 172.21.15.94:0/2503803415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 0x55d0c3419800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e931640 1 -- 172.21.15.94:0/2503803415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 msgr2=0x55d0c341c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e931640 1 --2- 172.21.15.94:0/2503803415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c341c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e931640 1 -- 172.21.15.94:0/2503803415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c33281f0 con 0x55d0c340eb60 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f463e931640 1 --2- 172.21.15.94:0/2503803415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c340ef40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4630005fb0 tx=0x7f46300093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d55326bfb6e0decf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.530+0000 7f462f7fe640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46240a33b0 0x7f46240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.531+0000 7f462cdea640 1 -- 172.21.15.94:0/2503803415 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630048070 con 0x55d0c340eb60 2024-09-19T04:28:29.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.531+0000 7f463e130640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46240a33b0 0x7f46240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.531+0000 7f462cdea640 1 -- 172.21.15.94:0/2503803415 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4630036070 con 0x55d0c340eb60 2024-09-19T04:28:29.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.531+0000 7f462cdea640 1 -- 172.21.15.94:0/2503803415 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630004720 con 0x55d0c340eb60 2024-09-19T04:28:29.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.531+0000 7f462f7fe640 1 -- 172.21.15.94:0/2849322091 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f46340cffd0 con 0x7f4638100a80 2024-09-19T04:28:29.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f464276d740 1 -- 172.21.15.94:0/2503803415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 msgr2=0x55d0c340ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f464276d740 1 --2- 172.21.15.94:0/2503803415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c340ef40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4630005fb0 tx=0x7f46300093e0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f463e130640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46240a33b0 0x7f46240a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46340cd790 tx=0x7f46340d3b20 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f464276d740 1 -- 172.21.15.94:0/2503803415 shutdown_connections 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f464276d740 1 --2- 172.21.15.94:0/2503803415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c341c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f464276d740 1 --2- 172.21.15.94:0/2503803415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 0x55d0c3419800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f464276d740 1 --2- 172.21.15.94:0/2503803415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c340ef40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f464276d740 1 -- 172.21.15.94:0/2503803415 >> 172.21.15.94:0/2503803415 conn(0x55d0c340b0f0 msgr2=0x55d0c340b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f464276d740 1 -- 172.21.15.94:0/2503803415 shutdown_connections 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.532+0000 7f464276d740 1 -- 172.21.15.94:0/2503803415 wait complete. 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.533+0000 7f464276d740 1 -- start start 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.533+0000 7f464276d740 1 Processor -- start 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.533+0000 7f464276d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 0x55d0c3416c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.533+0000 7f464276d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3417140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.533+0000 7f464276d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3417680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.533+0000 7f463d92f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3417140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.533+0000 7f463e931640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 0x55d0c3416c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.533+0000 7f463e931640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 0x55d0c3416c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51226/0 (socket says 172.21.15.94:51226) 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.533+0000 7f463d92f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3417140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54502/0 (socket says 172.21.15.94:54502) 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f463d92f640 1 -- 172.21.15.94:0/3801088272 learned_addr learned my addr 172.21.15.94:0/3801088272 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d0c341c910 con 0x55d0c3419d40 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d0c332ef10 con 0x55d0c340eb60 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d0c341ca90 con 0x55d0c340f480 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f464276d740 10 client.0 _ll_register_callbacks cb 0x55d0c32f7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f463e931640 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3417680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f463d92f640 1 -- 172.21.15.94:0/3801088272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 msgr2=0x55d0c3416c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f463d92f640 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 0x55d0c3416c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f463d92f640 1 -- 172.21.15.94:0/3801088272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 msgr2=0x55d0c3417680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f463d92f640 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3417680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f463d92f640 1 -- 172.21.15.94:0/3801088272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c30e56a0 con 0x55d0c340f480 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f463d92f640 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3417140 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f46280126e0 tx=0x7f4628012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be9221165d2ee4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f4621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f4621ffb640 1 -- 172.21.15.94:0/3801088272 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462800f920 con 0x55d0c340f480 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4630036070 con 0x55d0c340f480 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.534+0000 7f4621ffb640 1 -- 172.21.15.94:0/3801088272 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f462800fac0 con 0x55d0c340f480 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f4621ffb640 1 -- 172.21.15.94:0/3801088272 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f462800fcc0 con 0x55d0c340f480 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f464276d740 10 client.26656 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d0c3430d90 con 0x55d0c340f480 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f4621ffb640 1 -- 172.21.15.94:0/3801088272 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4630036070 con 0x55d0c340f480 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f4621ffb640 1 -- 172.21.15.94:0/3801088272 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f4628022030 con 0x55d0c340f480 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f464276d740 10 client.26656 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f464276d740 10 client.26656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f464276d740 20 client.26656 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:29.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f464276d740 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c3432260 0x55d0c3434670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f464276d740 4 client.26656 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0c33817f0 con 0x55d0c3432260 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.535+0000 7f463e931640 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c3432260 0x55d0c3434670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.536+0000 7f463e931640 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c3432260 0x55d0c3434670 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7f9edc1f30ac2 server_cookie=8ee947e42f3ad20a in_seq=0 out_seq=0 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.536+0000 7f4621ffb640 10 client.26656 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.536+0000 7f4621ffb640 1 -- 172.21.15.94:0/3801088272 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0c33817f0 con 0x55d0c3432260 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.536+0000 7f4621ffb640 10 client.26656 handle_command_reply: tid=0 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 client.26656 shutdown 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 msgr2=0x55d0c3417140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3417140 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f46280126e0 tx=0x7f4628012ba0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 shutdown_connections 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c3432260 0x55d0c3434670 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3417680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3417140 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 --2- 172.21.15.94:0/3801088272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 0x55d0c3416c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 >> 172.21.15.94:0/3801088272 conn(0x55d0c340b0f0 msgr2=0x55d0c340a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f4621ffb640 0 client.26656 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:29.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 shutdown_connections 2024-09-19T04:28:29.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 1 -- 172.21.15.94:0/3801088272 wait complete. 2024-09-19T04:28:29.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.537+0000 7f464276d740 20 client.26656 trim_cache size 0 max 16384 2024-09-19T04:28:29.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f464276d740 1 Processor -- start 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f464276d740 1 -- start start 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f464276d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c34115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f464276d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 0x55d0c3415ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f464276d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3439f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f464276d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46381adfd0 con 0x55d0c3419d40 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f464276d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46381ab7e0 con 0x55d0c340f480 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f464276d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46381a9630 con 0x55d0c340eb60 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f463e931640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3439f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f463e931640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3439f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54770/0 (socket says 172.21.15.94:54770) 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f463e931640 1 -- 172.21.15.94:0/406961317 learned_addr learned my addr 172.21.15.94:0/406961317 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:29.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f463e931640 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c34115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f463d92f640 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 0x55d0c3415ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f463d92f640 1 -- 172.21.15.94:0/406961317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 msgr2=0x55d0c34115c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.560+0000 7f463d92f640 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c34115c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.561+0000 7f463d92f640 1 -- 172.21.15.94:0/406961317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 msgr2=0x55d0c3439f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.561+0000 7f463d92f640 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3439f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.561+0000 7f463d92f640 1 -- 172.21.15.94:0/406961317 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c3410070 con 0x55d0c340f480 2024-09-19T04:28:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.561+0000 7f463e931640 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3439f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.561+0000 7f463d92f640 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 0x55d0c3415ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f462801c5a0 tx=0x7f462801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b399852b7a1c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.561+0000 7f4623fff640 1 -- 172.21.15.94:0/406961317 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462801c790 con 0x55d0c340f480 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.561+0000 7f4623fff640 1 -- 172.21.15.94:0/406961317 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f462801c930 con 0x55d0c340f480 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.561+0000 7f4623fff640 1 -- 172.21.15.94:0/406961317 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628003870 con 0x55d0c340f480 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 -- 172.21.15.94:0/406961317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 msgr2=0x55d0c3415ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 0x55d0c3415ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f462801c5a0 tx=0x7f462801c4a0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 -- 172.21.15.94:0/406961317 shutdown_connections 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3439f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340f480 0x55d0c3415ab0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 --2- 172.21.15.94:0/406961317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340eb60 0x55d0c34115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 -- 172.21.15.94:0/406961317 >> 172.21.15.94:0/406961317 conn(0x55d0c340b0f0 msgr2=0x55d0c34153a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 -- 172.21.15.94:0/406961317 shutdown_connections 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 -- 172.21.15.94:0/406961317 wait complete. 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.562+0000 7f464276d740 1 -- start start 2024-09-19T04:28:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f464276d740 1 Processor -- start 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f464276d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 0x55d0c342aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f464276d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3413710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f464276d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3413c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f464276d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46381a9ca0 con 0x55d0c3419d40 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f464276d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46381ab110 con 0x55d0c340eb60 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f464276d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46381aa370 con 0x55d0c340f480 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f464276d740 10 client.0 _ll_register_callbacks cb 0x7f4628009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463e931640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3413c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463e931640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3413c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54784/0 (socket says 172.21.15.94:54784) 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463e931640 1 -- 172.21.15.94:0/3277432826 learned_addr learned my addr 172.21.15.94:0/3277432826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463d92f640 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3413710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463e931640 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 0x55d0c342aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463e931640 1 -- 172.21.15.94:0/3277432826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 msgr2=0x55d0c3413710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463e931640 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3413710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463e931640 1 -- 172.21.15.94:0/3277432826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 msgr2=0x55d0c342aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463e931640 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 0x55d0c342aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.563+0000 7f463e931640 1 -- 172.21.15.94:0/3277432826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c3414240 con 0x55d0c3419d40 2024-09-19T04:28:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.564+0000 7f463e931640 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3413c50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4630005e30 tx=0x7f4630004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b28eff06c9e1c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.564+0000 7f4621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:28:29.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.564+0000 7f4621ffb640 1 -- 172.21.15.94:0/3277432826 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630048040 con 0x55d0c3419d40 2024-09-19T04:28:29.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.564+0000 7f4621ffb640 1 -- 172.21.15.94:0/3277432826 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f4630003b70 con 0x55d0c3419d40 2024-09-19T04:28:29.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.564+0000 7f464276d740 1 -- 172.21.15.94:0/3277432826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f462801c930 con 0x55d0c3419d40 2024-09-19T04:28:29.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.564+0000 7f4621ffb640 1 -- 172.21.15.94:0/3277432826 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4630004830 con 0x55d0c3419d40 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.564+0000 7f464276d740 10 client.17067 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.564+0000 7f464276d740 1 -- 172.21.15.94:0/3277432826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d0c349cd40 con 0x55d0c3419d40 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f4621ffb640 1 -- 172.21.15.94:0/3277432826 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f462801c930 con 0x55d0c3419d40 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f4621ffb640 1 -- 172.21.15.94:0/3277432826 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f4630067030 con 0x55d0c3419d40 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f464276d740 10 client.17067 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f464276d740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f464276d740 20 client.17067 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f464276d740 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c349cfb0 0x55d0c349f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f464276d740 4 client.17067 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f464276d740 1 -- 172.21.15.94:0/3277432826 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d0c337bd00 con 0x55d0c349cfb0 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f463d92f640 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c349cfb0 0x55d0c349f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f4621ffb640 10 client.17067 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.565+0000 7f463d92f640 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c349cfb0 0x55d0c349f3a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a717889357b36c13 server_cookie=28605aa4fa3e561 in_seq=0 out_seq=0 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f4621ffb640 1 -- 172.21.15.94:0/3277432826 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d0c337bd00 con 0x55d0c349cfb0 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f4621ffb640 10 client.17067 handle_command_reply: tid=0 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 client.17067 shutdown 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 -- 172.21.15.94:0/3277432826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 msgr2=0x55d0c3413c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3413c50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4630005e30 tx=0x7f4630004340 comp rx=0 tx=0).stop 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 -- 172.21.15.94:0/3277432826 shutdown_connections 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55d0c349cfb0 0x55d0c349f3a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d0c3419d40 0x55d0c3413c50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f4621ffb640 0 client.17067 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0c340f480 0x55d0c3413710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 --2- 172.21.15.94:0/3277432826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0c340eb60 0x55d0c342aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 -- 172.21.15.94:0/3277432826 >> 172.21.15.94:0/3277432826 conn(0x55d0c340b0f0 msgr2=0x55d0c3415bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.567+0000 7f464276d740 1 -- 172.21.15.94:0/3277432826 shutdown_connections 2024-09-19T04:28:29.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.568+0000 7f464276d740 1 -- 172.21.15.94:0/3277432826 wait complete. 2024-09-19T04:28:29.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.568+0000 7f464276d740 20 client.17067 trim_cache size 0 max 16384 2024-09-19T04:28:29.573 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:28:29.573 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:28:29.573 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "33b6eec6-12d6-499e-8995-77f3512fcf01", 2024-09-19T04:28:29.573 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:28:29.574 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:28:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.568+0000 7f4640394640 1 -- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46240a33b0 msgr2=0x7f46240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.568+0000 7f4640394640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46240a33b0 0x7f46240a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46340cd790 tx=0x7f46340d3b20 comp rx=0 tx=0).stop 2024-09-19T04:28:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.568+0000 7f4640394640 1 -- 172.21.15.94:0/2849322091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4638100a80 msgr2=0x7f4638198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.568+0000 7f4640394640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4638100a80 0x7f4638198330 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f463400e9f0 tx=0x7f463400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.569+0000 7f4640394640 1 -- 172.21.15.94:0/2849322091 shutdown_connections 2024-09-19T04:28:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.569+0000 7f4640394640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f46240a33b0 0x7f46240a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.569+0000 7f4640394640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4638106a80 0x7f463819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.569+0000 7f4640394640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4638101420 0x7f4638198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.569+0000 7f4640394640 1 --2- 172.21.15.94:0/2849322091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4638100a80 0x7f4638198330 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.569+0000 7f4640394640 1 -- 172.21.15.94:0/2849322091 >> 172.21.15.94:0/2849322091 conn(0x7f46380fc7a0 msgr2=0x7f4638104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.569+0000 7f4640394640 1 -- 172.21.15.94:0/2849322091 shutdown_connections 2024-09-19T04:28:29.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:29.569+0000 7f4640394640 1 -- 172.21.15.94:0/2849322091 wait complete. 2024-09-19T04:28:30.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:29 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:28:30.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:29 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:28:30.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:29 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:28:30.328 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:28:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 735.0 s, 364.0 tps, lat 273.923 ms stddev 114.898 2024-09-19T04:28:30.663 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:30.960 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:28:30.960 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:30.960 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:28:30.960 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:28:30.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:30.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:28:30.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:30.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:28:30.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:28:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: pgmap v421: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.9 MiB/s wr, 654 op/s 2024-09-19T04:28:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:28:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:28:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:30 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: pgmap v421: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.9 MiB/s wr, 654 op/s 2024-09-19T04:28:31.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:28:31.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:28:31.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:30 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:28:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:28:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: pgmap v421: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.9 MiB/s wr, 654 op/s 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:31.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:28:31.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:28:31.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:30 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:28:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 -- 172.21.15.94:0/503147657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5104106a90 msgr2=0x7f5104106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 --2- 172.21.15.94:0/503147657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5104106a90 0x7f5104106e70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009930 tx=0x7f50f402a420 comp rx=0 tx=0).stop 2024-09-19T04:28:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 -- 172.21.15.94:0/503147657 shutdown_connections 2024-09-19T04:28:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 --2- 172.21.15.94:0/503147657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5104101430 0x7f510410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 --2- 172.21.15.94:0/503147657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104100a90 0x7f5104100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 --2- 172.21.15.94:0/503147657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5104106a90 0x7f5104106e70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 -- 172.21.15.94:0/503147657 >> 172.21.15.94:0/503147657 conn(0x7f51040fc7d0 msgr2=0x7f51040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 -- 172.21.15.94:0/503147657 shutdown_connections 2024-09-19T04:28:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 -- 172.21.15.94:0/503147657 wait complete. 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.222+0000 7f51099b1640 1 Processor -- start 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f51099b1640 1 -- start start 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f51099b1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5104100a90 0x7f51041a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f51099b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104101430 0x7f51041a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f51099b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5104106a90 0x7f51041a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f51099b1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5104110d40 con 0x7f5104106a90 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f51099b1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5104110bc0 con 0x7f5104100a90 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f51099b1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5104110ec0 con 0x7f5104101430 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f5102ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5104100a90 0x7f51041a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f51037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5104106a90 0x7f51041a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f5102ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5104100a90 0x7f51041a3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51256/0 (socket says 172.21.15.94:51256) 2024-09-19T04:28:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f5102ffd640 1 -- 172.21.15.94:0/3701307788 learned_addr learned my addr 172.21.15.94:0/3701307788 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f5102ffd640 1 -- 172.21.15.94:0/3701307788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104101430 msgr2=0x7f51041a3aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:28:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f5102ffd640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104101430 0x7f51041a3aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f5102ffd640 1 -- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5104106a90 msgr2=0x7f51041a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f5102ffd640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5104106a90 0x7f51041a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f5102ffd640 1 -- 172.21.15.94:0/3701307788 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51041a4ab0 con 0x7f5104100a90 2024-09-19T04:28:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.223+0000 7f51037fe640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5104106a90 0x7f51041a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:31.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.224+0000 7f5102ffd640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5104100a90 0x7f51041a3560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009230 tx=0x7f50f40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.224+0000 7f51089af640 1 -- 172.21.15.94:0/3701307788 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f403f030 con 0x7f5104100a90 2024-09-19T04:28:31.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.224+0000 7f51089af640 1 -- 172.21.15.94:0/3701307788 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f40044c0 con 0x7f5104100a90 2024-09-19T04:28:31.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.224+0000 7f51089af640 1 -- 172.21.15.94:0/3701307788 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4035070 con 0x7f5104100a90 2024-09-19T04:28:31.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.224+0000 7f51099b1640 1 -- 172.21.15.94:0/3701307788 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51041a4d40 con 0x7f5104100a90 2024-09-19T04:28:31.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.224+0000 7f51099b1640 1 -- 172.21.15.94:0/3701307788 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510406bc10 con 0x7f5104100a90 2024-09-19T04:28:31.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.227+0000 7f51089af640 1 -- 172.21.15.94:0/3701307788 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f50f4042030 con 0x7f5104100a90 2024-09-19T04:28:31.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.227+0000 7f510bd8a740 1 Processor -- start 2024-09-19T04:28:31.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.227+0000 7f510bd8a740 1 -- start start 2024-09-19T04:28:31.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c68d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51089af640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f50ec0a3430 0x7f50ec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68ddc0 0x55926c698140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c69aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f510bd8a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55926c5f32e0 con 0x55926c68ddc0 2024-09-19T04:28:31.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f510bd8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55926c59ace0 con 0x55926c698680 2024-09-19T04:28:31.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f510bd8a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55926c5ef2a0 con 0x55926c68d4a0 2024-09-19T04:28:31.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51027fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68ddc0 0x55926c698140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c68d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51027fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68ddc0 0x55926c698140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54808/0 (socket says 172.21.15.94:54808) 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c68d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54546/0 (socket says 172.21.15.94:54546) 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51027fc640 1 -- 172.21.15.94:0/1385126331 learned_addr learned my addr 172.21.15.94:0/1385126331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51089af640 1 -- 172.21.15.94:0/3701307788 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f50f40f0000 con 0x7f5104100a90 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51037fe640 1 --2- 172.21.15.94:0/1385126331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c69aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f5102ffd640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f50ec0a3430 0x7f50ec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51037fe640 1 -- 172.21.15.94:0/1385126331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 msgr2=0x55926c69aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51037fe640 1 --2- 172.21.15.94:0/1385126331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c69aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51037fe640 1 -- 172.21.15.94:0/1385126331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68ddc0 msgr2=0x55926c698140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51037fe640 1 --2- 172.21.15.94:0/1385126331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68ddc0 0x55926c698140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51037fe640 1 -- 172.21.15.94:0/1385126331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926c57f440 con 0x55926c68d4a0 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.228+0000 7f51027fc640 1 --2- 172.21.15.94:0/1385126331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68ddc0 0x55926c698140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:31.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.229+0000 7f51037fe640 1 --2- 172.21.15.94:0/1385126331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c68d880 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55926c68adb0 tx=0x7f50fc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ea93a0bf5c22cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.229+0000 7f50e9ffb640 1 -- 172.21.15.94:0/1385126331 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00fc20 con 0x55926c68d4a0 2024-09-19T04:28:31.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.229+0000 7f50e9ffb640 1 -- 172.21.15.94:0/1385126331 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50fc00fdc0 con 0x55926c68d4a0 2024-09-19T04:28:31.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.229+0000 7f50e9ffb640 1 -- 172.21.15.94:0/1385126331 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc005940 con 0x55926c68d4a0 2024-09-19T04:28:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f510bd8a740 1 -- 172.21.15.94:0/1385126331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 msgr2=0x55926c68d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f5102ffd640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f50ec0a3430 0x7f50ec0a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50f4045fd0 tx=0x7f50f402a420 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f510bd8a740 1 --2- 172.21.15.94:0/1385126331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c68d880 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55926c68adb0 tx=0x7f50fc004ab0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f510bd8a740 1 -- 172.21.15.94:0/1385126331 shutdown_connections 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f510bd8a740 1 --2- 172.21.15.94:0/1385126331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c69aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f510bd8a740 1 --2- 172.21.15.94:0/1385126331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68ddc0 0x55926c698140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f510bd8a740 1 --2- 172.21.15.94:0/1385126331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c68d880 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f510bd8a740 1 -- 172.21.15.94:0/1385126331 >> 172.21.15.94:0/1385126331 conn(0x55926c689a30 msgr2=0x55926c689e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f510bd8a740 1 -- 172.21.15.94:0/1385126331 shutdown_connections 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.230+0000 7f510bd8a740 1 -- 172.21.15.94:0/1385126331 wait complete. 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f510bd8a740 1 -- start start 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f510bd8a740 1 Processor -- start 2024-09-19T04:28:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c695360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c695af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f510bd8a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55926c69b250 con 0x55926c68d4a0 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f510bd8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55926c5ae050 con 0x55926c698680 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f510bd8a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55926c69b3d0 con 0x55926c68ddc0 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f510bd8a740 10 client.0 _ll_register_callbacks cb 0x55926c5ee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f51037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f51027fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c695360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f51037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54824/0 (socket says 172.21.15.94:54824) 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.231+0000 7f51037fe640 1 -- 172.21.15.94:0/2897520819 learned_addr learned my addr 172.21.15.94:0/2897520819 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.232+0000 7f51037fe640 1 -- 172.21.15.94:0/2897520819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 msgr2=0x55926c695360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.232+0000 7f51037fe640 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c695360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.232+0000 7f51037fe640 1 -- 172.21.15.94:0/2897520819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 msgr2=0x55926c695af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:28:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.232+0000 7f51037fe640 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c695af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.232+0000 7f51037fe640 1 -- 172.21.15.94:0/2897520819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926c3643c0 con 0x55926c68d4a0 2024-09-19T04:28:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.232+0000 7f51027fc640 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c695360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:28:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.233+0000 7f51037fe640 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6980b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0058f0 tx=0x7f50fc005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427c63f0df1a3fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.233+0000 7f50e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:28:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.233+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2897520819 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc023070 con 0x55926c68d4a0 2024-09-19T04:28:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.233+0000 7f510bd8a740 1 -- 172.21.15.94:0/2897520819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50fc00fdc0 con 0x55926c68d4a0 2024-09-19T04:28:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.233+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2897520819 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f50fc004b20 con 0x55926c68d4a0 2024-09-19T04:28:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.233+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2897520819 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f50fc00fc20 con 0x55926c68d4a0 2024-09-19T04:28:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.233+0000 7f510bd8a740 10 client.17079 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.233+0000 7f510bd8a740 1 -- 172.21.15.94:0/2897520819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55926c6a6d20 con 0x55926c68d4a0 2024-09-19T04:28:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.233+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2897520819 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50fc00fdc0 con 0x55926c68d4a0 2024-09-19T04:28:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2897520819 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f50fc01d030 con 0x55926c68d4a0 2024-09-19T04:28:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f510bd8a740 10 client.17079 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f510bd8a740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:31.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f510bd8a740 20 client.17079 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55926c6a8070 0x55926c6aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f510bd8a740 4 client.17079 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f510bd8a740 1 -- 172.21.15.94:0/2897520819 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55926c600850 con 0x55926c6a8070 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f51027fc640 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55926c6a8070 0x55926c6aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f50e2ffd640 10 client.17079 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.234+0000 7f51027fc640 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55926c6a8070 0x55926c6aa460 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7371c26a958e166 server_cookie=ae32fec5731f7577 in_seq=0 out_seq=0 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2897520819 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55926c600850 con 0x55926c6a8070 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f50e2ffd640 10 client.17079 handle_command_reply: tid=0 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 client.17079 shutdown 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 -- 172.21.15.94:0/2897520819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 msgr2=0x55926c6980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6980b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0058f0 tx=0x7f50fc005920 comp rx=0 tx=0).stop 2024-09-19T04:28:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 -- 172.21.15.94:0/2897520819 shutdown_connections 2024-09-19T04:28:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55926c6a8070 0x55926c6aa460 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c695af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c695360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2897520819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6980b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f50e2ffd640 0 client.17079 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 -- 172.21.15.94:0/2897520819 >> 172.21.15.94:0/2897520819 conn(0x55926c689a30 msgr2=0x55926c68f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 -- 172.21.15.94:0/2897520819 shutdown_connections 2024-09-19T04:28:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.236+0000 7f510bd8a740 1 -- 172.21.15.94:0/2897520819 wait complete. 2024-09-19T04:28:31.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.237+0000 7f510bd8a740 20 client.17079 trim_cache size 0 max 16384 2024-09-19T04:28:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.262+0000 7f510bd8a740 1 Processor -- start 2024-09-19T04:28:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f510bd8a740 1 -- start start 2024-09-19T04:28:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c6b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c6925c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f510bd8a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51041bf070 con 0x55926c68d4a0 2024-09-19T04:28:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f510bd8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f51041bc880 con 0x55926c698680 2024-09-19T04:28:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f510bd8a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5104100ef0 con 0x55926c68ddc0 2024-09-19T04:28:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54828/0 (socket says 172.21.15.94:54828) 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51037fe640 1 -- 172.21.15.94:0/3737685815 learned_addr learned my addr 172.21.15.94:0/3737685815 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51027fc640 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c6b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51037fe640 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c6925c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51037fe640 1 -- 172.21.15.94:0/3737685815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 msgr2=0x55926c6b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51037fe640 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c6b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51037fe640 1 -- 172.21.15.94:0/3737685815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 msgr2=0x55926c6925c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51037fe640 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c6925c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51037fe640 1 -- 172.21.15.94:0/3737685815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926c68aae0 con 0x55926c68d4a0 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.263+0000 7f51027fc640 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c6b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.264+0000 7f51037fe640 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6afb80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009e20 tx=0x7f50fc011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804111ccdeed19b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.264+0000 7f50e97fa640 1 -- 172.21.15.94:0/3737685815 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc023040 con 0x55926c68d4a0 2024-09-19T04:28:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.264+0000 7f50e97fa640 1 -- 172.21.15.94:0/3737685815 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50fc00fe70 con 0x55926c68d4a0 2024-09-19T04:28:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.264+0000 7f50e97fa640 1 -- 172.21.15.94:0/3737685815 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc004130 con 0x55926c68d4a0 2024-09-19T04:28:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.265+0000 7f510bd8a740 1 -- 172.21.15.94:0/3737685815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 msgr2=0x55926c6afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.265+0000 7f510bd8a740 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6afb80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009e20 tx=0x7f50fc011cc0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.265+0000 7f510bd8a740 1 -- 172.21.15.94:0/3737685815 shutdown_connections 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.265+0000 7f510bd8a740 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c698680 0x55926c6925c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.265+0000 7f510bd8a740 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68ddc0 0x55926c6b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.265+0000 7f510bd8a740 1 --2- 172.21.15.94:0/3737685815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c68d4a0 0x55926c6afb80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.265+0000 7f510bd8a740 1 -- 172.21.15.94:0/3737685815 >> 172.21.15.94:0/3737685815 conn(0x55926c689a30 msgr2=0x55926c693c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.265+0000 7f510bd8a740 1 -- 172.21.15.94:0/3737685815 shutdown_connections 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.265+0000 7f510bd8a740 1 -- 172.21.15.94:0/3737685815 wait complete. 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f510bd8a740 1 -- start start 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f510bd8a740 1 Processor -- start 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c6a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c68ddc0 0x55926c69df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f510bd8a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c698680 0x55926c69e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f51037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c6a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f51037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c6a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:54568/0 (socket says 172.21.15.94:54568) 2024-09-19T04:28:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f51037fe640 1 -- 172.21.15.94:0/2663018010 learned_addr learned my addr 172.21.15.94:0/2663018010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:28:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5104072cc0 con 0x55926c698680 2024-09-19T04:28:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5104074110 con 0x55926c68ddc0 2024-09-19T04:28:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f51027fc640 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c68ddc0 0x55926c69df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.266+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5104073370 con 0x55926c68d4a0 2024-09-19T04:28:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f51037fe640 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c698680 0x55926c69e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f51037fe640 1 -- 172.21.15.94:0/2663018010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c68ddc0 msgr2=0x55926c69df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f51037fe640 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c68ddc0 0x55926c69df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f51037fe640 1 -- 172.21.15.94:0/2663018010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c698680 msgr2=0x55926c69e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f51037fe640 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c698680 0x55926c69e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f51037fe640 1 -- 172.21.15.94:0/2663018010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926c69ea40 con 0x55926c68d4a0 2024-09-19T04:28:31.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f510bd8a740 10 client.0 _ll_register_callbacks cb 0x7f50f8009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:31.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f51037fe640 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c6a0b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f50fc005d30 tx=0x7f50fc000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f624a68fe97962 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f50e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:28:31.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50fc00fe70 con 0x55926c68d4a0 2024-09-19T04:28:31.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2663018010 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc023040 con 0x55926c68d4a0 2024-09-19T04:28:31.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.267+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2663018010 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f50fc003b80 con 0x55926c68d4a0 2024-09-19T04:28:31.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.268+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2663018010 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f50fc003d80 con 0x55926c68d4a0 2024-09-19T04:28:31.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.268+0000 7f510bd8a740 10 client.26674 fetch_fsmap learned FSMap version 48 2024-09-19T04:28:31.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.268+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=48}) -- 0x55926c7126e0 con 0x55926c68d4a0 2024-09-19T04:28:31.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.268+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2663018010 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f50fc04b020 con 0x55926c68d4a0 2024-09-19T04:28:31.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.268+0000 7f510bd8a740 10 client.26674 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:28:31.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.268+0000 7f510bd8a740 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:31.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.268+0000 7f510bd8a740 20 client.26674 populate_metadata read hostname 'smithi094' 2024-09-19T04:28:31.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.268+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55926c712d10 0x55926c715190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.268+0000 7f510bd8a740 4 client.26674 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:28:31.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.269+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55926c5fad60 con 0x55926c712d10 2024-09-19T04:28:31.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.269+0000 7f51027fc640 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55926c712d10 0x55926c715190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.269+0000 7f51027fc640 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55926c712d10 0x55926c715190 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962a7d7aa6805a7c server_cookie=5167ad6f95b98ad4 in_seq=0 out_seq=0 2024-09-19T04:28:31.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.269+0000 7f50e2ffd640 10 client.26674 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:31.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.269+0000 7f50e2ffd640 1 -- 172.21.15.94:0/2663018010 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55926c5fad60 con 0x55926c712d10 2024-09-19T04:28:31.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.269+0000 7f50e2ffd640 10 client.26674 handle_command_reply: tid=0 2024-09-19T04:28:31.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.269+0000 7f510bd8a740 1 client.26674 shutdown 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 msgr2=0x55926c6a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c6a0b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f50fc005d30 tx=0x7f50fc000f40 comp rx=0 tx=0).stop 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 shutdown_connections 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55926c712d10 0x55926c715190 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55926c698680 0x55926c69e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f50e2ffd640 0 client.26674 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55926c68ddc0 0x55926c69df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 --2- 172.21.15.94:0/2663018010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55926c68d4a0 0x55926c6a0b50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 >> 172.21.15.94:0/2663018010 conn(0x55926c689a30 msgr2=0x55926c694020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 shutdown_connections 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 1 -- 172.21.15.94:0/2663018010 wait complete. 2024-09-19T04:28:31.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.270+0000 7f510bd8a740 20 client.26674 trim_cache size 0 max 16384 2024-09-19T04:28:31.281 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:28:31.281 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:28:31.281 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:28:31.281 INFO:teuthology.orchestra.run.smithi094.stdout: "33b6eec6-12d6-499e-8995-77f3512fcf01": { 2024-09-19T04:28:31.281 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:28:31.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "33b6eec6-12d6-499e-8995-77f3512fcf01", 2024-09-19T04:28:31.282 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:28:31.282 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:28:31.282 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:28:31.282 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:28:31.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 -- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f50ec0a3430 msgr2=0x7f50ec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f50ec0a3430 0x7f50ec0a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50f4045fd0 tx=0x7f50f402a420 comp rx=0 tx=0).stop 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 -- 172.21.15.94:0/3701307788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5104100a90 msgr2=0x7f51041a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5104100a90 0x7f51041a3560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009230 tx=0x7f50f40040c0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 -- 172.21.15.94:0/3701307788 shutdown_connections 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f50ec0a3430 0x7f50ec0a58f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5104106a90 0x7f51041a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104101430 0x7f51041a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 --2- 172.21.15.94:0/3701307788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5104100a90 0x7f51041a3560 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 -- 172.21.15.94:0/3701307788 >> 172.21.15.94:0/3701307788 conn(0x7f51040fc7d0 msgr2=0x7f510406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 -- 172.21.15.94:0/3701307788 shutdown_connections 2024-09-19T04:28:31.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:28:31.271+0000 7f51099b1640 1 -- 172.21.15.94:0/3701307788 wait complete. 2024-09-19T04:28:31.999 INFO:tasks.cephfs.filesystem:scrub status for tag:33b6eec6-12d6-499e-8995-77f3512fcf01 - {'path': '/', 'tag': '33b6eec6-12d6-499e-8995-77f3512fcf01', 'options': 'recursive,force'} 2024-09-19T04:28:33.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:32 smithi112 ceph-mon[35051]: pgmap v422: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.8 MiB/s wr, 505 op/s 2024-09-19T04:28:33.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:32 smithi094 ceph-mon[30972]: pgmap v422: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.8 MiB/s wr, 505 op/s 2024-09-19T04:28:33.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:32 smithi104 ceph-mon[34145]: pgmap v422: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.8 MiB/s wr, 505 op/s 2024-09-19T04:28:35.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:34 smithi112 ceph-mon[35051]: pgmap v423: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.5 MiB/s wr, 824 op/s 2024-09-19T04:28:35.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:34 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:28:35.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:34 smithi094 ceph-mon[30972]: pgmap v423: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.5 MiB/s wr, 824 op/s 2024-09-19T04:28:35.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:34 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:28:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:34 smithi104 ceph-mon[34145]: pgmap v423: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.5 MiB/s wr, 824 op/s 2024-09-19T04:28:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:34 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:28:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 740.0 s, 373.4 tps, lat 268.559 ms stddev 53.422 2024-09-19T04:28:37.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:36 smithi112 ceph-mon[35051]: pgmap v424: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.8 MiB/s wr, 588 op/s 2024-09-19T04:28:37.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:36 smithi094 ceph-mon[30972]: pgmap v424: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.8 MiB/s wr, 588 op/s 2024-09-19T04:28:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:36 smithi104 ceph-mon[34145]: pgmap v424: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.8 MiB/s wr, 588 op/s 2024-09-19T04:28:38.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:37 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:28:38.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:37 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:28:38.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:37 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:28:38.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:37 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:28:38.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:37 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:28:38.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:37 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:28:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:37 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:28:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:37 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:28:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:37 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:28:39.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:38 smithi112 ceph-mon[35051]: pgmap v425: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 5.3 MiB/s wr, 675 op/s 2024-09-19T04:28:39.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:38 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:28:39.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:38 smithi094 ceph-mon[30972]: pgmap v425: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 5.3 MiB/s wr, 675 op/s 2024-09-19T04:28:39.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:38 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:28:39.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:38 smithi104 ceph-mon[34145]: pgmap v425: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 5.3 MiB/s wr, 675 op/s 2024-09-19T04:28:39.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:38 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:28:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 745.0 s, 421.2 tps, lat 238.119 ms stddev 24.376 2024-09-19T04:28:41.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:40 smithi112 ceph-mon[35051]: pgmap v426: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 798 op/s 2024-09-19T04:28:41.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:40 smithi094 ceph-mon[30972]: pgmap v426: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 798 op/s 2024-09-19T04:28:41.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:40 smithi104 ceph-mon[34145]: pgmap v426: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 798 op/s 2024-09-19T04:28:43.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:42 smithi112 ceph-mon[35051]: pgmap v427: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 5.5 MiB/s wr, 609 op/s 2024-09-19T04:28:43.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:42 smithi094 ceph-mon[30972]: pgmap v427: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 5.5 MiB/s wr, 609 op/s 2024-09-19T04:28:43.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:42 smithi104 ceph-mon[34145]: pgmap v427: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 5.5 MiB/s wr, 609 op/s 2024-09-19T04:28:45.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:44 smithi112 ceph-mon[35051]: pgmap v428: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.5 MiB/s wr, 874 op/s 2024-09-19T04:28:45.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:44 smithi094 ceph-mon[30972]: pgmap v428: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.5 MiB/s wr, 874 op/s 2024-09-19T04:28:45.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:44 smithi104 ceph-mon[34145]: pgmap v428: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.5 MiB/s wr, 874 op/s 2024-09-19T04:28:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 750.0 s, 438.4 tps, lat 227.676 ms stddev 24.640 2024-09-19T04:28:47.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:46 smithi112 ceph-mon[35051]: pgmap v429: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 4.8 MiB/s wr, 555 op/s 2024-09-19T04:28:47.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:46 smithi094 ceph-mon[30972]: pgmap v429: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 4.8 MiB/s wr, 555 op/s 2024-09-19T04:28:47.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:46 smithi104 ceph-mon[34145]: pgmap v429: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 4.8 MiB/s wr, 555 op/s 2024-09-19T04:28:49.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:48 smithi112 ceph-mon[35051]: pgmap v430: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 6.2 MiB/s wr, 736 op/s 2024-09-19T04:28:49.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:48 smithi094 ceph-mon[30972]: pgmap v430: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 6.2 MiB/s wr, 736 op/s 2024-09-19T04:28:49.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:48 smithi104 ceph-mon[34145]: pgmap v430: 97 pgs: 97 active+clean; 3.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 6.2 MiB/s wr, 736 op/s 2024-09-19T04:28:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 755.0 s, 459.0 tps, lat 217.221 ms stddev 27.166 2024-09-19T04:28:51.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:50 smithi112 ceph-mon[35051]: pgmap v431: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 6.6 MiB/s wr, 781 op/s 2024-09-19T04:28:51.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:50 smithi094 ceph-mon[30972]: pgmap v431: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 6.6 MiB/s wr, 781 op/s 2024-09-19T04:28:51.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:50 smithi104 ceph-mon[34145]: pgmap v431: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 6.6 MiB/s wr, 781 op/s 2024-09-19T04:28:53.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:52 smithi112 ceph-mon[35051]: pgmap v432: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 4.3 MiB/s wr, 578 op/s 2024-09-19T04:28:53.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:52 smithi094 ceph-mon[30972]: pgmap v432: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 4.3 MiB/s wr, 578 op/s 2024-09-19T04:28:53.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:52 smithi104 ceph-mon[34145]: pgmap v432: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 4.3 MiB/s wr, 578 op/s 2024-09-19T04:28:54.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:53 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:28:54.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:53 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:28:54.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:53 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:28:55.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:54 smithi112 ceph-mon[35051]: pgmap v433: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.9 MiB/s wr, 798 op/s 2024-09-19T04:28:55.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:54 smithi094 ceph-mon[30972]: pgmap v433: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.9 MiB/s wr, 798 op/s 2024-09-19T04:28:55.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:54 smithi104 ceph-mon[34145]: pgmap v433: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.9 MiB/s wr, 798 op/s 2024-09-19T04:28:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 760.0 s, 401.8 tps, lat 249.361 ms stddev 31.819 2024-09-19T04:28:57.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:56 smithi112 ceph-mon[35051]: pgmap v434: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 4.0 MiB/s wr, 533 op/s 2024-09-19T04:28:57.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:56 smithi094 ceph-mon[30972]: pgmap v434: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 4.0 MiB/s wr, 533 op/s 2024-09-19T04:28:57.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:56 smithi104 ceph-mon[34145]: pgmap v434: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 4.0 MiB/s wr, 533 op/s 2024-09-19T04:28:59.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:28:58 smithi112 ceph-mon[35051]: pgmap v435: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 5.3 MiB/s wr, 701 op/s 2024-09-19T04:28:59.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:28:58 smithi094 ceph-mon[30972]: pgmap v435: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 5.3 MiB/s wr, 701 op/s 2024-09-19T04:28:59.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:28:58 smithi104 ceph-mon[34145]: pgmap v435: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 5.3 MiB/s wr, 701 op/s 2024-09-19T04:29:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 765.0 s, 417.8 tps, lat 235.674 ms stddev 22.770 2024-09-19T04:29:01.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:00 smithi112 ceph-mon[35051]: pgmap v436: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 4.6 MiB/s wr, 643 op/s 2024-09-19T04:29:01.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:00 smithi094 ceph-mon[30972]: pgmap v436: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 4.6 MiB/s wr, 643 op/s 2024-09-19T04:29:01.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:00 smithi104 ceph-mon[34145]: pgmap v436: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 4.6 MiB/s wr, 643 op/s 2024-09-19T04:29:01.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:29:02.361 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:03.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:02 smithi112 ceph-mon[35051]: pgmap v437: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.8 MiB/s wr, 510 op/s 2024-09-19T04:29:03.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:02 smithi094 ceph-mon[30972]: pgmap v437: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.8 MiB/s wr, 510 op/s 2024-09-19T04:29:03.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:02 smithi104 ceph-mon[34145]: pgmap v437: 97 pgs: 97 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.8 MiB/s wr, 510 op/s 2024-09-19T04:29:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.126+0000 7f2e049ea640 1 -- 172.21.15.94:0/537103028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e00102570 msgr2=0x7f2e001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.127+0000 7f2e049ea640 1 --2- 172.21.15.94:0/537103028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e00102570 0x7f2e001029b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009900 tx=0x7f2df002a480 comp rx=0 tx=0).stop 2024-09-19T04:29:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.128+0000 7f2e049ea640 1 -- 172.21.15.94:0/537103028 shutdown_connections 2024-09-19T04:29:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.128+0000 7f2e049ea640 1 --2- 172.21.15.94:0/537103028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e00102ef0 0x7f2e0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.128+0000 7f2e049ea640 1 --2- 172.21.15.94:0/537103028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e00102570 0x7f2e001029b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.128+0000 7f2e049ea640 1 --2- 172.21.15.94:0/537103028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e00100e20 0x7f2e00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.128+0000 7f2e049ea640 1 -- 172.21.15.94:0/537103028 >> 172.21.15.94:0/537103028 conn(0x7f2e000fc7d0 msgr2=0x7f2e000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.128+0000 7f2e049ea640 1 -- 172.21.15.94:0/537103028 shutdown_connections 2024-09-19T04:29:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.128+0000 7f2e049ea640 1 -- 172.21.15.94:0/537103028 wait complete. 2024-09-19T04:29:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.129+0000 7f2e049ea640 1 Processor -- start 2024-09-19T04:29:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.130+0000 7f2e049ea640 1 -- start start 2024-09-19T04:29:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.130+0000 7f2e049ea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e00100e20 0x7f2e001972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.131+0000 7f2e049ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e00102570 0x7f2e00197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.131+0000 7f2e049ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e00102ef0 0x7f2e0019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.131+0000 7f2e049ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00110e80 con 0x7f2e00102570 2024-09-19T04:29:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.131+0000 7f2e049ea640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00110d00 con 0x7f2e00100e20 2024-09-19T04:29:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.131+0000 7f2e049ea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00111000 con 0x7f2e00102ef0 2024-09-19T04:29:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.131+0000 7f2dfed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e00102ef0 0x7f2e0019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.131+0000 7f2dfed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e00102ef0 0x7f2e0019bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37062/0 (socket says 172.21.15.94:37062) 2024-09-19T04:29:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.131+0000 7f2dfed9d640 1 -- 172.21.15.94:0/1505383389 learned_addr learned my addr 172.21.15.94:0/1505383389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.131+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e00100e20 0x7f2e001972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.132+0000 7f2dfed9d640 1 -- 172.21.15.94:0/1505383389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e00100e20 msgr2=0x7f2e001972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.133+0000 7f2dfed9d640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e00100e20 0x7f2e001972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.133+0000 7f2dfed9d640 1 -- 172.21.15.94:0/1505383389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e00102570 msgr2=0x7f2e00197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.133+0000 7f2dfed9d640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e00102570 0x7f2e00197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.133+0000 7f2dfed9d640 1 -- 172.21.15.94:0/1505383389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0019c290 con 0x7f2e00102ef0 2024-09-19T04:29:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.134+0000 7f2dfed9d640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e00102ef0 0x7f2e0019bbb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2df800eca0 tx=0x7f2df800b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.134+0000 7f2def7fe640 1 -- 172.21.15.94:0/1505383389 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8004b20 con 0x7f2e00102ef0 2024-09-19T04:29:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.134+0000 7f2def7fe640 1 -- 172.21.15.94:0/1505383389 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df8004cc0 con 0x7f2e00102ef0 2024-09-19T04:29:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.135+0000 7f2e049ea640 1 -- 172.21.15.94:0/1505383389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0019c580 con 0x7f2e00102ef0 2024-09-19T04:29:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.135+0000 7f2e049ea640 1 -- 172.21.15.94:0/1505383389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e00069f70 con 0x7f2e00102ef0 2024-09-19T04:29:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.135+0000 7f2def7fe640 1 -- 172.21.15.94:0/1505383389 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8017670 con 0x7f2e00102ef0 2024-09-19T04:29:03.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.138+0000 7f2def7fe640 1 -- 172.21.15.94:0/1505383389 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2df8017810 con 0x7f2e00102ef0 2024-09-19T04:29:03.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.139+0000 7f2def7fe640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2de80a35c0 0x7f2de80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.140+0000 7f2def7fe640 1 -- 172.21.15.94:0/1505383389 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f2df80d2750 con 0x7f2e00102ef0 2024-09-19T04:29:03.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.140+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2de80a35c0 0x7f2de80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.146+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2de80a35c0 0x7f2de80a5a80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009bd0 tx=0x7f2df4005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.147+0000 7f2e06dc3740 1 Processor -- start 2024-09-19T04:29:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.147+0000 7f2e06dc3740 1 -- start start 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.147+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23221f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.147+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f23222a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.147+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f23222ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23221f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23221f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37076/0 (socket says 172.21.15.94:37076) 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2e06dc3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f2321852e0 con 0x55f23222a680 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2e06dc3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f23212cce0 con 0x55f23221fdc0 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2e06dc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f2321812a0 con 0x55f23221f4a0 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfdd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f23222a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfe59c640 1 -- 172.21.15.94:0/3262180587 learned_addr learned my addr 172.21.15.94:0/3262180587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfed9d640 1 --2- 172.21.15.94:0/3262180587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f23222ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfe59c640 1 -- 172.21.15.94:0/3262180587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 msgr2=0x55f23222a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/3262180587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f23222a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfe59c640 1 -- 172.21.15.94:0/3262180587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 msgr2=0x55f23222ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/3262180587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f23222ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.148+0000 7f2dfe59c640 1 -- 172.21.15.94:0/3262180587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f232111440 con 0x55f23221f4a0 2024-09-19T04:29:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.149+0000 7f2dfed9d640 1 --2- 172.21.15.94:0/3262180587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f23222ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.149+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/3262180587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23221f880 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2df400e630 tx=0x7f2df4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92352d47798d0eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.149+0000 7f2decff9640 1 -- 172.21.15.94:0/3262180587 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4004020 con 0x55f23221f4a0 2024-09-19T04:29:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.149+0000 7f2decff9640 1 -- 172.21.15.94:0/3262180587 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df40041c0 con 0x55f23221f4a0 2024-09-19T04:29:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.149+0000 7f2decff9640 1 -- 172.21.15.94:0/3262180587 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df401c690 con 0x55f23221f4a0 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.150+0000 7f2e06dc3740 1 -- 172.21.15.94:0/3262180587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 msgr2=0x55f23221f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.150+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/3262180587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23221f880 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2df400e630 tx=0x7f2df4014000 comp rx=0 tx=0).stop 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.151+0000 7f2e06dc3740 1 -- 172.21.15.94:0/3262180587 shutdown_connections 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.151+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/3262180587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f23222ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.151+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/3262180587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f23222a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.151+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/3262180587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23221f880 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.151+0000 7f2e06dc3740 1 -- 172.21.15.94:0/3262180587 >> 172.21.15.94:0/3262180587 conn(0x55f23221ba30 msgr2=0x55f23221be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.151+0000 7f2e06dc3740 1 -- 172.21.15.94:0/3262180587 shutdown_connections 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.151+0000 7f2e06dc3740 1 -- 172.21.15.94:0/3262180587 wait complete. 2024-09-19T04:29:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.151+0000 7f2e06dc3740 1 -- start start 2024-09-19T04:29:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.152+0000 7f2e06dc3740 1 Processor -- start 2024-09-19T04:29:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.152+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23222a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.152+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232227320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.152+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f232227910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.152+0000 7f2dfe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23222a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.152+0000 7f2dfe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23222a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37080/0 (socket says 172.21.15.94:37080) 2024-09-19T04:29:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.152+0000 7f2dfdd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232227320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.152+0000 7f2dfdd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232227320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:47080/0 (socket says 172.21.15.94:47080) 2024-09-19T04:29:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2dfe59c640 1 -- 172.21.15.94:0/497575517 learned_addr learned my addr 172.21.15.94:0/497575517 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f23222d250 con 0x55f23222a680 2024-09-19T04:29:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f232140050 con 0x55f23221fdc0 2024-09-19T04:29:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f23222d3d0 con 0x55f23221f4a0 2024-09-19T04:29:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2dfed9d640 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f232227910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/497575517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 msgr2=0x55f23222a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:03.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2dfdd9b640 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23222a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/497575517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 msgr2=0x55f232227910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:03.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2dfdd9b640 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f232227910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.153+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/497575517 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f231ef63c0 con 0x55f23221fdc0 2024-09-19T04:29:03.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.154+0000 7f2dfdd9b640 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232227320 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009230 tx=0x7f2df0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=961736d343c67923 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.154+0000 7f2e06dc3740 10 client.0 _ll_register_callbacks cb 0x55f232180160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.154+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df40041c0 con 0x55f23221fdc0 2024-09-19T04:29:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.154+0000 7f2de5ffb640 10 client.26708 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:29:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.154+0000 7f2de5ffb640 1 -- 172.21.15.94:0/497575517 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0035040 con 0x55f23221fdc0 2024-09-19T04:29:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.154+0000 7f2de5ffb640 1 -- 172.21.15.94:0/497575517 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f2df0005450 con 0x55f23221fdc0 2024-09-19T04:29:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.155+0000 7f2de5ffb640 1 -- 172.21.15.94:0/497575517 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2df00044a0 con 0x55f23221fdc0 2024-09-19T04:29:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.155+0000 7f2e06dc3740 10 client.26708 fetch_fsmap learned FSMap version 51 2024-09-19T04:29:03.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.155+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=51}) -- 0x55f232241220 con 0x55f23221fdc0 2024-09-19T04:29:03.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.157+0000 7f2de5ffb640 1 -- 172.21.15.94:0/497575517 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 51) ==== 8358+0+0 (secure 0 0 0) 0x7f2df00046c0 con 0x55f23221fdc0 2024-09-19T04:29:03.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.157+0000 7f2e06dc3740 10 client.26708 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T04:29:03.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.157+0000 7f2e06dc3740 10 client.26708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:03.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.157+0000 7f2e06dc3740 20 client.26708 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:03.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.157+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f232242b00 0x55f232244fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.158+0000 7f2e06dc3740 4 client.26708 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:03.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.158+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f232192850 con 0x55f232242b00 2024-09-19T04:29:03.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.158+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f232242b00 0x55f232244fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.159+0000 7f2de5ffb640 10 client.26708 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:03.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.159+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f232242b00 0x55f232244fc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e2b6de772075c2 server_cookie=c1bb31c44a9705dd in_seq=0 out_seq=0 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2de5ffb640 1 -- 172.21.15.94:0/497575517 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f232192850 con 0x55f232242b00 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2de5ffb640 10 client.26708 handle_command_reply: tid=0 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2e06dc3740 1 client.26708 shutdown 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 msgr2=0x55f232227320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232227320 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009230 tx=0x7f2df0004270 comp rx=0 tx=0).stop 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 shutdown_connections 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f232242b00 0x55f232244fc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f232227910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232227320 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/497575517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23222a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 >> 172.21.15.94:0/497575517 conn(0x55f23221ba30 msgr2=0x55f232221710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.160+0000 7f2de5ffb640 0 client.26708 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.161+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 shutdown_connections 2024-09-19T04:29:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.161+0000 7f2e06dc3740 1 -- 172.21.15.94:0/497575517 wait complete. 2024-09-19T04:29:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.161+0000 7f2e06dc3740 20 client.26708 trim_cache size 0 max 16384 2024-09-19T04:29:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.192+0000 7f2e06dc3740 1 Processor -- start 2024-09-19T04:29:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.192+0000 7f2e06dc3740 1 -- start start 2024-09-19T04:29:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23223ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232221600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f23224a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2dfe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23223ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2dfdd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232221600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2dfdd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232221600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:47088/0 (socket says 172.21.15.94:47088) 2024-09-19T04:29:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/704234169 learned_addr learned my addr 172.21.15.94:0/704234169 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2e06dc3740 1 -- 172.21.15.94:0/704234169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001ae0e0 con 0x55f23222a680 2024-09-19T04:29:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2e06dc3740 1 -- 172.21.15.94:0/704234169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001ab920 con 0x55f23221fdc0 2024-09-19T04:29:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2e06dc3740 1 -- 172.21.15.94:0/704234169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001a9790 con 0x55f23221f4a0 2024-09-19T04:29:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.193+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/704234169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 msgr2=0x55f23223ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.194+0000 7f2dfdd9b640 1 --2- 172.21.15.94:0/704234169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23223ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.194+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/704234169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 msgr2=0x55f23224a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.194+0000 7f2dfdd9b640 1 --2- 172.21.15.94:0/704234169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f23224a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.194+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/704234169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f23221b860 con 0x55f23221fdc0 2024-09-19T04:29:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.194+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/704234169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23223ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.194+0000 7f2dfdd9b640 1 --2- 172.21.15.94:0/704234169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232221600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009900 tx=0x7f2df003a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2d2263c9a89d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.194+0000 7f2de7fff640 1 -- 172.21.15.94:0/704234169 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0035070 con 0x55f23221fdc0 2024-09-19T04:29:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.194+0000 7f2de7fff640 1 -- 172.21.15.94:0/704234169 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df40041c0 con 0x55f23221fdc0 2024-09-19T04:29:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.194+0000 7f2de7fff640 1 -- 172.21.15.94:0/704234169 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0003aa0 con 0x55f23221fdc0 2024-09-19T04:29:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.195+0000 7f2e06dc3740 1 -- 172.21.15.94:0/704234169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 msgr2=0x55f232221600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.195+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/704234169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232221600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009900 tx=0x7f2df003a000 comp rx=0 tx=0).stop 2024-09-19T04:29:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.196+0000 7f2e06dc3740 1 -- 172.21.15.94:0/704234169 shutdown_connections 2024-09-19T04:29:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.196+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/704234169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f23224a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.196+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/704234169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f232221600 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.196+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/704234169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f23223ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.196+0000 7f2e06dc3740 1 -- 172.21.15.94:0/704234169 >> 172.21.15.94:0/704234169 conn(0x55f23221ba30 msgr2=0x55f232220ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.196+0000 7f2e06dc3740 1 -- 172.21.15.94:0/704234169 shutdown_connections 2024-09-19T04:29:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.196+0000 7f2e06dc3740 1 -- 172.21.15.94:0/704234169 wait complete. 2024-09-19T04:29:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.197+0000 7f2e06dc3740 1 -- start start 2024-09-19T04:29:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.198+0000 7f2e06dc3740 1 Processor -- start 2024-09-19T04:29:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.198+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f232235e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.198+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f2322363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.198+0000 7f2e06dc3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f2322368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.198+0000 7f2e06dc3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001a9de0 con 0x55f23222a680 2024-09-19T04:29:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.198+0000 7f2e06dc3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001ab250 con 0x55f23221fdc0 2024-09-19T04:29:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.198+0000 7f2e06dc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001aa4b0 con 0x55f23221f4a0 2024-09-19T04:29:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.198+0000 7f2e06dc3740 10 client.0 _ll_register_callbacks cb 0x7f2df4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.199+0000 7f2dfdd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f2322363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.199+0000 7f2dfdd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f2322363b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:47092/0 (socket says 172.21.15.94:47092) 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.199+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/1935270432 learned_addr learned my addr 172.21.15.94:0/1935270432 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.199+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f232235e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.199+0000 7f2dfed9d640 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f2322368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.199+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/1935270432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 msgr2=0x55f232235e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.199+0000 7f2dfdd9b640 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f232235e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.199+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/1935270432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 msgr2=0x55f2322368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.199+0000 7f2dfdd9b640 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f2322368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.200+0000 7f2dfdd9b640 1 -- 172.21.15.94:0/1935270432 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2322ab9c0 con 0x55f23221fdc0 2024-09-19T04:29:03.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.200+0000 7f2dfdd9b640 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f2322363b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2df0004cc0 tx=0x7f2df0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04cb5ea974646e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:03.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.200+0000 7f2de5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:29:03.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.201+0000 7f2de5ffb640 1 -- 172.21.15.94:0/1935270432 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0035040 con 0x55f23221fdc0 2024-09-19T04:29:03.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.201+0000 7f2de5ffb640 1 -- 172.21.15.94:0/1935270432 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f2df00057f0 con 0x55f23221fdc0 2024-09-19T04:29:03.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.201+0000 7f2e06dc3740 1 -- 172.21.15.94:0/1935270432 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df40041c0 con 0x55f23221fdc0 2024-09-19T04:29:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.201+0000 7f2de5ffb640 1 -- 172.21.15.94:0/1935270432 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2df0065050 con 0x55f23221fdc0 2024-09-19T04:29:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.201+0000 7f2e06dc3740 10 client.26714 fetch_fsmap learned FSMap version 51 2024-09-19T04:29:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.201+0000 7f2e06dc3740 1 -- 172.21.15.94:0/1935270432 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f23222d660 con 0x55f23221fdc0 2024-09-19T04:29:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.202+0000 7f2de5ffb640 1 -- 172.21.15.94:0/1935270432 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df40041c0 con 0x55f23221fdc0 2024-09-19T04:29:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.203+0000 7f2de5ffb640 1 -- 172.21.15.94:0/1935270432 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 51) ==== 8358+0+0 (secure 0 0 0) 0x7f2df003c440 con 0x55f23221fdc0 2024-09-19T04:29:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.203+0000 7f2e06dc3740 10 client.26714 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T04:29:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.203+0000 7f2e06dc3740 10 client.26714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.203+0000 7f2e06dc3740 20 client.26714 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.203+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f23222dd80 0x55f2322301a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.203+0000 7f2e06dc3740 4 client.26714 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:29:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.203+0000 7f2e06dc3740 1 -- 172.21.15.94:0/1935270432 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f23218cd60 con 0x55f23222dd80 2024-09-19T04:29:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.203+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f23222dd80 0x55f2322301a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:03.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.204+0000 7f2de5ffb640 10 client.26714 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:03.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.204+0000 7f2dfe59c640 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f23222dd80 0x55f2322301a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948af1ab89d701fc server_cookie=4631a3f3b1404d7c in_seq=0 out_seq=0 2024-09-19T04:29:03.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.204+0000 7f2de5ffb640 1 -- 172.21.15.94:0/1935270432 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f23218cd60 con 0x55f23222dd80 2024-09-19T04:29:03.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.204+0000 7f2de5ffb640 10 client.26714 handle_command_reply: tid=0 2024-09-19T04:29:03.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.204+0000 7f2e06dc3740 1 client.26714 shutdown 2024-09-19T04:29:03.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.205+0000 7f2e06dc3740 1 -- 172.21.15.94:0/1935270432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 msgr2=0x55f2322363b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:03.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.205+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f2322363b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2df0004cc0 tx=0x7f2df0003b90 comp rx=0 tx=0).stop 2024-09-19T04:29:03.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2e06dc3740 1 -- 172.21.15.94:0/1935270432 shutdown_connections 2024-09-19T04:29:03.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f23222dd80 0x55f2322301a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f23222a680 0x55f2322368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2de5ffb640 0 client.26714 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:03.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f23221fdc0 0x55f2322363b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2e06dc3740 1 --2- 172.21.15.94:0/1935270432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f23221f4a0 0x55f232235e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2e06dc3740 1 -- 172.21.15.94:0/1935270432 >> 172.21.15.94:0/1935270432 conn(0x55f23221ba30 msgr2=0x55f23221ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:03.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2e06dc3740 1 -- 172.21.15.94:0/1935270432 shutdown_connections 2024-09-19T04:29:03.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2e06dc3740 1 -- 172.21.15.94:0/1935270432 wait complete. 2024-09-19T04:29:03.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.206+0000 7f2e06dc3740 20 client.26714 trim_cache size 0 max 16384 2024-09-19T04:29:03.209 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:29:03.209 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:29:03.209 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:29:03.209 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 -- 172.21.15.94:0/1505383389 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2de80a35c0 msgr2=0x7f2de80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2de80a35c0 0x7f2de80a5a80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009bd0 tx=0x7f2df4005f90 comp rx=0 tx=0).stop 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 -- 172.21.15.94:0/1505383389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e00102ef0 msgr2=0x7f2e0019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e00102ef0 0x7f2e0019bbb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2df800eca0 tx=0x7f2df800b690 comp rx=0 tx=0).stop 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 -- 172.21.15.94:0/1505383389 shutdown_connections 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2de80a35c0 0x7f2de80a5a80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e00102ef0 0x7f2e0019bbb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e00102570 0x7f2e00197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 --2- 172.21.15.94:0/1505383389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e00100e20 0x7f2e001972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.207+0000 7f2e049ea640 1 -- 172.21.15.94:0/1505383389 >> 172.21.15.94:0/1505383389 conn(0x7f2e000fc7d0 msgr2=0x7f2e000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.208+0000 7f2e049ea640 1 -- 172.21.15.94:0/1505383389 shutdown_connections 2024-09-19T04:29:03.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:03.208+0000 7f2e049ea640 1 -- 172.21.15.94:0/1505383389 wait complete. 2024-09-19T04:29:03.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:03 smithi094 ceph-mon[30972]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:29:04.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:03 smithi112 ceph-mon[35051]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:29:04.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:03 smithi104 ceph-mon[34145]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:29:04.294 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:29:04.294 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:29:04.636 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:05.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:04 smithi112 ceph-mon[35051]: pgmap v439: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 5.4 MiB/s wr, 732 op/s 2024-09-19T04:29:05.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:04 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:29:05.050 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:04 smithi094 ceph-mon[30972]: pgmap v439: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 5.4 MiB/s wr, 732 op/s 2024-09-19T04:29:05.050 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:04 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:29:05.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:04 smithi104 ceph-mon[34145]: pgmap v439: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 5.4 MiB/s wr, 732 op/s 2024-09-19T04:29:05.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:04 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:29:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.367+0000 7fb3a3e04640 1 -- 172.21.15.94:0/3663231904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c1012f0 msgr2=0x7fb39c10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.367+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/3663231904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c1012f0 0x7fb39c10dfb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009900 tx=0x7fb38c02a410 comp rx=0 tx=0).stop 2024-09-19T04:29:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.367+0000 7fb3a3e04640 1 -- 172.21.15.94:0/3663231904 shutdown_connections 2024-09-19T04:29:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.367+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/3663231904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c10e4f0 0x7fb39c1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.367+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/3663231904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c1012f0 0x7fb39c10dfb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.367+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/3663231904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb39c100940 0x7fb39c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.367+0000 7fb3a3e04640 1 -- 172.21.15.94:0/3663231904 >> 172.21.15.94:0/3663231904 conn(0x7fb39c0fc7d0 msgr2=0x7fb39c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.367+0000 7fb3a3e04640 1 -- 172.21.15.94:0/3663231904 shutdown_connections 2024-09-19T04:29:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.367+0000 7fb3a3e04640 1 -- 172.21.15.94:0/3663231904 wait complete. 2024-09-19T04:29:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.368+0000 7fb3a3e04640 1 Processor -- start 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.368+0000 7fb3a3e04640 1 -- start start 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a3e04640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb39c100940 0x7fb39c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a3e04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c1012f0 0x7fb39c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a3e04640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c10e4f0 0x7fb39c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a1ba0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb39c100940 0x7fb39c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a3e04640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c114090 con 0x7fb39c1012f0 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a3e04640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c113f10 con 0x7fb39c100940 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a3e04640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c114210 con 0x7fb39c10e4f0 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a1ba0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb39c100940 0x7fb39c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:47114/0 (socket says 172.21.15.94:47114) 2024-09-19T04:29:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a139f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c1012f0 0x7fb39c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a1ba0640 1 -- 172.21.15.94:0/1686604637 learned_addr learned my addr 172.21.15.94:0/1686604637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a23a1640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c10e4f0 0x7fb39c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a1ba0640 1 -- 172.21.15.94:0/1686604637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c10e4f0 msgr2=0x7fb39c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a1ba0640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c10e4f0 0x7fb39c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a1ba0640 1 -- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c1012f0 msgr2=0x7fb39c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a1ba0640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c1012f0 0x7fb39c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a1ba0640 1 -- 172.21.15.94:0/1686604637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c19f5e0 con 0x7fb39c100940 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a139f640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c1012f0 0x7fb39c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.369+0000 7fb3a1ba0640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb39c100940 0x7fb39c19a630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb39800b520 tx=0x7fb39800b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.370+0000 7fb392ffd640 1 -- 172.21.15.94:0/1686604637 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398004020 con 0x7fb39c100940 2024-09-19T04:29:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.370+0000 7fb392ffd640 1 -- 172.21.15.94:0/1686604637 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3980041c0 con 0x7fb39c100940 2024-09-19T04:29:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.370+0000 7fb392ffd640 1 -- 172.21.15.94:0/1686604637 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39800a8b0 con 0x7fb39c100940 2024-09-19T04:29:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.370+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c19f8d0 con 0x7fb39c100940 2024-09-19T04:29:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.370+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c102b50 con 0x7fb39c100940 2024-09-19T04:29:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.371+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c06aff0 con 0x7fb39c100940 2024-09-19T04:29:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.373+0000 7fb392ffd640 1 -- 172.21.15.94:0/1686604637 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fb39800aa50 con 0x7fb39c100940 2024-09-19T04:29:05.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.374+0000 7fb392ffd640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb3880a3580 0x7fb3880a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:05.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.374+0000 7fb392ffd640 1 -- 172.21.15.94:0/1686604637 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fb3980d7cb0 con 0x7fb39c100940 2024-09-19T04:29:05.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.374+0000 7fb3a139f640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb3880a3580 0x7fb3880a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:05.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.375+0000 7fb3a139f640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb3880a3580 0x7fb3880a5a40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000c00 tx=0x7fb38c005a20 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:05.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.377+0000 7fb392ffd640 1 -- 172.21.15.94:0/1686604637 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb3980a46c0 con 0x7fb39c100940 2024-09-19T04:29:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 770.0 s, 379.4 tps, lat 265.284 ms stddev 57.814 2024-09-19T04:29:05.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.701+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb39c109680 con 0x7fb39c100940 2024-09-19T04:29:05.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.702+0000 7fb392ffd640 1 -- 172.21.15.94:0/1686604637 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11918 (secure 0 0 0) 0x7fb3980a3fc0 con 0x7fb39c100940 2024-09-19T04:29:05.705 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:29:05.705 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":52,"btime":"2024-09-19T04:29:03:759777+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:29:02.792668+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:29:05.705 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 52 2024-09-19T04:29:05.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb3880a3580 msgr2=0x7fb3880a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:05.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb3880a3580 0x7fb3880a5a40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000c00 tx=0x7fb38c005a20 comp rx=0 tx=0).stop 2024-09-19T04:29:05.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb39c100940 msgr2=0x7fb39c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:05.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb39c100940 0x7fb39c19a630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb39800b520 tx=0x7fb39800b9e0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 shutdown_connections 2024-09-19T04:29:05.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb3880a3580 0x7fb3880a5a40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c10e4f0 0x7fb39c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c1012f0 0x7fb39c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 --2- 172.21.15.94:0/1686604637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb39c100940 0x7fb39c19a630 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:05.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 >> 172.21.15.94:0/1686604637 conn(0x7fb39c0fc7d0 msgr2=0x7fb39c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:05.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.708+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 shutdown_connections 2024-09-19T04:29:05.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:05.709+0000 7fb3a3e04640 1 -- 172.21.15.94:0/1686604637 wait complete. 2024-09-19T04:29:06.031 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:05 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1686604637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:29:06.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:05 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1686604637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:29:06.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:05 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1686604637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:29:06.507 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:29:06.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:06 smithi094 ceph-mon[30972]: pgmap v440: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 5.4 MiB/s wr, 732 op/s 2024-09-19T04:29:06.858 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:07.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:06 smithi104 ceph-mon[34145]: pgmap v440: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 5.4 MiB/s wr, 732 op/s 2024-09-19T04:29:07.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:06 smithi112 ceph-mon[35051]: pgmap v440: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 5.4 MiB/s wr, 732 op/s 2024-09-19T04:29:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.450+0000 7f8f2fea4640 1 -- 172.21.15.94:0/507914801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f28102570 msgr2=0x7f8f281029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.450+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/507914801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f28102570 0x7f8f281029b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009900 tx=0x7f8f1802a490 comp rx=0 tx=0).stop 2024-09-19T04:29:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.450+0000 7f8f2fea4640 1 -- 172.21.15.94:0/507914801 shutdown_connections 2024-09-19T04:29:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.450+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/507914801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f28102ef0 0x7f8f2810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.450+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/507914801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f28102570 0x7f8f281029b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.450+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/507914801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f28100e20 0x7f8f28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.450+0000 7f8f2fea4640 1 -- 172.21.15.94:0/507914801 >> 172.21.15.94:0/507914801 conn(0x7f8f280fc7d0 msgr2=0x7f8f280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.450+0000 7f8f2fea4640 1 -- 172.21.15.94:0/507914801 shutdown_connections 2024-09-19T04:29:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.450+0000 7f8f2fea4640 1 -- 172.21.15.94:0/507914801 wait complete. 2024-09-19T04:29:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.451+0000 7f8f2fea4640 1 Processor -- start 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.451+0000 7f8f2fea4640 1 -- start start 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2fea4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f28100e20 0x7f8f28196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2fea4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f28102570 0x7f8f28196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2fea4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f28102ef0 0x7f8f28196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2fea4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810fd70 con 0x7f8f28100e20 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2fea4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810fbf0 con 0x7f8f28102ef0 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2fea4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810fef0 con 0x7f8f28102570 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2dc40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f28100e20 0x7f8f28196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2dc40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f28100e20 0x7f8f28196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39530/0 (socket says 172.21.15.94:39530) 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2dc40640 1 -- 172.21.15.94:0/2317574188 learned_addr learned my addr 172.21.15.94:0/2317574188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f28102ef0 0x7f8f28196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.452+0000 7f8f2d43f640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f28102570 0x7f8f28196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.453+0000 7f8f2e441640 1 -- 172.21.15.94:0/2317574188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f28102570 msgr2=0x7f8f28196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.453+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f28102570 0x7f8f28196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.453+0000 7f8f2e441640 1 -- 172.21.15.94:0/2317574188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f28100e20 msgr2=0x7f8f28196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.453+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f28100e20 0x7f8f28196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.453+0000 7f8f2e441640 1 -- 172.21.15.94:0/2317574188 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f281aaca0 con 0x7f8f28102ef0 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.453+0000 7f8f2d43f640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f28102570 0x7f8f28196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.453+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f28102ef0 0x7f8f28196aa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000eab0 tx=0x7f8f2000ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.453+0000 7f8f1effd640 1 -- 172.21.15.94:0/2317574188 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2000cea0 con 0x7f8f28102ef0 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.453+0000 7f8f1effd640 1 -- 172.21.15.94:0/2317574188 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f20005020 con 0x7f8f28102ef0 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.454+0000 7f8f1effd640 1 -- 172.21.15.94:0/2317574188 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20010670 con 0x7f8f28102ef0 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.454+0000 7f8f2fea4640 1 -- 172.21.15.94:0/2317574188 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f281aaf90 con 0x7f8f28102ef0 2024-09-19T04:29:07.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.454+0000 7f8f2fea4640 1 -- 172.21.15.94:0/2317574188 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281ab3f0 con 0x7f8f28102ef0 2024-09-19T04:29:07.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.457+0000 7f8f1effd640 1 -- 172.21.15.94:0/2317574188 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f8f200051d0 con 0x7f8f28102ef0 2024-09-19T04:29:07.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.458+0000 7f8f1effd640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8f140a3500 0x7f8f140a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.458+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8f140a3500 0x7f8f140a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.458+0000 7f8f1effd640 1 -- 172.21.15.94:0/2317574188 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f8f200d0400 con 0x7f8f28102ef0 2024-09-19T04:29:07.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.459+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8f140a3500 0x7f8f140a59c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400dc00 tx=0x7f8f2400f040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:07.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.460+0000 7f8f3227d740 1 Processor -- start 2024-09-19T04:29:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f3227d740 1 -- start start 2024-09-19T04:29:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308d8b40 0x55eb308d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308e2370 0x55eb308e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f3227d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55eb3083d2f0 con 0x55eb308d75d0 2024-09-19T04:29:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f3227d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eb307e4cf0 con 0x55eb308d8b40 2024-09-19T04:29:07.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f3227d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eb308392b0 con 0x55eb308e2370 2024-09-19T04:29:07.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f2dc40640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308d8b40 0x55eb308d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f2e441640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308e2370 0x55eb308e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f2dc40640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308d8b40 0x55eb308d7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:47144/0 (socket says 172.21.15.94:47144) 2024-09-19T04:29:07.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f2dc40640 1 -- 172.21.15.94:0/2656685208 learned_addr learned my addr 172.21.15.94:0/2656685208 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:07.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f2d43f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.462+0000 7f8f2dc40640 1 -- 172.21.15.94:0/2656685208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308e2370 msgr2=0x55eb308e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.461+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308e2370 0x55eb308e4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37156/0 (socket says 172.21.15.94:37156) 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.462+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308e2370 0x55eb308e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.462+0000 7f8f2dc40640 1 -- 172.21.15.94:0/2656685208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 msgr2=0x55eb308d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.462+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.462+0000 7f8f2dc40640 1 -- 172.21.15.94:0/2656685208 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb307c93a0 con 0x55eb308d8b40 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.462+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308d8b40 0x55eb308d7090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009ed0 tx=0x7f8f24015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0977d6010f52014 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.462+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308e2370 0x55eb308e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.463+0000 7f8f2d43f640 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.463+0000 7f8f13fff640 1 -- 172.21.15.94:0/2656685208 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24005220 con 0x55eb308d8b40 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.463+0000 7f8f13fff640 1 -- 172.21.15.94:0/2656685208 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f24019e20 con 0x55eb308d8b40 2024-09-19T04:29:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.463+0000 7f8f13fff640 1 -- 172.21.15.94:0/2656685208 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f240246f0 con 0x55eb308d8b40 2024-09-19T04:29:07.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.464+0000 7f8f3227d740 1 -- 172.21.15.94:0/2656685208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308d8b40 msgr2=0x55eb308d7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:07.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.464+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308d8b40 0x55eb308d7090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009ed0 tx=0x7f8f24015000 comp rx=0 tx=0).stop 2024-09-19T04:29:07.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.465+0000 7f8f3227d740 1 -- 172.21.15.94:0/2656685208 shutdown_connections 2024-09-19T04:29:07.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.465+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308e2370 0x55eb308e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.465+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.465+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2656685208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308d8b40 0x55eb308d7090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.465+0000 7f8f3227d740 1 -- 172.21.15.94:0/2656685208 >> 172.21.15.94:0/2656685208 conn(0x55eb308a0310 msgr2=0x55eb308d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:07.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.465+0000 7f8f3227d740 1 -- 172.21.15.94:0/2656685208 shutdown_connections 2024-09-19T04:29:07.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.465+0000 7f8f3227d740 1 -- 172.21.15.94:0/2656685208 wait complete. 2024-09-19T04:29:07.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.465+0000 7f8f3227d740 1 -- start start 2024-09-19T04:29:07.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.465+0000 7f8f3227d740 1 Processor -- start 2024-09-19T04:29:07.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308defc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308df560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f3227d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55eb308e5010 con 0x55eb308d75d0 2024-09-19T04:29:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f2dc40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f3227d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eb307f8060 con 0x55eb308e2370 2024-09-19T04:29:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f3227d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eb308e5190 con 0x55eb308d8b40 2024-09-19T04:29:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f3227d740 10 client.0 _ll_register_callbacks cb 0x55eb3054b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f2d43f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308defc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f2d43f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308defc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37168/0 (socket says 172.21.15.94:37168) 2024-09-19T04:29:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f2d43f640 1 -- 172.21.15.94:0/2111286440 learned_addr learned my addr 172.21.15.94:0/2111286440 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.466+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308df560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f2e441640 1 -- 172.21.15.94:0/2111286440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 msgr2=0x55eb308defc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308defc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f2e441640 1 -- 172.21.15.94:0/2111286440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 msgr2=0x55eb308e1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308e1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f2e441640 1 -- 172.21.15.94:0/2111286440 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb305ae3c0 con 0x55eb308e2370 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f2e441640 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308df560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8f200cda80 tx=0x7f8f2009cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc6aca1dbc728c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f117fa640 1 -- 172.21.15.94:0/2111286440 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200b4070 con 0x55eb308e2370 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f117fa640 1 -- 172.21.15.94:0/2111286440 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f8f20005540 con 0x55eb308e2370 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f3227d740 1 -- 172.21.15.94:0/2111286440 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f24019e20 con 0x55eb308e2370 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f117fa640 1 -- 172.21.15.94:0/2111286440 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8f200a5280 con 0x55eb308e2370 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f3227d740 10 client.26732 fetch_fsmap learned FSMap version 52 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.467+0000 7f8f3227d740 1 -- 172.21.15.94:0/2111286440 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55eb308f0a10 con 0x55eb308e2370 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.468+0000 7f8f117fa640 1 -- 172.21.15.94:0/2111286440 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f20098040 con 0x55eb308e2370 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.468+0000 7f8f117fa640 1 -- 172.21.15.94:0/2111286440 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f8f2009ba60 con 0x55eb308e2370 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.468+0000 7f8f3227d740 10 client.26732 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.468+0000 7f8f3227d740 10 client.26732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:29:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.468+0000 7f8f3227d740 20 client.26732 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.468+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55eb308f2250 0x55eb308f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.468+0000 7f8f3227d740 4 client.26732 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.468+0000 7f8f3227d740 1 -- 172.21.15.94:0/2111286440 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8f24019e20 con 0x55eb308f2250 2024-09-19T04:29:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.468+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55eb308f2250 0x55eb308f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.469+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55eb308f2250 0x55eb308f46d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a4c295e53300fea server_cookie=b52600684f345eb6 in_seq=0 out_seq=0 2024-09-19T04:29:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.469+0000 7f8f117fa640 10 client.26732 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:29:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.470+0000 7f8f117fa640 1 -- 172.21.15.94:0/2111286440 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8f24019e20 con 0x55eb308f2250 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.470+0000 7f8f117fa640 10 client.26732 handle_command_reply: tid=0 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.470+0000 7f8f3227d740 1 client.26732 shutdown 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.470+0000 7f8f3227d740 1 -- 172.21.15.94:0/2111286440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 msgr2=0x55eb308df560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.470+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308df560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8f200cda80 tx=0x7f8f2009cdd0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f3227d740 1 -- 172.21.15.94:0/2111286440 shutdown_connections 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55eb308f2250 0x55eb308f46d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308df560 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f117fa640 0 client.26732 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308defc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f3227d740 1 --2- 172.21.15.94:0/2111286440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308e1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f3227d740 1 -- 172.21.15.94:0/2111286440 >> 172.21.15.94:0/2111286440 conn(0x55eb308a0310 msgr2=0x55eb308d9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f3227d740 1 -- 172.21.15.94:0/2111286440 shutdown_connections 2024-09-19T04:29:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f3227d740 1 -- 172.21.15.94:0/2111286440 wait complete. 2024-09-19T04:29:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.471+0000 7f8f3227d740 20 client.26732 trim_cache size 0 max 16384 2024-09-19T04:29:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.486+0000 7f8f3227d740 1 Processor -- start 2024-09-19T04:29:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.486+0000 7f8f3227d740 1 -- start start 2024-09-19T04:29:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d75d0 0x55eb308e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d8b40 0x55eb308da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f3227d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b68b0 con 0x55eb308d8b40 2024-09-19T04:29:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f3227d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b4120 con 0x55eb308e2370 2024-09-19T04:29:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f3227d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b1f90 con 0x55eb308d75d0 2024-09-19T04:29:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2dc40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d75d0 0x55eb308e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2dc40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d75d0 0x55eb308e0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37178/0 (socket says 172.21.15.94:37178) 2024-09-19T04:29:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2dc40640 1 -- 172.21.15.94:0/3023925102 learned_addr learned my addr 172.21.15.94:0/3023925102 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2e441640 1 --2- 172.21.15.94:0/3023925102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2d43f640 1 --2- 172.21.15.94:0/3023925102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d8b40 0x55eb308da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2e441640 1 -- 172.21.15.94:0/3023925102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d75d0 msgr2=0x55eb308e0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2e441640 1 --2- 172.21.15.94:0/3023925102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d75d0 0x55eb308e0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2e441640 1 -- 172.21.15.94:0/3023925102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d8b40 msgr2=0x55eb308da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2e441640 1 --2- 172.21.15.94:0/3023925102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d8b40 0x55eb308da120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.487+0000 7f8f2e441640 1 -- 172.21.15.94:0/3023925102 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb308d7be0 con 0x55eb308e2370 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.488+0000 7f8f2e441640 1 --2- 172.21.15.94:0/3023925102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308da660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000c760 tx=0x7f8f200a0b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eefcffb48c18b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.488+0000 7f8f137fe640 1 -- 172.21.15.94:0/3023925102 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200b4070 con 0x55eb308e2370 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.488+0000 7f8f137fe640 1 -- 172.21.15.94:0/3023925102 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f20098040 con 0x55eb308e2370 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.488+0000 7f8f137fe640 1 -- 172.21.15.94:0/3023925102 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20098c80 con 0x55eb308e2370 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 -- 172.21.15.94:0/3023925102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 msgr2=0x55eb308da660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3023925102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308da660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000c760 tx=0x7f8f200a0b40 comp rx=0 tx=0).stop 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 -- 172.21.15.94:0/3023925102 shutdown_connections 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3023925102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308da660 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3023925102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d8b40 0x55eb308da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3023925102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d75d0 0x55eb308e0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 -- 172.21.15.94:0/3023925102 >> 172.21.15.94:0/3023925102 conn(0x55eb308a0310 msgr2=0x55eb308e0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:07.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 -- 172.21.15.94:0/3023925102 shutdown_connections 2024-09-19T04:29:07.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 -- 172.21.15.94:0/3023925102 wait complete. 2024-09-19T04:29:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.489+0000 7f8f3227d740 1 -- start start 2024-09-19T04:29:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.490+0000 7f8f3227d740 1 Processor -- start 2024-09-19T04:29:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.490+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308ea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.490+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.490+0000 7f8f3227d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.490+0000 7f8f3227d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b25e0 con 0x55eb308d75d0 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.490+0000 7f8f3227d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b3a50 con 0x55eb308e2370 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.490+0000 7f8f3227d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b2cb0 con 0x55eb308d8b40 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.490+0000 7f8f2dc40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2d43f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2e441640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2d43f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308ea960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37194/0 (socket says 172.21.15.94:37194) 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2d43f640 1 -- 172.21.15.94:0/3701279883 learned_addr learned my addr 172.21.15.94:0/3701279883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f3227d740 10 client.0 _ll_register_callbacks cb 0x7f8f200988b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2dc40640 1 -- 172.21.15.94:0/3701279883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 msgr2=0x55eb308ea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308ea960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2dc40640 1 -- 172.21.15.94:0/3701279883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 msgr2=0x55eb308e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308e73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2dc40640 1 -- 172.21.15.94:0/3701279883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb308e7930 con 0x55eb308d75d0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2e441640 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f2dc40640 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308ea420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f24000c00 tx=0x7f8f24005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d662813a621b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.491+0000 7f8f117fa640 1 -- 172.21.15.94:0/3701279883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2403b070 con 0x55eb308d75d0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f117fa640 1 -- 172.21.15.94:0/3701279883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f8f24005450 con 0x55eb308d75d0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f3227d740 1 -- 172.21.15.94:0/3701279883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f20098040 con 0x55eb308d75d0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f117fa640 1 -- 172.21.15.94:0/3701279883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8f24046050 con 0x55eb308d75d0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f3227d740 10 client.17115 fetch_fsmap learned FSMap version 52 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f3227d740 1 -- 172.21.15.94:0/3701279883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55eb3095d7a0 con 0x55eb308d75d0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f117fa640 1 -- 172.21.15.94:0/3701279883 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f20098040 con 0x55eb308d75d0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f117fa640 1 -- 172.21.15.94:0/3701279883 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f8f240037f0 con 0x55eb308d75d0 2024-09-19T04:29:07.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f3227d740 10 client.17115 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:29:07.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f3227d740 10 client.17115 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:29:07.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.492+0000 7f8f3227d740 20 client.17115 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:07.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.493+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55eb308f2250 0x55eb3095fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:07.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.493+0000 7f8f3227d740 4 client.17115 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:29:07.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.493+0000 7f8f3227d740 1 -- 172.21.15.94:0/3701279883 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb3084a860 con 0x55eb308f2250 2024-09-19T04:29:07.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.493+0000 7f8f2d43f640 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55eb308f2250 0x55eb3095fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.493+0000 7f8f117fa640 10 client.17115 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.493+0000 7f8f2d43f640 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55eb308f2250 0x55eb3095fcf0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abe82bcd7f898f6d server_cookie=9bef3338fa5d64f8 in_seq=0 out_seq=0 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f117fa640 1 -- 172.21.15.94:0/3701279883 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb3084a860 con 0x55eb308f2250 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f117fa640 10 client.17115 handle_command_reply: tid=0 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 client.17115 shutdown 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 -- 172.21.15.94:0/3701279883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 msgr2=0x55eb308ea420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308ea420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f24000c00 tx=0x7f8f24005fd0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 -- 172.21.15.94:0/3701279883 shutdown_connections 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55eb308f2250 0x55eb3095fcf0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb308e2370 0x55eb308e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb308d8b40 0x55eb308ea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 --2- 172.21.15.94:0/3701279883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb308d75d0 0x55eb308ea420 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f117fa640 0 client.17115 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:29:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 -- 172.21.15.94:0/3701279883 >> 172.21.15.94:0/3701279883 conn(0x55eb308a0310 msgr2=0x55eb308e1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:07.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 -- 172.21.15.94:0/3701279883 shutdown_connections 2024-09-19T04:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:29:07.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.494+0000 7f8f3227d740 1 -- 172.21.15.94:0/3701279883 wait complete. 2024-09-19T04:29:07.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f3227d740 20 client.17115 trim_cache size 0 max 16384 2024-09-19T04:29:07.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 -- 172.21.15.94:0/2317574188 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8f140a3500 msgr2=0x7f8f140a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:07.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8f140a3500 0x7f8f140a59c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400dc00 tx=0x7f8f2400f040 comp rx=0 tx=0).stop 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 -- 172.21.15.94:0/2317574188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f28102ef0 msgr2=0x7f8f28196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f28102ef0 0x7f8f28196aa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000eab0 tx=0x7f8f2000ef70 comp rx=0 tx=0).stop 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 -- 172.21.15.94:0/2317574188 shutdown_connections 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f8f140a3500 0x7f8f140a59c0 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400dc00 tx=0x7f8f2400f040 comp rx=0 tx=0).stop 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f28102ef0 0x7f8f28196aa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f28102570 0x7f8f28196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 --2- 172.21.15.94:0/2317574188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f28100e20 0x7f8f28196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.495+0000 7f8f2fea4640 1 -- 172.21.15.94:0/2317574188 >> 172.21.15.94:0/2317574188 conn(0x7f8f280fc7d0 msgr2=0x7f8f280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.496+0000 7f8f2fea4640 1 -- 172.21.15.94:0/2317574188 shutdown_connections 2024-09-19T04:29:07.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:07.496+0000 7f8f2fea4640 1 -- 172.21.15.94:0/2317574188 wait complete. 2024-09-19T04:29:08.346 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:29:08.708 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:09.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:08 smithi094 ceph-mon[30972]: pgmap v441: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 5.0 MiB/s wr, 686 op/s 2024-09-19T04:29:09.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:08 smithi104 ceph-mon[34145]: pgmap v441: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 5.0 MiB/s wr, 686 op/s 2024-09-19T04:29:09.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.253+0000 7fca538b0640 1 -- 172.21.15.94:0/475926545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c074620 msgr2=0x7fca4c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:09.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.253+0000 7fca538b0640 1 --2- 172.21.15.94:0/475926545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c074620 0x7fca4c074a00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fca44009ca0 tx=0x7fca4402a950 comp rx=0 tx=0).stop 2024-09-19T04:29:09.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.253+0000 7fca538b0640 1 -- 172.21.15.94:0/475926545 shutdown_connections 2024-09-19T04:29:09.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.253+0000 7fca538b0640 1 --2- 172.21.15.94:0/475926545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca4c10e3c0 0x7fca4c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.253+0000 7fca538b0640 1 --2- 172.21.15.94:0/475926545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca4c074f40 0x7fca4c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.253+0000 7fca538b0640 1 --2- 172.21.15.94:0/475926545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c074620 0x7fca4c074a00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.253+0000 7fca538b0640 1 -- 172.21.15.94:0/475926545 >> 172.21.15.94:0/475926545 conn(0x7fca4c0fd8e0 msgr2=0x7fca4c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:09.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.253+0000 7fca538b0640 1 -- 172.21.15.94:0/475926545 shutdown_connections 2024-09-19T04:29:09.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.253+0000 7fca538b0640 1 -- 172.21.15.94:0/475926545 wait complete. 2024-09-19T04:29:09.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca538b0640 1 Processor -- start 2024-09-19T04:29:09.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca538b0640 1 -- start start 2024-09-19T04:29:09.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca538b0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca4c074620 0x7fca4c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca538b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca4c074f40 0x7fca4c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca538b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c10e3c0 0x7fca4c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca538b0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c114000 con 0x7fca4c10e3c0 2024-09-19T04:29:09.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca538b0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c113e80 con 0x7fca4c074620 2024-09-19T04:29:09.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca538b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c114180 con 0x7fca4c074f40 2024-09-19T04:29:09.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca51e4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c10e3c0 0x7fca4c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca51e4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c10e3c0 0x7fca4c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39578/0 (socket says 172.21.15.94:39578) 2024-09-19T04:29:09.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.254+0000 7fca51e4d640 1 -- 172.21.15.94:0/3956227622 learned_addr learned my addr 172.21.15.94:0/3956227622 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.255+0000 7fca5164c640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca4c074620 0x7fca4c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.255+0000 7fca51e4d640 1 -- 172.21.15.94:0/3956227622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca4c074f40 msgr2=0x7fca4c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.255+0000 7fca51e4d640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca4c074f40 0x7fca4c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.255+0000 7fca51e4d640 1 -- 172.21.15.94:0/3956227622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca4c074620 msgr2=0x7fca4c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.255+0000 7fca51e4d640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca4c074620 0x7fca4c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.255+0000 7fca51e4d640 1 -- 172.21.15.94:0/3956227622 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c19f540 con 0x7fca4c10e3c0 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.255+0000 7fca51e4d640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c10e3c0 0x7fca4c19ee60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fca4800d750 tx=0x7fca4800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.255+0000 7fca427fc640 1 -- 172.21.15.94:0/3956227622 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48017070 con 0x7fca4c10e3c0 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.256+0000 7fca538b0640 1 -- 172.21.15.94:0/3956227622 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c19f830 con 0x7fca4c10e3c0 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.256+0000 7fca427fc640 1 -- 172.21.15.94:0/3956227622 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca480054d0 con 0x7fca4c10e3c0 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.256+0000 7fca427fc640 1 -- 172.21.15.94:0/3956227622 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48004c30 con 0x7fca4c10e3c0 2024-09-19T04:29:09.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.256+0000 7fca538b0640 1 -- 172.21.15.94:0/3956227622 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c1a7110 con 0x7fca4c10e3c0 2024-09-19T04:29:09.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.259+0000 7fca55c89740 1 Processor -- start 2024-09-19T04:29:09.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.259+0000 7fca427fc640 1 -- 172.21.15.94:0/3956227622 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fca48004dd0 con 0x7fca4c10e3c0 2024-09-19T04:29:09.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.259+0000 7fca55c89740 1 -- start start 2024-09-19T04:29:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca55c89740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 0x5610b4e838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca55c89740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e84220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca55c89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 0x5610b4e90f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca55c89740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5610b4de12f0 con 0x5610b4e83de0 2024-09-19T04:29:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca55c89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5610b4d88cf0 con 0x5610b4e8eb80 2024-09-19T04:29:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca55c89740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5610b4ddd2b0 con 0x5610b4e85350 2024-09-19T04:29:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca50e4b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 0x5610b4e838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca50e4b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 0x5610b4e90f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca50e4b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 0x5610b4e838a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37220/0 (socket says 172.21.15.94:37220) 2024-09-19T04:29:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca50e4b640 1 -- 172.21.15.94:0/3113745238 learned_addr learned my addr 172.21.15.94:0/3113745238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.260+0000 7fca5164c640 1 --2- 172.21.15.94:0/3113745238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e84220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca5164c640 1 -- 172.21.15.94:0/3113745238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 msgr2=0x5610b4e838a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca5164c640 1 --2- 172.21.15.94:0/3113745238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 0x5610b4e838a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca5164c640 1 -- 172.21.15.94:0/3113745238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 msgr2=0x5610b4e90f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca5164c640 1 --2- 172.21.15.94:0/3113745238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 0x5610b4e90f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca5164c640 1 -- 172.21.15.94:0/3113745238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b4d6d3a0 con 0x5610b4e83de0 2024-09-19T04:29:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca5164c640 1 --2- 172.21.15.94:0/3113745238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e84220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fca4402ae70 tx=0x7fca4402fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eea1c073c0558a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca377fe640 1 -- 172.21.15.94:0/3113745238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44048070 con 0x5610b4e83de0 2024-09-19T04:29:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca427fc640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fca380a3430 0x7fca380a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca377fe640 1 -- 172.21.15.94:0/3113745238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca44003e20 con 0x5610b4e83de0 2024-09-19T04:29:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.261+0000 7fca377fe640 1 -- 172.21.15.94:0/3113745238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44034040 con 0x5610b4e83de0 2024-09-19T04:29:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.262+0000 7fca50e4b640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fca380a3430 0x7fca380a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.262+0000 7fca427fc640 1 -- 172.21.15.94:0/3956227622 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fca480cf780 con 0x7fca4c10e3c0 2024-09-19T04:29:09.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.262+0000 7fca50e4b640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fca380a3430 0x7fca380a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5610b4e84880 tx=0x7fca3c0046c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:09.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.262+0000 7fca55c89740 1 -- 172.21.15.94:0/3113745238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 msgr2=0x5610b4e84220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:09.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 --2- 172.21.15.94:0/3113745238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e84220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fca4402ae70 tx=0x7fca4402fdf0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 -- 172.21.15.94:0/3113745238 shutdown_connections 2024-09-19T04:29:09.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 --2- 172.21.15.94:0/3113745238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 0x5610b4e90f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 --2- 172.21.15.94:0/3113745238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e84220 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 --2- 172.21.15.94:0/3113745238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 0x5610b4e838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 -- 172.21.15.94:0/3113745238 >> 172.21.15.94:0/3113745238 conn(0x5610b4e44310 msgr2=0x5610b4e80470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:09.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 -- 172.21.15.94:0/3113745238 shutdown_connections 2024-09-19T04:29:09.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 -- 172.21.15.94:0/3113745238 wait complete. 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 -- start start 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 Processor -- start 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.263+0000 7fca55c89740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca55c89740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 0x5610b4e8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca55c89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 0x5610b4e8bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca55c89740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5610b4e91820 con 0x5610b4e83de0 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca55c89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5610b4d9c060 con 0x5610b4e8eb80 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca55c89740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5610b4e919a0 con 0x5610b4e85350 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca5164c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca5164c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e8e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39592/0 (socket says 172.21.15.94:39592) 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca5164c640 1 -- 172.21.15.94:0/459058804 learned_addr learned my addr 172.21.15.94:0/459058804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca55c89740 10 client.0 _ll_register_callbacks cb 0x5610b4aef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca51e4d640 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 0x5610b4e8bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca5164c640 1 -- 172.21.15.94:0/459058804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 msgr2=0x5610b4e8b8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca5164c640 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 0x5610b4e8b8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca5164c640 1 -- 172.21.15.94:0/459058804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 msgr2=0x5610b4e8bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca5164c640 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 0x5610b4e8bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca5164c640 1 -- 172.21.15.94:0/459058804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b4b523c0 con 0x5610b4e83de0 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.264+0000 7fca51e4d640 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 0x5610b4e8bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.265+0000 7fca5164c640 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e8e5f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fca44005e30 tx=0x7fca4402faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae27da0c3c1beb4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.265+0000 7fca34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.265+0000 7fca34ff9640 1 -- 172.21.15.94:0/459058804 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44051060 con 0x5610b4e83de0 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.265+0000 7fca55c89740 1 -- 172.21.15.94:0/459058804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca44003e20 con 0x5610b4e83de0 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.265+0000 7fca34ff9640 1 -- 172.21.15.94:0/459058804 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fca44003b80 con 0x5610b4e83de0 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.265+0000 7fca34ff9640 1 -- 172.21.15.94:0/459058804 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fca44048070 con 0x5610b4e83de0 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.265+0000 7fca55c89740 10 client.17127 fetch_fsmap learned FSMap version 52 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.265+0000 7fca55c89740 1 -- 172.21.15.94:0/459058804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5610b4e9d350 con 0x5610b4e83de0 2024-09-19T04:29:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.266+0000 7fca34ff9640 1 -- 172.21.15.94:0/459058804 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca44003e20 con 0x5610b4e83de0 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.266+0000 7fca34ff9640 1 -- 172.21.15.94:0/459058804 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fca440304d0 con 0x5610b4e83de0 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.266+0000 7fca55c89740 10 client.17127 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.266+0000 7fca55c89740 10 client.17127 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.266+0000 7fca55c89740 20 client.17127 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.266+0000 7fca55c89740 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5610b4e9e830 0x5610b4ea0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.266+0000 7fca55c89740 4 client.17127 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.266+0000 7fca55c89740 1 -- 172.21.15.94:0/459058804 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610b4dee860 con 0x5610b4e9e830 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.266+0000 7fca50e4b640 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5610b4e9e830 0x5610b4ea0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.267+0000 7fca34ff9640 10 client.17127 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:29:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.267+0000 7fca50e4b640 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5610b4e9e830 0x5610b4ea0cd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1910521298ba1cc server_cookie=9905618a7c7a7a0a in_seq=0 out_seq=0 2024-09-19T04:29:09.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca34ff9640 1 -- 172.21.15.94:0/459058804 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610b4dee860 con 0x5610b4e9e830 2024-09-19T04:29:09.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca34ff9640 10 client.17127 handle_command_reply: tid=0 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca55c89740 1 client.17127 shutdown 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca55c89740 1 -- 172.21.15.94:0/459058804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 msgr2=0x5610b4e8e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca55c89740 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e8e5f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fca44005e30 tx=0x7fca4402faa0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca55c89740 1 -- 172.21.15.94:0/459058804 shutdown_connections 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca55c89740 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5610b4e9e830 0x5610b4ea0cd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca55c89740 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e8eb80 0x5610b4e8bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca34ff9640 0 client.17127 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca55c89740 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e85350 0x5610b4e8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca55c89740 1 --2- 172.21.15.94:0/459058804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e8e5f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.268+0000 7fca55c89740 1 -- 172.21.15.94:0/459058804 >> 172.21.15.94:0/459058804 conn(0x5610b4e44310 msgr2=0x5610b4e86170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.269+0000 7fca55c89740 1 -- 172.21.15.94:0/459058804 shutdown_connections 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.269+0000 7fca55c89740 1 -- 172.21.15.94:0/459058804 wait complete. 2024-09-19T04:29:09.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.269+0000 7fca55c89740 20 client.17127 trim_cache size 0 max 16384 2024-09-19T04:29:09.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:08 smithi112 ceph-mon[35051]: pgmap v441: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 5.0 MiB/s wr, 686 op/s 2024-09-19T04:29:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.290+0000 7fca55c89740 1 Processor -- start 2024-09-19T04:29:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.290+0000 7fca55c89740 1 -- start start 2024-09-19T04:29:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.291+0000 7fca55c89740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e9aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.291+0000 7fca55c89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e8a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.291+0000 7fca5164c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e9aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.291+0000 7fca5164c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e9aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39600/0 (socket says 172.21.15.94:39600) 2024-09-19T04:29:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.291+0000 7fca50e4b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e8a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.291+0000 7fca50e4b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e8a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:47206/0 (socket says 172.21.15.94:47206) 2024-09-19T04:29:09.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.292+0000 7fca5164c640 1 -- 172.21.15.94:0/3607809945 learned_addr learned my addr 172.21.15.94:0/3607809945 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:09.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.292+0000 7fca55c89740 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e8eb80 0x5610b4ea65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.292+0000 7fca55c89740 1 -- 172.21.15.94:0/3607809945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c1ae010 con 0x5610b4e83de0 2024-09-19T04:29:09.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.292+0000 7fca55c89740 1 -- 172.21.15.94:0/3607809945 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c1ab880 con 0x5610b4e85350 2024-09-19T04:29:09.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.292+0000 7fca51e4d640 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e8eb80 0x5610b4ea65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.292+0000 7fca55c89740 1 -- 172.21.15.94:0/3607809945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c1a96f0 con 0x5610b4e8eb80 2024-09-19T04:29:09.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca50e4b640 1 -- 172.21.15.94:0/3607809945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e8eb80 msgr2=0x5610b4ea65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:09.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca50e4b640 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e8eb80 0x5610b4ea65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca50e4b640 1 -- 172.21.15.94:0/3607809945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 msgr2=0x5610b4e9aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca50e4b640 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e9aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca50e4b640 1 -- 172.21.15.94:0/3607809945 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b4e9b000 con 0x5610b4e85350 2024-09-19T04:29:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca51e4d640 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e8eb80 0x5610b4ea65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca50e4b640 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e8a8a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5610b4e8ce50 tx=0x7fca3c016050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=801901bdd320c837 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca36ffd640 1 -- 172.21.15.94:0/3607809945 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c02f070 con 0x5610b4e85350 2024-09-19T04:29:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca36ffd640 1 -- 172.21.15.94:0/3607809945 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca3c013ab0 con 0x5610b4e85350 2024-09-19T04:29:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.293+0000 7fca36ffd640 1 -- 172.21.15.94:0/3607809945 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c01b070 con 0x5610b4e85350 2024-09-19T04:29:09.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.294+0000 7fca55c89740 1 -- 172.21.15.94:0/3607809945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 msgr2=0x5610b4e8a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:09.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.294+0000 7fca55c89740 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e8a8a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5610b4e8ce50 tx=0x7fca3c016050 comp rx=0 tx=0).stop 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.294+0000 7fca55c89740 1 -- 172.21.15.94:0/3607809945 shutdown_connections 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.294+0000 7fca55c89740 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e8eb80 0x5610b4ea65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.294+0000 7fca55c89740 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e8a8a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.294+0000 7fca55c89740 1 --2- 172.21.15.94:0/3607809945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e83de0 0x5610b4e9aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.294+0000 7fca55c89740 1 -- 172.21.15.94:0/3607809945 >> 172.21.15.94:0/3607809945 conn(0x5610b4e44310 msgr2=0x5610b4e8a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.295+0000 7fca55c89740 1 -- 172.21.15.94:0/3607809945 shutdown_connections 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.295+0000 7fca55c89740 1 -- 172.21.15.94:0/3607809945 wait complete. 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.295+0000 7fca55c89740 1 -- start start 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.295+0000 7fca55c89740 1 Processor -- start 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.295+0000 7fca55c89740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e83de0 0x5610b4e96e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.295+0000 7fca55c89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e97350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca55c89740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e8eb80 0x5610b4e88050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca55c89740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c1a9d40 con 0x5610b4e8eb80 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca55c89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c1ab1b0 con 0x5610b4e85350 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca5164c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e83de0 0x5610b4e96e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca55c89740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c1aa410 con 0x5610b4e83de0 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca5164c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e83de0 0x5610b4e96e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:37258/0 (socket says 172.21.15.94:37258) 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca5164c640 1 -- 172.21.15.94:0/2907943881 learned_addr learned my addr 172.21.15.94:0/2907943881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca55c89740 10 client.0 _ll_register_callbacks cb 0x7fca44046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca51e4d640 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e8eb80 0x5610b4e88050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca50e4b640 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e97350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca5164c640 1 -- 172.21.15.94:0/2907943881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 msgr2=0x5610b4e97350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca5164c640 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e97350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca5164c640 1 -- 172.21.15.94:0/2907943881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e8eb80 msgr2=0x5610b4e88050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca5164c640 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e8eb80 0x5610b4e88050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca5164c640 1 -- 172.21.15.94:0/2907943881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b4e88830 con 0x5610b4e83de0 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca50e4b640 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e97350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca5164c640 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e83de0 0x5610b4e96e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fca44009ca0 tx=0x7fca44030cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f4a8a5a907764d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.296+0000 7fca34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.297+0000 7fca34ff9640 1 -- 172.21.15.94:0/2907943881 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44051040 con 0x5610b4e83de0 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.297+0000 7fca34ff9640 1 -- 172.21.15.94:0/2907943881 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fca4404c660 con 0x5610b4e83de0 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.297+0000 7fca55c89740 1 -- 172.21.15.94:0/2907943881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca3c013ab0 con 0x5610b4e83de0 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.297+0000 7fca34ff9640 1 -- 172.21.15.94:0/2907943881 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fca44034050 con 0x5610b4e83de0 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.297+0000 7fca55c89740 10 client.26722 fetch_fsmap learned FSMap version 52 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.297+0000 7fca55c89740 1 -- 172.21.15.94:0/2907943881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5610b4f1a000 con 0x5610b4e83de0 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.297+0000 7fca34ff9640 1 -- 172.21.15.94:0/2907943881 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca3c013ab0 con 0x5610b4e83de0 2024-09-19T04:29:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.297+0000 7fca34ff9640 1 -- 172.21.15.94:0/2907943881 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fca4405b020 con 0x5610b4e83de0 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.298+0000 7fca55c89740 10 client.26722 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.298+0000 7fca55c89740 10 client.26722 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.298+0000 7fca55c89740 20 client.26722 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.298+0000 7fca55c89740 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5610b4e9e830 0x5610b4f1c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.298+0000 7fca55c89740 4 client.26722 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.298+0000 7fca55c89740 1 -- 172.21.15.94:0/2907943881 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610b4de8d70 con 0x5610b4e9e830 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.298+0000 7fca50e4b640 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5610b4e9e830 0x5610b4f1c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.298+0000 7fca34ff9640 10 client.26722 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.298+0000 7fca50e4b640 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5610b4e9e830 0x5610b4f1c630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a421baa0b442c971 server_cookie=33cb6dd012037db in_seq=0 out_seq=0 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca34ff9640 1 -- 172.21.15.94:0/2907943881 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610b4de8d70 con 0x5610b4e9e830 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca34ff9640 10 client.26722 handle_command_reply: tid=0 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 client.26722 shutdown 2024-09-19T04:29:09.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 -- 172.21.15.94:0/2907943881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e83de0 msgr2=0x5610b4e96e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:09.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e83de0 0x5610b4e96e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fca44009ca0 tx=0x7fca44030cc0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 -- 172.21.15.94:0/2907943881 shutdown_connections 2024-09-19T04:29:09.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x5610b4e9e830 0x5610b4f1c630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca34ff9640 0 client.26722 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:29:09.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610b4e8eb80 0x5610b4e88050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610b4e85350 0x5610b4e97350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 --2- 172.21.15.94:0/2907943881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610b4e83de0 0x5610b4e96e10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 -- 172.21.15.94:0/2907943881 >> 172.21.15.94:0/2907943881 conn(0x5610b4e44310 msgr2=0x5610b4e8a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:09.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 -- 172.21.15.94:0/2907943881 shutdown_connections 2024-09-19T04:29:09.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 1 -- 172.21.15.94:0/2907943881 wait complete. 2024-09-19T04:29:09.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.299+0000 7fca55c89740 20 client.26722 trim_cache size 0 max 16384 2024-09-19T04:29:09.306 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:29:09.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 -- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fca380a3430 msgr2=0x7fca380a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:09.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fca380a3430 0x7fca380a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5610b4e84880 tx=0x7fca3c0046c0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 -- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c10e3c0 msgr2=0x7fca4c19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:09.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c10e3c0 0x7fca4c19ee60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fca4800d750 tx=0x7fca4800dc10 comp rx=0 tx=0).stop 2024-09-19T04:29:09.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 -- 172.21.15.94:0/3956227622 shutdown_connections 2024-09-19T04:29:09.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fca380a3430 0x7fca380a58f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca4c10e3c0 0x7fca4c19ee60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca4c074f40 0x7fca4c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 --2- 172.21.15.94:0/3956227622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca4c074620 0x7fca4c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:09.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.300+0000 7fca538b0640 1 -- 172.21.15.94:0/3956227622 >> 172.21.15.94:0/3956227622 conn(0x7fca4c0fd8e0 msgr2=0x7fca4c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:09.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.301+0000 7fca538b0640 1 -- 172.21.15.94:0/3956227622 shutdown_connections 2024-09-19T04:29:09.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:09.301+0000 7fca538b0640 1 -- 172.21.15.94:0/3956227622 wait complete. 2024-09-19T04:29:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 775.0 s, 396.4 tps, lat 250.696 ms stddev 25.678 2024-09-19T04:29:11.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:10 smithi094 ceph-mon[30972]: pgmap v442: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 4.7 MiB/s wr, 633 op/s 2024-09-19T04:29:11.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:10 smithi094 ceph-mon[30972]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T04:29:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:10 smithi104 ceph-mon[34145]: pgmap v442: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 4.7 MiB/s wr, 633 op/s 2024-09-19T04:29:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:10 smithi104 ceph-mon[34145]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T04:29:11.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:10 smithi112 ceph-mon[35051]: pgmap v442: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 4.7 MiB/s wr, 633 op/s 2024-09-19T04:29:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:10 smithi112 ceph-mon[35051]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T04:29:13.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:12 smithi094 ceph-mon[30972]: pgmap v444: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.4 MiB/s wr, 337 op/s 2024-09-19T04:29:13.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:12 smithi104 ceph-mon[34145]: pgmap v444: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.4 MiB/s wr, 337 op/s 2024-09-19T04:29:13.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:12 smithi112 ceph-mon[35051]: pgmap v444: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.4 MiB/s wr, 337 op/s 2024-09-19T04:29:14.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:13 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:29:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:13 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:29:14.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:13 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:29:15.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:14 smithi094 ceph-mon[30972]: pgmap v445: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 4.0 MiB/s wr, 545 op/s 2024-09-19T04:29:15.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:14 smithi104 ceph-mon[34145]: pgmap v445: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 4.0 MiB/s wr, 545 op/s 2024-09-19T04:29:15.159 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:29:15.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:14 smithi112 ceph-mon[35051]: pgmap v445: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 4.0 MiB/s wr, 545 op/s 2024-09-19T04:29:15.516 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 780.0 s, 370.6 tps, lat 272.859 ms stddev 47.911 2024-09-19T04:29:16.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.064+0000 7fb54100f640 1 -- 172.21.15.94:0/1739291081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c101e20 msgr2=0x7fb53c1056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.064+0000 7fb54100f640 1 --2- 172.21.15.94:0/1739291081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c101e20 0x7fb53c1056b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb530009900 tx=0x7fb53002a410 comp rx=0 tx=0).stop 2024-09-19T04:29:16.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.064+0000 7fb54100f640 1 -- 172.21.15.94:0/1739291081 shutdown_connections 2024-09-19T04:29:16.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.064+0000 7fb54100f640 1 --2- 172.21.15.94:0/1739291081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c105de0 0x7fb53c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.064+0000 7fb54100f640 1 --2- 172.21.15.94:0/1739291081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c101e20 0x7fb53c1056b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.064+0000 7fb54100f640 1 --2- 172.21.15.94:0/1739291081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c101470 0x7fb53c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.064+0000 7fb54100f640 1 -- 172.21.15.94:0/1739291081 >> 172.21.15.94:0/1739291081 conn(0x7fb53c077230 msgr2=0x7fb53c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:16.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.064+0000 7fb54100f640 1 -- 172.21.15.94:0/1739291081 shutdown_connections 2024-09-19T04:29:16.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.064+0000 7fb54100f640 1 -- 172.21.15.94:0/1739291081 wait complete. 2024-09-19T04:29:16.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.065+0000 7fb54100f640 1 Processor -- start 2024-09-19T04:29:16.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.065+0000 7fb54100f640 1 -- start start 2024-09-19T04:29:16.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.065+0000 7fb54100f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c101470 0x7fb53c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.065+0000 7fb54100f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c101e20 0x7fb53c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.065+0000 7fb54100f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c105de0 0x7fb53c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb54100f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c110d40 con 0x7fb53c105de0 2024-09-19T04:29:16.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb54100f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c110bc0 con 0x7fb53c101470 2024-09-19T04:29:16.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb54100f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c110ec0 con 0x7fb53c101e20 2024-09-19T04:29:16.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c101470 0x7fb53c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c105de0 0x7fb53c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c101470 0x7fb53c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:34822/0 (socket says 172.21.15.94:34822) 2024-09-19T04:29:16.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53ad9d640 1 -- 172.21.15.94:0/2349131103 learned_addr learned my addr 172.21.15.94:0/2349131103 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:16.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53a59c640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c101e20 0x7fb53c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53b59e640 1 -- 172.21.15.94:0/2349131103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c101e20 msgr2=0x7fb53c19acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53b59e640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c101e20 0x7fb53c19acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53b59e640 1 -- 172.21.15.94:0/2349131103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c101470 msgr2=0x7fb53c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53b59e640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c101470 0x7fb53c19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53b59e640 1 -- 172.21.15.94:0/2349131103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c19f560 con 0x7fb53c105de0 2024-09-19T04:29:16.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c101470 0x7fb53c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:16.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.066+0000 7fb53b59e640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c105de0 0x7fb53c19ee80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb534009fd0 tx=0x7fb53400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:16.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.067+0000 7fb523fff640 1 -- 172.21.15.94:0/2349131103 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53400ce20 con 0x7fb53c105de0 2024-09-19T04:29:16.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.067+0000 7fb523fff640 1 -- 172.21.15.94:0/2349131103 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb534005020 con 0x7fb53c105de0 2024-09-19T04:29:16.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.067+0000 7fb523fff640 1 -- 172.21.15.94:0/2349131103 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5340216a0 con 0x7fb53c105de0 2024-09-19T04:29:16.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.067+0000 7fb54100f640 1 -- 172.21.15.94:0/2349131103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53c19f850 con 0x7fb53c105de0 2024-09-19T04:29:16.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.067+0000 7fb54100f640 1 -- 172.21.15.94:0/2349131103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53c19fc50 con 0x7fb53c105de0 2024-09-19T04:29:16.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.070+0000 7fb523fff640 1 -- 172.21.15.94:0/2349131103 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fb5340051d0 con 0x7fb53c105de0 2024-09-19T04:29:16.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.070+0000 7fb5433e8740 1 Processor -- start 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.070+0000 7fb5433e8740 1 -- start start 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 0x5624d43c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 0x5624d43d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb5433e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5624d432f2f0 con 0x5624d43d4370 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb5433e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5624d42d6cf0 con 0x5624d43cab40 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb5433e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5624d432b2b0 con 0x5624d43c95d0 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:36004/0 (socket says 172.21.15.94:36004) 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53ad9d640 1 -- 172.21.15.94:0/1881951280 learned_addr learned my addr 172.21.15.94:0/1881951280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb523fff640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5240a3430 0x7fb5240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53a59c640 1 --2- 172.21.15.94:0/1881951280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 0x5624d43c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53a59c640 1 --2- 172.21.15.94:0/1881951280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 0x5624d43d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb523fff640 1 -- 172.21.15.94:0/2349131103 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fb5340e1360 con 0x7fb53c105de0 2024-09-19T04:29:16.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53b59e640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5240a3430 0x7fb5240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53ad9d640 1 -- 172.21.15.94:0/1881951280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 msgr2=0x5624d43c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:16.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/1881951280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 0x5624d43c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53ad9d640 1 -- 172.21.15.94:0/1881951280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 msgr2=0x5624d43d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:16.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/1881951280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 0x5624d43d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.071+0000 7fb53ad9d640 1 -- 172.21.15.94:0/1881951280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d42bb3a0 con 0x5624d43c95d0 2024-09-19T04:29:16.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.072+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/1881951280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43c9a10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb52c011650 tx=0x7fb52c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ed1928ea6d0f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:16.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.072+0000 7fb5217fa640 1 -- 172.21.15.94:0/1881951280 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c01e070 con 0x5624d43c95d0 2024-09-19T04:29:16.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.072+0000 7fb53b59e640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5240a3430 0x7fb5240a58f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5340a9a60 tx=0x7fb534005f70 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.072+0000 7fb5217fa640 1 -- 172.21.15.94:0/1881951280 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52c00fb70 con 0x5624d43c95d0 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.072+0000 7fb5217fa640 1 -- 172.21.15.94:0/1881951280 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c019c30 con 0x5624d43c95d0 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.073+0000 7fb5433e8740 1 -- 172.21.15.94:0/1881951280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 msgr2=0x5624d43c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.073+0000 7fb5433e8740 1 --2- 172.21.15.94:0/1881951280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43c9a10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb52c011650 tx=0x7fb52c009f90 comp rx=0 tx=0).stop 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.073+0000 7fb5433e8740 1 -- 172.21.15.94:0/1881951280 shutdown_connections 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.073+0000 7fb5433e8740 1 --2- 172.21.15.94:0/1881951280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 0x5624d43d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.073+0000 7fb5433e8740 1 --2- 172.21.15.94:0/1881951280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43c9a10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.073+0000 7fb5433e8740 1 --2- 172.21.15.94:0/1881951280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 0x5624d43c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.073+0000 7fb5433e8740 1 -- 172.21.15.94:0/1881951280 >> 172.21.15.94:0/1881951280 conn(0x5624d4392310 msgr2=0x5624d43c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.073+0000 7fb5433e8740 1 -- 172.21.15.94:0/1881951280 shutdown_connections 2024-09-19T04:29:16.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.073+0000 7fb5433e8740 1 -- 172.21.15.94:0/1881951280 wait complete. 2024-09-19T04:29:16.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.074+0000 7fb5433e8740 1 -- start start 2024-09-19T04:29:16.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.074+0000 7fb5433e8740 1 Processor -- start 2024-09-19T04:29:16.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.074+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43c95d0 0x5624d43d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.074+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43cab40 0x5624d43d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.074+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 0x5624d43d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.074+0000 7fb53a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43c95d0 0x5624d43d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb53a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43c95d0 0x5624d43d3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51922/0 (socket says 172.21.15.94:51922) 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb53a59c640 1 -- 172.21.15.94:0/3749771155 learned_addr learned my addr 172.21.15.94:0/3749771155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5624d43d7070 con 0x5624d43c95d0 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5624d42ea060 con 0x5624d43d4370 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5624d43d71f0 con 0x5624d43cab40 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb53a59c640 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 0x5624d43d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb5433e8740 10 client.0 _ll_register_callbacks cb 0x5624d403d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb53a59c640 1 -- 172.21.15.94:0/3749771155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43cab40 msgr2=0x5624d43d1010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb53a59c640 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43cab40 0x5624d43d1010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb53a59c640 1 -- 172.21.15.94:0/3749771155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 msgr2=0x5624d43d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:16.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb53a59c640 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 0x5624d43d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb53a59c640 1 -- 172.21.15.94:0/3749771155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d40a03c0 con 0x5624d43c95d0 2024-09-19T04:29:16.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb53a59c640 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43c95d0 0x5624d43d3d60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb530006010 tx=0x7fb53002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1be13a6452f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:16.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb51eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:29:16.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.075+0000 7fb51eff5640 1 -- 172.21.15.94:0/3749771155 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530035070 con 0x5624d43c95d0 2024-09-19T04:29:16.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.076+0000 7fb51eff5640 1 -- 172.21.15.94:0/3749771155 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fb530003b70 con 0x5624d43c95d0 2024-09-19T04:29:16.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.076+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52c00fb70 con 0x5624d43c95d0 2024-09-19T04:29:16.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.076+0000 7fb51eff5640 1 -- 172.21.15.94:0/3749771155 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fb530003d70 con 0x5624d43c95d0 2024-09-19T04:29:16.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.077+0000 7fb5433e8740 10 client.17145 fetch_fsmap learned FSMap version 53 2024-09-19T04:29:16.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.077+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5624d43e2b10 con 0x5624d43c95d0 2024-09-19T04:29:16.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.077+0000 7fb51eff5640 1 -- 172.21.15.94:0/3749771155 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52c00fb70 con 0x5624d43c95d0 2024-09-19T04:29:16.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.077+0000 7fb51eff5640 1 -- 172.21.15.94:0/3749771155 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 53) ==== 8350+0+0 (secure 0 0 0) 0x7fb53003d450 con 0x5624d43c95d0 2024-09-19T04:29:16.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.078+0000 7fb5433e8740 10 client.17145 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T04:29:16.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.078+0000 7fb5433e8740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:16.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.078+0000 7fb5433e8740 20 client.17145 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:16.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.078+0000 7fb5433e8740 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5624d43e3f70 0x5624d43e6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.078+0000 7fb5433e8740 4 client.17145 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:16.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.078+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624d433c860 con 0x5624d43e3f70 2024-09-19T04:29:16.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.078+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5624d43e3f70 0x5624d43e6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.078+0000 7fb51eff5640 10 client.17145 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:16.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.078+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5624d43e3f70 0x5624d43e6400 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf5ce5d47a8f5e4 server_cookie=13c561d3381dd792 in_seq=0 out_seq=0 2024-09-19T04:29:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.079+0000 7fb51eff5640 1 -- 172.21.15.94:0/3749771155 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624d433c860 con 0x5624d43e3f70 2024-09-19T04:29:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.079+0000 7fb51eff5640 10 client.17145 handle_command_reply: tid=0 2024-09-19T04:29:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.079+0000 7fb5433e8740 1 client.17145 shutdown 2024-09-19T04:29:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43c95d0 msgr2=0x5624d43d3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb5433e8740 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43c95d0 0x5624d43d3d60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb530006010 tx=0x7fb53002a410 comp rx=0 tx=0).stop 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 shutdown_connections 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb5433e8740 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5624d43e3f70 0x5624d43e6400 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb51eff5640 0 client.17145 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb5433e8740 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 0x5624d43d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb5433e8740 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43cab40 0x5624d43d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb5433e8740 1 --2- 172.21.15.94:0/3749771155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43c95d0 0x5624d43d3d60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 >> 172.21.15.94:0/3749771155 conn(0x5624d4392310 msgr2=0x5624d43cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.080+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 shutdown_connections 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.081+0000 7fb5433e8740 1 -- 172.21.15.94:0/3749771155 wait complete. 2024-09-19T04:29:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.081+0000 7fb5433e8740 20 client.17145 trim_cache size 0 max 16384 2024-09-19T04:29:16.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.101+0000 7fb5433e8740 1 Processor -- start 2024-09-19T04:29:16.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.101+0000 7fb5433e8740 1 -- start start 2024-09-19T04:29:16.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.101+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 0x5624d43c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 0x5624d43cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb5433e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c1adf90 con 0x5624d43d4370 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb5433e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c1ab830 con 0x5624d43cab40 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb5433e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c1a96e0 con 0x5624d43c95d0 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:36018/0 (socket says 172.21.15.94:36018) 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53a59c640 1 -- 172.21.15.94:0/1426257745 learned_addr learned my addr 172.21.15.94:0/1426257745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/1426257745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 0x5624d43c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53a59c640 1 --2- 172.21.15.94:0/1426257745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 0x5624d43cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53a59c640 1 -- 172.21.15.94:0/1426257745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 msgr2=0x5624d43c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53a59c640 1 --2- 172.21.15.94:0/1426257745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 0x5624d43c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53a59c640 1 -- 172.21.15.94:0/1426257745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 msgr2=0x5624d43cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:16.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53a59c640 1 --2- 172.21.15.94:0/1426257745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 0x5624d43cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.102+0000 7fb53a59c640 1 -- 172.21.15.94:0/1426257745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d43d1930 con 0x5624d43c95d0 2024-09-19T04:29:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.103+0000 7fb53a59c640 1 --2- 172.21.15.94:0/1426257745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43d08d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb530003690 tx=0x7fb530000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc7ed630a460078 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.103+0000 7fb520ff9640 1 -- 172.21.15.94:0/1426257745 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530035040 con 0x5624d43c95d0 2024-09-19T04:29:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.103+0000 7fb520ff9640 1 -- 172.21.15.94:0/1426257745 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb53003d660 con 0x5624d43c95d0 2024-09-19T04:29:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.103+0000 7fb520ff9640 1 -- 172.21.15.94:0/1426257745 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530004770 con 0x5624d43c95d0 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 -- 172.21.15.94:0/1426257745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 msgr2=0x5624d43d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 --2- 172.21.15.94:0/1426257745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43d08d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb530003690 tx=0x7fb530000bc0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 -- 172.21.15.94:0/1426257745 shutdown_connections 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 --2- 172.21.15.94:0/1426257745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43d4370 0x5624d43cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 --2- 172.21.15.94:0/1426257745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43cab40 0x5624d43c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 --2- 172.21.15.94:0/1426257745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43d08d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 -- 172.21.15.94:0/1426257745 >> 172.21.15.94:0/1426257745 conn(0x5624d4392310 msgr2=0x5624d43c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 -- 172.21.15.94:0/1426257745 shutdown_connections 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 -- 172.21.15.94:0/1426257745 wait complete. 2024-09-19T04:29:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 -- start start 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.104+0000 7fb5433e8740 1 Processor -- start 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.105+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43dc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.105+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43cab40 0x5624d43dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.105+0000 7fb5433e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 0x5624d43cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.105+0000 7fb5433e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c1a9d30 con 0x5624d43cab40 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.105+0000 7fb5433e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c1ab160 con 0x5624d43d4370 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.105+0000 7fb53ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43cab40 0x5624d43dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43cab40 0x5624d43dcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51948/0 (socket says 172.21.15.94:51948) 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53ad9d640 1 -- 172.21.15.94:0/832865711 learned_addr learned my addr 172.21.15.94:0/832865711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb5433e8740 1 -- 172.21.15.94:0/832865711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c1aa3c0 con 0x5624d43c95d0 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb5433e8740 10 client.0 _ll_register_callbacks cb 0x7fb52c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53a59c640 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43dc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53a59c640 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 0x5624d43cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53ad9d640 1 -- 172.21.15.94:0/832865711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 msgr2=0x5624d43dc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43dc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53ad9d640 1 -- 172.21.15.94:0/832865711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 msgr2=0x5624d43cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 0x5624d43cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53ad9d640 1 -- 172.21.15.94:0/832865711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d43ce130 con 0x5624d43cab40 2024-09-19T04:29:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.106+0000 7fb53a59c640 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 0x5624d43cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.107+0000 7fb53ad9d640 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43cab40 0x5624d43dcc00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb52c01c040 tx=0x7fb52c00ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91df287edc3d12e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.107+0000 7fb51eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:29:16.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.108+0000 7fb51eff5640 1 -- 172.21.15.94:0/832865711 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c01e040 con 0x5624d43cab40 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.108+0000 7fb5433e8740 1 -- 172.21.15.94:0/832865711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb53003d660 con 0x5624d43cab40 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.108+0000 7fb51eff5640 1 -- 172.21.15.94:0/832865711 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fb52c009c90 con 0x5624d43cab40 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.108+0000 7fb51eff5640 1 -- 172.21.15.94:0/832865711 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fb52c019410 con 0x5624d43cab40 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.108+0000 7fb5433e8740 10 client.17151 fetch_fsmap learned FSMap version 53 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.108+0000 7fb5433e8740 1 -- 172.21.15.94:0/832865711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5624d444e810 con 0x5624d43cab40 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.109+0000 7fb51eff5640 1 -- 172.21.15.94:0/832865711 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52c0195b0 con 0x5624d43cab40 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.110+0000 7fb51eff5640 1 -- 172.21.15.94:0/832865711 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 53) ==== 8350+0+0 (secure 0 0 0) 0x7fb52c0198e0 con 0x5624d43cab40 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.110+0000 7fb5433e8740 10 client.17151 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.110+0000 7fb5433e8740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.110+0000 7fb5433e8740 20 client.17151 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.110+0000 7fb5433e8740 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5624d43e3f70 0x5624d4450db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.110+0000 7fb5433e8740 4 client.17151 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:29:16.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.110+0000 7fb5433e8740 1 -- 172.21.15.94:0/832865711 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb53003d660 con 0x5624d43e3f70 2024-09-19T04:29:16.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.110+0000 7fb53a59c640 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5624d43e3f70 0x5624d4450db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.111+0000 7fb53a59c640 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5624d43e3f70 0x5624d4450db0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73085ff30986d37b server_cookie=9d86c4f2ffbfb45e in_seq=0 out_seq=0 2024-09-19T04:29:16.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.111+0000 7fb51eff5640 10 client.17151 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:16.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.111+0000 7fb51eff5640 1 -- 172.21.15.94:0/832865711 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb53003d660 con 0x5624d43e3f70 2024-09-19T04:29:16.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.111+0000 7fb51eff5640 10 client.17151 handle_command_reply: tid=0 2024-09-19T04:29:16.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.111+0000 7fb5433e8740 1 client.17151 shutdown 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 -- 172.21.15.94:0/832865711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43cab40 msgr2=0x5624d43dcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43cab40 0x5624d43dcc00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb52c01c040 tx=0x7fb52c00ffa0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 -- 172.21.15.94:0/832865711 shutdown_connections 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5624d43e3f70 0x5624d4450db0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb51eff5640 0 client.17151 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d43d4370 0x5624d43cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624d43cab40 0x5624d43dcc00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 --2- 172.21.15.94:0/832865711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624d43c95d0 0x5624d43dc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 -- 172.21.15.94:0/832865711 >> 172.21.15.94:0/832865711 conn(0x5624d4392310 msgr2=0x5624d43c4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 -- 172.21.15.94:0/832865711 shutdown_connections 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 1 -- 172.21.15.94:0/832865711 wait complete. 2024-09-19T04:29:16.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.112+0000 7fb5433e8740 20 client.17151 trim_cache size 0 max 16384 2024-09-19T04:29:16.117 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 -- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5240a3430 msgr2=0x7fb5240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5240a3430 0x7fb5240a58f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5340a9a60 tx=0x7fb534005f70 comp rx=0 tx=0).stop 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 -- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c105de0 msgr2=0x7fb53c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c105de0 0x7fb53c19ee80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb534009fd0 tx=0x7fb53400ef10 comp rx=0 tx=0).stop 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 -- 172.21.15.94:0/2349131103 shutdown_connections 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb5240a3430 0x7fb5240a58f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb53c105de0 0x7fb53c19ee80 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb53c101e20 0x7fb53c19acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 --2- 172.21.15.94:0/2349131103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53c101470 0x7fb53c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 -- 172.21.15.94:0/2349131103 >> 172.21.15.94:0/2349131103 conn(0x7fb53c077230 msgr2=0x7fb53c103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 -- 172.21.15.94:0/2349131103 shutdown_connections 2024-09-19T04:29:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:16.113+0000 7fb54100f640 1 -- 172.21.15.94:0/2349131103 wait complete. 2024-09-19T04:29:17.086 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:16 smithi112 ceph-mon[35051]: pgmap v446: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 4.0 MiB/s wr, 545 op/s 2024-09-19T04:29:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:16 smithi094 ceph-mon[30972]: pgmap v446: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 4.0 MiB/s wr, 545 op/s 2024-09-19T04:29:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:16 smithi104 ceph-mon[34145]: pgmap v446: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 4.0 MiB/s wr, 545 op/s 2024-09-19T04:29:17.882 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:29:17.882 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:29:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:18.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.771+0000 7f24623a2640 1 -- 172.21.15.94:0/626402370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c101270 msgr2=0x7f245c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.771+0000 7f24623a2640 1 --2- 172.21.15.94:0/626402370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c101270 0x7f245c10d7a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2454009cd0 tx=0x7f245402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.771+0000 7f24623a2640 1 -- 172.21.15.94:0/626402370 shutdown_connections 2024-09-19T04:29:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.771+0000 7f24623a2640 1 --2- 172.21.15.94:0/626402370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c101270 0x7f245c10d7a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.771+0000 7f24623a2640 1 --2- 172.21.15.94:0/626402370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c1008d0 0x7f245c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.771+0000 7f24623a2640 1 --2- 172.21.15.94:0/626402370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c1068d0 0x7f245c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.773+0000 7f24623a2640 1 -- 172.21.15.94:0/626402370 >> 172.21.15.94:0/626402370 conn(0x7f245c0fc630 msgr2=0x7f245c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.773+0000 7f24623a2640 1 -- 172.21.15.94:0/626402370 shutdown_connections 2024-09-19T04:29:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.773+0000 7f24623a2640 1 -- 172.21.15.94:0/626402370 wait complete. 2024-09-19T04:29:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.773+0000 7f24623a2640 1 Processor -- start 2024-09-19T04:29:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.773+0000 7f24623a2640 1 -- start start 2024-09-19T04:29:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.773+0000 7f24623a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c1008d0 0x7f245c197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.773+0000 7f24623a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c101270 0x7f245c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24613a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c1008d0 0x7f245c197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24613a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c1008d0 0x7f245c197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:36050/0 (socket says 172.21.15.94:36050) 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24613a0640 1 -- 172.21.15.94:0/1816047676 learned_addr learned my addr 172.21.15.94:0/1816047676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f2460b9f640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c101270 0x7f245c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24623a2640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c1068d0 0x7f245c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f245c110b80 con 0x7f245c101270 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f245c110a00 con 0x7f245c1068d0 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f245c110d00 con 0x7f245c1008d0 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f2461ba1640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c1068d0 0x7f245c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24613a0640 1 -- 172.21.15.94:0/1816047676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c1068d0 msgr2=0x7f245c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24613a0640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c1068d0 0x7f245c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24613a0640 1 -- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c101270 msgr2=0x7f245c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:18.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24613a0640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c101270 0x7f245c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24613a0640 1 -- 172.21.15.94:0/1816047676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c19c030 con 0x7f245c1008d0 2024-09-19T04:29:18.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.774+0000 7f24613a0640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c1008d0 0x7f245c197080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f245800e9f0 tx=0x7f245800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:18.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.775+0000 7f24527fc640 1 -- 172.21.15.94:0/1816047676 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245800cd80 con 0x7f245c1008d0 2024-09-19T04:29:18.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.775+0000 7f24527fc640 1 -- 172.21.15.94:0/1816047676 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24580108e0 con 0x7f245c1008d0 2024-09-19T04:29:18.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.775+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c19c320 con 0x7f245c1008d0 2024-09-19T04:29:18.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.775+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c06a790 con 0x7f245c1008d0 2024-09-19T04:29:18.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.775+0000 7f24527fc640 1 -- 172.21.15.94:0/1816047676 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458018850 con 0x7f245c1008d0 2024-09-19T04:29:18.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.778+0000 7f24527fc640 1 -- 172.21.15.94:0/1816047676 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2458010a90 con 0x7f245c1008d0 2024-09-19T04:29:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.778+0000 7f24659dd740 1 Processor -- start 2024-09-19T04:29:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24659dd740 1 -- start start 2024-09-19T04:29:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24527fc640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24480a3500 0x7f24480a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24659dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a25560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24659dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25aa0 0x55eda0a2fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24659dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a30360 0x55eda0a32750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24659dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55eda0986280 con 0x55eda0a30360 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24659dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eda092db10 con 0x55eda0a25180 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24659dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eda0982240 con 0x55eda0a25aa0 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24527fc640 1 -- 172.21.15.94:0/1816047676 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7f24580cfec0 con 0x7f245c1008d0 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2461ba1640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24480a3500 0x7f24480a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2460b9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25aa0 0x55eda0a2fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2460b9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25aa0 0x55eda0a2fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:36058/0 (socket says 172.21.15.94:36058) 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2460b9f640 1 -- 172.21.15.94:0/2997068077 learned_addr learned my addr 172.21.15.94:0/2997068077 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24613a0640 1 --2- 172.21.15.94:0/2997068077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a25560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2460b9f640 1 -- 172.21.15.94:0/2997068077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 msgr2=0x55eda0a25560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2460b9f640 1 --2- 172.21.15.94:0/2997068077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a25560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2460b9f640 1 -- 172.21.15.94:0/2997068077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a30360 msgr2=0x55eda0a32750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2460b9f640 1 --2- 172.21.15.94:0/2997068077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a30360 0x55eda0a32750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2460b9f640 1 -- 172.21.15.94:0/2997068077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda093a1f0 con 0x55eda0a25aa0 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24613a0640 1 --2- 172.21.15.94:0/2997068077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a25560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2461ba1640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24480a3500 0x7f24480a59c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2454006010 tx=0x7f2454009940 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f2460b9f640 1 --2- 172.21.15.94:0/2997068077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25aa0 0x55eda0a2fe20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f244c009c10 tx=0x7f244c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d481f643f0125e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24477fe640 1 -- 172.21.15.94:0/2997068077 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c01b070 con 0x55eda0a25aa0 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24477fe640 1 -- 172.21.15.94:0/2997068077 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f244c0178b0 con 0x55eda0a25aa0 2024-09-19T04:29:18.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.780+0000 7f24477fe640 1 -- 172.21.15.94:0/2997068077 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c016870 con 0x55eda0a25aa0 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.781+0000 7f24659dd740 1 -- 172.21.15.94:0/2997068077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25aa0 msgr2=0x55eda0a2fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.781+0000 7f24659dd740 1 --2- 172.21.15.94:0/2997068077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25aa0 0x55eda0a2fe20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f244c009c10 tx=0x7f244c005f90 comp rx=0 tx=0).stop 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.781+0000 7f24659dd740 1 -- 172.21.15.94:0/2997068077 shutdown_connections 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.781+0000 7f24659dd740 1 --2- 172.21.15.94:0/2997068077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a30360 0x55eda0a32750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.781+0000 7f24659dd740 1 --2- 172.21.15.94:0/2997068077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25aa0 0x55eda0a2fe20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.781+0000 7f24659dd740 1 --2- 172.21.15.94:0/2997068077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a25560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.781+0000 7f24659dd740 1 -- 172.21.15.94:0/2997068077 >> 172.21.15.94:0/2997068077 conn(0x55eda0a21710 msgr2=0x55eda0a21af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.781+0000 7f24659dd740 1 -- 172.21.15.94:0/2997068077 shutdown_connections 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.781+0000 7f24659dd740 1 -- 172.21.15.94:0/2997068077 wait complete. 2024-09-19T04:29:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.782+0000 7f24659dd740 1 -- start start 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.782+0000 7f24659dd740 1 Processor -- start 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.782+0000 7f24659dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25180 0x55eda0a2d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.782+0000 7f24659dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f24659dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a30360 0x55eda0a2dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f24659dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55eda0a32f30 con 0x55eda0a25aa0 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2460b9f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f24613a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25180 0x55eda0a2d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2460b9f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51976/0 (socket says 172.21.15.94:51976) 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f24613a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25180 0x55eda0a2d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:36062/0 (socket says 172.21.15.94:36062) 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2460b9f640 1 -- 172.21.15.94:0/2017802202 learned_addr learned my addr 172.21.15.94:0/2017802202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eda0940f10 con 0x55eda0a30360 2024-09-19T04:29:18.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eda0a330b0 con 0x55eda0a25180 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2461ba1640 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a30360 0x55eda0a2dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f24659dd740 10 client.0 _ll_register_callbacks cb 0x55eda0909a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2461ba1640 1 -- 172.21.15.94:0/2017802202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25180 msgr2=0x55eda0a2d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2461ba1640 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25180 0x55eda0a2d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2461ba1640 1 -- 172.21.15.94:0/2017802202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 msgr2=0x55eda0a2d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2461ba1640 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2461ba1640 1 -- 172.21.15.94:0/2017802202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda06f76a0 con 0x55eda0a30360 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2460b9f640 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2461ba1640 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a30360 0x55eda0a2dc80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2454009ca0 tx=0x7f245402a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67cc60a316890ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.783+0000 7f2444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f2444ff9640 1 -- 172.21.15.94:0/2017802202 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245404c070 con 0x55eda0a30360 2024-09-19T04:29:18.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f2444ff9640 1 -- 172.21.15.94:0/2017802202 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7f2454004580 con 0x55eda0a30360 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f244c0178b0 con 0x55eda0a30360 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f2444ff9640 1 -- 172.21.15.94:0/2017802202 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f2454043070 con 0x55eda0a30360 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f24659dd740 10 client.26780 fetch_fsmap learned FSMap version 53 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55eda0a3eb00 con 0x55eda0a30360 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f2444ff9640 1 -- 172.21.15.94:0/2017802202 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f244c0178b0 con 0x55eda0a30360 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f2444ff9640 1 -- 172.21.15.94:0/2017802202 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 53) ==== 8350+0+0 (secure 0 0 0) 0x7f2454030070 con 0x55eda0a30360 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f24659dd740 10 client.26780 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f24659dd740 10 client.26780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.784+0000 7f24659dd740 20 client.26780 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.785+0000 7f24659dd740 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55eda0a40080 0x55eda0a42520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.785+0000 7f24659dd740 4 client.26780 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.785+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eda09937f0 con 0x55eda0a40080 2024-09-19T04:29:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.785+0000 7f24613a0640 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55eda0a40080 0x55eda0a42520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.785+0000 7f2444ff9640 10 client.26780 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:18.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.785+0000 7f24613a0640 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55eda0a40080 0x55eda0a42520 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe21b2e8b44ec92a server_cookie=565782bbbd335d82 in_seq=0 out_seq=0 2024-09-19T04:29:18.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f2444ff9640 1 -- 172.21.15.94:0/2017802202 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eda09937f0 con 0x55eda0a40080 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f2444ff9640 10 client.26780 handle_command_reply: tid=0 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 client.26780 shutdown 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a30360 msgr2=0x55eda0a2dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a30360 0x55eda0a2dc80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2454009ca0 tx=0x7f245402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 shutdown_connections 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55eda0a40080 0x55eda0a42520 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a30360 0x55eda0a2dc80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 --2- 172.21.15.94:0/2017802202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a25180 0x55eda0a2d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 >> 172.21.15.94:0/2017802202 conn(0x55eda0a21710 msgr2=0x55eda0a1c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f2444ff9640 0 client.26780 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:18.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 shutdown_connections 2024-09-19T04:29:18.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 1 -- 172.21.15.94:0/2017802202 wait complete. 2024-09-19T04:29:18.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.786+0000 7f24659dd740 20 client.26780 trim_cache size 0 max 16384 2024-09-19T04:29:18.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24659dd740 1 Processor -- start 2024-09-19T04:29:18.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24659dd740 1 -- start start 2024-09-19T04:29:18.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24659dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25180 0x55eda0a27be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24659dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25aa0 0x55eda0a2f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24659dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a47df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24659dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24400110d0 con 0x55eda0a25180 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24659dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f244000e970 con 0x55eda0a25aa0 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24659dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f244000c7e0 con 0x55eda0a30360 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24613a0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25180 0x55eda0a27be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f24613a0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25180 0x55eda0a27be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51978/0 (socket says 172.21.15.94:51978) 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f2461ba1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a47df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f24613a0640 1 -- 172.21.15.94:0/1361456724 learned_addr learned my addr 172.21.15.94:0/1361456724 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.814+0000 7f2460b9f640 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25aa0 0x55eda0a2f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2460b9f640 1 -- 172.21.15.94:0/1361456724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 msgr2=0x55eda0a47df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2460b9f640 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a47df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2460b9f640 1 -- 172.21.15.94:0/1361456724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25180 msgr2=0x55eda0a27be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:18.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2460b9f640 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25180 0x55eda0a27be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2460b9f640 1 -- 172.21.15.94:0/1361456724 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda0a21540 con 0x55eda0a25aa0 2024-09-19T04:29:18.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2461ba1640 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a47df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:18.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f24613a0640 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25180 0x55eda0a27be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2460b9f640 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25aa0 0x55eda0a2f060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f244c009890 tx=0x7f244c009970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f81eb731f0a35b19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2446ffd640 1 -- 172.21.15.94:0/1361456724 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c01b040 con 0x55eda0a25aa0 2024-09-19T04:29:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2446ffd640 1 -- 172.21.15.94:0/1361456724 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f244c004620 con 0x55eda0a25aa0 2024-09-19T04:29:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.815+0000 7f2446ffd640 1 -- 172.21.15.94:0/1361456724 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c016760 con 0x55eda0a25aa0 2024-09-19T04:29:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.816+0000 7f24659dd740 1 -- 172.21.15.94:0/1361456724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25aa0 msgr2=0x55eda0a2f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:18.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.816+0000 7f24659dd740 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25aa0 0x55eda0a2f060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f244c009890 tx=0x7f244c009970 comp rx=0 tx=0).stop 2024-09-19T04:29:18.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.817+0000 7f24659dd740 1 -- 172.21.15.94:0/1361456724 shutdown_connections 2024-09-19T04:29:18.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.817+0000 7f24659dd740 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a47df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.817+0000 7f24659dd740 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25aa0 0x55eda0a2f060 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.817+0000 7f24659dd740 1 --2- 172.21.15.94:0/1361456724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25180 0x55eda0a27be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.817+0000 7f24659dd740 1 -- 172.21.15.94:0/1361456724 >> 172.21.15.94:0/1361456724 conn(0x55eda0a21710 msgr2=0x55eda0a2e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:18.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.818+0000 7f24659dd740 1 -- 172.21.15.94:0/1361456724 shutdown_connections 2024-09-19T04:29:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.818+0000 7f24659dd740 1 -- 172.21.15.94:0/1361456724 wait complete. 2024-09-19T04:29:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.818+0000 7f24659dd740 1 -- start start 2024-09-19T04:29:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24659dd740 1 Processor -- start 2024-09-19T04:29:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24659dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a2a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24659dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24659dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a2adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24659dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f244000ce30 con 0x55eda0a25aa0 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24659dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f244000e2a0 con 0x55eda0a25180 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24659dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f244000d500 con 0x55eda0a30360 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24659dd740 10 client.0 _ll_register_callbacks cb 0x7f2458005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f2461ba1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a2adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24613a0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a2a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f2461ba1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a2adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:36084/0 (socket says 172.21.15.94:36084) 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f2461ba1640 1 -- 172.21.15.94:0/2225998247 learned_addr learned my addr 172.21.15.94:0/2225998247 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.819+0000 7f24613a0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a2a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:34926/0 (socket says 172.21.15.94:34926) 2024-09-19T04:29:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2460b9f640 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2461ba1640 1 -- 172.21.15.94:0/2225998247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 msgr2=0x55eda0a2a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:18.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2461ba1640 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a2a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2461ba1640 1 -- 172.21.15.94:0/2225998247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 msgr2=0x55eda0a2a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2461ba1640 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2461ba1640 1 -- 172.21.15.94:0/2225998247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda0aa9ef0 con 0x55eda0a30360 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2460b9f640 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2461ba1640 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a2adf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2454048560 tx=0x7f245403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf0b71141444477 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f2444ff9640 1 -- 172.21.15.94:0/2225998247 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245404c040 con 0x55eda0a30360 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.820+0000 7f24659dd740 1 -- 172.21.15.94:0/2225998247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f244c004620 con 0x55eda0a30360 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f2444ff9640 1 -- 172.21.15.94:0/2225998247 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7f24540307b0 con 0x55eda0a30360 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f2444ff9640 1 -- 172.21.15.94:0/2225998247 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f2454004b40 con 0x55eda0a30360 2024-09-19T04:29:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f24659dd740 10 client.26755 fetch_fsmap learned FSMap version 53 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f24659dd740 1 -- 172.21.15.94:0/2225998247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55eda0aab210 con 0x55eda0a30360 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f2444ff9640 1 -- 172.21.15.94:0/2225998247 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f244c004620 con 0x55eda0a30360 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f2444ff9640 1 -- 172.21.15.94:0/2225998247 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 53) ==== 8350+0+0 (secure 0 0 0) 0x7f2454030040 con 0x55eda0a30360 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f24659dd740 10 client.26755 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f24659dd740 10 client.26755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f24659dd740 20 client.26755 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f24659dd740 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55eda0aab930 0x55eda0aadd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f24659dd740 4 client.26755 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.821+0000 7f24659dd740 1 -- 172.21.15.94:0/2225998247 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eda098dd00 con 0x55eda0aab930 2024-09-19T04:29:18.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.823+0000 7f24613a0640 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55eda0aab930 0x55eda0aadd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.823+0000 7f24613a0640 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55eda0aab930 0x55eda0aadd50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a13606554209ea server_cookie=731502ce8484397b in_seq=0 out_seq=0 2024-09-19T04:29:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.823+0000 7f2444ff9640 10 client.26755 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.824+0000 7f2444ff9640 1 -- 172.21.15.94:0/2225998247 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eda098dd00 con 0x55eda0aab930 2024-09-19T04:29:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.824+0000 7f2444ff9640 10 client.26755 handle_command_reply: tid=0 2024-09-19T04:29:18.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.824+0000 7f24659dd740 1 client.26755 shutdown 2024-09-19T04:29:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.824+0000 7f24659dd740 1 -- 172.21.15.94:0/2225998247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 msgr2=0x55eda0a2adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.824+0000 7f24659dd740 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a2adf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2454048560 tx=0x7f245403f000 comp rx=0 tx=0).stop 2024-09-19T04:29:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.824+0000 7f24659dd740 1 -- 172.21.15.94:0/2225998247 shutdown_connections 2024-09-19T04:29:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.824+0000 7f24659dd740 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55eda0aab930 0x55eda0aadd50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24659dd740 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eda0a30360 0x55eda0a2adf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24659dd740 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eda0a25aa0 0x55eda0a2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24659dd740 1 --2- 172.21.15.94:0/2225998247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eda0a25180 0x55eda0a2a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f2444ff9640 0 client.26755 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24659dd740 1 -- 172.21.15.94:0/2225998247 >> 172.21.15.94:0/2225998247 conn(0x55eda0a21710 msgr2=0x55eda0a1c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:18.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24659dd740 1 -- 172.21.15.94:0/2225998247 shutdown_connections 2024-09-19T04:29:18.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24659dd740 1 -- 172.21.15.94:0/2225998247 wait complete. 2024-09-19T04:29:18.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24659dd740 20 client.26755 trim_cache size 0 max 16384 2024-09-19T04:29:18.829 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:29:18.829 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:29:18.829 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "88cfa4cf-30cf-4a89-b392-4f8dc17cf1cf", 2024-09-19T04:29:18.829 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:29:18.830 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24480a3500 msgr2=0x7f24480a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24480a3500 0x7f24480a59c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2454006010 tx=0x7f2454009940 comp rx=0 tx=0).stop 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c1008d0 msgr2=0x7f245c197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c1008d0 0x7f245c197080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f245800e9f0 tx=0x7f245800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 shutdown_connections 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f24480a3500 0x7f24480a59c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f245c1068d0 0x7f245c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f245c101270 0x7f245c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 --2- 172.21.15.94:0/1816047676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c1008d0 0x7f245c197080 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:18.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.826+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 >> 172.21.15.94:0/1816047676 conn(0x7f245c0fc630 msgr2=0x7f245c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.828+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 shutdown_connections 2024-09-19T04:29:18.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:18.828+0000 7f24623a2640 1 -- 172.21.15.94:0/1816047676 wait complete. 2024-09-19T04:29:19.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:18 smithi094 ceph-mon[30972]: pgmap v447: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 4.2 MiB/s wr, 562 op/s 2024-09-19T04:29:19.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:18 smithi104 ceph-mon[34145]: pgmap v447: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 4.2 MiB/s wr, 562 op/s 2024-09-19T04:29:19.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:18 smithi112 ceph-mon[35051]: pgmap v447: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 4.2 MiB/s wr, 562 op/s 2024-09-19T04:29:19.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:29:19.933 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:29:20.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:19 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:29:20.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:29:20.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:29:20.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:29:20.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:29:20.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:29:20.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:19 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:19 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:20.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.504+0000 7fd1595fb640 1 -- 172.21.15.94:0/117601138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd154102d30 msgr2=0x7fd154103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:20.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.504+0000 7fd1595fb640 1 --2- 172.21.15.94:0/117601138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd154102d30 0x7fd154103110 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd144009930 tx=0x7fd14402a420 comp rx=0 tx=0).stop 2024-09-19T04:29:20.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.504+0000 7fd1595fb640 1 -- 172.21.15.94:0/117601138 shutdown_connections 2024-09-19T04:29:20.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.504+0000 7fd1595fb640 1 --2- 172.21.15.94:0/117601138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1541082a0 0x7fd15410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.504+0000 7fd1595fb640 1 --2- 172.21.15.94:0/117601138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1541036e0 0x7fd154107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.504+0000 7fd1595fb640 1 --2- 172.21.15.94:0/117601138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd154102d30 0x7fd154103110 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.504+0000 7fd1595fb640 1 -- 172.21.15.94:0/117601138 >> 172.21.15.94:0/117601138 conn(0x7fd1540fc7d0 msgr2=0x7fd1540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.504+0000 7fd1595fb640 1 -- 172.21.15.94:0/117601138 shutdown_connections 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.505+0000 7fd1595fb640 1 -- 172.21.15.94:0/117601138 wait complete. 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.505+0000 7fd1595fb640 1 Processor -- start 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd1595fb640 1 -- start start 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd1595fb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd154102d30 0x7fd154196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd1595fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1541036e0 0x7fd1541968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd1595fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1541082a0 0x7fd15419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd1595fb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd15410fd90 con 0x7fd1541036e0 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd1595fb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd15410fc10 con 0x7fd154102d30 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd1595fb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd15410ff10 con 0x7fd1541082a0 2024-09-19T04:29:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd154102d30 0x7fd154196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd154102d30 0x7fd154196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:34944/0 (socket says 172.21.15.94:34944) 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd152ffd640 1 -- 172.21.15.94:0/1036833431 learned_addr learned my addr 172.21.15.94:0/1036833431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd1537fe640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1541082a0 0x7fd15419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd1527fc640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1541036e0 0x7fd1541968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd152ffd640 1 -- 172.21.15.94:0/1036833431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1541082a0 msgr2=0x7fd15419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd152ffd640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1541082a0 0x7fd15419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.506+0000 7fd152ffd640 1 -- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1541036e0 msgr2=0x7fd1541968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.507+0000 7fd152ffd640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1541036e0 0x7fd1541968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.507+0000 7fd152ffd640 1 -- 172.21.15.94:0/1036833431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15419b320 con 0x7fd154102d30 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.507+0000 7fd152ffd640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd154102d30 0x7fd154196370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd144005ab0 tx=0x7fd144003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.507+0000 7fd13bfff640 1 -- 172.21.15.94:0/1036833431 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14403f070 con 0x7fd154102d30 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.507+0000 7fd13bfff640 1 -- 172.21.15.94:0/1036833431 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd144031b50 con 0x7fd154102d30 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.507+0000 7fd1595fb640 1 -- 172.21.15.94:0/1036833431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15419b5b0 con 0x7fd154102d30 2024-09-19T04:29:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.507+0000 7fd13bfff640 1 -- 172.21.15.94:0/1036833431 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144035030 con 0x7fd154102d30 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.507+0000 7fd1595fb640 1 -- 172.21.15.94:0/1036833431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1541a2ef0 con 0x7fd154102d30 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.510+0000 7fd13bfff640 1 -- 172.21.15.94:0/1036833431 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd144042070 con 0x7fd154102d30 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.510+0000 7fd15b9d4740 1 Processor -- start 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd15b9d4740 1 -- start start 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd13bfff640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd13c0a3430 0x7fd13c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225f13c0 0x55f4225f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd13bfff640 1 -- 172.21.15.94:0/1036833431 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fd1440f0350 con 0x7fd154102d30 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f422604950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd1537fe640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd13c0a3430 0x7fd13c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd1527fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f422604950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f422604e90 0x55f422607280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd1527fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f422604950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52018/0 (socket says 172.21.15.94:52018) 2024-09-19T04:29:20.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd1527fc640 1 -- 172.21.15.94:0/3867054257 learned_addr learned my addr 172.21.15.94:0/3867054257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd15b9d4740 1 -- 172.21.15.94:0/3867054257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f4225572e0 con 0x55f4225f1ce0 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd15b9d4740 1 -- 172.21.15.94:0/3867054257 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f4224fece0 con 0x55f4225f13c0 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd15b9d4740 1 -- 172.21.15.94:0/3867054257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4225532a0 con 0x55f422604e90 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd152ffd640 1 --2- 172.21.15.94:0/3867054257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225f13c0 0x55f4225f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd1527fc640 1 -- 172.21.15.94:0/3867054257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f422604e90 msgr2=0x55f422607280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd1527fc640 1 --2- 172.21.15.94:0/3867054257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f422604e90 0x55f422607280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd1527fc640 1 -- 172.21.15.94:0/3867054257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225f13c0 msgr2=0x55f4225f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd1527fc640 1 --2- 172.21.15.94:0/3867054257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225f13c0 0x55f4225f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.511+0000 7fd1527fc640 1 -- 172.21.15.94:0/3867054257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4224e3440 con 0x55f4225f1ce0 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.512+0000 7fd1527fc640 1 --2- 172.21.15.94:0/3867054257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f422604950 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd14800bf50 tx=0x7fd1480096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc275bcfc7715fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.512+0000 7fd1537fe640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd13c0a3430 0x7fd13c0a58f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005fd0 tx=0x7fd14c022820 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.512+0000 7fd1397fa640 1 -- 172.21.15.94:0/3867054257 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14801d070 con 0x55f4225f1ce0 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.512+0000 7fd1397fa640 1 -- 172.21.15.94:0/3867054257 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd148018930 con 0x55f4225f1ce0 2024-09-19T04:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.512+0000 7fd1397fa640 1 -- 172.21.15.94:0/3867054257 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148016780 con 0x55f4225f1ce0 2024-09-19T04:29:20.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.513+0000 7fd15b9d4740 1 -- 172.21.15.94:0/3867054257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 msgr2=0x55f422604950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:20.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.513+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/3867054257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f422604950 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd14800bf50 tx=0x7fd1480096d0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.513+0000 7fd15b9d4740 1 -- 172.21.15.94:0/3867054257 shutdown_connections 2024-09-19T04:29:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.513+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/3867054257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f422604e90 0x55f422607280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.513+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/3867054257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f422604950 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.513+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/3867054257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225f13c0 0x55f4225f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.513+0000 7fd15b9d4740 1 -- 172.21.15.94:0/3867054257 >> 172.21.15.94:0/3867054257 conn(0x55f4225eda30 msgr2=0x55f4225ede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.513+0000 7fd15b9d4740 1 -- 172.21.15.94:0/3867054257 shutdown_connections 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.513+0000 7fd15b9d4740 1 -- 172.21.15.94:0/3867054257 wait complete. 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd15b9d4740 1 -- start start 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd15b9d4740 1 Processor -- start 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4225fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 0x55f4225fc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f422604e90 0x55f4225fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd15b9d4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f4225f4320 con 0x55f4225f13c0 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd15b9d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f422512050 con 0x55f422604e90 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd15b9d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4225f44a0 con 0x55f4225f1ce0 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd15b9d4740 10 client.0 _ll_register_callbacks cb 0x55f422552160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.514+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4225fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4225fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52030/0 (socket says 172.21.15.94:52030) 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd152ffd640 1 -- 172.21.15.94:0/537099855 learned_addr learned my addr 172.21.15.94:0/537099855 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd1537fe640 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f422604e90 0x55f4225fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd1527fc640 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 0x55f4225fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd152ffd640 1 -- 172.21.15.94:0/537099855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 msgr2=0x55f4225fc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd152ffd640 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 0x55f4225fc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd152ffd640 1 -- 172.21.15.94:0/537099855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f422604e90 msgr2=0x55f4225fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd152ffd640 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f422604e90 0x55f4225fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd152ffd640 1 -- 172.21.15.94:0/537099855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4222c83c0 con 0x55f4225f13c0 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd1537fe640 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f422604e90 0x55f4225fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd152ffd640 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4225fbf20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b8380 tx=0x7fd144031f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661bc6a5baafcd52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd136ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd136ff5640 1 -- 172.21.15.94:0/537099855 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1440d4020 con 0x55f4225f13c0 2024-09-19T04:29:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd136ff5640 1 -- 172.21.15.94:0/537099855 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fd1440b8d10 con 0x55f4225f13c0 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.515+0000 7fd15b9d4740 1 -- 172.21.15.94:0/537099855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd148018930 con 0x55f4225f13c0 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.516+0000 7fd136ff5640 1 -- 172.21.15.94:0/537099855 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fd144049050 con 0x55f4225f13c0 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.516+0000 7fd15b9d4740 10 client.17175 fetch_fsmap learned FSMap version 53 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.516+0000 7fd15b9d4740 1 -- 172.21.15.94:0/537099855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55f42260acc0 con 0x55f4225f13c0 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.517+0000 7fd136ff5640 1 -- 172.21.15.94:0/537099855 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd148018930 con 0x55f4225f13c0 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.517+0000 7fd136ff5640 1 -- 172.21.15.94:0/537099855 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 53) ==== 8350+0+0 (secure 0 0 0) 0x7fd1440c4e60 con 0x55f4225f13c0 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.517+0000 7fd15b9d4740 10 client.17175 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.517+0000 7fd15b9d4740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.517+0000 7fd15b9d4740 20 client.17175 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.517+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f42260c010 0x55f42260e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.517+0000 7fd15b9d4740 4 client.17175 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.517+0000 7fd15b9d4740 1 -- 172.21.15.94:0/537099855 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f422564850 con 0x55f42260c010 2024-09-19T04:29:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.517+0000 7fd1527fc640 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f42260c010 0x55f42260e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.518+0000 7fd1527fc640 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f42260c010 0x55f42260e460 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73c903535507d3c server_cookie=650fb4b4c0044995 in_seq=0 out_seq=0 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.518+0000 7fd136ff5640 10 client.17175 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd136ff5640 1 -- 172.21.15.94:0/537099855 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f422564850 con 0x55f42260c010 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd136ff5640 10 client.17175 handle_command_reply: tid=0 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 client.17175 shutdown 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 -- 172.21.15.94:0/537099855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 msgr2=0x55f4225fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4225fbf20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b8380 tx=0x7fd144031f30 comp rx=0 tx=0).stop 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 -- 172.21.15.94:0/537099855 shutdown_connections 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f42260c010 0x55f42260e460 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f422604e90 0x55f4225fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 0x55f4225fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/537099855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4225fbf20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 -- 172.21.15.94:0/537099855 >> 172.21.15.94:0/537099855 conn(0x55f4225eda30 msgr2=0x55f4225f36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:20.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd136ff5640 0 client.17175 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:20.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.519+0000 7fd15b9d4740 1 -- 172.21.15.94:0/537099855 shutdown_connections 2024-09-19T04:29:20.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.520+0000 7fd15b9d4740 1 -- 172.21.15.94:0/537099855 wait complete. 2024-09-19T04:29:20.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.520+0000 7fd15b9d4740 20 client.17175 trim_cache size 0 max 16384 2024-09-19T04:29:20.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.546+0000 7fd15b9d4740 1 Processor -- start 2024-09-19T04:29:20.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.546+0000 7fd15b9d4740 1 -- start start 2024-09-19T04:29:20.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.546+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f422613d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.546+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4226142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.546+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 0x55f4225f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.546+0000 7fd15b9d4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541ae110 con 0x55f4225f13c0 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.546+0000 7fd15b9d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541ab9b0 con 0x55f4225eda30 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.546+0000 7fd15b9d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541a9820 con 0x55f4225f1ce0 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.546+0000 7fd1527fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4226142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd1527fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4226142d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52036/0 (socket says 172.21.15.94:52036) 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd1527fc640 1 -- 172.21.15.94:0/879837193 learned_addr learned my addr 172.21.15.94:0/879837193 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd152ffd640 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f422613d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd1537fe640 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 0x55f4225f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd152ffd640 1 -- 172.21.15.94:0/879837193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 msgr2=0x55f4225f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:20.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd152ffd640 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 0x55f4225f9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd152ffd640 1 -- 172.21.15.94:0/879837193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 msgr2=0x55f4226142d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:20.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd152ffd640 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4226142d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd152ffd640 1 -- 172.21.15.94:0/879837193 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4225ed860 con 0x55f4225eda30 2024-09-19T04:29:20.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd1527fc640 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4226142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:20.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd152ffd640 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f422613d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd1440bc630 tx=0x7fd144000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b29ce869ec5da13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:20.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd138ff9640 1 -- 172.21.15.94:0/879837193 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1440d4040 con 0x55f4225eda30 2024-09-19T04:29:20.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd138ff9640 1 -- 172.21.15.94:0/879837193 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1440b8d10 con 0x55f4225eda30 2024-09-19T04:29:20.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.547+0000 7fd138ff9640 1 -- 172.21.15.94:0/879837193 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1440c4dd0 con 0x55f4225eda30 2024-09-19T04:29:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.548+0000 7fd15b9d4740 1 -- 172.21.15.94:0/879837193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 msgr2=0x55f422613d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.548+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f422613d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd1440bc630 tx=0x7fd144000bc0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.548+0000 7fd15b9d4740 1 -- 172.21.15.94:0/879837193 shutdown_connections 2024-09-19T04:29:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.548+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f1ce0 0x55f4225f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.548+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f13c0 0x55f4226142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.548+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/879837193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f422613d90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.548+0000 7fd15b9d4740 1 -- 172.21.15.94:0/879837193 >> 172.21.15.94:0/879837193 conn(0x55f4226003c0 msgr2=0x55f4225ecb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 -- 172.21.15.94:0/879837193 shutdown_connections 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 -- 172.21.15.94:0/879837193 wait complete. 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 -- start start 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 Processor -- start 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f42261c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f13c0 0x55f42261c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f42266fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541a9e70 con 0x55f4225f1ce0 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541ab2e0 con 0x55f4225eda30 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541aa540 con 0x55f4225f13c0 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd15b9d4740 10 client.0 _ll_register_callbacks cb 0x7fd1440b89d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd1527fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f13c0 0x55f42261c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd1527fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f13c0 0x55f42261c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:36140/0 (socket says 172.21.15.94:36140) 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd1527fc640 1 -- 172.21.15.94:0/2496034646 learned_addr learned my addr 172.21.15.94:0/2496034646 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd152ffd640 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f42261c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.549+0000 7fd1537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f42266fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd152ffd640 1 -- 172.21.15.94:0/2496034646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f13c0 msgr2=0x55f42261c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd152ffd640 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f13c0 0x55f42261c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd152ffd640 1 -- 172.21.15.94:0/2496034646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 msgr2=0x55f42266fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd152ffd640 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f42266fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd152ffd640 1 -- 172.21.15.94:0/2496034646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4226701f0 con 0x55f4225eda30 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd1527fc640 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f13c0 0x55f42261c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd152ffd640 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f42261c0d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd1440bc630 tx=0x7fd144031f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2831c33f154c59bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd136ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.550+0000 7fd136ff5640 1 -- 172.21.15.94:0/2496034646 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1440d4070 con 0x55f4225eda30 2024-09-19T04:29:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.551+0000 7fd136ff5640 1 -- 172.21.15.94:0/2496034646 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fd1440d0de0 con 0x55f4225eda30 2024-09-19T04:29:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.551+0000 7fd15b9d4740 1 -- 172.21.15.94:0/2496034646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1440b8d10 con 0x55f4225eda30 2024-09-19T04:29:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.551+0000 7fd136ff5640 1 -- 172.21.15.94:0/2496034646 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fd1440c4fe0 con 0x55f4225eda30 2024-09-19T04:29:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.551+0000 7fd15b9d4740 10 client.26804 fetch_fsmap learned FSMap version 53 2024-09-19T04:29:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.551+0000 7fd15b9d4740 1 -- 172.21.15.94:0/2496034646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55f4226701f0 con 0x55f4225eda30 2024-09-19T04:29:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.551+0000 7fd136ff5640 1 -- 172.21.15.94:0/2496034646 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1440b8d10 con 0x55f4225eda30 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.552+0000 7fd136ff5640 1 -- 172.21.15.94:0/2496034646 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 53) ==== 8350+0+0 (secure 0 0 0) 0x7fd1440b85f0 con 0x55f4225eda30 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.552+0000 7fd15b9d4740 10 client.26804 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.552+0000 7fd15b9d4740 10 client.26804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.552+0000 7fd15b9d4740 20 client.26804 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.552+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f422604e90 0x55f422679880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.552+0000 7fd15b9d4740 4 client.26804 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.552+0000 7fd15b9d4740 1 -- 172.21.15.94:0/2496034646 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f42255ed60 con 0x55f422604e90 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.552+0000 7fd1527fc640 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f422604e90 0x55f422679880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd1527fc640 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f422604e90 0x55f422679880 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4119a23cefa646d server_cookie=12e8ef9529eed51e in_seq=0 out_seq=0 2024-09-19T04:29:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd136ff5640 10 client.26804 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd136ff5640 1 -- 172.21.15.94:0/2496034646 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f42255ed60 con 0x55f422604e90 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd136ff5640 10 client.26804 handle_command_reply: tid=0 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd15b9d4740 1 client.26804 shutdown 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd15b9d4740 1 -- 172.21.15.94:0/2496034646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 msgr2=0x55f42261c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f42261c0d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd1440bc630 tx=0x7fd144031f30 comp rx=0 tx=0).stop 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd15b9d4740 1 -- 172.21.15.94:0/2496034646 shutdown_connections 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55f422604e90 0x55f422679880 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.554+0000 7fd136ff5640 0 client.26804 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.553+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4225f1ce0 0x55f42266fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.554+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4225f13c0 0x55f42261c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.554+0000 7fd15b9d4740 1 --2- 172.21.15.94:0/2496034646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f4225eda30 0x55f42261c0d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.554+0000 7fd15b9d4740 1 -- 172.21.15.94:0/2496034646 >> 172.21.15.94:0/2496034646 conn(0x55f4226003c0 msgr2=0x55f4225f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.554+0000 7fd15b9d4740 1 -- 172.21.15.94:0/2496034646 shutdown_connections 2024-09-19T04:29:20.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.554+0000 7fd15b9d4740 1 -- 172.21.15.94:0/2496034646 wait complete. 2024-09-19T04:29:20.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.554+0000 7fd15b9d4740 20 client.26804 trim_cache size 0 max 16384 2024-09-19T04:29:20.560 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:29:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:29:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:29:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "88cfa4cf-30cf-4a89-b392-4f8dc17cf1cf": { 2024-09-19T04:29:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:29:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "88cfa4cf-30cf-4a89-b392-4f8dc17cf1cf", 2024-09-19T04:29:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:29:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:29:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:29:20.561 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:29:20.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 -- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd13c0a3430 msgr2=0x7fd13c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd13c0a3430 0x7fd13c0a58f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005fd0 tx=0x7fd14c022820 comp rx=0 tx=0).stop 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 -- 172.21.15.94:0/1036833431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd154102d30 msgr2=0x7fd154196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd154102d30 0x7fd154196370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd144005ab0 tx=0x7fd144003b90 comp rx=0 tx=0).stop 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 -- 172.21.15.94:0/1036833431 shutdown_connections 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd13c0a3430 0x7fd13c0a58f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1541082a0 0x7fd15419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1541036e0 0x7fd1541968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 --2- 172.21.15.94:0/1036833431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd154102d30 0x7fd154196370 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 -- 172.21.15.94:0/1036833431 >> 172.21.15.94:0/1036833431 conn(0x7fd1540fc7d0 msgr2=0x7fd154105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 -- 172.21.15.94:0/1036833431 shutdown_connections 2024-09-19T04:29:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:20.555+0000 7fd1595fb640 1 -- 172.21.15.94:0/1036833431 wait complete. 2024-09-19T04:29:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 785.0 s, 396.8 tps, lat 252.633 ms stddev 34.876 2024-09-19T04:29:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:20 smithi094 ceph-mon[30972]: pgmap v448: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 4.2 MiB/s wr, 584 op/s 2024-09-19T04:29:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:20 smithi104 ceph-mon[34145]: pgmap v448: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 4.2 MiB/s wr, 584 op/s 2024-09-19T04:29:21.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:21.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:20 smithi112 ceph-mon[35051]: pgmap v448: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 4.2 MiB/s wr, 584 op/s 2024-09-19T04:29:22.379 INFO:tasks.cephfs.filesystem:scrub status for tag:88cfa4cf-30cf-4a89-b392-4f8dc17cf1cf - {'path': '/', 'tag': '88cfa4cf-30cf-4a89-b392-4f8dc17cf1cf', 'options': 'recursive,force'} 2024-09-19T04:29:23.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:22 smithi094 ceph-mon[30972]: pgmap v449: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.9 MiB/s wr, 534 op/s 2024-09-19T04:29:23.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:22 smithi104 ceph-mon[34145]: pgmap v449: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.9 MiB/s wr, 534 op/s 2024-09-19T04:29:23.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:22 smithi112 ceph-mon[35051]: pgmap v449: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.9 MiB/s wr, 534 op/s 2024-09-19T04:29:25.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:24 smithi094 ceph-mon[30972]: pgmap v450: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 5.6 MiB/s wr, 758 op/s 2024-09-19T04:29:25.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:24 smithi104 ceph-mon[34145]: pgmap v450: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 5.6 MiB/s wr, 758 op/s 2024-09-19T04:29:25.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:24 smithi112 ceph-mon[35051]: pgmap v450: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 5.6 MiB/s wr, 758 op/s 2024-09-19T04:29:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 790.0 s, 377.8 tps, lat 264.484 ms stddev 61.607 2024-09-19T04:29:27.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:26 smithi094 ceph-mon[30972]: pgmap v451: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-09-19T04:29:27.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:26 smithi104 ceph-mon[34145]: pgmap v451: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-09-19T04:29:27.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:26 smithi112 ceph-mon[35051]: pgmap v451: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-09-19T04:29:28.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:27 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:29:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:27 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:29:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:27 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:29:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:27 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:29:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:27 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:29:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:27 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:29:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:27 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:29:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:27 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:29:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:27 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:29:29.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:28 smithi094 ceph-mon[30972]: pgmap v452: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.2 MiB/s wr, 592 op/s 2024-09-19T04:29:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:28 smithi104 ceph-mon[34145]: pgmap v452: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.2 MiB/s wr, 592 op/s 2024-09-19T04:29:29.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:28 smithi112 ceph-mon[35051]: pgmap v452: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.2 MiB/s wr, 592 op/s 2024-09-19T04:29:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 795.0 s, 400.0 tps, lat 248.453 ms stddev 28.311 2024-09-19T04:29:31.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:30 smithi094 ceph-mon[30972]: pgmap v453: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.7 MiB/s wr, 657 op/s 2024-09-19T04:29:31.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:30 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:29:31.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:30 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:29:31.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:30 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:29:31.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:30 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:29:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:30 smithi104 ceph-mon[34145]: pgmap v453: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.7 MiB/s wr, 657 op/s 2024-09-19T04:29:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:30 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:29:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:30 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:29:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:30 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:29:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:30 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:29:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:30 smithi112 ceph-mon[35051]: pgmap v453: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.7 MiB/s wr, 657 op/s 2024-09-19T04:29:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:30 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:29:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:30 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:29:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:30 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:29:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:30 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:29:33.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:32 smithi094 ceph-mon[30972]: pgmap v454: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 4.1 MiB/s wr, 560 op/s 2024-09-19T04:29:33.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:32 smithi104 ceph-mon[34145]: pgmap v454: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 4.1 MiB/s wr, 560 op/s 2024-09-19T04:29:33.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:32 smithi112 ceph-mon[35051]: pgmap v454: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 4.1 MiB/s wr, 560 op/s 2024-09-19T04:29:34.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:33 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:29:34.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:33 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:29:34.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:33 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:29:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:34 smithi104 ceph-mon[34145]: pgmap v455: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.0 MiB/s wr, 812 op/s 2024-09-19T04:29:35.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:34 smithi112 ceph-mon[35051]: pgmap v455: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.0 MiB/s wr, 812 op/s 2024-09-19T04:29:35.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:34 smithi094 ceph-mon[30972]: pgmap v455: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.0 MiB/s wr, 812 op/s 2024-09-19T04:29:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 800.0 s, 398.6 tps, lat 252.252 ms stddev 27.692 2024-09-19T04:29:37.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:36 smithi104 ceph-mon[34145]: pgmap v456: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 4.0 MiB/s wr, 540 op/s 2024-09-19T04:29:37.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:36 smithi112 ceph-mon[35051]: pgmap v456: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 4.0 MiB/s wr, 540 op/s 2024-09-19T04:29:37.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:36 smithi094 ceph-mon[30972]: pgmap v456: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 4.0 MiB/s wr, 540 op/s 2024-09-19T04:29:39.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:38 smithi112 ceph-mon[35051]: pgmap v457: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 4.6 MiB/s wr, 631 op/s 2024-09-19T04:29:39.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:38 smithi094 ceph-mon[30972]: pgmap v457: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 4.6 MiB/s wr, 631 op/s 2024-09-19T04:29:39.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:38 smithi104 ceph-mon[34145]: pgmap v457: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 4.6 MiB/s wr, 631 op/s 2024-09-19T04:29:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 805.0 s, 399.4 tps, lat 250.848 ms stddev 30.783 2024-09-19T04:29:41.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:40 smithi112 ceph-mon[35051]: pgmap v458: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 5.5 MiB/s wr, 735 op/s 2024-09-19T04:29:41.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:40 smithi094 ceph-mon[30972]: pgmap v458: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 5.5 MiB/s wr, 735 op/s 2024-09-19T04:29:41.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:40 smithi104 ceph-mon[34145]: pgmap v458: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 5.5 MiB/s wr, 735 op/s 2024-09-19T04:29:43.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:42 smithi112 ceph-mon[35051]: pgmap v459: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 3.9 MiB/s wr, 526 op/s 2024-09-19T04:29:43.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:42 smithi094 ceph-mon[30972]: pgmap v459: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 3.9 MiB/s wr, 526 op/s 2024-09-19T04:29:43.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:42 smithi104 ceph-mon[34145]: pgmap v459: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 3.9 MiB/s wr, 526 op/s 2024-09-19T04:29:45.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:44 smithi112 ceph-mon[35051]: pgmap v460: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.8 MiB/s wr, 784 op/s 2024-09-19T04:29:45.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:44 smithi094 ceph-mon[30972]: pgmap v460: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.8 MiB/s wr, 784 op/s 2024-09-19T04:29:45.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:44 smithi104 ceph-mon[34145]: pgmap v460: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.8 MiB/s wr, 784 op/s 2024-09-19T04:29:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 810.0 s, 421.4 tps, lat 237.003 ms stddev 24.685 2024-09-19T04:29:47.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:46 smithi112 ceph-mon[35051]: pgmap v461: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 4.0 MiB/s wr, 532 op/s 2024-09-19T04:29:47.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:46 smithi094 ceph-mon[30972]: pgmap v461: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 4.0 MiB/s wr, 532 op/s 2024-09-19T04:29:47.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:46 smithi104 ceph-mon[34145]: pgmap v461: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 4.0 MiB/s wr, 532 op/s 2024-09-19T04:29:49.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:48 smithi112 ceph-mon[35051]: pgmap v462: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.4 MiB/s wr, 708 op/s 2024-09-19T04:29:49.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:48 smithi094 ceph-mon[30972]: pgmap v462: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.4 MiB/s wr, 708 op/s 2024-09-19T04:29:49.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:48 smithi104 ceph-mon[34145]: pgmap v462: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.4 MiB/s wr, 708 op/s 2024-09-19T04:29:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 815.0 s, 429.6 tps, lat 233.332 ms stddev 24.543 2024-09-19T04:29:51.142 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:50 smithi094 ceph-mon[30972]: pgmap v463: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.7 MiB/s wr, 762 op/s 2024-09-19T04:29:51.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:50 smithi112 ceph-mon[35051]: pgmap v463: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.7 MiB/s wr, 762 op/s 2024-09-19T04:29:51.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:50 smithi104 ceph-mon[34145]: pgmap v463: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.7 MiB/s wr, 762 op/s 2024-09-19T04:29:52.379 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:29:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:53.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:52 smithi104 ceph-mon[34145]: pgmap v464: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.3 MiB/s wr, 578 op/s 2024-09-19T04:29:53.198 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:52 smithi094 ceph-mon[30972]: pgmap v464: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.3 MiB/s wr, 578 op/s 2024-09-19T04:29:53.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:52 smithi112 ceph-mon[35051]: pgmap v464: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.3 MiB/s wr, 578 op/s 2024-09-19T04:29:53.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.312+0000 7fc95773e640 1 -- 172.21.15.94:0/619541693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc950100100 msgr2=0x7fc950100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:53.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.312+0000 7fc95773e640 1 --2- 172.21.15.94:0/619541693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc950100100 0x7fc950100560 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc940009900 tx=0x7fc94002a410 comp rx=0 tx=0).stop 2024-09-19T04:29:53.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.313+0000 7fc95773e640 1 -- 172.21.15.94:0/619541693 shutdown_connections 2024-09-19T04:29:53.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.313+0000 7fc95773e640 1 --2- 172.21.15.94:0/619541693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc950100aa0 0x7fc95010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.313+0000 7fc95773e640 1 --2- 172.21.15.94:0/619541693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc950100100 0x7fc950100560 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.313+0000 7fc95773e640 1 --2- 172.21.15.94:0/619541693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc950106b80 0x7fc950106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.313+0000 7fc95773e640 1 -- 172.21.15.94:0/619541693 >> 172.21.15.94:0/619541693 conn(0x7fc950077230 msgr2=0x7fc9500fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.313+0000 7fc95773e640 1 -- 172.21.15.94:0/619541693 shutdown_connections 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.313+0000 7fc95773e640 1 -- 172.21.15.94:0/619541693 wait complete. 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.314+0000 7fc95773e640 1 Processor -- start 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.314+0000 7fc95773e640 1 -- start start 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.314+0000 7fc95773e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc950100100 0x7fc9501012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.314+0000 7fc95773e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc950100aa0 0x7fc950101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.314+0000 7fc95773e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc950106b80 0x7fc950101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.314+0000 7fc9554da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc950100100 0x7fc9501012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.314+0000 7fc9554da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc950100100 0x7fc9501012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48778/0 (socket says 172.21.15.94:48778) 2024-09-19T04:29:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc9554da640 1 -- 172.21.15.94:0/377385430 learned_addr learned my addr 172.21.15.94:0/377385430 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc955cdb640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc950106b80 0x7fc950101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc950111f50 con 0x7fc950106b80 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc950111dd0 con 0x7fc950100aa0 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9501120d0 con 0x7fc950100100 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc954cd9640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc950100aa0 0x7fc950101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc9554da640 1 -- 172.21.15.94:0/377385430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc950100aa0 msgr2=0x7fc950101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc9554da640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc950100aa0 0x7fc950101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc9554da640 1 -- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc950106b80 msgr2=0x7fc950101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc9554da640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc950106b80 0x7fc950101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc9554da640 1 -- 172.21.15.94:0/377385430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9501a68b0 con 0x7fc950100100 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc954cd9640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc950100aa0 0x7fc950101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.315+0000 7fc9554da640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc950100100 0x7fc9501012e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00ea60 tx=0x7fc94c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.316+0000 7fc9467fc640 1 -- 172.21.15.94:0/377385430 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00cde0 con 0x7fc950100100 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.316+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9501a6ba0 con 0x7fc950100100 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.316+0000 7fc9467fc640 1 -- 172.21.15.94:0/377385430 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc94c0035f0 con 0x7fc950100100 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.316+0000 7fc9467fc640 1 -- 172.21.15.94:0/377385430 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c010690 con 0x7fc950100100 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.316+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9501a7100 con 0x7fc950100100 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.319+0000 7fc9467fc640 1 -- 172.21.15.94:0/377385430 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc94c004020 con 0x7fc950100100 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.319+0000 7fc959b17740 1 Processor -- start 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc959b17740 1 -- start start 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc959b17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 0x562ee03d5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc959b17740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc959b17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03e0680 0x562ee03e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc959b17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562ee033b2e0 con 0x562ee03d54a0 2024-09-19T04:29:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc959b17740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562ee02e2ce0 con 0x562ee03d5dc0 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc959b17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562ee03372a0 con 0x562ee03e0680 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc955cdb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 0x562ee03d5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc954cd9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03e0680 0x562ee03e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc954cd9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03e0680 0x562ee03e2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48786/0 (socket says 172.21.15.94:48786) 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc954cd9640 1 -- 172.21.15.94:0/3874493423 learned_addr learned my addr 172.21.15.94:0/3874493423 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc9554da640 1 --2- 172.21.15.94:0/3874493423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.320+0000 7fc9467fc640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc93c0a33b0 0x7fc93c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.321+0000 7fc9554da640 1 -- 172.21.15.94:0/3874493423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03e0680 msgr2=0x562ee03e2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.321+0000 7fc9554da640 1 --2- 172.21.15.94:0/3874493423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03e0680 0x562ee03e2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.321+0000 7fc9554da640 1 -- 172.21.15.94:0/3874493423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 msgr2=0x562ee03d5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.321+0000 7fc9554da640 1 --2- 172.21.15.94:0/3874493423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 0x562ee03d5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.321+0000 7fc9554da640 1 -- 172.21.15.94:0/3874493423 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee02c7440 con 0x562ee03d5dc0 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.321+0000 7fc954cd9640 1 --2- 172.21.15.94:0/3874493423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03e0680 0x562ee03e2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.321+0000 7fc955cdb640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc93c0a33b0 0x7fc93c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.321+0000 7fc9467fc640 1 -- 172.21.15.94:0/377385430 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fc94c0d0010 con 0x7fc950100100 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.321+0000 7fc9554da640 1 --2- 172.21.15.94:0/3874493423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03e0140 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0cd720 tx=0x7fc94c091740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7641a0ad4432fc33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.322+0000 7fc93b7fe640 1 -- 172.21.15.94:0/3874493423 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c003b30 con 0x562ee03d5dc0 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.322+0000 7fc955cdb640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc93c0a33b0 0x7fc93c0a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc9501077c0 tx=0x7fc948014040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.322+0000 7fc93b7fe640 1 -- 172.21.15.94:0/3874493423 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc94c004310 con 0x562ee03d5dc0 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.322+0000 7fc93b7fe640 1 -- 172.21.15.94:0/3874493423 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c067100 con 0x562ee03d5dc0 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.323+0000 7fc959b17740 1 -- 172.21.15.94:0/3874493423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 msgr2=0x562ee03e0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.323+0000 7fc959b17740 1 --2- 172.21.15.94:0/3874493423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03e0140 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0cd720 tx=0x7fc94c091740 comp rx=0 tx=0).stop 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.323+0000 7fc959b17740 1 -- 172.21.15.94:0/3874493423 shutdown_connections 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.323+0000 7fc959b17740 1 --2- 172.21.15.94:0/3874493423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03e0680 0x562ee03e2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.323+0000 7fc959b17740 1 --2- 172.21.15.94:0/3874493423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03e0140 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.323+0000 7fc959b17740 1 --2- 172.21.15.94:0/3874493423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 0x562ee03d5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.323+0000 7fc959b17740 1 -- 172.21.15.94:0/3874493423 >> 172.21.15.94:0/3874493423 conn(0x562ee03d1a30 msgr2=0x562ee03d1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.323+0000 7fc959b17740 1 -- 172.21.15.94:0/3874493423 shutdown_connections 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.323+0000 7fc959b17740 1 -- 172.21.15.94:0/3874493423 wait complete. 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.324+0000 7fc959b17740 1 -- start start 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.324+0000 7fc959b17740 1 Processor -- start 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.324+0000 7fc959b17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 0x562ee03dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc959b17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d5dc0 0x562ee03dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc959b17740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc959b17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562ee03e31f0 con 0x562ee03d54a0 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc959b17740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562ee02f6050 con 0x562ee03e0680 2024-09-19T04:29:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc959b17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562ee03e3370 con 0x562ee03d5dc0 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc959b17740 10 client.0 _ll_register_callbacks cb 0x562ee0336160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc954cd9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d5dc0 0x562ee03dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc9554da640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 0x562ee03dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc954cd9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d5dc0 0x562ee03dd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48794/0 (socket says 172.21.15.94:48794) 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc9554da640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 0x562ee03dffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57854/0 (socket says 172.21.15.94:57854) 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc954cd9640 1 -- 172.21.15.94:0/2903012695 learned_addr learned my addr 172.21.15.94:0/2903012695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc955cdb640 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc954cd9640 1 -- 172.21.15.94:0/2903012695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 msgr2=0x562ee03dd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc954cd9640 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03dd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc954cd9640 1 -- 172.21.15.94:0/2903012695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 msgr2=0x562ee03dffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc954cd9640 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 0x562ee03dffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc954cd9640 1 -- 172.21.15.94:0/2903012695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee00ac3c0 con 0x562ee03d5dc0 2024-09-19T04:29:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.325+0000 7fc955cdb640 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03dd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.326+0000 7fc954cd9640 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d5dc0 0x562ee03dd280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc94002aa00 tx=0x7fc940005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=730a9240a9e70664 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.326+0000 7fc938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.326+0000 7fc938ff9640 1 -- 172.21.15.94:0/2903012695 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940035040 con 0x562ee03d5dc0 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.326+0000 7fc959b17740 1 -- 172.21.15.94:0/2903012695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ee0348850 con 0x562ee03d5dc0 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.326+0000 7fc938ff9640 1 -- 172.21.15.94:0/2903012695 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fc9400048c0 con 0x562ee03d5dc0 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.326+0000 7fc938ff9640 1 -- 172.21.15.94:0/2903012695 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc940004ac0 con 0x562ee03d5dc0 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.326+0000 7fc959b17740 10 client.26782 fetch_fsmap learned FSMap version 54 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.326+0000 7fc959b17740 1 -- 172.21.15.94:0/2903012695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562ee03f7420 con 0x562ee03d5dc0 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.326+0000 7fc938ff9640 1 -- 172.21.15.94:0/2903012695 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562ee0348850 con 0x562ee03d5dc0 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.327+0000 7fc938ff9640 1 -- 172.21.15.94:0/2903012695 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fc9400314a0 con 0x562ee03d5dc0 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.327+0000 7fc959b17740 10 client.26782 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:29:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.327+0000 7fc959b17740 10 client.26782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.327+0000 7fc959b17740 20 client.26782 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.327+0000 7fc959b17740 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x562ee03f88e0 0x562ee03fad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.327+0000 7fc959b17740 4 client.26782 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.327+0000 7fc959b17740 1 -- 172.21.15.94:0/2903012695 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ee0342d60 con 0x562ee03f88e0 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.328+0000 7fc9554da640 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x562ee03f88e0 0x562ee03fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.328+0000 7fc9554da640 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x562ee03f88e0 0x562ee03fad00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98203897764849e1 server_cookie=793e1ff6d6526702 in_seq=0 out_seq=0 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.328+0000 7fc938ff9640 10 client.26782 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc938ff9640 1 -- 172.21.15.94:0/2903012695 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ee0342d60 con 0x562ee03f88e0 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc938ff9640 10 client.26782 handle_command_reply: tid=0 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc959b17740 1 client.26782 shutdown 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc959b17740 1 -- 172.21.15.94:0/2903012695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d5dc0 msgr2=0x562ee03dd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc959b17740 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d5dc0 0x562ee03dd280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc94002aa00 tx=0x7fc940005270 comp rx=0 tx=0).stop 2024-09-19T04:29:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc959b17740 1 -- 172.21.15.94:0/2903012695 shutdown_connections 2024-09-19T04:29:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc959b17740 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x562ee03f88e0 0x562ee03fad00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc959b17740 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03dd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc959b17740 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d5dc0 0x562ee03dd280 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc959b17740 1 --2- 172.21.15.94:0/2903012695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d54a0 0x562ee03dffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc959b17740 1 -- 172.21.15.94:0/2903012695 >> 172.21.15.94:0/2903012695 conn(0x562ee03d1a30 msgr2=0x562ee03d7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.329+0000 7fc938ff9640 0 client.26782 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.330+0000 7fc959b17740 1 -- 172.21.15.94:0/2903012695 shutdown_connections 2024-09-19T04:29:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.330+0000 7fc959b17740 1 -- 172.21.15.94:0/2903012695 wait complete. 2024-09-19T04:29:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.330+0000 7fc959b17740 20 client.26782 trim_cache size 0 max 16384 2024-09-19T04:29:53.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.350+0000 7fc959b17740 1 Processor -- start 2024-09-19T04:29:53.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.350+0000 7fc959b17740 1 -- start start 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc959b17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc959b17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d5dc0 0x562ee03dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc959b17740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc959b17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9501adf20 con 0x562ee03d5dc0 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc959b17740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9501ab7f0 con 0x562ee03e0680 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc959b17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9501a9660 con 0x562ee03d54a0 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc954cd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d5dc0 0x562ee03dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc9554da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc9554da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48808/0 (socket says 172.21.15.94:48808) 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc954cd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d5dc0 0x562ee03dbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57862/0 (socket says 172.21.15.94:57862) 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc9554da640 1 -- 172.21.15.94:0/3071090984 learned_addr learned my addr 172.21.15.94:0/3071090984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc955cdb640 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc9554da640 1 -- 172.21.15.94:0/3071090984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 msgr2=0x562ee03d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc9554da640 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc9554da640 1 -- 172.21.15.94:0/3071090984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d5dc0 msgr2=0x562ee03dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc9554da640 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d5dc0 0x562ee03dbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc9554da640 1 -- 172.21.15.94:0/3071090984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee03dc230 con 0x562ee03d54a0 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc954cd9640 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d5dc0 0x562ee03dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc955cdb640 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.351+0000 7fc9554da640 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03f4ce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc94c000c00 tx=0x7fc94c0a07e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25168fc344410f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.352+0000 7fc93affd640 1 -- 172.21.15.94:0/3071090984 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0abe80 con 0x562ee03d54a0 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.352+0000 7fc93affd640 1 -- 172.21.15.94:0/3071090984 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc94c003b30 con 0x562ee03d54a0 2024-09-19T04:29:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.352+0000 7fc93affd640 1 -- 172.21.15.94:0/3071090984 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c098920 con 0x562ee03d54a0 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.352+0000 7fc959b17740 1 -- 172.21.15.94:0/3071090984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 msgr2=0x562ee03f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.352+0000 7fc959b17740 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03f4ce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc94c000c00 tx=0x7fc94c0a07e0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.353+0000 7fc959b17740 1 -- 172.21.15.94:0/3071090984 shutdown_connections 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.353+0000 7fc959b17740 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03e0680 0x562ee03d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.353+0000 7fc959b17740 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03d5dc0 0x562ee03dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.353+0000 7fc959b17740 1 --2- 172.21.15.94:0/3071090984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03f4ce0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.353+0000 7fc959b17740 1 -- 172.21.15.94:0/3071090984 >> 172.21.15.94:0/3071090984 conn(0x562ee03d1a30 msgr2=0x562ee03d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.353+0000 7fc959b17740 1 -- 172.21.15.94:0/3071090984 shutdown_connections 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.353+0000 7fc959b17740 1 -- 172.21.15.94:0/3071090984 wait complete. 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.354+0000 7fc959b17740 1 -- start start 2024-09-19T04:29:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.354+0000 7fc959b17740 1 Processor -- start 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.354+0000 7fc959b17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03eceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.354+0000 7fc959b17740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03ed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc959b17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03e0680 0x562ee03ed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc954cd9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03ed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc954cd9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03ed3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44924/0 (socket says 172.21.15.94:44924) 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc9554da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03eceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc9554da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03eceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48822/0 (socket says 172.21.15.94:48822) 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc955cdb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03e0680 0x562ee03ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc955cdb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03e0680 0x562ee03ed970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57866/0 (socket says 172.21.15.94:57866) 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc954cd9640 1 -- 172.21.15.94:0/3352038807 learned_addr learned my addr 172.21.15.94:0/3352038807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9501a9cb0 con 0x562ee03e0680 2024-09-19T04:29:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9501ab120 con 0x562ee03d5dc0 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9501aa380 con 0x562ee03d54a0 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc959b17740 10 client.0 _ll_register_callbacks cb 0x7fc9480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc954cd9640 1 -- 172.21.15.94:0/3352038807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 msgr2=0x562ee03eceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc954cd9640 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03eceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc954cd9640 1 -- 172.21.15.94:0/3352038807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03e0680 msgr2=0x562ee03ed970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc954cd9640 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03e0680 0x562ee03ed970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.355+0000 7fc954cd9640 1 -- 172.21.15.94:0/3352038807 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee0462890 con 0x562ee03d5dc0 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.356+0000 7fc954cd9640 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03ed3f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc9400318f0 tx=0x7fc940031d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25fefabaf602d554 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.356+0000 7fc938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.356+0000 7fc938ff9640 1 -- 172.21.15.94:0/3352038807 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940051070 con 0x562ee03d5dc0 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.356+0000 7fc938ff9640 1 -- 172.21.15.94:0/3352038807 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fc9400048c0 con 0x562ee03d5dc0 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.356+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ee02bf210 con 0x562ee03d5dc0 2024-09-19T04:29:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.356+0000 7fc938ff9640 1 -- 172.21.15.94:0/3352038807 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc940035040 con 0x562ee03d5dc0 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.356+0000 7fc959b17740 10 client.26822 fetch_fsmap learned FSMap version 54 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.356+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562ee0463b40 con 0x562ee03d5dc0 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.356+0000 7fc938ff9640 1 -- 172.21.15.94:0/3352038807 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562ee02bf210 con 0x562ee03d5dc0 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc938ff9640 1 -- 172.21.15.94:0/3352038807 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fc940003aa0 con 0x562ee03d5dc0 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc959b17740 10 client.26822 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc959b17740 10 client.26822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc959b17740 20 client.26822 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc959b17740 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x562ee0464210 0x562ee0466630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc959b17740 4 client.26822 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ee030c9a0 con 0x562ee0464210 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc9554da640 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x562ee0464210 0x562ee0466630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc938ff9640 10 client.26822 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.357+0000 7fc9554da640 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x562ee0464210 0x562ee0466630 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef84f416c55a1b79 server_cookie=4a1bcf01a6f8ed8c in_seq=0 out_seq=0 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc938ff9640 1 -- 172.21.15.94:0/3352038807 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562ee030c9a0 con 0x562ee0464210 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc938ff9640 10 client.26822 handle_command_reply: tid=0 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 client.26822 shutdown 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 msgr2=0x562ee03ed3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03ed3f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc9400318f0 tx=0x7fc940031d10 comp rx=0 tx=0).stop 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 shutdown_connections 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x562ee0464210 0x562ee0466630 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ee03e0680 0x562ee03ed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ee03d5dc0 0x562ee03ed3f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc938ff9640 0 client.26822 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 --2- 172.21.15.94:0/3352038807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ee03d54a0 0x562ee03eceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 >> 172.21.15.94:0/3352038807 conn(0x562ee03d1a30 msgr2=0x562ee03df300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 shutdown_connections 2024-09-19T04:29:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 1 -- 172.21.15.94:0/3352038807 wait complete. 2024-09-19T04:29:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.358+0000 7fc959b17740 20 client.26822 trim_cache size 0 max 16384 2024-09-19T04:29:53.365 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:29:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:29:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:29:53.365 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:29:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc93c0a33b0 msgr2=0x7fc93c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc93c0a33b0 0x7fc93c0a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc9501077c0 tx=0x7fc948014040 comp rx=0 tx=0).stop 2024-09-19T04:29:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc950100100 msgr2=0x7fc9501012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc950100100 0x7fc9501012e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00ea60 tx=0x7fc94c00ef20 comp rx=0 tx=0).stop 2024-09-19T04:29:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 shutdown_connections 2024-09-19T04:29:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc93c0a33b0 0x7fc93c0a5870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc950106b80 0x7fc950101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc950100aa0 0x7fc950101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 --2- 172.21.15.94:0/377385430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc950100100 0x7fc9501012e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 >> 172.21.15.94:0/377385430 conn(0x7fc950077230 msgr2=0x7fc950104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.359+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 shutdown_connections 2024-09-19T04:29:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:53.360+0000 7fc95773e640 1 -- 172.21.15.94:0/377385430 wait complete. 2024-09-19T04:29:54.220 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:29:54.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:29:54.565 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:54.996 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:54 smithi094 ceph-mon[30972]: pgmap v465: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.8 MiB/s wr, 792 op/s 2024-09-19T04:29:55.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 -- 172.21.15.94:0/3664242981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f4100aa0 msgr2=0x7ff1f410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:55.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 --2- 172.21.15.94:0/3664242981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f4100aa0 0x7ff1f410ea20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009be0 tx=0x7ff1ec02a6c0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 -- 172.21.15.94:0/3664242981 shutdown_connections 2024-09-19T04:29:55.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 --2- 172.21.15.94:0/3664242981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f4100aa0 0x7ff1f410ea20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 --2- 172.21.15.94:0/3664242981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4100100 0x7ff1f4100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 --2- 172.21.15.94:0/3664242981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f4106b80 0x7ff1f4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 -- 172.21.15.94:0/3664242981 >> 172.21.15.94:0/3664242981 conn(0x7ff1f4077230 msgr2=0x7ff1f40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:55.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 -- 172.21.15.94:0/3664242981 shutdown_connections 2024-09-19T04:29:55.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 -- 172.21.15.94:0/3664242981 wait complete. 2024-09-19T04:29:55.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.103+0000 7ff1fa629640 1 Processor -- start 2024-09-19T04:29:55.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1fa629640 1 -- start start 2024-09-19T04:29:55.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1fa629640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4100100 0x7ff1f41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:55.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1fa629640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f4100aa0 0x7ff1f41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:55.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1fa629640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f4106b80 0x7ff1f419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:55.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1fa629640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4111f00 con 0x7ff1f4100aa0 2024-09-19T04:29:55.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1fa629640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4111d80 con 0x7ff1f4106b80 2024-09-19T04:29:55.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1fa629640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4112080 con 0x7ff1f4100100 2024-09-19T04:29:55.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1f3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4100100 0x7ff1f41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:55.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1f3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4100100 0x7ff1f41983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48840/0 (socket says 172.21.15.94:48840) 2024-09-19T04:29:55.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1f3fff640 1 -- 172.21.15.94:0/4230879433 learned_addr learned my addr 172.21.15.94:0/4230879433 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:55.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.104+0000 7ff1f8bc6640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f4106b80 0x7ff1f419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:55.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f37fe640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f4100aa0 0x7ff1f41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:55.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f3fff640 1 -- 172.21.15.94:0/4230879433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f4106b80 msgr2=0x7ff1f419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:55.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f3fff640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f4106b80 0x7ff1f419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f3fff640 1 -- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f4100aa0 msgr2=0x7ff1f41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:55.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f3fff640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f4100aa0 0x7ff1f41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f3fff640 1 -- 172.21.15.94:0/4230879433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f419d350 con 0x7ff1f4100100 2024-09-19T04:29:55.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f37fe640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f4100aa0 0x7ff1f41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:55.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f3fff640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4100100 0x7ff1f41983a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e9f0 tx=0x7ff1e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:55.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f17fa640 1 -- 172.21.15.94:0/4230879433 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400cd80 con 0x7ff1f4100100 2024-09-19T04:29:55.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f419d640 con 0x7ff1f4100100 2024-09-19T04:29:55.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f17fa640 1 -- 172.21.15.94:0/4230879433 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1e40108e0 con 0x7ff1f4100100 2024-09-19T04:29:55.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1f17fa640 1 -- 172.21.15.94:0/4230879433 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4018920 con 0x7ff1f4100100 2024-09-19T04:29:55.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.105+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f4101210 con 0x7ff1f4100100 2024-09-19T04:29:55.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.107+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f4199700 con 0x7ff1f4100100 2024-09-19T04:29:55.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.108+0000 7ff1f17fa640 1 -- 172.21.15.94:0/4230879433 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff1e4010a90 con 0x7ff1f4100100 2024-09-19T04:29:55.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.109+0000 7ff1f17fa640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1dc0a33b0 0x7ff1dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:55.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.109+0000 7ff1f37fe640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1dc0a33b0 0x7ff1dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:55.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.109+0000 7ff1f17fa640 1 -- 172.21.15.94:0/4230879433 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7ff1e40d0230 con 0x7ff1f4100100 2024-09-19T04:29:55.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.110+0000 7ff1f37fe640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1dc0a33b0 0x7ff1dc0a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40ff810 tx=0x7ff1e8009260 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:55.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.112+0000 7ff1f17fa640 1 -- 172.21.15.94:0/4230879433 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff1e409cbd0 con 0x7ff1f4100100 2024-09-19T04:29:55.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:54 smithi112 ceph-mon[35051]: pgmap v465: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.8 MiB/s wr, 792 op/s 2024-09-19T04:29:55.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:54 smithi104 ceph-mon[34145]: pgmap v465: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.8 MiB/s wr, 792 op/s 2024-09-19T04:29:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.410+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1f402d7c0 con 0x7ff1f4100100 2024-09-19T04:29:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.411+0000 7ff1f17fa640 1 -- 172.21.15.94:0/4230879433 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11915 (secure 0 0 0) 0x7ff1e409c4d0 con 0x7ff1f4100100 2024-09-19T04:29:55.413 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:29:55.413 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":54,"btime":"2024-09-19T04:29:32:831739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:29:32.764738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:29:55.413 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 54 2024-09-19T04:29:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1dc0a33b0 msgr2=0x7ff1dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1dc0a33b0 0x7ff1dc0a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40ff810 tx=0x7ff1e8009260 comp rx=0 tx=0).stop 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4100100 msgr2=0x7ff1f41983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4100100 0x7ff1f41983a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e9f0 tx=0x7ff1e400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 shutdown_connections 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff1dc0a33b0 0x7ff1dc0a5870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f4106b80 0x7ff1f419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f4100aa0 0x7ff1f41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 --2- 172.21.15.94:0/4230879433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4100100 0x7ff1f41983a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 >> 172.21.15.94:0/4230879433 conn(0x7ff1f4077230 msgr2=0x7ff1f4104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 shutdown_connections 2024-09-19T04:29:55.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:55.414+0000 7ff1fa629640 1 -- 172.21.15.94:0/4230879433 wait complete. 2024-09-19T04:29:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 820.0 s, 393.2 tps, lat 253.294 ms stddev 30.602 2024-09-19T04:29:56.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:55 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4230879433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:29:56.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:29:56.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:55 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4230879433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:29:56.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:55 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4230879433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:29:56.483 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:57.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.032+0000 7f60a315e640 1 -- 172.21.15.94:0/2498264782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c101df0 msgr2=0x7f609c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.032+0000 7f60a315e640 1 --2- 172.21.15.94:0/2498264782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c101df0 0x7f609c1021d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6090009860 tx=0x7f609002a340 comp rx=0 tx=0).stop 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.032+0000 7f60a315e640 1 -- 172.21.15.94:0/2498264782 shutdown_connections 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.032+0000 7f60a315e640 1 --2- 172.21.15.94:0/2498264782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f609c106f20 0x7f609c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.032+0000 7f60a315e640 1 --2- 172.21.15.94:0/2498264782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c1027a0 0x7f609c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.032+0000 7f60a315e640 1 --2- 172.21.15.94:0/2498264782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c101df0 0x7f609c1021d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.032+0000 7f60a315e640 1 -- 172.21.15.94:0/2498264782 >> 172.21.15.94:0/2498264782 conn(0x7f609c0fd8f0 msgr2=0x7f609c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.032+0000 7f60a315e640 1 -- 172.21.15.94:0/2498264782 shutdown_connections 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.032+0000 7f60a315e640 1 -- 172.21.15.94:0/2498264782 wait complete. 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a315e640 1 Processor -- start 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a315e640 1 -- start start 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a315e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c101df0 0x7f609c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a315e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f609c1027a0 0x7f609c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a315e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c106f20 0x7f609c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a315e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f609c111df0 con 0x7f609c106f20 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a315e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f609c111c70 con 0x7f609c1027a0 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a315e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f609c111f70 con 0x7f609c101df0 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a16fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c106f20 0x7f609c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a0efa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c101df0 0x7f609c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a16fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c106f20 0x7f609c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57908/0 (socket says 172.21.15.94:57908) 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a16fb640 1 -- 172.21.15.94:0/2046852429 learned_addr learned my addr 172.21.15.94:0/2046852429 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f609bfff640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f609c1027a0 0x7f609c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a16fb640 1 -- 172.21.15.94:0/2046852429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c101df0 msgr2=0x7f609c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a16fb640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c101df0 0x7f609c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a16fb640 1 -- 172.21.15.94:0/2046852429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f609c1027a0 msgr2=0x7f609c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:57.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a16fb640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f609c1027a0 0x7f609c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.033+0000 7f60a16fb640 1 -- 172.21.15.94:0/2046852429 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c19d360 con 0x7f609c106f20 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.034+0000 7f60a16fb640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c106f20 0x7f609c19cc80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f609400eb10 tx=0x7f609400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.034+0000 7f6099ffb640 1 -- 172.21.15.94:0/2046852429 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094004b20 con 0x7f609c106f20 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.034+0000 7f60a315e640 1 -- 172.21.15.94:0/2046852429 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c19d650 con 0x7f609c106f20 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.034+0000 7f6099ffb640 1 -- 172.21.15.94:0/2046852429 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6094004cc0 con 0x7f609c106f20 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.034+0000 7f6099ffb640 1 -- 172.21.15.94:0/2046852429 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094010690 con 0x7f609c106f20 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.034+0000 7f60a315e640 1 -- 172.21.15.94:0/2046852429 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c1a4f30 con 0x7f609c106f20 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.037+0000 7f6099ffb640 1 -- 172.21.15.94:0/2046852429 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6094005020 con 0x7f609c106f20 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a5537740 1 Processor -- start 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f6099ffb640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f608c0a33b0 0x7f608c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f609bfff640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f608c0a33b0 0x7f608c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a5537740 1 -- start start 2024-09-19T04:29:57.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f6099ffb640 1 -- 172.21.15.94:0/2046852429 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7f60940d0270 con 0x7f609c106f20 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a5537740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f9f50 0x55a1954f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a5537740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a1954f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a5537740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a195505b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a5537740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a19545a2f0 con 0x55a1954f89e0 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a5537740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a195401cf0 con 0x55a195503780 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a5537740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a1954562b0 con 0x55a1954f9f50 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a16fb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a195505b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a16fb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a195505b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44962/0 (socket says 172.21.15.94:44962) 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a16fb640 1 -- 172.21.15.94:0/1886538155 learned_addr learned my addr 172.21.15.94:0/1886538155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a16fb640 1 -- 172.21.15.94:0/1886538155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f9f50 msgr2=0x55a1954f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.038+0000 7f60a0efa640 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f9f50 0x55a1954f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f60a16fb640 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f9f50 0x55a1954f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f60a16fb640 1 -- 172.21.15.94:0/1886538155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 msgr2=0x55a1954f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f609bfff640 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a1954f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f60a16fb640 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a1954f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f60a16fb640 1 -- 172.21.15.94:0/1886538155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1953e63a0 con 0x55a195503780 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f60a0efa640 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f9f50 0x55a1954f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f609bfff640 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a1954f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f60a16fb640 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a195505b70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6094098ee0 tx=0x7f6094098880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461808168f172880 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f6086ffd640 1 -- 172.21.15.94:0/1886538155 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940b4020 con 0x55a195503780 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f609bfff640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f608c0a33b0 0x7f608c0a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f609c0fff10 tx=0x7f6088011810 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f6086ffd640 1 -- 172.21.15.94:0/1886538155 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f609409cc10 con 0x55a195503780 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.039+0000 7f6086ffd640 1 -- 172.21.15.94:0/1886538155 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094098b80 con 0x55a195503780 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.040+0000 7f60a5537740 1 -- 172.21.15.94:0/1886538155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 msgr2=0x55a195505b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:57.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.040+0000 7f60a5537740 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a195505b70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6094098ee0 tx=0x7f6094098880 comp rx=0 tx=0).stop 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.040+0000 7f60a5537740 1 -- 172.21.15.94:0/1886538155 shutdown_connections 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.040+0000 7f60a5537740 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a195505b70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.040+0000 7f60a5537740 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a1954f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.040+0000 7f60a5537740 1 --2- 172.21.15.94:0/1886538155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f9f50 0x55a1954f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.040+0000 7f60a5537740 1 -- 172.21.15.94:0/1886538155 >> 172.21.15.94:0/1886538155 conn(0x55a1954bd310 msgr2=0x55a1954f5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.040+0000 7f60a5537740 1 -- 172.21.15.94:0/1886538155 shutdown_connections 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.041+0000 7f60a5537740 1 -- 172.21.15.94:0/1886538155 wait complete. 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.041+0000 7f60a5537740 1 -- start start 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.041+0000 7f60a5537740 1 Processor -- start 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a5537740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a195503160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a5537740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a195500410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a5537740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a195503780 0x55a1955009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a5537740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a195506420 con 0x55a1954f89e0 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a5537740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a195415060 con 0x55a1954f9f50 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a5537740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a1955065a0 con 0x55a195503780 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f609bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a195500410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f609bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a195500410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44964/0 (socket says 172.21.15.94:44964) 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f609bfff640 1 -- 172.21.15.94:0/376201703 learned_addr learned my addr 172.21.15.94:0/376201703 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a16fb640 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a195503780 0x55a1955009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a5537740 10 client.0 _ll_register_callbacks cb 0x55a1951688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a0efa640 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a195503160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f609bfff640 1 -- 172.21.15.94:0/376201703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a195503780 msgr2=0x55a1955009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f609bfff640 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a195503780 0x55a1955009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f609bfff640 1 -- 172.21.15.94:0/376201703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 msgr2=0x55a195503160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f609bfff640 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a195503160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f609bfff640 1 -- 172.21.15.94:0/376201703 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1951cb3c0 con 0x55a1954f9f50 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.042+0000 7f60a0efa640 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a195503160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f609bfff640 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a195500410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6088005ab0 tx=0x7f6088004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8500af65cbf5f7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f607bfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f607bfff640 1 -- 172.21.15.94:0/376201703 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608801db90 con 0x55a1954f9f50 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f60a5537740 1 -- 172.21.15.94:0/376201703 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f609409cc10 con 0x55a1954f9f50 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f607bfff640 1 -- 172.21.15.94:0/376201703 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7f608801dd30 con 0x55a1954f9f50 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f607bfff640 1 -- 172.21.15.94:0/376201703 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6088005020 con 0x55a1954f9f50 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f60a5537740 10 client.26840 fetch_fsmap learned FSMap version 54 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f60a5537740 1 -- 172.21.15.94:0/376201703 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a1954f1f70 con 0x55a1954f9f50 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f607bfff640 1 -- 172.21.15.94:0/376201703 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f609409cc10 con 0x55a1954f9f50 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.043+0000 7f607bfff640 1 -- 172.21.15.94:0/376201703 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f6088027480 con 0x55a1954f9f50 2024-09-19T04:29:57.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.044+0000 7f60a5537740 10 client.26840 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.044+0000 7f60a5537740 10 client.26840 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.044+0000 7f60a5537740 20 client.26840 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.044+0000 7f60a5537740 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a1954f3380 0x55a1954f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.044+0000 7f60a5537740 4 client.26840 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.044+0000 7f60a5537740 1 -- 172.21.15.94:0/376201703 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a195467860 con 0x55a1954f3380 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.044+0000 7f60a0efa640 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a1954f3380 0x55a1954f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.044+0000 7f60a0efa640 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a1954f3380 0x55a1954f37b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27faaa1b5381a91c server_cookie=ea41bd3c5ada6b6a in_seq=0 out_seq=0 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.044+0000 7f607bfff640 10 client.26840 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f607bfff640 1 -- 172.21.15.94:0/376201703 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a195467860 con 0x55a1954f3380 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f607bfff640 10 client.26840 handle_command_reply: tid=0 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 client.26840 shutdown 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 -- 172.21.15.94:0/376201703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 msgr2=0x55a195500410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:57.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a195500410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6088005ab0 tx=0x7f6088004730 comp rx=0 tx=0).stop 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 -- 172.21.15.94:0/376201703 shutdown_connections 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a1954f3380 0x55a1954f37b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a195503780 0x55a1955009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a195500410 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 --2- 172.21.15.94:0/376201703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f89e0 0x55a195503160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f607bfff640 0 client.26840 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 -- 172.21.15.94:0/376201703 >> 172.21.15.94:0/376201703 conn(0x55a1954bd310 msgr2=0x55a1954fadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.045+0000 7f60a5537740 1 -- 172.21.15.94:0/376201703 shutdown_connections 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.046+0000 7f60a5537740 1 -- 172.21.15.94:0/376201703 wait complete. 2024-09-19T04:29:57.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.046+0000 7f60a5537740 20 client.26840 trim_cache size 0 max 16384 2024-09-19T04:29:57.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f60a5537740 1 Processor -- start 2024-09-19T04:29:57.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f60a5537740 1 -- start start 2024-09-19T04:29:57.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f60a5537740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 0x55a1954fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f60a5537740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f9f50 0x55a1954fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f609bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f9f50 0x55a1954fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f60a0efa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 0x55a1954fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f60a5537740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a19551fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f60a5537740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1adfa0 con 0x55a1954f9f50 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f60a0efa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 0x55a1954fd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48886/0 (socket says 172.21.15.94:48886) 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.062+0000 7f60a0efa640 1 -- 172.21.15.94:0/3734085881 learned_addr learned my addr 172.21.15.94:0/3734085881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f60a5537740 1 -- 172.21.15.94:0/3734085881 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1ab870 con 0x55a195503780 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f60a5537740 1 -- 172.21.15.94:0/3734085881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1a96e0 con 0x55a1954f89e0 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f60a16fb640 1 --2- 172.21.15.94:0/3734085881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a19551fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f609bfff640 1 -- 172.21.15.94:0/3734085881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 msgr2=0x55a1954fd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f609bfff640 1 --2- 172.21.15.94:0/3734085881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 0x55a1954fd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f609bfff640 1 -- 172.21.15.94:0/3734085881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 msgr2=0x55a19551fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f609bfff640 1 --2- 172.21.15.94:0/3734085881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a19551fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f609bfff640 1 -- 172.21.15.94:0/3734085881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1954f52e0 con 0x55a1954f9f50 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f609bfff640 1 --2- 172.21.15.94:0/3734085881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f9f50 0x55a1954fdbf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6088009b60 tx=0x7f6088011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337db9c2b867337c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:57.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f60867fc640 1 -- 172.21.15.94:0/3734085881 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088033070 con 0x55a1954f9f50 2024-09-19T04:29:57.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f60867fc640 1 -- 172.21.15.94:0/3734085881 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6088027a90 con 0x55a1954f9f50 2024-09-19T04:29:57.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.063+0000 7f60867fc640 1 -- 172.21.15.94:0/3734085881 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088027e60 con 0x55a1954f9f50 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.064+0000 7f60a5537740 1 -- 172.21.15.94:0/3734085881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f9f50 msgr2=0x55a1954fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.064+0000 7f60a5537740 1 --2- 172.21.15.94:0/3734085881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f9f50 0x55a1954fdbf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6088009b60 tx=0x7f6088011db0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.064+0000 7f60a5537740 1 -- 172.21.15.94:0/3734085881 shutdown_connections 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.064+0000 7f60a5537740 1 --2- 172.21.15.94:0/3734085881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a195503780 0x55a19551fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.064+0000 7f60a5537740 1 --2- 172.21.15.94:0/3734085881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1954f9f50 0x55a1954fdbf0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.064+0000 7f60a5537740 1 --2- 172.21.15.94:0/3734085881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 0x55a1954fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.064+0000 7f60a5537740 1 -- 172.21.15.94:0/3734085881 >> 172.21.15.94:0/3734085881 conn(0x55a1954bd310 msgr2=0x55a1954ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.064+0000 7f60a5537740 1 -- 172.21.15.94:0/3734085881 shutdown_connections 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.064+0000 7f60a5537740 1 -- 172.21.15.94:0/3734085881 wait complete. 2024-09-19T04:29:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.065+0000 7f60a5537740 1 -- start start 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.065+0000 7f60a5537740 1 Processor -- start 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.065+0000 7f60a5537740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 0x55a1955200b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.065+0000 7f60a5537740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a1955205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.065+0000 7f60a5537740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a195503780 0x55a19551b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.065+0000 7f60a5537740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1a9d30 con 0x55a195503780 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.065+0000 7f60a5537740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1ab1a0 con 0x55a1954f9f50 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.065+0000 7f60a5537740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1aa400 con 0x55a1954f89e0 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.065+0000 7f60a5537740 10 client.0 _ll_register_callbacks cb 0x7f6088005a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f60a0efa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 0x55a1955200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f609bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a1955205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f609bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a1955205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:44988/0 (socket says 172.21.15.94:44988) 2024-09-19T04:29:57.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f609bfff640 1 -- 172.21.15.94:0/2420738816 learned_addr learned my addr 172.21.15.94:0/2420738816 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f60a16fb640 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a195503780 0x55a19551b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f609bfff640 1 -- 172.21.15.94:0/2420738816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 msgr2=0x55a1955200b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f609bfff640 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 0x55a1955200b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f609bfff640 1 -- 172.21.15.94:0/2420738816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a195503780 msgr2=0x55a19551b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f609bfff640 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a195503780 0x55a19551b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f609bfff640 1 -- 172.21.15.94:0/2420738816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19551bc70 con 0x55a1954f9f50 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f609bfff640 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a1955205f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f60880278a0 tx=0x7f6088027730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98ed03417d68cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.066+0000 7f607bfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f607bfff640 1 -- 172.21.15.94:0/2420738816 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088033050 con 0x55a1954f9f50 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f607bfff640 1 -- 172.21.15.94:0/2420738816 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7f608801d990 con 0x55a1954f9f50 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f60a5537740 1 -- 172.21.15.94:0/2420738816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6088027a90 con 0x55a1954f9f50 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f607bfff640 1 -- 172.21.15.94:0/2420738816 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6088045050 con 0x55a1954f9f50 2024-09-19T04:29:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f60a5537740 10 client.26846 fetch_fsmap learned FSMap version 54 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f60a5537740 1 -- 172.21.15.94:0/2420738816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a195582940 con 0x55a1954f9f50 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f607bfff640 1 -- 172.21.15.94:0/2420738816 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6088027a90 con 0x55a1954f9f50 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f607bfff640 1 -- 172.21.15.94:0/2420738816 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f6088038460 con 0x55a1954f9f50 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f60a5537740 10 client.26846 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f60a5537740 10 client.26846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f60a5537740 20 client.26846 populate_metadata read hostname 'smithi094' 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f60a5537740 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a195582fd0 0x55a1955853f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f60a5537740 4 client.26846 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.067+0000 7f60a5537740 1 -- 172.21.15.94:0/2420738816 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a195461d70 con 0x55a195582fd0 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.068+0000 7f60a0efa640 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a195582fd0 0x55a1955853f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.068+0000 7f607bfff640 10 client.26846 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.068+0000 7f60a0efa640 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a195582fd0 0x55a1955853f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f79208394a6f9d34 server_cookie=b81139318541a6ce in_seq=0 out_seq=0 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.068+0000 7f607bfff640 1 -- 172.21.15.94:0/2420738816 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a195461d70 con 0x55a195582fd0 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.068+0000 7f607bfff640 10 client.26846 handle_command_reply: tid=0 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.068+0000 7f60a5537740 1 client.26846 shutdown 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 -- 172.21.15.94:0/2420738816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 msgr2=0x55a1955205f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a1955205f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f60880278a0 tx=0x7f6088027730 comp rx=0 tx=0).stop 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 -- 172.21.15.94:0/2420738816 shutdown_connections 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55a195582fd0 0x55a1955853f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a195503780 0x55a19551b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1954f9f50 0x55a1955205f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 --2- 172.21.15.94:0/2420738816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1954f89e0 0x55a1955200b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f607bfff640 0 client.26846 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:29:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 -- 172.21.15.94:0/2420738816 >> 172.21.15.94:0/2420738816 conn(0x55a1954bd310 msgr2=0x55a195501c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:57.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 -- 172.21.15.94:0/2420738816 shutdown_connections 2024-09-19T04:29:57.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 1 -- 172.21.15.94:0/2420738816 wait complete. 2024-09-19T04:29:57.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.069+0000 7f60a5537740 20 client.26846 trim_cache size 0 max 16384 2024-09-19T04:29:57.076 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 -- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f608c0a33b0 msgr2=0x7f608c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f608c0a33b0 0x7f608c0a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f609c0fff10 tx=0x7f6088011810 comp rx=0 tx=0).stop 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 -- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c106f20 msgr2=0x7f609c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c106f20 0x7f609c19cc80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f609400eb10 tx=0x7f609400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 -- 172.21.15.94:0/2046852429 shutdown_connections 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f608c0a33b0 0x7f608c0a5870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c106f20 0x7f609c19cc80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f609c1027a0 0x7f609c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 --2- 172.21.15.94:0/2046852429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c101df0 0x7f609c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 -- 172.21.15.94:0/2046852429 >> 172.21.15.94:0/2046852429 conn(0x7f609c0fd8f0 msgr2=0x7f609c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 -- 172.21.15.94:0/2046852429 shutdown_connections 2024-09-19T04:29:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:29:57.070+0000 7f60a315e640 1 -- 172.21.15.94:0/2046852429 wait complete. 2024-09-19T04:29:57.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:56 smithi112 ceph-mon[35051]: pgmap v466: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 3.9 MiB/s wr, 534 op/s 2024-09-19T04:29:57.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:56 smithi094 ceph-mon[30972]: pgmap v466: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 3.9 MiB/s wr, 534 op/s 2024-09-19T04:29:57.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:56 smithi104 ceph-mon[34145]: pgmap v466: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 3.9 MiB/s wr, 534 op/s 2024-09-19T04:29:59.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:29:58 smithi112 ceph-mon[35051]: pgmap v467: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 4.4 MiB/s wr, 620 op/s 2024-09-19T04:29:59.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:29:58 smithi094 ceph-mon[30972]: pgmap v467: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 4.4 MiB/s wr, 620 op/s 2024-09-19T04:29:59.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:29:58 smithi104 ceph-mon[34145]: pgmap v467: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 4.4 MiB/s wr, 620 op/s 2024-09-19T04:30:00.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:30:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 825.0 s, 399.8 tps, lat 249.584 ms stddev 32.625 2024-09-19T04:30:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:01.234 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:00 smithi094 ceph-mon[30972]: pgmap v468: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 4.4 MiB/s wr, 628 op/s 2024-09-19T04:30:01.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:00 smithi094 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T04:30:01.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:00 smithi112 ceph-mon[35051]: pgmap v468: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 4.4 MiB/s wr, 628 op/s 2024-09-19T04:30:01.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:00 smithi112 ceph-mon[35051]: overall HEALTH_OK 2024-09-19T04:30:01.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:00 smithi104 ceph-mon[34145]: pgmap v468: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 4.4 MiB/s wr, 628 op/s 2024-09-19T04:30:01.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:00 smithi104 ceph-mon[34145]: overall HEALTH_OK 2024-09-19T04:30:01.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.487+0000 7fa88b025640 1 -- 172.21.15.94:0/3134655070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa884106a80 msgr2=0x7fa884106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:01.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.487+0000 7fa88b025640 1 --2- 172.21.15.94:0/3134655070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa884106a80 0x7fa884106e60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa878009930 tx=0x7fa87802a420 comp rx=0 tx=0).stop 2024-09-19T04:30:01.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.487+0000 7fa88b025640 1 -- 172.21.15.94:0/3134655070 shutdown_connections 2024-09-19T04:30:01.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.487+0000 7fa88b025640 1 --2- 172.21.15.94:0/3134655070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa884101420 0x7fa88410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.487+0000 7fa88b025640 1 --2- 172.21.15.94:0/3134655070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884100a80 0x7fa884100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.487+0000 7fa88b025640 1 --2- 172.21.15.94:0/3134655070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa884106a80 0x7fa884106e60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.487+0000 7fa88b025640 1 -- 172.21.15.94:0/3134655070 >> 172.21.15.94:0/3134655070 conn(0x7fa8840fc7a0 msgr2=0x7fa8840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:01.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.487+0000 7fa88b025640 1 -- 172.21.15.94:0/3134655070 shutdown_connections 2024-09-19T04:30:01.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.488+0000 7fa88b025640 1 -- 172.21.15.94:0/3134655070 wait complete. 2024-09-19T04:30:01.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.488+0000 7fa88b025640 1 Processor -- start 2024-09-19T04:30:01.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa88b025640 1 -- start start 2024-09-19T04:30:01.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa88b025640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa884100a80 0x7fa884073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa88b025640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884101420 0x7fa88406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa88b025640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa884106a80 0x7fa88406d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa88b025640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa884110d30 con 0x7fa884106a80 2024-09-19T04:30:01.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa88b025640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa884110bb0 con 0x7fa884100a80 2024-09-19T04:30:01.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa88b025640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa884110eb0 con 0x7fa884101420 2024-09-19T04:30:01.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa888dc1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa884100a80 0x7fa884073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa883fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884101420 0x7fa88406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa883fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884101420 0x7fa88406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48918/0 (socket says 172.21.15.94:48918) 2024-09-19T04:30:01.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa883fff640 1 -- 172.21.15.94:0/2760908519 learned_addr learned my addr 172.21.15.94:0/2760908519 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:01.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.489+0000 7fa8895c2640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa884106a80 0x7fa88406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.490+0000 7fa883fff640 1 -- 172.21.15.94:0/2760908519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa884100a80 msgr2=0x7fa884073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:01.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.490+0000 7fa883fff640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa884100a80 0x7fa884073ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.490+0000 7fa883fff640 1 -- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa884106a80 msgr2=0x7fa88406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:01.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.490+0000 7fa883fff640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa884106a80 0x7fa88406d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.490+0000 7fa883fff640 1 -- 172.21.15.94:0/2760908519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88406d7d0 con 0x7fa884101420 2024-09-19T04:30:01.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.490+0000 7fa883fff640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884101420 0x7fa88406cbf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa870009d40 tx=0x7fa87000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:01.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.491+0000 7fa881ffb640 1 -- 172.21.15.94:0/2760908519 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870005220 con 0x7fa884101420 2024-09-19T04:30:01.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.491+0000 7fa881ffb640 1 -- 172.21.15.94:0/2760908519 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa870004c50 con 0x7fa884101420 2024-09-19T04:30:01.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.491+0000 7fa881ffb640 1 -- 172.21.15.94:0/2760908519 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87001a840 con 0x7fa884101420 2024-09-19T04:30:01.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.491+0000 7fa88b025640 1 -- 172.21.15.94:0/2760908519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8840715f0 con 0x7fa884101420 2024-09-19T04:30:01.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.491+0000 7fa88b025640 1 -- 172.21.15.94:0/2760908519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa884071ae0 con 0x7fa884101420 2024-09-19T04:30:01.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.494+0000 7fa881ffb640 1 -- 172.21.15.94:0/2760908519 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa8700046c0 con 0x7fa884101420 2024-09-19T04:30:01.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.494+0000 7fa88d3fe740 1 Processor -- start 2024-09-19T04:30:01.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.494+0000 7fa88d3fe740 1 -- start start 2024-09-19T04:30:01.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.494+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f3b40 0x55623d4f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa8895c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f3b40 0x55623d4f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa888dc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa8895c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f3b40 0x55623d4f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57984/0 (socket says 172.21.15.94:57984) 2024-09-19T04:30:01.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa888dc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48934/0 (socket says 172.21.15.94:48934) 2024-09-19T04:30:01.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa881ffb640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8740a33b0 0x7fa8740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 0x55623d4ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa8895c2640 1 -- 172.21.15.94:0/782492465 learned_addr learned my addr 172.21.15.94:0/782492465 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa88d3fe740 1 -- 172.21.15.94:0/782492465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55623d4582f0 con 0x55623d4f3b40 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa88d3fe740 1 -- 172.21.15.94:0/782492465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55623d3ffcf0 con 0x55623d4fd370 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa88d3fe740 1 -- 172.21.15.94:0/782492465 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55623d4542b0 con 0x55623d4f25d0 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.495+0000 7fa881ffb640 1 -- 172.21.15.94:0/2760908519 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fa8700d40c0 con 0x7fa884101420 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa883fff640 1 --2- 172.21.15.94:0/782492465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 0x55623d4ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa888dc1640 1 -- 172.21.15.94:0/782492465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 msgr2=0x55623d4ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa888dc1640 1 --2- 172.21.15.94:0/782492465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 0x55623d4ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa888dc1640 1 -- 172.21.15.94:0/782492465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f3b40 msgr2=0x55623d4f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa888dc1640 1 --2- 172.21.15.94:0/782492465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f3b40 0x55623d4f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa888dc1640 1 -- 172.21.15.94:0/782492465 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623d3e43a0 con 0x55623d4f25d0 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa8895c2640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8740a33b0 0x7fa8740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa883fff640 1 --2- 172.21.15.94:0/782492465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 0x55623d4ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:30:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa888dc1640 1 --2- 172.21.15.94:0/782492465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f2a10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa878031440 tx=0x7fa878005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db7d29dc7ea2c38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa8895c2640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8740a33b0 0x7fa8740a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa87c013840 tx=0x7fa87c004ab0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa86effd640 1 -- 172.21.15.94:0/782492465 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878040070 con 0x55623d4f25d0 2024-09-19T04:30:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa86effd640 1 -- 172.21.15.94:0/782492465 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa878004030 con 0x55623d4f25d0 2024-09-19T04:30:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.496+0000 7fa86effd640 1 -- 172.21.15.94:0/782492465 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878035040 con 0x55623d4f25d0 2024-09-19T04:30:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.497+0000 7fa88d3fe740 1 -- 172.21.15.94:0/782492465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 msgr2=0x55623d4f2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.497+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/782492465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f2a10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa878031440 tx=0x7fa878005be0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.497+0000 7fa88d3fe740 1 -- 172.21.15.94:0/782492465 shutdown_connections 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.497+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/782492465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 0x55623d4ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.497+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/782492465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f2a10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.497+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/782492465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f3b40 0x55623d4f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.497+0000 7fa88d3fe740 1 -- 172.21.15.94:0/782492465 >> 172.21.15.94:0/782492465 conn(0x55623d4bb310 msgr2=0x55623d4eec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.497+0000 7fa88d3fe740 1 -- 172.21.15.94:0/782492465 shutdown_connections 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.497+0000 7fa88d3fe740 1 -- 172.21.15.94:0/782492465 wait complete. 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa88d3fe740 1 -- start start 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa88d3fe740 1 Processor -- start 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f25d0 0x55623d4fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f3b40 0x55623d4fa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 0x55623d4fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa88d3fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55623d500010 con 0x55623d4f25d0 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa88d3fe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55623d413060 con 0x55623d4fd370 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa88d3fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55623d500190 con 0x55623d4f3b40 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa883fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f3b40 0x55623d4fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa883fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f3b40 0x55623d4fa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48944/0 (socket says 172.21.15.94:48944) 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa883fff640 1 -- 172.21.15.94:0/3335119957 learned_addr learned my addr 172.21.15.94:0/3335119957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.498+0000 7fa88d3fe740 10 client.0 _ll_register_callbacks cb 0x55623d1668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa888dc1640 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f25d0 0x55623d4fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa8895c2640 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 0x55623d4fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa883fff640 1 -- 172.21.15.94:0/3335119957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 msgr2=0x55623d4fa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa883fff640 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 0x55623d4fa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa883fff640 1 -- 172.21.15.94:0/3335119957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f25d0 msgr2=0x55623d4fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa883fff640 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f25d0 0x55623d4fcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa883fff640 1 -- 172.21.15.94:0/3335119957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55623d1c93c0 con 0x55623d4f3b40 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa888dc1640 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f25d0 0x55623d4fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa883fff640 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f3b40 0x55623d4fa010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa87009cde0 tx=0x7fa87007d8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad950933f0cd0446 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa863fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa863fff640 1 -- 172.21.15.94:0/3335119957 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87007d3e0 con 0x55623d4f3b40 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3335119957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa878004030 con 0x55623d4f3b40 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.499+0000 7fa863fff640 1 -- 172.21.15.94:0/3335119957 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fa8700d1a50 con 0x55623d4f3b40 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa863fff640 1 -- 172.21.15.94:0/3335119957 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa8700a0410 con 0x55623d4f3b40 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa88d3fe740 10 client.26821 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3335119957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55623d50ba80 con 0x55623d4f3b40 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa863fff640 1 -- 172.21.15.94:0/3335119957 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa878004030 con 0x55623d4f3b40 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa863fff640 1 -- 172.21.15.94:0/3335119957 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fa87009c160 con 0x55623d4f3b40 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa88d3fe740 10 client.26821 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa88d3fe740 10 client.26821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:30:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa88d3fe740 20 client.26821 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55623d50cdd0 0x55623d50f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa88d3fe740 4 client.26821 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.500+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3335119957 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55623d465860 con 0x55623d50cdd0 2024-09-19T04:30:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.501+0000 7fa888dc1640 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55623d50cdd0 0x55623d50f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.501+0000 7fa863fff640 10 client.26821 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:30:01.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.501+0000 7fa888dc1640 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55623d50cdd0 0x55623d50f200 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90b12da47d94bf03 server_cookie=141c821366cdea6b in_seq=0 out_seq=0 2024-09-19T04:30:01.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.502+0000 7fa863fff640 1 -- 172.21.15.94:0/3335119957 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55623d465860 con 0x55623d50cdd0 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.502+0000 7fa863fff640 10 client.26821 handle_command_reply: tid=0 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.502+0000 7fa88d3fe740 1 client.26821 shutdown 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.502+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3335119957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f3b40 msgr2=0x55623d4fa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.502+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f3b40 0x55623d4fa010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa87009cde0 tx=0x7fa87007d8e0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.502+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3335119957 shutdown_connections 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.502+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55623d50cdd0 0x55623d50f200 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.502+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4fd370 0x55623d4fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.502+0000 7fa863fff640 0 client.26821 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.503+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f3b40 0x55623d4fa010 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.503+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3335119957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4f25d0 0x55623d4fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.503+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3335119957 >> 172.21.15.94:0/3335119957 conn(0x55623d4bb310 msgr2=0x55623d4f4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.503+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3335119957 shutdown_connections 2024-09-19T04:30:01.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.503+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3335119957 wait complete. 2024-09-19T04:30:01.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.503+0000 7fa88d3fe740 20 client.26821 trim_cache size 0 max 16384 2024-09-19T04:30:01.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.519+0000 7fa88d3fe740 1 Processor -- start 2024-09-19T04:30:01.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.519+0000 7fa88d3fe740 1 -- start start 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.519+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d5092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.519+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 0x55623d4f9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.519+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 0x55623d514ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.519+0000 7fa88d3fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8841b6760 con 0x55623d4fd370 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.519+0000 7fa88d3fe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8841b4030 con 0x55623d4f3b40 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.519+0000 7fa88d3fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8841b1ea0 con 0x55623d4f25d0 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa888dc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d5092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa888dc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d5092b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48956/0 (socket says 172.21.15.94:48956) 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa888dc1640 1 -- 172.21.15.94:0/813537916 learned_addr learned my addr 172.21.15.94:0/813537916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:01.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa8895c2640 1 --2- 172.21.15.94:0/813537916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 0x55623d514ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa883fff640 1 --2- 172.21.15.94:0/813537916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 0x55623d4f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa888dc1640 1 -- 172.21.15.94:0/813537916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 msgr2=0x55623d4f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa888dc1640 1 --2- 172.21.15.94:0/813537916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 0x55623d4f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa888dc1640 1 -- 172.21.15.94:0/813537916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 msgr2=0x55623d514ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa888dc1640 1 --2- 172.21.15.94:0/813537916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 0x55623d514ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa888dc1640 1 -- 172.21.15.94:0/813537916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623d4efdd0 con 0x55623d4f25d0 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa888dc1640 1 --2- 172.21.15.94:0/813537916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d5092b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa878031440 tx=0x7fa878005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ceb3cd97c23a528 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.520+0000 7fa86e7fc640 1 -- 172.21.15.94:0/813537916 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878040040 con 0x55623d4f25d0 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.521+0000 7fa86e7fc640 1 -- 172.21.15.94:0/813537916 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8780055e0 con 0x55623d4f25d0 2024-09-19T04:30:01.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.521+0000 7fa86e7fc640 1 -- 172.21.15.94:0/813537916 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878035040 con 0x55623d4f25d0 2024-09-19T04:30:01.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.521+0000 7fa88d3fe740 1 -- 172.21.15.94:0/813537916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 msgr2=0x55623d5092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:01.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.521+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/813537916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d5092b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa878031440 tx=0x7fa878005020 comp rx=0 tx=0).stop 2024-09-19T04:30:01.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.521+0000 7fa88d3fe740 1 -- 172.21.15.94:0/813537916 shutdown_connections 2024-09-19T04:30:01.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.521+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/813537916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 0x55623d514ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.521+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/813537916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 0x55623d4f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.521+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/813537916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d5092b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.521+0000 7fa88d3fe740 1 -- 172.21.15.94:0/813537916 >> 172.21.15.94:0/813537916 conn(0x55623d4bb310 msgr2=0x55623d4f8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.522+0000 7fa88d3fe740 1 -- 172.21.15.94:0/813537916 shutdown_connections 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.522+0000 7fa88d3fe740 1 -- 172.21.15.94:0/813537916 wait complete. 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.522+0000 7fa88d3fe740 1 -- start start 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.522+0000 7fa88d3fe740 1 Processor -- start 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 0x55623d4f7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa88d3fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 0x55623d4f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa88d3fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8841b24f0 con 0x55623d4fd370 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa88d3fe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8841b3960 con 0x55623d4f3b40 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa88d3fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8841b2bc0 con 0x55623d4f25d0 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa88d3fe740 10 client.0 _ll_register_callbacks cb 0x7fa87009c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:01.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa888dc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa888dc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48962/0 (socket says 172.21.15.94:48962) 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa888dc1640 1 -- 172.21.15.94:0/3347028008 learned_addr learned my addr 172.21.15.94:0/3347028008 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa883fff640 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 0x55623d4f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.523+0000 7fa8895c2640 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 0x55623d4f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.524+0000 7fa8895c2640 1 -- 172.21.15.94:0/3347028008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 msgr2=0x55623d4f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.524+0000 7fa8895c2640 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.524+0000 7fa8895c2640 1 -- 172.21.15.94:0/3347028008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 msgr2=0x55623d4f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.524+0000 7fa8895c2640 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 0x55623d4f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.524+0000 7fa8895c2640 1 -- 172.21.15.94:0/3347028008 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55623d57f2b0 con 0x55623d4fd370 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.524+0000 7fa888dc1640 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.524+0000 7fa8895c2640 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 0x55623d4f7a60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa87c005c40 tx=0x7fa87c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aefac29ce635a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.524+0000 7fa863fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:30:01.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.525+0000 7fa863fff640 1 -- 172.21.15.94:0/3347028008 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c020400 con 0x55623d4fd370 2024-09-19T04:30:01.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.525+0000 7fa863fff640 1 -- 172.21.15.94:0/3347028008 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fa87c0205a0 con 0x55623d4fd370 2024-09-19T04:30:01.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.525+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3347028008 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8780055e0 con 0x55623d4fd370 2024-09-19T04:30:01.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.525+0000 7fa863fff640 1 -- 172.21.15.94:0/3347028008 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa87c0207a0 con 0x55623d4fd370 2024-09-19T04:30:01.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.525+0000 7fa88d3fe740 10 client.17238 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:01.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.525+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3347028008 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55623d580550 con 0x55623d4fd370 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.525+0000 7fa863fff640 1 -- 172.21.15.94:0/3347028008 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8780055e0 con 0x55623d4fd370 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.525+0000 7fa863fff640 1 -- 172.21.15.94:0/3347028008 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fa87c029070 con 0x55623d4fd370 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.526+0000 7fa88d3fe740 10 client.17238 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.526+0000 7fa88d3fe740 10 client.17238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.526+0000 7fa88d3fe740 20 client.17238 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.526+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55623d50cdd0 0x55623d582d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.526+0000 7fa88d3fe740 4 client.17238 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.526+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3347028008 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55623d45fd70 con 0x55623d50cdd0 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.526+0000 7fa888dc1640 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55623d50cdd0 0x55623d582d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.526+0000 7fa863fff640 10 client.17238 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.526+0000 7fa888dc1640 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55623d50cdd0 0x55623d582d60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e83c3a25732dbfb1 server_cookie=bf0df067dc0f873a in_seq=0 out_seq=0 2024-09-19T04:30:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa863fff640 1 -- 172.21.15.94:0/3347028008 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55623d45fd70 con 0x55623d50cdd0 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa863fff640 10 client.17238 handle_command_reply: tid=0 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 client.17238 shutdown 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3347028008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 msgr2=0x55623d4f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 0x55623d4f7a60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa87c005c40 tx=0x7fa87c004a40 comp rx=0 tx=0).stop 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3347028008 shutdown_connections 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55623d50cdd0 0x55623d582d60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55623d4fd370 0x55623d4f7a60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55623d4f3b40 0x55623d4f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa863fff640 0 client.17238 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 --2- 172.21.15.94:0/3347028008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55623d4f25d0 0x55623d4f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3347028008 >> 172.21.15.94:0/3347028008 conn(0x55623d4bb310 msgr2=0x55623d4f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3347028008 shutdown_connections 2024-09-19T04:30:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.527+0000 7fa88d3fe740 1 -- 172.21.15.94:0/3347028008 wait complete. 2024-09-19T04:30:01.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.528+0000 7fa88d3fe740 20 client.17238 trim_cache size 0 max 16384 2024-09-19T04:30:01.533 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:30:01.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.528+0000 7fa88b025640 1 -- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8740a33b0 msgr2=0x7fa8740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.528+0000 7fa88b025640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8740a33b0 0x7fa8740a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa87c013840 tx=0x7fa87c004ab0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.528+0000 7fa88b025640 1 -- 172.21.15.94:0/2760908519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884101420 msgr2=0x7fa88406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.528+0000 7fa88b025640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884101420 0x7fa88406cbf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa870009d40 tx=0x7fa87000c6a0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.529+0000 7fa88b025640 1 -- 172.21.15.94:0/2760908519 shutdown_connections 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.529+0000 7fa88b025640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fa8740a33b0 0x7fa8740a5870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.529+0000 7fa88b025640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa884106a80 0x7fa88406d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.529+0000 7fa88b025640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884101420 0x7fa88406cbf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.529+0000 7fa88b025640 1 --2- 172.21.15.94:0/2760908519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa884100a80 0x7fa884073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.529+0000 7fa88b025640 1 -- 172.21.15.94:0/2760908519 >> 172.21.15.94:0/2760908519 conn(0x7fa8840fc7a0 msgr2=0x7fa8840fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.529+0000 7fa88b025640 1 -- 172.21.15.94:0/2760908519 shutdown_connections 2024-09-19T04:30:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:01.529+0000 7fa88b025640 1 -- 172.21.15.94:0/2760908519 wait complete. 2024-09-19T04:30:02.646 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:30:02.895 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:02 smithi094 ceph-mon[30972]: pgmap v469: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 3.5 MiB/s wr, 483 op/s 2024-09-19T04:30:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:03.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:02 smithi112 ceph-mon[35051]: pgmap v469: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 3.5 MiB/s wr, 483 op/s 2024-09-19T04:30:03.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:02 smithi104 ceph-mon[34145]: pgmap v469: 97 pgs: 97 active+clean; 3.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 3.5 MiB/s wr, 483 op/s 2024-09-19T04:30:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.498+0000 7f4017fe6640 1 -- 172.21.15.94:0/2725147986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010102570 msgr2=0x7f40101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.498+0000 7f4017fe6640 1 --2- 172.21.15.94:0/2725147986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010102570 0x7f40101029b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4000009930 tx=0x7f400002a4d0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.498+0000 7f4017fe6640 1 -- 172.21.15.94:0/2725147986 shutdown_connections 2024-09-19T04:30:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.498+0000 7f4017fe6640 1 --2- 172.21.15.94:0/2725147986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4010102ef0 0x7f401010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.498+0000 7f4017fe6640 1 --2- 172.21.15.94:0/2725147986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010102570 0x7f40101029b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.498+0000 7f4017fe6640 1 --2- 172.21.15.94:0/2725147986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4010100e20 0x7f4010101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.498+0000 7f4017fe6640 1 -- 172.21.15.94:0/2725147986 >> 172.21.15.94:0/2725147986 conn(0x7f40100fc7d0 msgr2=0x7f40100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.498+0000 7f4017fe6640 1 -- 172.21.15.94:0/2725147986 shutdown_connections 2024-09-19T04:30:03.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.498+0000 7f4017fe6640 1 -- 172.21.15.94:0/2725147986 wait complete. 2024-09-19T04:30:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.499+0000 7f4017fe6640 1 Processor -- start 2024-09-19T04:30:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.499+0000 7f4017fe6640 1 -- start start 2024-09-19T04:30:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.499+0000 7f4017fe6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010100e20 0x7f401019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.499+0000 7f4017fe6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4010102570 0x7f401019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.499+0000 7f4017fe6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4010102ef0 0x7f401019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.499+0000 7f4017fe6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f401010fd30 con 0x7f4010102ef0 2024-09-19T04:30:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.499+0000 7f4017fe6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f401010fbb0 con 0x7f4010102570 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4017fe6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f401010feb0 con 0x7f4010100e20 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015581640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4010102570 0x7f401019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015581640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4010102570 0x7f401019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51944/0 (socket says 172.21.15.94:51944) 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4016583640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4010102ef0 0x7f401019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015581640 1 -- 172.21.15.94:0/981494447 learned_addr learned my addr 172.21.15.94:0/981494447 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015d82640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010100e20 0x7f401019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015d82640 1 -- 172.21.15.94:0/981494447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4010102570 msgr2=0x7f401019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015d82640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4010102570 0x7f401019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015d82640 1 -- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4010102ef0 msgr2=0x7f401019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015d82640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4010102ef0 0x7f401019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015d82640 1 -- 172.21.15.94:0/981494447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401019f5a0 con 0x7f4010100e20 2024-09-19T04:30:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4016583640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4010102ef0 0x7f401019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4015d82640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010100e20 0x7f401019a5f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f400c00e9f0 tx=0x7f400c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.500+0000 7f4006ffd640 1 -- 172.21.15.94:0/981494447 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c00ccc0 con 0x7f4010100e20 2024-09-19T04:30:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.501+0000 7f4017fe6640 1 -- 172.21.15.94:0/981494447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401019f890 con 0x7f4010100e20 2024-09-19T04:30:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.501+0000 7f4006ffd640 1 -- 172.21.15.94:0/981494447 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f400c0108e0 con 0x7f4010100e20 2024-09-19T04:30:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.501+0000 7f4006ffd640 1 -- 172.21.15.94:0/981494447 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c018880 con 0x7f4010100e20 2024-09-19T04:30:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.501+0000 7f4017fe6640 1 -- 172.21.15.94:0/981494447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40101a7170 con 0x7f4010100e20 2024-09-19T04:30:03.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f401a3bf740 1 Processor -- start 2024-09-19T04:30:03.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f401a3bf740 1 -- start start 2024-09-19T04:30:03.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f4006ffd640 1 -- 172.21.15.94:0/981494447 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f400c010a90 con 0x7f4010100e20 2024-09-19T04:30:03.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 0x55b8394b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b75d0 0x55b8394b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394c2370 0x55b8394c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f401a3bf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b83941d2f0 con 0x55b8394b8b40 2024-09-19T04:30:03.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f401a3bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b8393c4cf0 con 0x55b8394c2370 2024-09-19T04:30:03.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f401a3bf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b8394192b0 con 0x55b8394b75d0 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f4015d82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b75d0 0x55b8394b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f4015d82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b75d0 0x55b8394b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47876/0 (socket says 172.21.15.94:47876) 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f4015d82640 1 -- 172.21.15.94:0/95317387 learned_addr learned my addr 172.21.15.94:0/95317387 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.504+0000 7f4015581640 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394c2370 0x55b8394c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f4016583640 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 0x55b8394b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f4015d82640 1 -- 172.21.15.94:0/95317387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394c2370 msgr2=0x55b8394c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f4015d82640 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394c2370 0x55b8394c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f4015d82640 1 -- 172.21.15.94:0/95317387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 msgr2=0x55b8394b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f4015d82640 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 0x55b8394b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f4015d82640 1 -- 172.21.15.94:0/95317387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8393a93a0 con 0x55b8394b75d0 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f4015581640 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394c2370 0x55b8394c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f4015d82640 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b75d0 0x55b8394b7a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f400c0cd690 tx=0x7f400c0d8700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b98c89360cd91720 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f3ffbfff640 1 -- 172.21.15.94:0/95317387 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0e3070 con 0x55b8394b75d0 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f3ffbfff640 1 -- 172.21.15.94:0/95317387 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f400c0d0970 con 0x55b8394b75d0 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.505+0000 7f3ffbfff640 1 -- 172.21.15.94:0/95317387 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0de980 con 0x55b8394b75d0 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f4006ffd640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f401a3bf740 1 -- 172.21.15.94:0/95317387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b75d0 msgr2=0x55b8394b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f401a3bf740 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b75d0 0x55b8394b7a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f400c0cd690 tx=0x7f400c0d8700 comp rx=0 tx=0).stop 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f401a3bf740 1 -- 172.21.15.94:0/95317387 shutdown_connections 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f401a3bf740 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394c2370 0x55b8394c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f401a3bf740 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b75d0 0x55b8394b7a10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f401a3bf740 1 --2- 172.21.15.94:0/95317387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 0x55b8394b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f401a3bf740 1 -- 172.21.15.94:0/95317387 >> 172.21.15.94:0/95317387 conn(0x55b839480310 msgr2=0x55b8394b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f4006ffd640 1 -- 172.21.15.94:0/981494447 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7f400c0d0010 con 0x7f4010100e20 2024-09-19T04:30:03.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.506+0000 7f4016583640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.507+0000 7f401a3bf740 1 -- 172.21.15.94:0/95317387 shutdown_connections 2024-09-19T04:30:03.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.507+0000 7f401a3bf740 1 -- 172.21.15.94:0/95317387 wait complete. 2024-09-19T04:30:03.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.507+0000 7f401a3bf740 1 -- start start 2024-09-19T04:30:03.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.507+0000 7f401a3bf740 1 Processor -- start 2024-09-19T04:30:03.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.507+0000 7f4016583640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f400800bfd0 tx=0x7f400800d040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 0x55b8394bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f401a3bf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b8394c5070 con 0x55b8394b8b40 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f401a3bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b8393d8060 con 0x55b8394b75d0 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f401a3bf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b8394c51f0 con 0x55b8394c2370 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f401a3bf740 10 client.0 _ll_register_callbacks cb 0x55b83912b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f4016583640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f4016583640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394bf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47884/0 (socket says 172.21.15.94:47884) 2024-09-19T04:30:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f4016583640 1 -- 172.21.15.94:0/3048431399 learned_addr learned my addr 172.21.15.94:0/3048431399 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f4015581640 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 0x55b8394bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.508+0000 7f4015d82640 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f4016583640 1 -- 172.21.15.94:0/3048431399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 msgr2=0x55b8394c1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f4016583640 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394c1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f4016583640 1 -- 172.21.15.94:0/3048431399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 msgr2=0x55b8394bf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f4016583640 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 0x55b8394bf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f4016583640 1 -- 172.21.15.94:0/3048431399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b83918e3c0 con 0x55b8394c2370 2024-09-19T04:30:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f4016583640 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394bf7f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4008019c40 tx=0x7f4008004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9260db276026f19c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f3ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f3ff97fa640 1 -- 172.21.15.94:0/3048431399 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400801f050 con 0x55b8394c2370 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f401a3bf740 1 -- 172.21.15.94:0/3048431399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f400c0d0970 con 0x55b8394c2370 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f3ff97fa640 1 -- 172.21.15.94:0/3048431399 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7f4008024640 con 0x55b8394c2370 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f3ff97fa640 1 -- 172.21.15.94:0/3048431399 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4008004b20 con 0x55b8394c2370 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f401a3bf740 10 client.26839 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.509+0000 7f401a3bf740 1 -- 172.21.15.94:0/3048431399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b8394d0c10 con 0x55b8394c2370 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.510+0000 7f3ff97fa640 1 -- 172.21.15.94:0/3048431399 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4008029070 con 0x55b8394c2370 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.510+0000 7f3ff97fa640 1 -- 172.21.15.94:0/3048431399 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f40080157a0 con 0x55b8394c2370 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.510+0000 7f401a3bf740 10 client.26839 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.510+0000 7f401a3bf740 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.510+0000 7f401a3bf740 20 client.26839 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.510+0000 7f401a3bf740 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b8394d20d0 0x55b8394d44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.510+0000 7f401a3bf740 4 client.26839 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.510+0000 7f401a3bf740 1 -- 172.21.15.94:0/3048431399 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f400c0d0970 con 0x55b8394d20d0 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.510+0000 7f4015d82640 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b8394d20d0 0x55b8394d44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.511+0000 7f4015d82640 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b8394d20d0 0x55b8394d44c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451ba1a71db67cc2 server_cookie=8428870af0d3fd49 in_seq=0 out_seq=0 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.511+0000 7f3ff97fa640 10 client.26839 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.511+0000 7f3ff97fa640 1 -- 172.21.15.94:0/3048431399 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f400c0d0970 con 0x55b8394d20d0 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.511+0000 7f3ff97fa640 10 client.26839 handle_command_reply: tid=0 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 client.26839 shutdown 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 -- 172.21.15.94:0/3048431399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 msgr2=0x55b8394bf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394bf7f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4008019c40 tx=0x7f4008004ab0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 -- 172.21.15.94:0/3048431399 shutdown_connections 2024-09-19T04:30:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b8394d20d0 0x55b8394d44c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394bf7f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b8b40 0x55b8394bf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f3ff97fa640 0 client.26839 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 --2- 172.21.15.94:0/3048431399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394c1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 -- 172.21.15.94:0/3048431399 >> 172.21.15.94:0/3048431399 conn(0x55b839480310 msgr2=0x55b8394b9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 -- 172.21.15.94:0/3048431399 shutdown_connections 2024-09-19T04:30:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.512+0000 7f401a3bf740 1 -- 172.21.15.94:0/3048431399 wait complete. 2024-09-19T04:30:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.513+0000 7f401a3bf740 20 client.26839 trim_cache size 0 max 16384 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.529+0000 7f401a3bf740 1 Processor -- start 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.529+0000 7f401a3bf740 1 -- start start 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b8b40 0x55b8394b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394c2370 0x55b8394aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f401a3bf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40101b23b0 con 0x55b8394c2370 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f401a3bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40101afc80 con 0x55b8394b75d0 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f401a3bf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40101adaf0 con 0x55b8394b8b40 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4015d82640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4015d82640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51970/0 (socket says 172.21.15.94:51970) 2024-09-19T04:30:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4015d82640 1 -- 172.21.15.94:0/304112970 learned_addr learned my addr 172.21.15.94:0/304112970 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4016583640 1 --2- 172.21.15.94:0/304112970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394c2370 0x55b8394aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4015581640 1 --2- 172.21.15.94:0/304112970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b8b40 0x55b8394b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4015d82640 1 -- 172.21.15.94:0/304112970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b8b40 msgr2=0x55b8394b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4015d82640 1 --2- 172.21.15.94:0/304112970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b8b40 0x55b8394b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4015d82640 1 -- 172.21.15.94:0/304112970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394c2370 msgr2=0x55b8394aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4015d82640 1 --2- 172.21.15.94:0/304112970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394c2370 0x55b8394aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.530+0000 7f4015d82640 1 -- 172.21.15.94:0/304112970 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8394b4dd0 con 0x55b8394b75d0 2024-09-19T04:30:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.531+0000 7f4015d82640 1 --2- 172.21.15.94:0/304112970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394b3e10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f400c0987c0 tx=0x7f400c009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a57f4cc9fb30d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.531+0000 7f3ffb7fe640 1 -- 172.21.15.94:0/304112970 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c098600 con 0x55b8394b75d0 2024-09-19T04:30:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.531+0000 7f3ffb7fe640 1 -- 172.21.15.94:0/304112970 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f400c010e00 con 0x55b8394b75d0 2024-09-19T04:30:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.531+0000 7f3ffb7fe640 1 -- 172.21.15.94:0/304112970 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c09bcc0 con 0x55b8394b75d0 2024-09-19T04:30:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 -- 172.21.15.94:0/304112970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 msgr2=0x55b8394b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 --2- 172.21.15.94:0/304112970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394b3e10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f400c0987c0 tx=0x7f400c009700 comp rx=0 tx=0).stop 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 -- 172.21.15.94:0/304112970 shutdown_connections 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 --2- 172.21.15.94:0/304112970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394c2370 0x55b8394aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 --2- 172.21.15.94:0/304112970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394b8b40 0x55b8394b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 --2- 172.21.15.94:0/304112970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b75d0 0x55b8394b3e10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 -- 172.21.15.94:0/304112970 >> 172.21.15.94:0/304112970 conn(0x55b839480310 msgr2=0x55b8394b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 -- 172.21.15.94:0/304112970 shutdown_connections 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 -- 172.21.15.94:0/304112970 wait complete. 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.532+0000 7f401a3bf740 1 -- start start 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f401a3bf740 1 Processor -- start 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b75d0 0x55b8394cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b8b40 0x55b8394c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f401a3bf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f401a3bf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ae140 con 0x55b8394b75d0 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f401a3bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40101af5b0 con 0x55b8394b8b40 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f4015d82640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b75d0 0x55b8394cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f401a3bf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ae810 con 0x55b8394c2370 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f4015d82640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b75d0 0x55b8394cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57872/0 (socket says 172.21.15.94:57872) 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f4015d82640 1 -- 172.21.15.94:0/4005933828 learned_addr learned my addr 172.21.15.94:0/4005933828 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f401a3bf740 10 client.0 _ll_register_callbacks cb 0x7f4008003b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f4016583640 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.533+0000 7f4015581640 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b8b40 0x55b8394c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f4015d82640 1 -- 172.21.15.94:0/4005933828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 msgr2=0x55b8394c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f4015d82640 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f4015d82640 1 -- 172.21.15.94:0/4005933828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b8b40 msgr2=0x55b8394c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f4015d82640 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b8b40 0x55b8394c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f4015d82640 1 -- 172.21.15.94:0/4005933828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8394c6bf0 con 0x55b8394b75d0 2024-09-19T04:30:03.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f4015d82640 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b75d0 0x55b8394cab50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f400c0d8240 tx=0x7f400c0d0390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd19bce9552698a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f3ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f3ff97fa640 1 -- 172.21.15.94:0/4005933828 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c09bea0 con 0x55b8394b75d0 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f3ff97fa640 1 -- 172.21.15.94:0/4005933828 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7f400c0b5250 con 0x55b8394b75d0 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f401a3bf740 1 -- 172.21.15.94:0/4005933828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b83942a860 con 0x55b8394b75d0 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f3ff97fa640 1 -- 172.21.15.94:0/4005933828 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f400c09c130 con 0x55b8394b75d0 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f401a3bf740 10 client.17256 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.534+0000 7f401a3bf740 1 -- 172.21.15.94:0/4005933828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b83953ccf0 con 0x55b8394b75d0 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.535+0000 7f3ff97fa640 1 -- 172.21.15.94:0/4005933828 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b83942a860 con 0x55b8394b75d0 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.535+0000 7f3ff97fa640 1 -- 172.21.15.94:0/4005933828 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f400c0cd9a0 con 0x55b8394b75d0 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.535+0000 7f401a3bf740 10 client.17256 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.535+0000 7f401a3bf740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.535+0000 7f401a3bf740 20 client.17256 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.535+0000 7f401a3bf740 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b83953cd60 0x55b83953f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.535+0000 7f401a3bf740 4 client.17256 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.535+0000 7f401a3bf740 1 -- 172.21.15.94:0/4005933828 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b839424d70 con 0x55b83953cd60 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.535+0000 7f4015581640 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b83953cd60 0x55b83953f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f3ff97fa640 10 client.17256 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f4015581640 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b83953cd60 0x55b83953f150 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff106df1233d14e server_cookie=4f4f4f9682a85cf4 in_seq=0 out_seq=0 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f3ff97fa640 1 -- 172.21.15.94:0/4005933828 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b839424d70 con 0x55b83953cd60 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f3ff97fa640 10 client.17256 handle_command_reply: tid=0 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f401a3bf740 1 client.17256 shutdown 2024-09-19T04:30:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f401a3bf740 1 -- 172.21.15.94:0/4005933828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b75d0 msgr2=0x55b8394cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f401a3bf740 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b75d0 0x55b8394cab50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f400c0d8240 tx=0x7f400c0d0390 comp rx=0 tx=0).stop 2024-09-19T04:30:03.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f401a3bf740 1 -- 172.21.15.94:0/4005933828 shutdown_connections 2024-09-19T04:30:03.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f401a3bf740 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55b83953cd60 0x55b83953f150 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f401a3bf740 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8394c2370 0x55b8394c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f401a3bf740 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b8394b8b40 0x55b8394c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f401a3bf740 1 --2- 172.21.15.94:0/4005933828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b8394b75d0 0x55b8394cab50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f401a3bf740 1 -- 172.21.15.94:0/4005933828 >> 172.21.15.94:0/4005933828 conn(0x55b839480310 msgr2=0x55b8394b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.536+0000 7f3ff97fa640 0 client.17256 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.537+0000 7f401a3bf740 1 -- 172.21.15.94:0/4005933828 shutdown_connections 2024-09-19T04:30:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.537+0000 7f401a3bf740 1 -- 172.21.15.94:0/4005933828 wait complete. 2024-09-19T04:30:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.537+0000 7f401a3bf740 20 client.17256 trim_cache size 0 max 16384 2024-09-19T04:30:03.543 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:30:03.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.537+0000 7f4017fe6640 1 -- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3ffc0a3430 msgr2=0x7f3ffc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.537+0000 7f4017fe6640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f400800bfd0 tx=0x7f400800d040 comp rx=0 tx=0).stop 2024-09-19T04:30:03.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.537+0000 7f4017fe6640 1 -- 172.21.15.94:0/981494447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010100e20 msgr2=0x7f401019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.537+0000 7f4017fe6640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010100e20 0x7f401019a5f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f400c00e9f0 tx=0x7f400c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.538+0000 7f4017fe6640 1 -- 172.21.15.94:0/981494447 shutdown_connections 2024-09-19T04:30:03.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.538+0000 7f4017fe6640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.538+0000 7f4017fe6640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4010102ef0 0x7f401019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.538+0000 7f4017fe6640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4010102570 0x7f401019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.538+0000 7f4017fe6640 1 --2- 172.21.15.94:0/981494447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010100e20 0x7f401019a5f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.538+0000 7f4017fe6640 1 -- 172.21.15.94:0/981494447 >> 172.21.15.94:0/981494447 conn(0x7f40100fc7d0 msgr2=0x7f40101048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.538+0000 7f4017fe6640 1 -- 172.21.15.94:0/981494447 shutdown_connections 2024-09-19T04:30:03.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:03.538+0000 7f4017fe6640 1 -- 172.21.15.94:0/981494447 wait complete. 2024-09-19T04:30:05.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:04 smithi112 ceph-mon[35051]: pgmap v470: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.6 MiB/s wr, 784 op/s 2024-09-19T04:30:05.334 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:30:05.335 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:30:05.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:04 smithi094 ceph-mon[30972]: pgmap v470: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.6 MiB/s wr, 784 op/s 2024-09-19T04:30:05.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:04 smithi104 ceph-mon[34145]: pgmap v470: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.6 MiB/s wr, 784 op/s 2024-09-19T04:30:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 830.0 s, 417.2 tps, lat 241.711 ms stddev 50.932 2024-09-19T04:30:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.244+0000 7fb965fe3640 1 -- 172.21.15.94:0/3116609060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960101df0 msgr2=0x7fb9601021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.244+0000 7fb965fe3640 1 --2- 172.21.15.94:0/3116609060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960101df0 0x7fb9601021d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb950009900 tx=0x7fb95002a410 comp rx=0 tx=0).stop 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.246+0000 7fb965fe3640 1 -- 172.21.15.94:0/3116609060 shutdown_connections 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.246+0000 7fb965fe3640 1 --2- 172.21.15.94:0/3116609060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb960106f20 0x7fb96010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.246+0000 7fb965fe3640 1 --2- 172.21.15.94:0/3116609060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9601027a0 0x7fb9601067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.246+0000 7fb965fe3640 1 --2- 172.21.15.94:0/3116609060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960101df0 0x7fb9601021d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.246+0000 7fb965fe3640 1 -- 172.21.15.94:0/3116609060 >> 172.21.15.94:0/3116609060 conn(0x7fb9600fd8b0 msgr2=0x7fb9600ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.247+0000 7fb965fe3640 1 -- 172.21.15.94:0/3116609060 shutdown_connections 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.247+0000 7fb965fe3640 1 -- 172.21.15.94:0/3116609060 wait complete. 2024-09-19T04:30:06.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.248+0000 7fb965fe3640 1 Processor -- start 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.248+0000 7fb965fe3640 1 -- start start 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.248+0000 7fb965fe3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb960101df0 0x7fb960198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.248+0000 7fb965fe3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9601027a0 0x7fb960198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.248+0000 7fb965fe3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960106f20 0x7fb96019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9601027a0 0x7fb960198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960106f20 0x7fb96019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960106f20 0x7fb96019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57884/0 (socket says 172.21.15.94:57884) 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9601027a0 0x7fb960198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:51996/0 (socket says 172.21.15.94:51996) 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95ffff640 1 -- 172.21.15.94:0/4112084067 learned_addr learned my addr 172.21.15.94:0/4112084067 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb960111e80 con 0x7fb960106f20 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb960111d00 con 0x7fb9601027a0 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb960112000 con 0x7fb960101df0 2024-09-19T04:30:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95effd640 1 -- 172.21.15.94:0/4112084067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb960101df0 msgr2=0x7fb960198340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95effd640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb960101df0 0x7fb960198340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95effd640 1 -- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960106f20 msgr2=0x7fb96019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95effd640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960106f20 0x7fb96019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.249+0000 7fb95effd640 1 -- 172.21.15.94:0/4112084067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96019d2f0 con 0x7fb9601027a0 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.250+0000 7fb95ffff640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960106f20 0x7fb96019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.250+0000 7fb95effd640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9601027a0 0x7fb960198880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb95400e9f0 tx=0x7fb95400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.250+0000 7fb95cff9640 1 -- 172.21.15.94:0/4112084067 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95400cd80 con 0x7fb9601027a0 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.250+0000 7fb95cff9640 1 -- 172.21.15.94:0/4112084067 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9540035f0 con 0x7fb9601027a0 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.251+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96019d5e0 con 0x7fb9601027a0 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.251+0000 7fb95cff9640 1 -- 172.21.15.94:0/4112084067 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954010670 con 0x7fb9601027a0 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.251+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9601a4ec0 con 0x7fb9601027a0 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.254+0000 7fb95cff9640 1 -- 172.21.15.94:0/4112084067 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fb9540037a0 con 0x7fb9601027a0 2024-09-19T04:30:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.255+0000 7fb95cff9640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9480a3430 0x7fb9480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.255+0000 7fb9683bc740 1 Processor -- start 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.255+0000 7fb9683bc740 1 -- start start 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.255+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9480a3430 0x7fb9480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.255+0000 7fb95cff9640 1 -- 172.21.15.94:0/4112084067 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fb9540d0010 con 0x7fb9601027a0 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc479de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc47a320 0x5617fc488c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 0x5617fc48b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb9683bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5617fc3df280 con 0x5617fc47a320 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb9683bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5617fc386b10 con 0x5617fc489150 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb9683bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5617fc3db240 con 0x5617fc479a00 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb95effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc47a320 0x5617fc488c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb95ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 0x5617fc48b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb95effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc47a320 0x5617fc488c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57900/0 (socket says 172.21.15.94:57900) 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb95ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 0x5617fc48b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52000/0 (socket says 172.21.15.94:52000) 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb95effd640 1 -- 172.21.15.94:0/2932829945 learned_addr learned my addr 172.21.15.94:0/2932829945 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.256+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9480a3430 0x7fb9480a58f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9500096c0 tx=0x7fb950005fd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.257+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/2932829945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc479de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.257+0000 7fb95effd640 1 -- 172.21.15.94:0/2932829945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 msgr2=0x5617fc479de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.257+0000 7fb95effd640 1 --2- 172.21.15.94:0/2932829945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc479de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.257+0000 7fb95effd640 1 -- 172.21.15.94:0/2932829945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 msgr2=0x5617fc48b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.257+0000 7fb95effd640 1 --2- 172.21.15.94:0/2932829945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 0x5617fc48b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.257+0000 7fb95effd640 1 -- 172.21.15.94:0/2932829945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617fc3931f0 con 0x5617fc47a320 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.258+0000 7fb95effd640 1 --2- 172.21.15.94:0/2932829945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc47a320 0x5617fc488c10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb954098c80 tx=0x7fb95409bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6f092dddccfb3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.258+0000 7fb945ffb640 1 -- 172.21.15.94:0/2932829945 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540b5070 con 0x5617fc47a320 2024-09-19T04:30:06.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.258+0000 7fb945ffb640 1 -- 172.21.15.94:0/2932829945 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb95409c2b0 con 0x5617fc47a320 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.259+0000 7fb945ffb640 1 -- 172.21.15.94:0/2932829945 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954098920 con 0x5617fc47a320 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.259+0000 7fb9683bc740 1 -- 172.21.15.94:0/2932829945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc47a320 msgr2=0x5617fc488c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.259+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2932829945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc47a320 0x5617fc488c10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb954098c80 tx=0x7fb95409bd10 comp rx=0 tx=0).stop 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.259+0000 7fb9683bc740 1 -- 172.21.15.94:0/2932829945 shutdown_connections 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.259+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2932829945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 0x5617fc48b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.259+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2932829945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc47a320 0x5617fc488c10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.259+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2932829945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc479de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.259+0000 7fb9683bc740 1 -- 172.21.15.94:0/2932829945 >> 172.21.15.94:0/2932829945 conn(0x5617fc4760f0 msgr2=0x5617fc4764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.259+0000 7fb9683bc740 1 -- 172.21.15.94:0/2932829945 shutdown_connections 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.260+0000 7fb9683bc740 1 -- 172.21.15.94:0/2932829945 wait complete. 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.260+0000 7fb9683bc740 1 -- start start 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.260+0000 7fb9683bc740 1 Processor -- start 2024-09-19T04:30:06.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc488bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc489150 0x5617fc481e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc482550 0x5617fc4866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb9683bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5617fc48bd70 con 0x5617fc489150 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb9683bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5617fc399f10 con 0x5617fc482550 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb9683bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5617fc48bef0 con 0x5617fc479a00 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb9683bc740 10 client.0 _ll_register_callbacks cb 0x5617fc362a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc488bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc488bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47954/0 (socket says 172.21.15.94:47954) 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb95f7fe640 1 -- 172.21.15.94:0/2627034011 learned_addr learned my addr 172.21.15.94:0/2627034011 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb95effd640 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc489150 0x5617fc481e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb95ffff640 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc482550 0x5617fc4866f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb95f7fe640 1 -- 172.21.15.94:0/2627034011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc482550 msgr2=0x5617fc4866f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:06.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc482550 0x5617fc4866f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.261+0000 7fb95f7fe640 1 -- 172.21.15.94:0/2627034011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc489150 msgr2=0x5617fc481e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.262+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc489150 0x5617fc481e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.262+0000 7fb95f7fe640 1 -- 172.21.15.94:0/2627034011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617fc1506a0 con 0x5617fc479a00 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.262+0000 7fb95effd640 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc489150 0x5617fc481e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.262+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc488bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb950033000 tx=0x7fb950031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d39bf897fb72710 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.262+0000 7fb93effd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.262+0000 7fb93effd640 1 -- 172.21.15.94:0/2627034011 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950035040 con 0x5617fc479a00 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.262+0000 7fb9683bc740 1 -- 172.21.15.94:0/2627034011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb95409c2b0 con 0x5617fc479a00 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.262+0000 7fb93effd640 1 -- 172.21.15.94:0/2627034011 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fb950003aa0 con 0x5617fc479a00 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb93effd640 1 -- 172.21.15.94:0/2627034011 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb950003ca0 con 0x5617fc479a00 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb9683bc740 10 client.26854 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb9683bc740 1 -- 172.21.15.94:0/2627034011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5617fc49c4a0 con 0x5617fc479a00 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb93effd640 1 -- 172.21.15.94:0/2627034011 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb95409c2b0 con 0x5617fc479a00 2024-09-19T04:30:06.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb93effd640 1 -- 172.21.15.94:0/2627034011 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fb9500584b0 con 0x5617fc479a00 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb9683bc740 10 client.26854 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb9683bc740 10 client.26854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb9683bc740 20 client.26854 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5617fc49d8b0 0x5617fc49fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb9683bc740 4 client.26854 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb9683bc740 1 -- 172.21.15.94:0/2627034011 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617fc3ec7f0 con 0x5617fc49d8b0 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.263+0000 7fb95effd640 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5617fc49d8b0 0x5617fc49fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.264+0000 7fb95effd640 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5617fc49d8b0 0x5617fc49fca0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b5294bdc829878 server_cookie=e09ddf43b6e5cc8c in_seq=0 out_seq=0 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.264+0000 7fb93effd640 10 client.26854 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb93effd640 1 -- 172.21.15.94:0/2627034011 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617fc3ec7f0 con 0x5617fc49d8b0 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb93effd640 10 client.26854 handle_command_reply: tid=0 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 client.26854 shutdown 2024-09-19T04:30:06.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 -- 172.21.15.94:0/2627034011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 msgr2=0x5617fc488bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc488bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb950033000 tx=0x7fb950031400 comp rx=0 tx=0).stop 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 -- 172.21.15.94:0/2627034011 shutdown_connections 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5617fc49d8b0 0x5617fc49fca0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb93effd640 0 client.26854 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc482550 0x5617fc4866f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc489150 0x5617fc481e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2627034011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc479a00 0x5617fc488bd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 -- 172.21.15.94:0/2627034011 >> 172.21.15.94:0/2627034011 conn(0x5617fc4760f0 msgr2=0x5617fc47be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 -- 172.21.15.94:0/2627034011 shutdown_connections 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.265+0000 7fb9683bc740 1 -- 172.21.15.94:0/2627034011 wait complete. 2024-09-19T04:30:06.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.266+0000 7fb9683bc740 20 client.26854 trim_cache size 0 max 16384 2024-09-19T04:30:06.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb9683bc740 1 Processor -- start 2024-09-19T04:30:06.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb9683bc740 1 -- start start 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc479a00 0x5617fc4866e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc482550 0x5617fc475160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc489150 0x5617fc4a5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb9683bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9601adff0 con 0x5617fc482550 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb9683bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9601ab8c0 con 0x5617fc479a00 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb9683bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9601a9730 con 0x5617fc489150 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb95f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc479a00 0x5617fc4866e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb95f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc479a00 0x5617fc4866e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52020/0 (socket says 172.21.15.94:52020) 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb95f7fe640 1 -- 172.21.15.94:0/2120225465 learned_addr learned my addr 172.21.15.94:0/2120225465 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb95ffff640 1 --2- 172.21.15.94:0/2120225465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc489150 0x5617fc4a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.290+0000 7fb95effd640 1 --2- 172.21.15.94:0/2120225465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc482550 0x5617fc475160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.291+0000 7fb95f7fe640 1 -- 172.21.15.94:0/2120225465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc489150 msgr2=0x5617fc4a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.291+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/2120225465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc489150 0x5617fc4a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.291+0000 7fb95f7fe640 1 -- 172.21.15.94:0/2120225465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc482550 msgr2=0x5617fc475160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.291+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/2120225465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc482550 0x5617fc475160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.291+0000 7fb95f7fe640 1 -- 172.21.15.94:0/2120225465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617fc488f40 con 0x5617fc479a00 2024-09-19T04:30:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.291+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/2120225465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc479a00 0x5617fc4866e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb950009800 tx=0x7fb9500045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df6c57d5065a559 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.291+0000 7fb9457fa640 1 -- 172.21.15.94:0/2120225465 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95005d030 con 0x5617fc479a00 2024-09-19T04:30:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.291+0000 7fb9457fa640 1 -- 172.21.15.94:0/2120225465 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb950040030 con 0x5617fc479a00 2024-09-19T04:30:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.291+0000 7fb9457fa640 1 -- 172.21.15.94:0/2120225465 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950035070 con 0x5617fc479a00 2024-09-19T04:30:06.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.292+0000 7fb9683bc740 1 -- 172.21.15.94:0/2120225465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc479a00 msgr2=0x5617fc4866e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.292+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2120225465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc479a00 0x5617fc4866e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb950009800 tx=0x7fb9500045a0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.292+0000 7fb9683bc740 1 -- 172.21.15.94:0/2120225465 shutdown_connections 2024-09-19T04:30:06.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.292+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2120225465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc489150 0x5617fc4a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.292+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2120225465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc482550 0x5617fc475160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.292+0000 7fb9683bc740 1 --2- 172.21.15.94:0/2120225465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc479a00 0x5617fc4866e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.292+0000 7fb9683bc740 1 -- 172.21.15.94:0/2120225465 >> 172.21.15.94:0/2120225465 conn(0x5617fc4760f0 msgr2=0x5617fc474a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.292+0000 7fb9683bc740 1 -- 172.21.15.94:0/2120225465 shutdown_connections 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.292+0000 7fb9683bc740 1 -- 172.21.15.94:0/2120225465 wait complete. 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.293+0000 7fb9683bc740 1 -- start start 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.293+0000 7fb9683bc740 1 Processor -- start 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.293+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc479a00 0x5617fc495a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.293+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc482550 0x5617fc495f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.293+0000 7fb9683bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 0x5617fc4964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.293+0000 7fb9683bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9601a9d80 con 0x5617fc479a00 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.293+0000 7fb9683bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9601ab1f0 con 0x5617fc489150 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.293+0000 7fb9683bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9601aa450 con 0x5617fc482550 2024-09-19T04:30:06.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.293+0000 7fb9683bc740 10 client.0 _ll_register_callbacks cb 0x7fb9580093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc482550 0x5617fc495f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc479a00 0x5617fc495a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc482550 0x5617fc495f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:47972/0 (socket says 172.21.15.94:47972) 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95effd640 1 -- 172.21.15.94:0/206649133 learned_addr learned my addr 172.21.15.94:0/206649133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95ffff640 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 0x5617fc4964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95effd640 1 -- 172.21.15.94:0/206649133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 msgr2=0x5617fc4964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95effd640 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 0x5617fc4964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95effd640 1 -- 172.21.15.94:0/206649133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc479a00 msgr2=0x5617fc495a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95effd640 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc479a00 0x5617fc495a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95effd640 1 -- 172.21.15.94:0/206649133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617fc507650 con 0x5617fc482550 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb95effd640 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc482550 0x5617fc495f60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb95409bfb0 tx=0x7fb954010810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d80ed7ae0ccf33f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb93effd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb93effd640 1 -- 172.21.15.94:0/206649133 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540b5040 con 0x5617fc482550 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb93effd640 1 -- 172.21.15.94:0/206649133 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8585+0+0 (secure 0 0 0) 0x7fb9540cd9a0 con 0x5617fc482550 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.294+0000 7fb9683bc740 1 -- 172.21.15.94:0/206649133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb950040030 con 0x5617fc482550 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb93effd640 1 -- 172.21.15.94:0/206649133 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb9540a0660 con 0x5617fc482550 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb9683bc740 10 client.26860 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb9683bc740 1 -- 172.21.15.94:0/206649133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5617fc508970 con 0x5617fc482550 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb93effd640 1 -- 172.21.15.94:0/206649133 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb95409c2b0 con 0x5617fc482550 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb93effd640 1 -- 172.21.15.94:0/206649133 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fb9540bc0b0 con 0x5617fc482550 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb9683bc740 10 client.26860 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb9683bc740 10 client.26860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb9683bc740 20 client.26860 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb9683bc740 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5617fc49d8b0 0x5617fc50ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:06.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb9683bc740 4 client.26860 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:30:06.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb9683bc740 1 -- 172.21.15.94:0/206649133 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb950040030 con 0x5617fc49d8b0 2024-09-19T04:30:06.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.295+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5617fc49d8b0 0x5617fc50ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:06.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.296+0000 7fb93effd640 10 client.26860 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:06.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.296+0000 7fb95f7fe640 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5617fc49d8b0 0x5617fc50ad10 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7108069fc7b78670 server_cookie=5ebf095c25047206 in_seq=0 out_seq=0 2024-09-19T04:30:06.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb93effd640 1 -- 172.21.15.94:0/206649133 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb950040030 con 0x5617fc49d8b0 2024-09-19T04:30:06.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb93effd640 10 client.26860 handle_command_reply: tid=0 2024-09-19T04:30:06.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 client.26860 shutdown 2024-09-19T04:30:06.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 -- 172.21.15.94:0/206649133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc482550 msgr2=0x5617fc495f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc482550 0x5617fc495f60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb95409bfb0 tx=0x7fb954010810 comp rx=0 tx=0).stop 2024-09-19T04:30:06.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 -- 172.21.15.94:0/206649133 shutdown_connections 2024-09-19T04:30:06.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5617fc49d8b0 0x5617fc50ad10 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617fc489150 0x5617fc4964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617fc482550 0x5617fc495f60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 --2- 172.21.15.94:0/206649133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5617fc479a00 0x5617fc495a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 -- 172.21.15.94:0/206649133 >> 172.21.15.94:0/206649133 conn(0x5617fc4760f0 msgr2=0x5617fc47bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:06.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb93effd640 0 client.26860 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:06.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 -- 172.21.15.94:0/206649133 shutdown_connections 2024-09-19T04:30:06.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.297+0000 7fb9683bc740 1 -- 172.21.15.94:0/206649133 wait complete. 2024-09-19T04:30:06.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb9683bc740 20 client.26860 trim_cache size 0 max 16384 2024-09-19T04:30:06.303 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:30:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:30:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "1b1ca263-bb93-4025-a2a2-a0a77f9c2f42", 2024-09-19T04:30:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:30:06.304 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:30:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9480a3430 msgr2=0x7fb9480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9480a3430 0x7fb9480a58f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9500096c0 tx=0x7fb950005fd0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9601027a0 msgr2=0x7fb960198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9601027a0 0x7fb960198880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb95400e9f0 tx=0x7fb95400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 shutdown_connections 2024-09-19T04:30:06.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb9480a3430 0x7fb9480a58f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960106f20 0x7fb96019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9601027a0 0x7fb960198880 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 --2- 172.21.15.94:0/4112084067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb960101df0 0x7fb960198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.298+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 >> 172.21.15.94:0/4112084067 conn(0x7fb9600fd8b0 msgr2=0x7fb9600fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:06.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.299+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 shutdown_connections 2024-09-19T04:30:06.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:06.299+0000 7fb965fe3640 1 -- 172.21.15.94:0/4112084067 wait complete. 2024-09-19T04:30:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:06 smithi094 ceph-mon[30972]: pgmap v471: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-19T04:30:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:06 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:06 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:06 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:06 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:06 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:06 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:06 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:06 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:30:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:06 smithi112 ceph-mon[35051]: pgmap v471: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-19T04:30:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:06 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:06 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:06 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:06 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:06 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:06 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:06 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:06 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:06 smithi104 ceph-mon[34145]: pgmap v471: 97 pgs: 97 active+clean; 3.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-19T04:30:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:06 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:06 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:06 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:07.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:06 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:06 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:07.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:06 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:07.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:06 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:06 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.601 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:07.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:07 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:30:07.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:07 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:07 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:07.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:07 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:07.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:07 smithi094 ceph-mon[30972]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T04:30:08.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:07 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:30:08.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:07 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:08.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:07 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:08.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:07 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:08.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:07 smithi112 ceph-mon[35051]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T04:30:08.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.349+0000 7f1f22e1b640 1 -- 172.21.15.94:0/844356702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f1c1082a0 msgr2=0x7f1f1c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:08.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.349+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/844356702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f1c1082a0 0x7f1f1c10ec40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009cd0 tx=0x7f1f1402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.349+0000 7f1f22e1b640 1 -- 172.21.15.94:0/844356702 shutdown_connections 2024-09-19T04:30:08.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.349+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/844356702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f1c1082a0 0x7f1f1c10ec40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.349+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/844356702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f1c1036e0 0x7f1f1c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.349+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/844356702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f1c102d30 0x7f1f1c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.349+0000 7f1f22e1b640 1 -- 172.21.15.94:0/844356702 >> 172.21.15.94:0/844356702 conn(0x7f1f1c0fc7d0 msgr2=0x7f1f1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:08.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.350+0000 7f1f22e1b640 1 -- 172.21.15.94:0/844356702 shutdown_connections 2024-09-19T04:30:08.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.350+0000 7f1f22e1b640 1 -- 172.21.15.94:0/844356702 wait complete. 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.350+0000 7f1f22e1b640 1 Processor -- start 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f22e1b640 1 -- start start 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f22e1b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f1c102d30 0x7f1f1c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f22e1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f1c1036e0 0x7f1f1c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f22e1b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f1c1082a0 0x7f1f1c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f20bb7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f1c102d30 0x7f1f1c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f20bb7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f1c102d30 0x7f1f1c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52046/0 (socket says 172.21.15.94:52046) 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f1bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f1c1036e0 0x7f1f1c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f20bb7640 1 -- 172.21.15.94:0/1182376825 learned_addr learned my addr 172.21.15.94:0/1182376825 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c111f50 con 0x7f1f1c1082a0 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f213b8640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f1c1082a0 0x7f1f1c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c111dd0 con 0x7f1f1c102d30 2024-09-19T04:30:08.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.351+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1120d0 con 0x7f1f1c1036e0 2024-09-19T04:30:08.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.352+0000 7f1f20bb7640 1 -- 172.21.15.94:0/1182376825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f1c1036e0 msgr2=0x7f1f1c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:08.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.352+0000 7f1f20bb7640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f1c1036e0 0x7f1f1c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.352+0000 7f1f20bb7640 1 -- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f1c1082a0 msgr2=0x7f1f1c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:08.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.352+0000 7f1f20bb7640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f1c1082a0 0x7f1f1c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.352+0000 7f1f20bb7640 1 -- 172.21.15.94:0/1182376825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c19f660 con 0x7f1f1c102d30 2024-09-19T04:30:08.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.352+0000 7f1f20bb7640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f1c102d30 0x7f1f1c19a6b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e9f0 tx=0x7f1f1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:08.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.352+0000 7f1f19ffb640 1 -- 172.21.15.94:0/1182376825 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000cd80 con 0x7f1f1c102d30 2024-09-19T04:30:08.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:07 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:30:08.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:07 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:08.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:07 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:08.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:07 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:08.356 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:07 smithi104 ceph-mon[34145]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T04:30:08.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.352+0000 7f1f19ffb640 1 -- 172.21.15.94:0/1182376825 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f100108e0 con 0x7f1f1c102d30 2024-09-19T04:30:08.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.353+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c19f950 con 0x7f1f1c102d30 2024-09-19T04:30:08.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.353+0000 7f1f19ffb640 1 -- 172.21.15.94:0/1182376825 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10018920 con 0x7f1f1c102d30 2024-09-19T04:30:08.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.353+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c1a7230 con 0x7f1f1c102d30 2024-09-19T04:30:08.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.356+0000 7f1f19ffb640 1 -- 172.21.15.94:0/1182376825 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1f10010a90 con 0x7f1f1c102d30 2024-09-19T04:30:08.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.356+0000 7f1f251f4740 1 Processor -- start 2024-09-19T04:30:08.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.356+0000 7f1f251f4740 1 -- start start 2024-09-19T04:30:08.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.356+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c74a0 0x55912e7c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.356+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c7dc0 0x55912e7d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.356+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.356+0000 7f1f251f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55912e72d2e0 con 0x55912e7d2680 2024-09-19T04:30:08.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.356+0000 7f1f251f4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55912e6d4ce0 con 0x55912e7c74a0 2024-09-19T04:30:08.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.356+0000 7f1f251f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55912e7292a0 con 0x55912e7c7dc0 2024-09-19T04:30:08.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f213b8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c74a0 0x55912e7c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f213b8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c74a0 0x55912e7c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:52058/0 (socket says 172.21.15.94:52058) 2024-09-19T04:30:08.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f213b8640 1 -- 172.21.15.94:0/2471320831 learned_addr learned my addr 172.21.15.94:0/2471320831 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:08.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7d4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57960/0 (socket says 172.21.15.94:57960) 2024-09-19T04:30:08.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f20bb7640 1 --2- 172.21.15.94:0/2471320831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c7dc0 0x55912e7d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f213b8640 1 -- 172.21.15.94:0/2471320831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c7dc0 msgr2=0x55912e7d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:08.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f213b8640 1 --2- 172.21.15.94:0/2471320831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c7dc0 0x55912e7d2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f213b8640 1 -- 172.21.15.94:0/2471320831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 msgr2=0x55912e7d4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:08.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f213b8640 1 --2- 172.21.15.94:0/2471320831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7d4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f213b8640 1 -- 172.21.15.94:0/2471320831 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55912e6b9440 con 0x55912e7c74a0 2024-09-19T04:30:08.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.357+0000 7f1f1bfff640 1 --2- 172.21.15.94:0/2471320831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:08.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.358+0000 7f1f213b8640 1 --2- 172.21.15.94:0/2471320831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c74a0 0x55912e7c7880 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009cd0 tx=0x7f1f14009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811b77d8e046be0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:08.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.358+0000 7f1f06ffd640 1 -- 172.21.15.94:0/2471320831 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14049070 con 0x55912e7c74a0 2024-09-19T04:30:08.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.358+0000 7f1f19ffb640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1f0c0a3430 0x7f1f0c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.358+0000 7f1f06ffd640 1 -- 172.21.15.94:0/2471320831 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f14036040 con 0x55912e7c74a0 2024-09-19T04:30:08.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.358+0000 7f1f06ffd640 1 -- 172.21.15.94:0/2471320831 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14004040 con 0x55912e7c74a0 2024-09-19T04:30:08.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.358+0000 7f1f19ffb640 1 -- 172.21.15.94:0/1182376825 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f1f100d00c0 con 0x7f1f1c102d30 2024-09-19T04:30:08.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.358+0000 7f1f20bb7640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1f0c0a3430 0x7f1f0c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f251f4740 1 -- 172.21.15.94:0/2471320831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c74a0 msgr2=0x55912e7c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2471320831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c74a0 0x55912e7c7880 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009cd0 tx=0x7f1f14009a20 comp rx=0 tx=0).stop 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f251f4740 1 -- 172.21.15.94:0/2471320831 shutdown_connections 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2471320831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2471320831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c7dc0 0x55912e7d2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2471320831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c74a0 0x55912e7c7880 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f251f4740 1 -- 172.21.15.94:0/2471320831 >> 172.21.15.94:0/2471320831 conn(0x55912e7c3a30 msgr2=0x55912e7c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f20bb7640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1f0c0a3430 0x7f1f0c0a58f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55912e7c4890 tx=0x7f1f10000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f251f4740 1 -- 172.21.15.94:0/2471320831 shutdown_connections 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.359+0000 7f1f251f4740 1 -- 172.21.15.94:0/2471320831 wait complete. 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.360+0000 7f1f251f4740 1 -- start start 2024-09-19T04:30:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.360+0000 7f1f251f4740 1 Processor -- start 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.360+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c74a0 0x55912e7d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.360+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.360+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7d2680 0x55912e7cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.360+0000 7f1f251f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55912e7d5250 con 0x55912e7c74a0 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.360+0000 7f1f251f4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55912e6e8050 con 0x55912e7c7dc0 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.360+0000 7f1f251f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55912e7d53d0 con 0x55912e7d2680 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f251f4740 10 client.0 _ll_register_callbacks cb 0x55912e728160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f213b8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c74a0 0x55912e7d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f213b8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c74a0 0x55912e7d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57966/0 (socket says 172.21.15.94:57966) 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f213b8640 1 -- 172.21.15.94:0/1477014340 learned_addr learned my addr 172.21.15.94:0/1477014340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f1bfff640 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f213b8640 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7d2680 0x55912e7cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f1bfff640 1 -- 172.21.15.94:0/1477014340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7d2680 msgr2=0x55912e7cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f1bfff640 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7d2680 0x55912e7cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f1bfff640 1 -- 172.21.15.94:0/1477014340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c74a0 msgr2=0x55912e7d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f1bfff640 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c74a0 0x55912e7d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.361+0000 7f1f1bfff640 1 -- 172.21.15.94:0/1477014340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55912e49e3c0 con 0x55912e7c7dc0 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.362+0000 7f1f1bfff640 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7cf2a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c1027d0 tx=0x7f1f08012a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6abd8d0259f4bff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.362+0000 7f1efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.362+0000 7f1efbfff640 1 -- 172.21.15.94:0/1477014340 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08004050 con 0x55912e7c7dc0 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.362+0000 7f1f251f4740 1 -- 172.21.15.94:0/1477014340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f14036040 con 0x55912e7c7dc0 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.362+0000 7f1efbfff640 1 -- 172.21.15.94:0/1477014340 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f1f080041f0 con 0x55912e7c7dc0 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.362+0000 7f1efbfff640 1 -- 172.21.15.94:0/1477014340 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1f080043f0 con 0x55912e7c7dc0 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.362+0000 7f1f251f4740 10 client.26909 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:08.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.362+0000 7f1f251f4740 1 -- 172.21.15.94:0/1477014340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55912e7e0cd0 con 0x55912e7c7dc0 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.362+0000 7f1efbfff640 1 -- 172.21.15.94:0/1477014340 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f14036040 con 0x55912e7c7dc0 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.363+0000 7f1efbfff640 1 -- 172.21.15.94:0/1477014340 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f1f0800f960 con 0x55912e7c7dc0 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.363+0000 7f1f251f4740 10 client.26909 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.363+0000 7f1f251f4740 10 client.26909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.363+0000 7f1f251f4740 20 client.26909 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.363+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55912e7e2550 0x55912e7e4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.363+0000 7f1f251f4740 4 client.26909 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.363+0000 7f1f251f4740 1 -- 172.21.15.94:0/1477014340 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55912e73a850 con 0x55912e7e2550 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.363+0000 7f1f213b8640 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55912e7e2550 0x55912e7e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.363+0000 7f1efbfff640 10 client.26909 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.364+0000 7f1f213b8640 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55912e7e2550 0x55912e7e4970 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf3e80aee407c98 server_cookie=145f2b32169d6900 in_seq=0 out_seq=0 2024-09-19T04:30:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.364+0000 7f1efbfff640 1 -- 172.21.15.94:0/1477014340 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55912e73a850 con 0x55912e7e2550 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.364+0000 7f1efbfff640 10 client.26909 handle_command_reply: tid=0 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.364+0000 7f1f251f4740 1 client.26909 shutdown 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 -- 172.21.15.94:0/1477014340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 msgr2=0x55912e7cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7cf2a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c1027d0 tx=0x7f1f08012a80 comp rx=0 tx=0).stop 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 -- 172.21.15.94:0/1477014340 shutdown_connections 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55912e7e2550 0x55912e7e4970 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7d2680 0x55912e7cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7cf2a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1efbfff640 0 client.26909 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1477014340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c74a0 0x55912e7d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 -- 172.21.15.94:0/1477014340 >> 172.21.15.94:0/1477014340 conn(0x55912e7c3a30 msgr2=0x55912e7c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 -- 172.21.15.94:0/1477014340 shutdown_connections 2024-09-19T04:30:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 1 -- 172.21.15.94:0/1477014340 wait complete. 2024-09-19T04:30:08.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.365+0000 7f1f251f4740 20 client.26909 trim_cache size 0 max 16384 2024-09-19T04:30:08.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.387+0000 7f1f251f4740 1 Processor -- start 2024-09-19T04:30:08.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.387+0000 7f1f251f4740 1 -- start start 2024-09-19T04:30:08.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.387+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 0x55912e7de4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.387+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.387+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7ea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.387+0000 7f1f251f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1ae090 con 0x55912e7d2680 2024-09-19T04:30:08.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.387+0000 7f1f251f4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1ab960 con 0x55912e7c7dc0 2024-09-19T04:30:08.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.387+0000 7f1f251f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1a97f0 con 0x55912e7c74a0 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7ea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7ea240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57974/0 (socket says 172.21.15.94:57974) 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 -- 172.21.15.94:0/1468685213 learned_addr learned my addr 172.21.15.94:0/1468685213 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f1bfff640 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 0x55912e7de4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 -- 172.21.15.94:0/1468685213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 msgr2=0x55912e7de4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 0x55912e7de4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 -- 172.21.15.94:0/1468685213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 msgr2=0x55912e7ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 -- 172.21.15.94:0/1468685213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55912e7ce250 con 0x55912e7d2680 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f1bfff640 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f213b8640 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7ea240 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1f1402ae00 tx=0x7f1f14046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392e39b9aee79bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f067fc640 1 -- 172.21.15.94:0/1468685213 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14059040 con 0x55912e7d2680 2024-09-19T04:30:08.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.388+0000 7f1f067fc640 1 -- 172.21.15.94:0/1468685213 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f14036070 con 0x55912e7d2680 2024-09-19T04:30:08.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.389+0000 7f1f067fc640 1 -- 172.21.15.94:0/1468685213 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f140307a0 con 0x55912e7d2680 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.389+0000 7f1f251f4740 1 -- 172.21.15.94:0/1468685213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 msgr2=0x55912e7ea240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.389+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7ea240 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1f1402ae00 tx=0x7f1f14046040 comp rx=0 tx=0).stop 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.389+0000 7f1f251f4740 1 -- 172.21.15.94:0/1468685213 shutdown_connections 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.389+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7d2680 0x55912e7ea240 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.389+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7c7dc0 0x55912e7ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.389+0000 7f1f251f4740 1 --2- 172.21.15.94:0/1468685213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 0x55912e7de4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.389+0000 7f1f251f4740 1 -- 172.21.15.94:0/1468685213 >> 172.21.15.94:0/1468685213 conn(0x55912e7c3a30 msgr2=0x55912e7ca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.390+0000 7f1f251f4740 1 -- 172.21.15.94:0/1468685213 shutdown_connections 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.390+0000 7f1f251f4740 1 -- 172.21.15.94:0/1468685213 wait complete. 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.390+0000 7f1f251f4740 1 -- start start 2024-09-19T04:30:08.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.390+0000 7f1f251f4740 1 Processor -- start 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 0x55912e7dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c7dc0 0x55912e7d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f251f4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7d2680 0x55912e7d63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f251f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1a9e40 con 0x55912e7c7dc0 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f251f4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1ab290 con 0x55912e7d2680 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f251f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1aa4f0 con 0x55912e7c74a0 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f251f4740 10 client.0 _ll_register_callbacks cb 0x7f1f08009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c7dc0 0x55912e7d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f213b8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7d2680 0x55912e7d63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c7dc0 0x55912e7d5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57978/0 (socket says 172.21.15.94:57978) 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f1bfff640 1 -- 172.21.15.94:0/2181613149 learned_addr learned my addr 172.21.15.94:0/2181613149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:08.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f213b8640 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 0x55912e7dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f213b8640 1 -- 172.21.15.94:0/2181613149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 msgr2=0x55912e7dab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f213b8640 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 0x55912e7dab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f213b8640 1 -- 172.21.15.94:0/2181613149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c7dc0 msgr2=0x55912e7d5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f213b8640 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c7dc0 0x55912e7d5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f213b8640 1 -- 172.21.15.94:0/2181613149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55912e7d2360 con 0x55912e7d2680 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1f213b8640 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7d2680 0x55912e7d63d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55912e772820 tx=0x7f1f14030640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb9193cc56c4438 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.391+0000 7f1efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.392+0000 7f1efbfff640 1 -- 172.21.15.94:0/2181613149 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14031a80 con 0x55912e7d2680 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.392+0000 7f1efbfff640 1 -- 172.21.15.94:0/2181613149 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f1f14041040 con 0x55912e7d2680 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.392+0000 7f1f251f4740 1 -- 172.21.15.94:0/2181613149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f14036070 con 0x55912e7d2680 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.392+0000 7f1efbfff640 1 -- 172.21.15.94:0/2181613149 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1f14031c20 con 0x55912e7d2680 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.392+0000 7f1f251f4740 10 client.26915 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.392+0000 7f1f251f4740 1 -- 172.21.15.94:0/2181613149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55912e84ce60 con 0x55912e7d2680 2024-09-19T04:30:08.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.392+0000 7f1efbfff640 1 -- 172.21.15.94:0/2181613149 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f14050e30 con 0x55912e7d2680 2024-09-19T04:30:08.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.393+0000 7f1efbfff640 1 -- 172.21.15.94:0/2181613149 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f1f14030070 con 0x55912e7d2680 2024-09-19T04:30:08.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.393+0000 7f1f251f4740 10 client.26915 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:08.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.393+0000 7f1f251f4740 10 client.26915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:08.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.393+0000 7f1f251f4740 20 client.26915 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:08.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.393+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55912e7e2550 0x55912e84f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:08.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.393+0000 7f1f251f4740 4 client.26915 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:30:08.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.393+0000 7f1f251f4740 1 -- 172.21.15.94:0/2181613149 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1f14036070 con 0x55912e7e2550 2024-09-19T04:30:08.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.393+0000 7f1f1bfff640 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55912e7e2550 0x55912e84f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:08.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f1bfff640 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55912e7e2550 0x55912e84f150 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a41ac339243f8a server_cookie=162fa7c5cca129a5 in_seq=0 out_seq=0 2024-09-19T04:30:08.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1efbfff640 10 client.26915 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1efbfff640 1 -- 172.21.15.94:0/2181613149 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f1f14036070 con 0x55912e7e2550 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1efbfff640 10 client.26915 handle_command_reply: tid=0 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f251f4740 1 client.26915 shutdown 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f251f4740 1 -- 172.21.15.94:0/2181613149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7d2680 msgr2=0x55912e7d63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7d2680 0x55912e7d63d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55912e772820 tx=0x7f1f14030640 comp rx=0 tx=0).stop 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f251f4740 1 -- 172.21.15.94:0/2181613149 shutdown_connections 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55912e7e2550 0x55912e84f150 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1efbfff640 0 client.26915 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912e7d2680 0x55912e7d63d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912e7c7dc0 0x55912e7d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f251f4740 1 --2- 172.21.15.94:0/2181613149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55912e7c74a0 0x55912e7dab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout: "1b1ca263-bb93-4025-a2a2-a0a77f9c2f42": { 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "1b1ca263-bb93-4025-a2a2-a0a77f9c2f42", 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:30:08.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.394+0000 7f1f251f4740 1 -- 172.21.15.94:0/2181613149 >> 172.21.15.94:0/2181613149 conn(0x55912e7c3a30 msgr2=0x55912e7d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.395+0000 7f1f251f4740 1 -- 172.21.15.94:0/2181613149 shutdown_connections 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.395+0000 7f1f251f4740 1 -- 172.21.15.94:0/2181613149 wait complete. 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.395+0000 7f1f251f4740 20 client.26915 trim_cache size 0 max 16384 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.395+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1f0c0a3430 msgr2=0x7f1f0c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.395+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1f0c0a3430 0x7f1f0c0a58f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55912e7c4890 tx=0x7f1f10000bc0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f1c102d30 msgr2=0x7f1f1c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f1c102d30 0x7f1f1c19a6b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e9f0 tx=0x7f1f1000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 shutdown_connections 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1f0c0a3430 0x7f1f0c0a58f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f1c1082a0 0x7f1f1c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f1c1036e0 0x7f1f1c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 --2- 172.21.15.94:0/1182376825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f1c102d30 0x7f1f1c19a6b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:08.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 >> 172.21.15.94:0/1182376825 conn(0x7f1f1c0fc7d0 msgr2=0x7f1f1c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 shutdown_connections 2024-09-19T04:30:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:08.396+0000 7f1f22e1b640 1 -- 172.21.15.94:0/1182376825 wait complete. 2024-09-19T04:30:09.193 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:08 smithi094 ceph-mon[30972]: pgmap v473: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 5.4 MiB/s wr, 735 op/s 2024-09-19T04:30:09.195 INFO:tasks.cephfs.filesystem:scrub status for tag:1b1ca263-bb93-4025-a2a2-a0a77f9c2f42 - {'path': '/', 'tag': '1b1ca263-bb93-4025-a2a2-a0a77f9c2f42', 'options': 'recursive,force'} 2024-09-19T04:30:09.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:08 smithi112 ceph-mon[35051]: pgmap v473: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 5.4 MiB/s wr, 735 op/s 2024-09-19T04:30:09.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:08 smithi104 ceph-mon[34145]: pgmap v473: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 5.4 MiB/s wr, 735 op/s 2024-09-19T04:30:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 835.0 s, 392.4 tps, lat 252.911 ms stddev 56.112 2024-09-19T04:30:11.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:10 smithi112 ceph-mon[35051]: pgmap v474: 97 pgs: 97 active+clean; 3.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 4.1 MiB/s wr, 589 op/s 2024-09-19T04:30:11.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:10 smithi112 ceph-mon[35051]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T04:30:11.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:10 smithi094 ceph-mon[30972]: pgmap v474: 97 pgs: 97 active+clean; 3.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 4.1 MiB/s wr, 589 op/s 2024-09-19T04:30:11.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:10 smithi094 ceph-mon[30972]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T04:30:11.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:10 smithi104 ceph-mon[34145]: pgmap v474: 97 pgs: 97 active+clean; 3.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 4.1 MiB/s wr, 589 op/s 2024-09-19T04:30:11.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:10 smithi104 ceph-mon[34145]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T04:30:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:11 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:30:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:11 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:30:12.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:11 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:30:12.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:11 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:30:12.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:11 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:30:12.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:11 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:30:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:11 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:30:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:11 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:30:12.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:11 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:30:13.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:12 smithi112 ceph-mon[35051]: pgmap v476: 97 pgs: 97 active+clean; 3.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-09-19T04:30:13.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:12 smithi094 ceph-mon[30972]: pgmap v476: 97 pgs: 97 active+clean; 3.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-09-19T04:30:13.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:12 smithi104 ceph-mon[34145]: pgmap v476: 97 pgs: 97 active+clean; 3.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-09-19T04:30:15.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:14 smithi112 ceph-mon[35051]: pgmap v477: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 4.8 MiB/s wr, 678 op/s 2024-09-19T04:30:15.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:14 smithi094 ceph-mon[30972]: pgmap v477: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 4.8 MiB/s wr, 678 op/s 2024-09-19T04:30:15.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:14 smithi104 ceph-mon[34145]: pgmap v477: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 4.8 MiB/s wr, 678 op/s 2024-09-19T04:30:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 840.0 s, 403.0 tps, lat 248.952 ms stddev 24.991 2024-09-19T04:30:17.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:16 smithi112 ceph-mon[35051]: pgmap v478: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 4.4 MiB/s wr, 621 op/s 2024-09-19T04:30:17.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:16 smithi094 ceph-mon[30972]: pgmap v478: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 4.4 MiB/s wr, 621 op/s 2024-09-19T04:30:17.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:16 smithi104 ceph-mon[34145]: pgmap v478: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 4.4 MiB/s wr, 621 op/s 2024-09-19T04:30:19.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:18 smithi112 ceph-mon[35051]: pgmap v479: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 4.3 MiB/s wr, 599 op/s 2024-09-19T04:30:19.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:18 smithi094 ceph-mon[30972]: pgmap v479: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 4.3 MiB/s wr, 599 op/s 2024-09-19T04:30:19.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:18 smithi104 ceph-mon[34145]: pgmap v479: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 4.3 MiB/s wr, 599 op/s 2024-09-19T04:30:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 845.0 s, 416.8 tps, lat 239.377 ms stddev 30.065 2024-09-19T04:30:21.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:20 smithi112 ceph-mon[35051]: pgmap v480: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.8 MiB/s wr, 660 op/s 2024-09-19T04:30:21.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:20 smithi094 ceph-mon[30972]: pgmap v480: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.8 MiB/s wr, 660 op/s 2024-09-19T04:30:21.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:20 smithi104 ceph-mon[34145]: pgmap v480: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.8 MiB/s wr, 660 op/s 2024-09-19T04:30:23.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:22 smithi112 ceph-mon[35051]: pgmap v481: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 4.4 MiB/s wr, 604 op/s 2024-09-19T04:30:23.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:22 smithi094 ceph-mon[30972]: pgmap v481: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 4.4 MiB/s wr, 604 op/s 2024-09-19T04:30:23.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:22 smithi104 ceph-mon[34145]: pgmap v481: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 4.4 MiB/s wr, 604 op/s 2024-09-19T04:30:25.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:24 smithi112 ceph-mon[35051]: pgmap v482: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.9 MiB/s wr, 821 op/s 2024-09-19T04:30:25.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:24 smithi094 ceph-mon[30972]: pgmap v482: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.9 MiB/s wr, 821 op/s 2024-09-19T04:30:25.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:24 smithi104 ceph-mon[34145]: pgmap v482: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.9 MiB/s wr, 821 op/s 2024-09-19T04:30:25.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 850.0 s, 398.6 tps, lat 251.041 ms stddev 36.877 2024-09-19T04:30:27.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:26 smithi112 ceph-mon[35051]: pgmap v483: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 4.0 MiB/s wr, 558 op/s 2024-09-19T04:30:27.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:26 smithi094 ceph-mon[30972]: pgmap v483: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 4.0 MiB/s wr, 558 op/s 2024-09-19T04:30:27.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:26 smithi104 ceph-mon[34145]: pgmap v483: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 4.0 MiB/s wr, 558 op/s 2024-09-19T04:30:29.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:28 smithi112 ceph-mon[35051]: pgmap v484: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 4.6 MiB/s wr, 652 op/s 2024-09-19T04:30:29.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:28 smithi094 ceph-mon[30972]: pgmap v484: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 4.6 MiB/s wr, 652 op/s 2024-09-19T04:30:29.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:28 smithi104 ceph-mon[34145]: pgmap v484: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 4.6 MiB/s wr, 652 op/s 2024-09-19T04:30:30.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 855.0 s, 434.0 tps, lat 230.387 ms stddev 30.057 2024-09-19T04:30:31.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:30 smithi112 ceph-mon[35051]: pgmap v485: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 4.7 MiB/s wr, 653 op/s 2024-09-19T04:30:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:30 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:30:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:30 smithi094 ceph-mon[30972]: pgmap v485: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 4.7 MiB/s wr, 653 op/s 2024-09-19T04:30:31.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:30 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:30:31.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:30 smithi104 ceph-mon[34145]: pgmap v485: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 4.7 MiB/s wr, 653 op/s 2024-09-19T04:30:31.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:30 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:30:32.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:31 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:30:32.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:31 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:30:32.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:31 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:30:32.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:31 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:30:32.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:31 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:30:32.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:31 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:30:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:31 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:30:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:31 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:30:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:31 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:30:33.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:32 smithi112 ceph-mon[35051]: pgmap v486: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 4.0 MiB/s wr, 540 op/s 2024-09-19T04:30:33.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:32 smithi094 ceph-mon[30972]: pgmap v486: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 4.0 MiB/s wr, 540 op/s 2024-09-19T04:30:33.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:32 smithi104 ceph-mon[34145]: pgmap v486: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 4.0 MiB/s wr, 540 op/s 2024-09-19T04:30:35.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:34 smithi112 ceph-mon[35051]: pgmap v487: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.4 MiB/s wr, 828 op/s 2024-09-19T04:30:35.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:34 smithi094 ceph-mon[30972]: pgmap v487: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.4 MiB/s wr, 828 op/s 2024-09-19T04:30:35.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:34 smithi104 ceph-mon[34145]: pgmap v487: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.4 MiB/s wr, 828 op/s 2024-09-19T04:30:35.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 860.0 s, 408.2 tps, lat 245.508 ms stddev 25.970 2024-09-19T04:30:37.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:36 smithi112 ceph-mon[35051]: pgmap v488: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 4.5 MiB/s wr, 558 op/s 2024-09-19T04:30:37.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:36 smithi094 ceph-mon[30972]: pgmap v488: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 4.5 MiB/s wr, 558 op/s 2024-09-19T04:30:37.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:36 smithi104 ceph-mon[34145]: pgmap v488: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 4.5 MiB/s wr, 558 op/s 2024-09-19T04:30:39.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:30:39.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:38 smithi094 ceph-mon[30972]: pgmap v489: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.0 MiB/s wr, 744 op/s 2024-09-19T04:30:39.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:38 smithi112 ceph-mon[35051]: pgmap v489: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.0 MiB/s wr, 744 op/s 2024-09-19T04:30:39.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:38 smithi104 ceph-mon[34145]: pgmap v489: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.0 MiB/s wr, 744 op/s 2024-09-19T04:30:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:40.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.078+0000 7f9cc3651640 1 -- 172.21.15.94:0/3469248793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc100a70 msgr2=0x7f9cbc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:40.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.078+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3469248793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc100a70 0x7f9cbc100ed0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009900 tx=0x7f9cac02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.078+0000 7f9cc3651640 1 -- 172.21.15.94:0/3469248793 shutdown_connections 2024-09-19T04:30:40.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.078+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3469248793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cbc101410 0x7f9cbc10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.078+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3469248793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc100a70 0x7f9cbc100ed0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.078+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3469248793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc106a70 0x7f9cbc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.078+0000 7f9cc3651640 1 -- 172.21.15.94:0/3469248793 >> 172.21.15.94:0/3469248793 conn(0x7f9cbc0fc7d0 msgr2=0x7f9cbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:40.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.078+0000 7f9cc3651640 1 -- 172.21.15.94:0/3469248793 shutdown_connections 2024-09-19T04:30:40.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.078+0000 7f9cc3651640 1 -- 172.21.15.94:0/3469248793 wait complete. 2024-09-19T04:30:40.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.079+0000 7f9cc3651640 1 Processor -- start 2024-09-19T04:30:40.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc3651640 1 -- start start 2024-09-19T04:30:40.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc3651640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100a70 0x7f9cbc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc3651640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cbc101410 0x7f9cbc198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc3651640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc106a70 0x7f9cbc19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc13ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100a70 0x7f9cbc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc3651640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc110d70 con 0x7f9cbc101410 2024-09-19T04:30:40.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc3651640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc110bf0 con 0x7f9cbc106a70 2024-09-19T04:30:40.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc3651640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc110ef0 con 0x7f9cbc100a70 2024-09-19T04:30:40.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc13ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100a70 0x7f9cbc198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44046/0 (socket says 172.21.15.94:44046) 2024-09-19T04:30:40.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc13ed640 1 -- 172.21.15.94:0/3597926968 learned_addr learned my addr 172.21.15.94:0/3597926968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:40.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.080+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cbc101410 0x7f9cbc198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.081+0000 7f9cc1bee640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc106a70 0x7f9cbc19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.081+0000 7f9cc13ed640 1 -- 172.21.15.94:0/3597926968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc106a70 msgr2=0x7f9cbc19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:40.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.081+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc106a70 0x7f9cbc19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.081+0000 7f9cc13ed640 1 -- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cbc101410 msgr2=0x7f9cbc198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:40.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.081+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cbc101410 0x7f9cbc198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.081+0000 7f9cc13ed640 1 -- 172.21.15.94:0/3597926968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc19d2d0 con 0x7f9cbc100a70 2024-09-19T04:30:40.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.081+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100a70 0x7f9cbc198320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9f0 tx=0x7f9cb800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:40.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.082+0000 7f9cb27fc640 1 -- 172.21.15.94:0/3597926968 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800cd80 con 0x7f9cbc100a70 2024-09-19T04:30:40.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.082+0000 7f9cb27fc640 1 -- 172.21.15.94:0/3597926968 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cb80054d0 con 0x7f9cbc100a70 2024-09-19T04:30:40.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.083+0000 7f9cb27fc640 1 -- 172.21.15.94:0/3597926968 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8010690 con 0x7f9cbc100a70 2024-09-19T04:30:40.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.085+0000 7f9cc1bee640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc106a70 0x7f9cbc19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:40.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.090+0000 7f9cc3651640 1 -- 172.21.15.94:0/3597926968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc19d5c0 con 0x7f9cbc100a70 2024-09-19T04:30:40.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.090+0000 7f9cc3651640 1 -- 172.21.15.94:0/3597926968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc19da20 con 0x7f9cbc100a70 2024-09-19T04:30:40.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.100+0000 7f9cc5a2a740 1 Processor -- start 2024-09-19T04:30:40.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.100+0000 7f9cc5a2a740 1 -- start start 2024-09-19T04:30:40.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.100+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bbae6e0 0x56073bbaeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.100+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbaf000 0x56073bba9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.100+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bba96a0 0x56073bba9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.100+0000 7f9cc5a2a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56073bb052e0 con 0x56073bba96a0 2024-09-19T04:30:40.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.100+0000 7f9cc5a2a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56073baacce0 con 0x56073bbae6e0 2024-09-19T04:30:40.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.100+0000 7f9cc5a2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56073bb012a0 con 0x56073bbaf000 2024-09-19T04:30:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.101+0000 7f9cc0bec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bba96a0 0x56073bba9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.101+0000 7f9cc0bec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bba96a0 0x56073bba9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46036/0 (socket says 172.21.15.94:46036) 2024-09-19T04:30:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.101+0000 7f9cc0bec640 1 -- 172.21.15.94:0/3221892780 learned_addr learned my addr 172.21.15.94:0/3221892780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.101+0000 7f9cc1bee640 1 --2- 172.21.15.94:0/3221892780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bbae6e0 0x56073bbaeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.102+0000 7f9cc0bec640 1 -- 172.21.15.94:0/3221892780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbaf000 msgr2=0x56073bba9030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:30:40.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.102+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/3221892780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbaf000 0x56073bba9030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.102+0000 7f9cc0bec640 1 -- 172.21.15.94:0/3221892780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bbae6e0 msgr2=0x56073bbaeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:40.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.102+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/3221892780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bbae6e0 0x56073bbaeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.102+0000 7f9cc0bec640 1 -- 172.21.15.94:0/3221892780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073ba91440 con 0x56073bba96a0 2024-09-19T04:30:40.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.104+0000 7f9cb27fc640 1 -- 172.21.15.94:0/3597926968 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9cb8004680 con 0x7f9cbc100a70 2024-09-19T04:30:40.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.104+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/3221892780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bba96a0 0x56073bba9b20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009900 tx=0x7f9cac004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d49931e3d8e9904 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:40.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.105+0000 7f9cb27fc640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ca80a3390 0x7f9ca80a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.106+0000 7f9cb27fc640 1 -- 172.21.15.94:0/3597926968 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f9cb80d00c0 con 0x7f9cbc100a70 2024-09-19T04:30:40.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.106+0000 7f9ca77fe640 1 -- 172.21.15.94:0/3221892780 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac035040 con 0x56073bba96a0 2024-09-19T04:30:40.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.106+0000 7f9ca77fe640 1 -- 172.21.15.94:0/3221892780 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cac0054f0 con 0x56073bba96a0 2024-09-19T04:30:40.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.106+0000 7f9cc1bee640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ca80a3390 0x7f9ca80a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.107+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/3221892780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bba96a0 msgr2=0x56073bba9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:40.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.107+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/3221892780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bba96a0 0x56073bba9b20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009900 tx=0x7f9cac004180 comp rx=0 tx=0).stop 2024-09-19T04:30:40.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.107+0000 7f9cc1bee640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ca80a3390 0x7f9ca80a5850 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4005fd0 tx=0x7f9cb4009600 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:40.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.108+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/3221892780 shutdown_connections 2024-09-19T04:30:40.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.108+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/3221892780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bba96a0 0x56073bba9b20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.108+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/3221892780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbaf000 0x56073bba9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.108+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/3221892780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bbae6e0 0x56073bbaeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.108+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/3221892780 >> 172.21.15.94:0/3221892780 conn(0x56073bb9b4e0 msgr2=0x56073bb9b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:40.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.111+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/3221892780 shutdown_connections 2024-09-19T04:30:40.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.111+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/3221892780 wait complete. 2024-09-19T04:30:40.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.112+0000 7f9cc5a2a740 1 -- start start 2024-09-19T04:30:40.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.112+0000 7f9cc5a2a740 1 Processor -- start 2024-09-19T04:30:40.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbb4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbb21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbb2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc5a2a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56073bbc6a20 con 0x56073bbaf000 2024-09-19T04:30:40.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc5a2a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56073bac0050 con 0x56073bba96a0 2024-09-19T04:30:40.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc5a2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56073bbc6ba0 con 0x56073bbae6e0 2024-09-19T04:30:40.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc5a2a740 10 client.0 _ll_register_callbacks cb 0x56073bb00160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:40.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc0bec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbb21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc0bec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbb21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44060/0 (socket says 172.21.15.94:44060) 2024-09-19T04:30:40.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.113+0000 7f9cc0bec640 1 -- 172.21.15.94:0/91241089 learned_addr learned my addr 172.21.15.94:0/91241089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:40.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.114+0000 7f9cc1bee640 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbb2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.114+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbb4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.114+0000 7f9cc0bec640 1 -- 172.21.15.94:0/91241089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 msgr2=0x56073bbb4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:40.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.114+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbb4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.114+0000 7f9cc0bec640 1 -- 172.21.15.94:0/91241089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 msgr2=0x56073bbb2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:40.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.114+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbb2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.114+0000 7f9cc0bec640 1 -- 172.21.15.94:0/91241089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073b8763c0 con 0x56073bbae6e0 2024-09-19T04:30:40.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.114+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbb21a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9cac006010 tx=0x7f9cac0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a17590cd55177510 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:40.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.115+0000 7f9ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:30:40.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.116+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/91241089 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac035070 con 0x56073bbae6e0 2024-09-19T04:30:40.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.116+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/91241089 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f9cac003ab0 con 0x56073bbae6e0 2024-09-19T04:30:40.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.117+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/91241089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cac0054f0 con 0x56073bbae6e0 2024-09-19T04:30:40.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.117+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/91241089 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9cac009520 con 0x56073bbae6e0 2024-09-19T04:30:40.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.117+0000 7f9cc5a2a740 10 client.26887 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:40.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.117+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/91241089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56073bbc6a20 con 0x56073bbae6e0 2024-09-19T04:30:40.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.118+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/91241089 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cac0054f0 con 0x56073bbae6e0 2024-09-19T04:30:40.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.119+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/91241089 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f9cac03c540 con 0x56073bbae6e0 2024-09-19T04:30:40.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.119+0000 7f9cc5a2a740 10 client.26887 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:40.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.119+0000 7f9cc5a2a740 10 client.26887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:40.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.119+0000 7f9cc5a2a740 20 client.26887 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:40.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.119+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56073bbd3630 0x56073bbd5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.119+0000 7f9cc5a2a740 4 client.26887 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:40.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.119+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/91241089 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56073bb12850 con 0x56073bbd3630 2024-09-19T04:30:40.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.119+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56073bbd3630 0x56073bbd5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.120+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56073bbd3630 0x56073bbd5a20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16896b07557e1f70 server_cookie=82c7a92a7949f77a in_seq=0 out_seq=0 2024-09-19T04:30:40.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.120+0000 7f9ca4ff9640 10 client.26887 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:40.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.121+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/91241089 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56073bb12850 con 0x56073bbd3630 2024-09-19T04:30:40.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.121+0000 7f9ca4ff9640 10 client.26887 handle_command_reply: tid=0 2024-09-19T04:30:40.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.121+0000 7f9cc5a2a740 1 client.26887 shutdown 2024-09-19T04:30:40.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.121+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/91241089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 msgr2=0x56073bbb21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:40.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.121+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbb21a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9cac006010 tx=0x7f9cac0045e0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.121+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/91241089 shutdown_connections 2024-09-19T04:30:40.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.122+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56073bbd3630 0x56073bbd5a20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.122+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbb2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.122+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbb21a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.122+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/91241089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbb4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.122+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/91241089 >> 172.21.15.94:0/91241089 conn(0x56073bb9b4e0 msgr2=0x56073bb9abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:40.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.122+0000 7f9ca4ff9640 0 client.26887 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:40.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.122+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/91241089 shutdown_connections 2024-09-19T04:30:40.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.122+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/91241089 wait complete. 2024-09-19T04:30:40.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.122+0000 7f9cc5a2a740 20 client.26887 trim_cache size 0 max 16384 2024-09-19T04:30:40.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.150+0000 7f9cc5a2a740 1 Processor -- start 2024-09-19T04:30:40.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.150+0000 7f9cc5a2a740 1 -- start start 2024-09-19T04:30:40.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bba9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbadb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc5a2a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc1a54f0 con 0x56073bbaf000 2024-09-19T04:30:40.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc5a2a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc072280 con 0x56073bba96a0 2024-09-19T04:30:40.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc5a2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc0700f0 con 0x56073bbae6e0 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc1bee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbadb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc0bec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc0bec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:44062/0 (socket says 172.21.15.94:44062) 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc1bee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbadb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46046/0 (socket says 172.21.15.94:46046) 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc0bec640 1 -- 172.21.15.94:0/1970614257 learned_addr learned my addr 172.21.15.94:0/1970614257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/1970614257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bba9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc0bec640 1 -- 172.21.15.94:0/1970614257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 msgr2=0x56073bba9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/1970614257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bba9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc0bec640 1 -- 172.21.15.94:0/1970614257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 msgr2=0x56073bbadb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/1970614257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbadb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc0bec640 1 -- 172.21.15.94:0/1970614257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073bb9c290 con 0x56073bbae6e0 2024-09-19T04:30:40.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.151+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/1970614257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bba9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:40.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.152+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/1970614257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcfd40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9cac03c900 tx=0x7f9cac03cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56b7a6c50b95e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:40.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.152+0000 7f9ca6ffd640 1 -- 172.21.15.94:0/1970614257 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac035070 con 0x56073bbae6e0 2024-09-19T04:30:40.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.152+0000 7f9ca6ffd640 1 -- 172.21.15.94:0/1970614257 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cac03cd60 con 0x56073bbae6e0 2024-09-19T04:30:40.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.152+0000 7f9ca6ffd640 1 -- 172.21.15.94:0/1970614257 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac004d30 con 0x56073bbae6e0 2024-09-19T04:30:40.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.153+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/1970614257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 msgr2=0x56073bbcfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:40.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.153+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/1970614257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcfd40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9cac03c900 tx=0x7f9cac03cb70 comp rx=0 tx=0).stop 2024-09-19T04:30:40.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.153+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/1970614257 shutdown_connections 2024-09-19T04:30:40.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.153+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/1970614257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbadb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.153+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/1970614257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcfd40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.153+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/1970614257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bba9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.153+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/1970614257 >> 172.21.15.94:0/1970614257 conn(0x56073bb9b4e0 msgr2=0x56073bbb3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:40.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.154+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/1970614257 shutdown_connections 2024-09-19T04:30:40.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.154+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/1970614257 wait complete. 2024-09-19T04:30:40.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.154+0000 7f9cc5a2a740 1 -- start start 2024-09-19T04:30:40.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.154+0000 7f9cc5a2a740 1 Processor -- start 2024-09-19T04:30:40.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.155+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbcc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.155+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.155+0000 7f9cc5a2a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbda560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc13ed640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbcc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc13ed640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbcc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:42226/0 (socket says 172.21.15.94:42226) 2024-09-19T04:30:40.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc13ed640 1 -- 172.21.15.94:0/630015358 learned_addr learned my addr 172.21.15.94:0/630015358 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:40.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc070740 con 0x56073bbaf000 2024-09-19T04:30:40.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc071bb0 con 0x56073bba96a0 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc070e10 con 0x56073bbae6e0 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc1bee640 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbda560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc13ed640 1 -- 172.21.15.94:0/630015358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 msgr2=0x56073bbcc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc13ed640 1 -- 172.21.15.94:0/630015358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 msgr2=0x56073bbda560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbda560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc13ed640 1 -- 172.21.15.94:0/630015358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073bbdaca0 con 0x56073bba96a0 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.156+0000 7f9cc1bee640 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbda560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9cc5a2a740 10 client.0 _ll_register_callbacks cb 0x7f9cb4009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56073bb0cd60 con 0x56073bba96a0 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9cc13ed640 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbcc090 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009380 tx=0x7f9cb8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f884a02d345166b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:40.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9ca4ff9640 10 client.26930 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/630015358 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb809c300 con 0x56073bba96a0 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/630015358 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f9cb8098970 con 0x56073bba96a0 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/630015358 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9cb80a06b0 con 0x56073bba96a0 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9cc5a2a740 10 client.26930 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=54}) -- 0x56073bc3da30 con 0x56073bba96a0 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/630015358 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f9cb80cd9f0 con 0x56073bba96a0 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9cc5a2a740 10 client.26930 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9cc5a2a740 10 client.26930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9cc5a2a740 20 client.26930 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.157+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56073bc3e050 0x56073bc404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.158+0000 7f9cc5a2a740 4 client.26930 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.158+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56073ba89210 con 0x56073bc3e050 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.158+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56073bc3e050 0x56073bc404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:40.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.158+0000 7f9cc0bec640 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56073bc3e050 0x56073bc404f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8513660be4d5b5de server_cookie=42cea927904d670b in_seq=0 out_seq=0 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.158+0000 7f9ca4ff9640 10 client.26930 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.158+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/630015358 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56073ba89210 con 0x56073bc3e050 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9ca4ff9640 10 client.26930 handle_command_reply: tid=0 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 client.26930 shutdown 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 msgr2=0x56073bbcc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbcc090 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009380 tx=0x7f9cb8009310 comp rx=0 tx=0).stop 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 shutdown_connections 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x56073bc3e050 0x56073bc404f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56073bbaf000 0x56073bbda560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9ca4ff9640 0 client.26930 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56073bbae6e0 0x56073bbcc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 --2- 172.21.15.94:0/630015358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56073bba96a0 0x56073bbcc090 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 >> 172.21.15.94:0/630015358 conn(0x56073bb9b4e0 msgr2=0x56073bb9a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:40.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 shutdown_connections 2024-09-19T04:30:40.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.159+0000 7f9cc5a2a740 1 -- 172.21.15.94:0/630015358 wait complete. 2024-09-19T04:30:40.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc5a2a740 20 client.26930 trim_cache size 0 max 16384 2024-09-19T04:30:40.165 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:30:40.166 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:30:40.166 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:30:40.166 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:30:40.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc3651640 1 -- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ca80a3390 msgr2=0x7f9ca80a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ca80a3390 0x7f9ca80a5850 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4005fd0 tx=0x7f9cb4009600 comp rx=0 tx=0).stop 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc3651640 1 -- 172.21.15.94:0/3597926968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100a70 msgr2=0x7f9cbc198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100a70 0x7f9cbc198320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9f0 tx=0x7f9cb800edf0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc3651640 1 -- 172.21.15.94:0/3597926968 shutdown_connections 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9ca80a3390 0x7f9ca80a5850 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc106a70 0x7f9cbc19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cbc101410 0x7f9cbc198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.160+0000 7f9cc3651640 1 --2- 172.21.15.94:0/3597926968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100a70 0x7f9cbc198320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.161+0000 7f9cc3651640 1 -- 172.21.15.94:0/3597926968 >> 172.21.15.94:0/3597926968 conn(0x7f9cbc0fc7d0 msgr2=0x7f9cbc06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.161+0000 7f9cc3651640 1 -- 172.21.15.94:0/3597926968 shutdown_connections 2024-09-19T04:30:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:40.161+0000 7f9cc3651640 1 -- 172.21.15.94:0/3597926968 wait complete. 2024-09-19T04:30:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 865.0 s, 405.0 tps, lat 247.037 ms stddev 27.144 2024-09-19T04:30:41.204 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:30:41.205 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:30:41.271 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:40 smithi094 ceph-mon[30972]: pgmap v490: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.7 MiB/s wr, 785 op/s 2024-09-19T04:30:41.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:40 smithi112 ceph-mon[35051]: pgmap v490: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.7 MiB/s wr, 785 op/s 2024-09-19T04:30:41.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:40 smithi104 ceph-mon[34145]: pgmap v490: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.7 MiB/s wr, 785 op/s 2024-09-19T04:30:41.581 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.108+0000 7fd4945fb640 1 -- 172.21.15.94:0/367884211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 msgr2=0x7fd48c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.108+0000 7fd4945fb640 1 --2- 172.21.15.94:0/367884211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 0x7fd48c101200 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd488009900 tx=0x7fd48802a480 comp rx=0 tx=0).stop 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.109+0000 7fd4945fb640 1 -- 172.21.15.94:0/367884211 shutdown_connections 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.109+0000 7fd4945fb640 1 --2- 172.21.15.94:0/367884211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd48c102ef0 0x7fd48c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.109+0000 7fd4945fb640 1 --2- 172.21.15.94:0/367884211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c102570 0x7fd48c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.109+0000 7fd4945fb640 1 --2- 172.21.15.94:0/367884211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 0x7fd48c101200 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.109+0000 7fd4945fb640 1 -- 172.21.15.94:0/367884211 >> 172.21.15.94:0/367884211 conn(0x7fd48c0fc7d0 msgr2=0x7fd48c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.109+0000 7fd4945fb640 1 -- 172.21.15.94:0/367884211 shutdown_connections 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.109+0000 7fd4945fb640 1 -- 172.21.15.94:0/367884211 wait complete. 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.109+0000 7fd4945fb640 1 Processor -- start 2024-09-19T04:30:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.109+0000 7fd4945fb640 1 -- start start 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd4945fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 0x7fd48c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd4945fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd48c102570 0x7fd48c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd4945fb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c102ef0 0x7fd48c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd4945fb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c110e80 con 0x7fd48c100e20 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd4945fb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c110d00 con 0x7fd48c102ef0 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd4945fb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c111000 con 0x7fd48c102570 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd492397640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 0x7fd48c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd492397640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 0x7fd48c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38280/0 (socket says 172.21.15.94:38280) 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd492397640 1 -- 172.21.15.94:0/1012457033 learned_addr learned my addr 172.21.15.94:0/1012457033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd491b96640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd48c102570 0x7fd48c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd492b98640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c102ef0 0x7fd48c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd492397640 1 -- 172.21.15.94:0/1012457033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd48c102570 msgr2=0x7fd48c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:42.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd492397640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd48c102570 0x7fd48c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:42.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd492397640 1 -- 172.21.15.94:0/1012457033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c102ef0 msgr2=0x7fd48c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:42.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd492397640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c102ef0 0x7fd48c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:42.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.110+0000 7fd492397640 1 -- 172.21.15.94:0/1012457033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1a6890 con 0x7fd48c100e20 2024-09-19T04:30:42.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.111+0000 7fd492397640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 0x7fd48c0fcc80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd48802aa70 tx=0x7fd488003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:42.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.111+0000 7fd4837fe640 1 -- 172.21.15.94:0/1012457033 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48803f070 con 0x7fd48c100e20 2024-09-19T04:30:42.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.111+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c1a6b20 con 0x7fd48c100e20 2024-09-19T04:30:42.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.111+0000 7fd4837fe640 1 -- 172.21.15.94:0/1012457033 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd48803bb20 con 0x7fd48c100e20 2024-09-19T04:30:42.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.111+0000 7fd4837fe640 1 -- 172.21.15.94:0/1012457033 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488035030 con 0x7fd48c100e20 2024-09-19T04:30:42.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.111+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a7050 con 0x7fd48c100e20 2024-09-19T04:30:42.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.113+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c101200 con 0x7fd48c100e20 2024-09-19T04:30:42.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.115+0000 7fd4837fe640 1 -- 172.21.15.94:0/1012457033 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd488042070 con 0x7fd48c100e20 2024-09-19T04:30:42.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.115+0000 7fd4837fe640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd4780a3430 0x7fd4780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:42.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.116+0000 7fd4837fe640 1 -- 172.21.15.94:0/1012457033 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7fd4880f0780 con 0x7fd48c100e20 2024-09-19T04:30:42.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.116+0000 7fd491b96640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd4780a3430 0x7fd4780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:42.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.116+0000 7fd491b96640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd4780a3430 0x7fd4780a58f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0fe710 tx=0x7fd47c00b390 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:42.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.120+0000 7fd4837fe640 1 -- 172.21.15.94:0/1012457033 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd4880bcad0 con 0x7fd48c100e20 2024-09-19T04:30:42.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.388+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd48c0fe1b0 con 0x7fd48c100e20 2024-09-19T04:30:42.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.389+0000 7fd4837fe640 1 -- 172.21.15.94:0/1012457033 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11915 (secure 0 0 0) 0x7fd48c0fe1b0 con 0x7fd48c100e20 2024-09-19T04:30:42.391 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:30:42.391 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":54,"btime":"2024-09-19T04:29:32:831739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:29:32.764738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:30:42.391 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 54 2024-09-19T04:30:42.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd4780a3430 msgr2=0x7fd4780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd4780a3430 0x7fd4780a58f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0fe710 tx=0x7fd47c00b390 comp rx=0 tx=0).stop 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 msgr2=0x7fd48c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 0x7fd48c0fcc80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd48802aa70 tx=0x7fd488003c10 comp rx=0 tx=0).stop 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 shutdown_connections 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd4780a3430 0x7fd4780a58f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c102ef0 0x7fd48c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd48c102570 0x7fd48c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 --2- 172.21.15.94:0/1012457033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c100e20 0x7fd48c0fcc80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 >> 172.21.15.94:0/1012457033 conn(0x7fd48c0fc7d0 msgr2=0x7fd48c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 shutdown_connections 2024-09-19T04:30:42.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:42.392+0000 7fd4945fb640 1 -- 172.21.15.94:0/1012457033 wait complete. 2024-09-19T04:30:43.018 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:42 smithi094 ceph-mon[30972]: pgmap v491: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 6.2 MiB/s wr, 609 op/s 2024-09-19T04:30:43.018 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:42 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1012457033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:30:43.050 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:30:43.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:42 smithi112 ceph-mon[35051]: pgmap v491: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 6.2 MiB/s wr, 609 op/s 2024-09-19T04:30:43.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:42 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1012457033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:30:43.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:42 smithi104 ceph-mon[34145]: pgmap v491: 97 pgs: 97 active+clean; 3.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 6.2 MiB/s wr, 609 op/s 2024-09-19T04:30:43.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:42 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1012457033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:30:43.433 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:44.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.245+0000 7f5a649ad640 1 -- 172.21.15.94:0/1285721975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 msgr2=0x7f5a6010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:44.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.245+0000 7f5a649ad640 1 --2- 172.21.15.94:0/1285721975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 0x7f5a6010e610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009ca0 tx=0x7f5a5802a950 comp rx=0 tx=0).stop 2024-09-19T04:30:44.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.246+0000 7f5a649ad640 1 -- 172.21.15.94:0/1285721975 shutdown_connections 2024-09-19T04:30:44.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.246+0000 7f5a649ad640 1 --2- 172.21.15.94:0/1285721975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 0x7f5a6010e610 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.246+0000 7f5a649ad640 1 --2- 172.21.15.94:0/1285721975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a60103660 0x7f5a6010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.246+0000 7f5a649ad640 1 --2- 172.21.15.94:0/1285721975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a60102d40 0x7f5a60103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.246+0000 7f5a649ad640 1 -- 172.21.15.94:0/1285721975 >> 172.21.15.94:0/1285721975 conn(0x7f5a600fc7a0 msgr2=0x7f5a600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:44.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.246+0000 7f5a649ad640 1 -- 172.21.15.94:0/1285721975 shutdown_connections 2024-09-19T04:30:44.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.246+0000 7f5a649ad640 1 -- 172.21.15.94:0/1285721975 wait complete. 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.247+0000 7f5a649ad640 1 Processor -- start 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.247+0000 7f5a649ad640 1 -- start start 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.247+0000 7f5a649ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a60102d40 0x7f5a6019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.247+0000 7f5a649ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a60103660 0x7f5a6019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.247+0000 7f5a649ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 0x7f5a6019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.247+0000 7f5a649ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a60111e70 con 0x7f5a6010c170 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.247+0000 7f5a649ad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a60111cf0 con 0x7f5a60102d40 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.247+0000 7f5a649ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a60111ff0 con 0x7f5a60103660 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 0x7f5a6019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a60103660 0x7f5a6019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 0x7f5a6019ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38302/0 (socket says 172.21.15.94:38302) 2024-09-19T04:30:44.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5ed9d640 1 -- 172.21.15.94:0/3424452470 learned_addr learned my addr 172.21.15.94:0/3424452470 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a60102d40 0x7f5a6019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5ed9d640 1 -- 172.21.15.94:0/3424452470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a60103660 msgr2=0x7f5a6019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5ed9d640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a60103660 0x7f5a6019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5ed9d640 1 -- 172.21.15.94:0/3424452470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a60102d40 msgr2=0x7f5a6019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5ed9d640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a60102d40 0x7f5a6019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5ed9d640 1 -- 172.21.15.94:0/3424452470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6019f340 con 0x7f5a6010c170 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a60102d40 0x7f5a6019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.248+0000 7f5a5ed9d640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 0x7f5a6019ec60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5a5802af40 tx=0x7f5a58003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.249+0000 7f5a577fe640 1 -- 172.21.15.94:0/3424452470 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58034040 con 0x7f5a6010c170 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.249+0000 7f5a649ad640 1 -- 172.21.15.94:0/3424452470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6019f5d0 con 0x7f5a6010c170 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.249+0000 7f5a649ad640 1 -- 172.21.15.94:0/3424452470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a601a6f10 con 0x7f5a6010c170 2024-09-19T04:30:44.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.249+0000 7f5a577fe640 1 -- 172.21.15.94:0/3424452470 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a580308d0 con 0x7f5a6010c170 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.249+0000 7f5a577fe640 1 -- 172.21.15.94:0/3424452470 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58004520 con 0x7f5a6010c170 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.252+0000 7f5a577fe640 1 -- 172.21.15.94:0/3424452470 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5a58037020 con 0x7f5a6010c170 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.252+0000 7f5a66d86740 1 Processor -- start 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a66d86740 1 -- start start 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2aab40 0x563bcf2a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a66d86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563bcf20f2f0 con 0x563bcf2a95d0 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a66d86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563bcf1b6cf0 con 0x563bcf2b4370 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a66d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563bcf20b2b0 con 0x563bcf2aab40 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a5e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2aab40 0x563bcf2a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a5e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38304/0 (socket says 172.21.15.94:38304) 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a5e59c640 1 -- 172.21.15.94:0/3048544561 learned_addr learned my addr 172.21.15.94:0/3048544561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2aab40 0x563bcf2a9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:39968/0 (socket says 172.21.15.94:39968) 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a577fe640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a4c0a34b0 0x7f5a4c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.253+0000 7f5a57fff640 1 --2- 172.21.15.94:0/3048544561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a57fff640 1 -- 172.21.15.94:0/3048544561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 msgr2=0x563bcf2b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a57fff640 1 --2- 172.21.15.94:0/3048544561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a57fff640 1 -- 172.21.15.94:0/3048544561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 msgr2=0x563bcf2a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a577fe640 1 -- 172.21.15.94:0/3424452470 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f5a580f0bc0 con 0x7f5a6010c170 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a57fff640 1 --2- 172.21.15.94:0/3048544561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a57fff640 1 -- 172.21.15.94:0/3048544561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bcf19b3a0 con 0x563bcf2aab40 2024-09-19T04:30:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/3048544561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a57fff640 1 --2- 172.21.15.94:0/3048544561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2aab40 0x563bcf2a9090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ff150 tx=0x7f5a48009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf06ac0da226cc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a5ed9d640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a4c0a34b0 0x7f5a4c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a54ff9640 1 -- 172.21.15.94:0/3048544561 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48011050 con 0x563bcf2aab40 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a54ff9640 1 -- 172.21.15.94:0/3048544561 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a480111f0 con 0x563bcf2aab40 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.254+0000 7f5a54ff9640 1 -- 172.21.15.94:0/3048544561 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48004dd0 con 0x563bcf2aab40 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.255+0000 7f5a66d86740 1 -- 172.21.15.94:0/3048544561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2aab40 msgr2=0x563bcf2a9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.255+0000 7f5a5ed9d640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a4c0a34b0 0x7f5a4c0a5970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5a5802af10 tx=0x7f5a58009360 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.255+0000 7f5a66d86740 1 --2- 172.21.15.94:0/3048544561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2aab40 0x563bcf2a9090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ff150 tx=0x7f5a48009f90 comp rx=0 tx=0).stop 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.256+0000 7f5a66d86740 1 -- 172.21.15.94:0/3048544561 shutdown_connections 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.256+0000 7f5a66d86740 1 --2- 172.21.15.94:0/3048544561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.256+0000 7f5a66d86740 1 --2- 172.21.15.94:0/3048544561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.256+0000 7f5a66d86740 1 --2- 172.21.15.94:0/3048544561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2aab40 0x563bcf2a9090 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.256+0000 7f5a66d86740 1 -- 172.21.15.94:0/3048544561 >> 172.21.15.94:0/3048544561 conn(0x563bcf272310 msgr2=0x563bcf2a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:44.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.256+0000 7f5a66d86740 1 -- 172.21.15.94:0/3048544561 shutdown_connections 2024-09-19T04:30:44.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.256+0000 7f5a66d86740 1 -- 172.21.15.94:0/3048544561 wait complete. 2024-09-19T04:30:44.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.256+0000 7f5a66d86740 1 -- start start 2024-09-19T04:30:44.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.256+0000 7f5a66d86740 1 Processor -- start 2024-09-19T04:30:44.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 0x563bcf2c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a66d86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563bcf2b7070 con 0x563bcf2a95d0 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a66d86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563bcf1ca060 con 0x563bcf2b4370 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a66d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563bcf2b71f0 con 0x563bcf2b1c40 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a66d86740 10 client.0 _ll_register_callbacks cb 0x563bcef1d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a5e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38306/0 (socket says 172.21.15.94:38306) 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a57fff640 1 -- 172.21.15.94:0/4141771358 learned_addr learned my addr 172.21.15.94:0/4141771358 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.257+0000 7f5a57fff640 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 0x563bcf2c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.258+0000 7f5a5e59c640 1 -- 172.21.15.94:0/4141771358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 msgr2=0x563bcf2c05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.258+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 0x563bcf2c05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.258+0000 7f5a5e59c640 1 -- 172.21.15.94:0/4141771358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 msgr2=0x563bcf2b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:44.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.258+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.258+0000 7f5a5e59c640 1 -- 172.21.15.94:0/4141771358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bcef803c0 con 0x563bcf2b4370 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.258+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b1700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5a50004650 tx=0x7f5a50012f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=204423c7b2021290 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.258+0000 7f5a45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.258+0000 7f5a45ffb640 1 -- 172.21.15.94:0/4141771358 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5000fb00 con 0x563bcf2b4370 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.259+0000 7f5a45ffb640 1 -- 172.21.15.94:0/4141771358 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f5a5000fca0 con 0x563bcf2b4370 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.259+0000 7f5a66d86740 1 -- 172.21.15.94:0/4141771358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a480111f0 con 0x563bcf2b4370 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.259+0000 7f5a45ffb640 1 -- 172.21.15.94:0/4141771358 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5a5000fea0 con 0x563bcf2b4370 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.259+0000 7f5a66d86740 10 client.26948 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.259+0000 7f5a66d86740 1 -- 172.21.15.94:0/4141771358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x563bcf2c31b0 con 0x563bcf2b4370 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.260+0000 7f5a45ffb640 1 -- 172.21.15.94:0/4141771358 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a480111f0 con 0x563bcf2b4370 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.260+0000 7f5a45ffb640 1 -- 172.21.15.94:0/4141771358 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f5a500105d0 con 0x563bcf2b4370 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.260+0000 7f5a66d86740 10 client.26948 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:44.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.260+0000 7f5a66d86740 10 client.26948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.260+0000 7f5a66d86740 20 client.26948 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.260+0000 7f5a66d86740 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x563bcf2c4670 0x563bcf2c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.260+0000 7f5a66d86740 4 client.26948 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.260+0000 7f5a66d86740 1 -- 172.21.15.94:0/4141771358 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bcf21c860 con 0x563bcf2c4670 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.260+0000 7f5a57fff640 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x563bcf2c4670 0x563bcf2c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.261+0000 7f5a45ffb640 10 client.26948 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.261+0000 7f5a57fff640 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x563bcf2c4670 0x563bcf2c6a60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0259330c531b31a server_cookie=d95cfc6aea0f2761 in_seq=0 out_seq=0 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.262+0000 7f5a45ffb640 1 -- 172.21.15.94:0/4141771358 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563bcf21c860 con 0x563bcf2c4670 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.262+0000 7f5a45ffb640 10 client.26948 handle_command_reply: tid=0 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.262+0000 7f5a66d86740 1 client.26948 shutdown 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.262+0000 7f5a66d86740 1 -- 172.21.15.94:0/4141771358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 msgr2=0x563bcf2b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.262+0000 7f5a66d86740 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b1700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5a50004650 tx=0x7f5a50012f70 comp rx=0 tx=0).stop 2024-09-19T04:30:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a66d86740 1 -- 172.21.15.94:0/4141771358 shutdown_connections 2024-09-19T04:30:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a66d86740 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x563bcf2c4670 0x563bcf2c6a60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a66d86740 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 0x563bcf2c05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a66d86740 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b1700 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a45ffb640 0 client.26948 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:30:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a66d86740 1 --2- 172.21.15.94:0/4141771358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a66d86740 1 -- 172.21.15.94:0/4141771358 >> 172.21.15.94:0/4141771358 conn(0x563bcf272310 msgr2=0x563bcf2ab930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a66d86740 1 -- 172.21.15.94:0/4141771358 shutdown_connections 2024-09-19T04:30:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a66d86740 1 -- 172.21.15.94:0/4141771358 wait complete. 2024-09-19T04:30:44.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.263+0000 7f5a66d86740 20 client.26948 trim_cache size 0 max 16384 2024-09-19T04:30:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.280+0000 7f5a66d86740 1 Processor -- start 2024-09-19T04:30:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.280+0000 7f5a66d86740 1 -- start start 2024-09-19T04:30:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.280+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2a95d0 0x563bcf2b3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.280+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b1c40 0x563bcf2ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.280+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2b4370 0x563bcf2ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.280+0000 7f5a66d86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a400111b0 con 0x563bcf2b4370 2024-09-19T04:30:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.280+0000 7f5a66d86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4000ea80 con 0x563bcf2b1c40 2024-09-19T04:30:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.280+0000 7f5a66d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4000c8f0 con 0x563bcf2a95d0 2024-09-19T04:30:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2a95d0 0x563bcf2b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2a95d0 0x563bcf2b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:39984/0 (socket says 172.21.15.94:39984) 2024-09-19T04:30:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 -- 172.21.15.94:0/2475907664 learned_addr learned my addr 172.21.15.94:0/2475907664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/2475907664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b1c40 0x563bcf2ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 --2- 172.21.15.94:0/2475907664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2b4370 0x563bcf2ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 -- 172.21.15.94:0/2475907664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b1c40 msgr2=0x563bcf2ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 --2- 172.21.15.94:0/2475907664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b1c40 0x563bcf2ac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 -- 172.21.15.94:0/2475907664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2b4370 msgr2=0x563bcf2ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 --2- 172.21.15.94:0/2475907664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2b4370 0x563bcf2ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 -- 172.21.15.94:0/2475907664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bcf2c70c0 con 0x563bcf2a95d0 2024-09-19T04:30:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.281+0000 7f5a57fff640 1 --2- 172.21.15.94:0/2475907664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2a95d0 0x563bcf2b3130 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5a48011680 tx=0x7f5a480049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c181b247504690 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.282+0000 7f5a47fff640 1 -- 172.21.15.94:0/2475907664 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48011020 con 0x563bcf2a95d0 2024-09-19T04:30:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.282+0000 7f5a47fff640 1 -- 172.21.15.94:0/2475907664 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a48035070 con 0x563bcf2a95d0 2024-09-19T04:30:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.282+0000 7f5a47fff640 1 -- 172.21.15.94:0/2475907664 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a480319a0 con 0x563bcf2a95d0 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 -- 172.21.15.94:0/2475907664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2a95d0 msgr2=0x563bcf2b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 --2- 172.21.15.94:0/2475907664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2a95d0 0x563bcf2b3130 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5a48011680 tx=0x7f5a480049b0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 -- 172.21.15.94:0/2475907664 shutdown_connections 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 --2- 172.21.15.94:0/2475907664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2b4370 0x563bcf2ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 --2- 172.21.15.94:0/2475907664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b1c40 0x563bcf2ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 --2- 172.21.15.94:0/2475907664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2a95d0 0x563bcf2b3130 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 -- 172.21.15.94:0/2475907664 >> 172.21.15.94:0/2475907664 conn(0x563bcf272310 msgr2=0x563bcf2b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 -- 172.21.15.94:0/2475907664 shutdown_connections 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 -- 172.21.15.94:0/2475907664 wait complete. 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.283+0000 7f5a66d86740 1 -- start start 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a66d86740 1 Processor -- start 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2bc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 0x563bcf2bce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a66d86740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a66d86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4000cf40 con 0x563bcf2a95d0 2024-09-19T04:30:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a66d86740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4000e3b0 con 0x563bcf2b4370 2024-09-19T04:30:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a66d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4000d610 con 0x563bcf2b1c40 2024-09-19T04:30:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a66d86740 10 client.0 _ll_register_callbacks cb 0x7f5a50009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2bc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2bc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38322/0 (socket says 172.21.15.94:38322) 2024-09-19T04:30:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a57fff640 1 -- 172.21.15.94:0/1820619976 learned_addr learned my addr 172.21.15.94:0/1820619976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.284+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 0x563bcf2bce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.285+0000 7f5a57fff640 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.285+0000 7f5a57fff640 1 -- 172.21.15.94:0/1820619976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 msgr2=0x563bcf2bce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.285+0000 7f5a57fff640 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 0x563bcf2bce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.285+0000 7f5a57fff640 1 -- 172.21.15.94:0/1820619976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 msgr2=0x563bcf2b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.285+0000 7f5a57fff640 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.285+0000 7f5a57fff640 1 -- 172.21.15.94:0/1820619976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bcf2b86b0 con 0x563bcf2a95d0 2024-09-19T04:30:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.285+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 0x563bcf2bce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.285+0000 7f5a57fff640 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2bc900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5a48013040 tx=0x7f5a48009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927998f7906f3fd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.285+0000 7f5a45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.286+0000 7f5a45ffb640 1 -- 172.21.15.94:0/1820619976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48031820 con 0x563bcf2a95d0 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.286+0000 7f5a45ffb640 1 -- 172.21.15.94:0/1820619976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f5a480319c0 con 0x563bcf2a95d0 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.286+0000 7f5a66d86740 1 -- 172.21.15.94:0/1820619976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a48035070 con 0x563bcf2a95d0 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.286+0000 7f5a45ffb640 1 -- 172.21.15.94:0/1820619976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5a48031bc0 con 0x563bcf2a95d0 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.286+0000 7f5a66d86740 10 client.17334 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.286+0000 7f5a66d86740 1 -- 172.21.15.94:0/1820619976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x563bcf32efd0 con 0x563bcf2a95d0 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.286+0000 7f5a45ffb640 1 -- 172.21.15.94:0/1820619976 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a48049920 con 0x563bcf2a95d0 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.287+0000 7f5a45ffb640 1 -- 172.21.15.94:0/1820619976 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f5a4801f6d0 con 0x563bcf2a95d0 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.287+0000 7f5a66d86740 10 client.17334 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.287+0000 7f5a66d86740 10 client.17334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.287+0000 7f5a66d86740 20 client.17334 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.287+0000 7f5a66d86740 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x563bcf32f360 0x563bcf331750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.287+0000 7f5a66d86740 4 client.17334 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.287+0000 7f5a66d86740 1 -- 172.21.15.94:0/1820619976 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5a48035070 con 0x563bcf32f360 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.287+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x563bcf32f360 0x563bcf331750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.287+0000 7f5a45ffb640 10 client.17334 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a5e59c640 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x563bcf32f360 0x563bcf331750 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ca3c24513c224cf server_cookie=51bae0b8c104fae9 in_seq=0 out_seq=0 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a45ffb640 1 -- 172.21.15.94:0/1820619976 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5a48035070 con 0x563bcf32f360 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a45ffb640 10 client.17334 handle_command_reply: tid=0 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 client.17334 shutdown 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 -- 172.21.15.94:0/1820619976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 msgr2=0x563bcf2bc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2bc900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5a48013040 tx=0x7f5a48009a00 comp rx=0 tx=0).stop 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 -- 172.21.15.94:0/1820619976 shutdown_connections 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x563bcf32f360 0x563bcf331750 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563bcf2b4370 0x563bcf2b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcf2b1c40 0x563bcf2bce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a45ffb640 0 client.17334 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:30:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 --2- 172.21.15.94:0/1820619976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563bcf2a95d0 0x563bcf2bc900 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 -- 172.21.15.94:0/1820619976 >> 172.21.15.94:0/1820619976 conn(0x563bcf272310 msgr2=0x563bcf2b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 -- 172.21.15.94:0/1820619976 shutdown_connections 2024-09-19T04:30:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 1 -- 172.21.15.94:0/1820619976 wait complete. 2024-09-19T04:30:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.288+0000 7f5a66d86740 20 client.17334 trim_cache size 0 max 16384 2024-09-19T04:30:44.295 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.289+0000 7f5a649ad640 1 -- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a4c0a34b0 msgr2=0x7f5a4c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.289+0000 7f5a649ad640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a4c0a34b0 0x7f5a4c0a5970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5a5802af10 tx=0x7f5a58009360 comp rx=0 tx=0).stop 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.289+0000 7f5a649ad640 1 -- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 msgr2=0x7f5a6019ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.289+0000 7f5a649ad640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 0x7f5a6019ec60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5a5802af40 tx=0x7f5a58003f10 comp rx=0 tx=0).stop 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.290+0000 7f5a649ad640 1 -- 172.21.15.94:0/3424452470 shutdown_connections 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.290+0000 7f5a649ad640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5a4c0a34b0 0x7f5a4c0a5970 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.290+0000 7f5a649ad640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a6010c170 0x7f5a6019ec60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.290+0000 7f5a649ad640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a60103660 0x7f5a6019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.290+0000 7f5a649ad640 1 --2- 172.21.15.94:0/3424452470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a60102d40 0x7f5a6019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.290+0000 7f5a649ad640 1 -- 172.21.15.94:0/3424452470 >> 172.21.15.94:0/3424452470 conn(0x7f5a600fc7a0 msgr2=0x7f5a6010e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.290+0000 7f5a649ad640 1 -- 172.21.15.94:0/3424452470 shutdown_connections 2024-09-19T04:30:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:44.290+0000 7f5a649ad640 1 -- 172.21.15.94:0/3424452470 wait complete. 2024-09-19T04:30:45.084 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:30:45.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:44 smithi112 ceph-mon[35051]: pgmap v492: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.4 MiB/s wr, 877 op/s 2024-09-19T04:30:45.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:44 smithi094 ceph-mon[30972]: pgmap v492: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.4 MiB/s wr, 877 op/s 2024-09-19T04:30:45.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:44 smithi104 ceph-mon[34145]: pgmap v492: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.4 MiB/s wr, 877 op/s 2024-09-19T04:30:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:45.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 870.0 s, 390.8 tps, lat 254.116 ms stddev 36.120 2024-09-19T04:30:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.910+0000 7f72b12a8640 1 -- 172.21.15.94:0/3326433781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72ac1049a0 msgr2=0x7f72ac1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.910+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3326433781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72ac1049a0 0x7f72ac1089f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7294009900 tx=0x7f729402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.910+0000 7f72b12a8640 1 -- 172.21.15.94:0/3326433781 shutdown_connections 2024-09-19T04:30:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.910+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3326433781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac109120 0x7f72ac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.910+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3326433781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72ac1049a0 0x7f72ac1089f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.910+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3326433781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72ac103ff0 0x7f72ac1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.910+0000 7f72b12a8640 1 -- 172.21.15.94:0/3326433781 >> 172.21.15.94:0/3326433781 conn(0x7f72ac0ffab0 msgr2=0x7f72ac101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.910+0000 7f72b12a8640 1 -- 172.21.15.94:0/3326433781 shutdown_connections 2024-09-19T04:30:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.910+0000 7f72b12a8640 1 -- 172.21.15.94:0/3326433781 wait complete. 2024-09-19T04:30:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72b12a8640 1 Processor -- start 2024-09-19T04:30:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72b12a8640 1 -- start start 2024-09-19T04:30:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72b12a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72ac103ff0 0x7f72ac19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72b12a8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72ac1049a0 0x7f72ac19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72b12a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac109120 0x7f72ac19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72b12a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac114020 con 0x7f72ac103ff0 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72b12a8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac113ea0 con 0x7f72ac1049a0 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72b12a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1141a0 con 0x7f72ac109120 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72ab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac109120 0x7f72ac19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72ab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac109120 0x7f72ac19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:40006/0 (socket says 172.21.15.94:40006) 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72ab7fe640 1 -- 172.21.15.94:0/3358042398 learned_addr learned my addr 172.21.15.94:0/3358042398 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72a3fff640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72ac1049a0 0x7f72ac19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.911+0000 7f72aaffd640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72ac103ff0 0x7f72ac19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72ab7fe640 1 -- 172.21.15.94:0/3358042398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72ac1049a0 msgr2=0x7f72ac19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72ab7fe640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72ac1049a0 0x7f72ac19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72ab7fe640 1 -- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72ac103ff0 msgr2=0x7f72ac19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72ab7fe640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72ac103ff0 0x7f72ac19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72ab7fe640 1 -- 172.21.15.94:0/3358042398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac19f4f0 con 0x7f72ac109120 2024-09-19T04:30:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72ab7fe640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac109120 0x7f72ac19ee10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72a400efd0 tx=0x7f72a400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72a8ff9640 1 -- 172.21.15.94:0/3358042398 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4003b70 con 0x7f72ac109120 2024-09-19T04:30:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72a8ff9640 1 -- 172.21.15.94:0/3358042398 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72a4003d10 con 0x7f72ac109120 2024-09-19T04:30:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72a8ff9640 1 -- 172.21.15.94:0/3358042398 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4010980 con 0x7f72ac109120 2024-09-19T04:30:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72b12a8640 1 -- 172.21.15.94:0/3358042398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac19f7e0 con 0x7f72ac109120 2024-09-19T04:30:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.912+0000 7f72b12a8640 1 -- 172.21.15.94:0/3358042398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac1a70c0 con 0x7f72ac109120 2024-09-19T04:30:45.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.915+0000 7f72a8ff9640 1 -- 172.21.15.94:0/3358042398 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f72a4004020 con 0x7f72ac109120 2024-09-19T04:30:45.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.915+0000 7f72b3681740 1 Processor -- start 2024-09-19T04:30:45.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72b3681740 1 -- start start 2024-09-19T04:30:45.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72b3681740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182cb40 0x55d40182b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72b3681740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d40182ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72b3681740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40183eb80 0x55d401840f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72b3681740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d4017912f0 con 0x55d40182cb40 2024-09-19T04:30:45.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72b3681740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d401738cf0 con 0x55d40183eb80 2024-09-19T04:30:45.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72b3681740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d40178d2b0 con 0x55d40182b5d0 2024-09-19T04:30:45.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d40182ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182cb40 0x55d40182b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182cb40 0x55d40182b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38350/0 (socket says 172.21.15.94:38350) 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.916+0000 7f72a3fff640 1 -- 172.21.15.94:0/3300345771 learned_addr learned my addr 172.21.15.94:0/3300345771 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.917+0000 7f72a3fff640 1 --2- 172.21.15.94:0/3300345771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40183eb80 0x55d401840f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.917+0000 7f72a3fff640 1 -- 172.21.15.94:0/3300345771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 msgr2=0x55d40182ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.917+0000 7f72a3fff640 1 --2- 172.21.15.94:0/3300345771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d40182ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.917+0000 7f72a3fff640 1 -- 172.21.15.94:0/3300345771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182cb40 msgr2=0x55d40182b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.917+0000 7f72a3fff640 1 --2- 172.21.15.94:0/3300345771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182cb40 0x55d40182b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.917+0000 7f72a3fff640 1 -- 172.21.15.94:0/3300345771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40171d3a0 con 0x55d40183eb80 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.917+0000 7f72a3fff640 1 --2- 172.21.15.94:0/3300345771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40183eb80 0x55d401840f70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7294006010 tx=0x7f7294005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8737dae9ab2d1a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.917+0000 7f72a8ff9640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f72980a3430 0x7f72980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.918+0000 7f72a17fa640 1 -- 172.21.15.94:0/3300345771 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294041070 con 0x55d40183eb80 2024-09-19T04:30:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.918+0000 7f72aaffd640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f72980a3430 0x7f72980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.918+0000 7f72a17fa640 1 -- 172.21.15.94:0/3300345771 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7294003de0 con 0x55d40183eb80 2024-09-19T04:30:45.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.918+0000 7f72a17fa640 1 -- 172.21.15.94:0/3300345771 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294035070 con 0x55d40183eb80 2024-09-19T04:30:45.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.918+0000 7f72a8ff9640 1 -- 172.21.15.94:0/3358042398 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f72a40d0070 con 0x7f72ac109120 2024-09-19T04:30:45.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.919+0000 7f72b3681740 1 -- 172.21.15.94:0/3300345771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40183eb80 msgr2=0x55d401840f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:45.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.919+0000 7f72b3681740 1 --2- 172.21.15.94:0/3300345771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40183eb80 0x55d401840f70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7294006010 tx=0x7f7294005a20 comp rx=0 tx=0).stop 2024-09-19T04:30:45.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.919+0000 7f72b3681740 1 -- 172.21.15.94:0/3300345771 shutdown_connections 2024-09-19T04:30:45.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.919+0000 7f72b3681740 1 --2- 172.21.15.94:0/3300345771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40183eb80 0x55d401840f70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.919+0000 7f72b3681740 1 --2- 172.21.15.94:0/3300345771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d40182ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.919+0000 7f72b3681740 1 --2- 172.21.15.94:0/3300345771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182cb40 0x55d40182b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.919+0000 7f72b3681740 1 -- 172.21.15.94:0/3300345771 >> 172.21.15.94:0/3300345771 conn(0x55d4017f4310 msgr2=0x55d401827c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:45.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.920+0000 7f72b3681740 1 -- 172.21.15.94:0/3300345771 shutdown_connections 2024-09-19T04:30:45.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.920+0000 7f72aaffd640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f72980a3430 0x7f72980a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55d401828ac0 tx=0x7f729c00b410 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:45.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.920+0000 7f72b3681740 1 -- 172.21.15.94:0/3300345771 wait complete. 2024-09-19T04:30:45.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.921+0000 7f72b3681740 1 -- start start 2024-09-19T04:30:45.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.921+0000 7f72b3681740 1 Processor -- start 2024-09-19T04:30:45.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.921+0000 7f72b3681740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182b5d0 0x55d40183e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.921+0000 7f72b3681740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182cb40 0x55d40183b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72b3681740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 0x55d40183bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72b3681740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d401841820 con 0x55d40183eb80 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72b3681740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d40174c060 con 0x55d40182b5d0 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72b3681740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d4018419a0 con 0x55d40182cb40 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182b5d0 0x55d40183e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182cb40 0x55d40183b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182b5d0 0x55d40183e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59196/0 (socket says 172.21.15.94:59196) 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182cb40 0x55d40183b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:40024/0 (socket says 172.21.15.94:40024) 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72aaffd640 1 -- 172.21.15.94:0/826039010 learned_addr learned my addr 172.21.15.94:0/826039010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72b3681740 10 client.0 _ll_register_callbacks cb 0x55d40149f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:45.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72ab7fe640 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 0x55d40183bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72aaffd640 1 -- 172.21.15.94:0/826039010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182cb40 msgr2=0x55d40183b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:45.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72aaffd640 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182cb40 0x55d40183b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72aaffd640 1 -- 172.21.15.94:0/826039010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 msgr2=0x55d40183bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72aaffd640 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 0x55d40183bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.922+0000 7f72aaffd640 1 -- 172.21.15.94:0/826039010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4015023c0 con 0x55d40182b5d0 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.923+0000 7f72a3fff640 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182cb40 0x55d40183b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.923+0000 7f72aaffd640 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182b5d0 0x55d40183e560 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f729c010790 tx=0x7f729c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147de286574ea50d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.923+0000 7f72927fc640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.923+0000 7f72927fc640 1 -- 172.21.15.94:0/826039010 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c023070 con 0x55d40182b5d0 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.923+0000 7f72927fc640 1 -- 172.21.15.94:0/826039010 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f729c00fb00 con 0x55d40182b5d0 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.923+0000 7f72b3681740 1 -- 172.21.15.94:0/826039010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7294003de0 con 0x55d40182b5d0 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72927fc640 1 -- 172.21.15.94:0/826039010 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f729c02f050 con 0x55d40182b5d0 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72b3681740 10 client.26966 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72b3681740 1 -- 172.21.15.94:0/826039010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d40184d360 con 0x55d40182b5d0 2024-09-19T04:30:45.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72927fc640 1 -- 172.21.15.94:0/826039010 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f729c00fd00 con 0x55d40182b5d0 2024-09-19T04:30:45.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72927fc640 1 -- 172.21.15.94:0/826039010 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f729c027440 con 0x55d40182b5d0 2024-09-19T04:30:45.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72b3681740 10 client.26966 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:45.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72b3681740 10 client.26966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:30:45.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72b3681740 20 client.26966 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:45.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72b3681740 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d40184eba0 0x55d401851020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72b3681740 4 client.26966 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:45.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.924+0000 7f72b3681740 1 -- 172.21.15.94:0/826039010 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7294003de0 con 0x55d40184eba0 2024-09-19T04:30:45.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.925+0000 7f72a3fff640 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d40184eba0 0x55d401851020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.926+0000 7f72927fc640 10 client.26966 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:30:45.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.926+0000 7f72a3fff640 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d40184eba0 0x55d401851020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efea07b29b9a5953 server_cookie=7de9f7c69f6e3ded in_seq=0 out_seq=0 2024-09-19T04:30:45.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72927fc640 1 -- 172.21.15.94:0/826039010 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7294003de0 con 0x55d40184eba0 2024-09-19T04:30:45.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72927fc640 10 client.26966 handle_command_reply: tid=0 2024-09-19T04:30:45.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72b3681740 1 client.26966 shutdown 2024-09-19T04:30:45.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72b3681740 1 -- 172.21.15.94:0/826039010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182b5d0 msgr2=0x55d40183e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:45.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72b3681740 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182b5d0 0x55d40183e560 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f729c010790 tx=0x7f729c0046c0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72b3681740 1 -- 172.21.15.94:0/826039010 shutdown_connections 2024-09-19T04:30:45.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72b3681740 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d40184eba0 0x55d401851020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72b3681740 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 0x55d40183bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72b3681740 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182cb40 0x55d40183b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72b3681740 1 --2- 172.21.15.94:0/826039010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182b5d0 0x55d40183e560 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72b3681740 1 -- 172.21.15.94:0/826039010 >> 172.21.15.94:0/826039010 conn(0x55d4017f4310 msgr2=0x55d40182d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:45.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.928+0000 7f72927fc640 0 client.26966 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:30:45.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.929+0000 7f72b3681740 1 -- 172.21.15.94:0/826039010 shutdown_connections 2024-09-19T04:30:45.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.929+0000 7f72b3681740 1 -- 172.21.15.94:0/826039010 wait complete. 2024-09-19T04:30:45.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.929+0000 7f72b3681740 20 client.26966 trim_cache size 0 max 16384 2024-09-19T04:30:45.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72b3681740 1 Processor -- start 2024-09-19T04:30:45.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72b3681740 1 -- start start 2024-09-19T04:30:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72b3681740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182b5d0 0x55d40184aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72b3681740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 0x55d401827700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72b3681740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40183eb80 0x55d401827c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72b3681740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1adfb0 con 0x55d40182b5d0 2024-09-19T04:30:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72b3681740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1ab880 con 0x55d40182cb40 2024-09-19T04:30:45.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72b3681740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1a96f0 con 0x55d40183eb80 2024-09-19T04:30:45.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182b5d0 0x55d40184aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72ab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40183eb80 0x55d401827c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.953+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182b5d0 0x55d40184aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38368/0 (socket says 172.21.15.94:38368) 2024-09-19T04:30:45.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72ab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40183eb80 0x55d401827c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:40034/0 (socket says 172.21.15.94:40034) 2024-09-19T04:30:45.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72aaffd640 1 -- 172.21.15.94:0/774972441 learned_addr learned my addr 172.21.15.94:0/774972441 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:45.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72a3fff640 1 --2- 172.21.15.94:0/774972441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 0x55d401827700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72ab7fe640 1 -- 172.21.15.94:0/774972441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 msgr2=0x55d401827700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72ab7fe640 1 --2- 172.21.15.94:0/774972441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 0x55d401827700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72ab7fe640 1 -- 172.21.15.94:0/774972441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182b5d0 msgr2=0x55d40184aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72ab7fe640 1 --2- 172.21.15.94:0/774972441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182b5d0 0x55d40184aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72ab7fe640 1 -- 172.21.15.94:0/774972441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d401828180 con 0x55d40183eb80 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72ab7fe640 1 --2- 172.21.15.94:0/774972441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40183eb80 0x55d401827c40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f72a40cd780 tx=0x7f72a409bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e933a3f6d5e1857 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72a0ff9640 1 -- 172.21.15.94:0/774972441 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a40b5070 con 0x55d40183eb80 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72a0ff9640 1 -- 172.21.15.94:0/774972441 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72a409ca10 con 0x55d40183eb80 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.954+0000 7f72a0ff9640 1 -- 172.21.15.94:0/774972441 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a40cda00 con 0x55d40183eb80 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.955+0000 7f72b3681740 1 -- 172.21.15.94:0/774972441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40183eb80 msgr2=0x55d401827c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.955+0000 7f72b3681740 1 --2- 172.21.15.94:0/774972441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40183eb80 0x55d401827c40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f72a40cd780 tx=0x7f72a409bc70 comp rx=0 tx=0).stop 2024-09-19T04:30:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.955+0000 7f72b3681740 1 -- 172.21.15.94:0/774972441 shutdown_connections 2024-09-19T04:30:45.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.955+0000 7f72b3681740 1 --2- 172.21.15.94:0/774972441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40183eb80 0x55d401827c40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.955+0000 7f72b3681740 1 --2- 172.21.15.94:0/774972441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 0x55d401827700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.955+0000 7f72b3681740 1 --2- 172.21.15.94:0/774972441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40182b5d0 0x55d40184aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.955+0000 7f72b3681740 1 -- 172.21.15.94:0/774972441 >> 172.21.15.94:0/774972441 conn(0x55d4017f4310 msgr2=0x55d401836b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:45.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.955+0000 7f72b3681740 1 -- 172.21.15.94:0/774972441 shutdown_connections 2024-09-19T04:30:45.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.955+0000 7f72b3681740 1 -- 172.21.15.94:0/774972441 wait complete. 2024-09-19T04:30:45.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.956+0000 7f72b3681740 1 -- start start 2024-09-19T04:30:45.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.956+0000 7f72b3681740 1 Processor -- start 2024-09-19T04:30:45.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.956+0000 7f72b3681740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d401846fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.956+0000 7f72b3681740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 0x55d401847510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.956+0000 7f72b3681740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 0x55d4018383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d401846fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d401846fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:40048/0 (socket says 172.21.15.94:40048) 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72aaffd640 1 -- 172.21.15.94:0/3141480247 learned_addr learned my addr 172.21.15.94:0/3141480247 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1a9d40 con 0x55d40183eb80 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1ab1b0 con 0x55d40182cb40 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1aa410 con 0x55d40182b5d0 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72ab7fe640 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 0x55d4018383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72b3681740 10 client.0 _ll_register_callbacks cb 0x7f729c00ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72a3fff640 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 0x55d401847510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72aaffd640 1 -- 172.21.15.94:0/3141480247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 msgr2=0x55d401847510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72aaffd640 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 0x55d401847510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72aaffd640 1 -- 172.21.15.94:0/3141480247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 msgr2=0x55d4018383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72aaffd640 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 0x55d4018383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72aaffd640 1 -- 172.21.15.94:0/3141480247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d401838c00 con 0x55d40182b5d0 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72aaffd640 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d401846fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f729c00f600 tx=0x7f729c0275e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2799951389aebe0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72927fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72927fc640 1 -- 172.21.15.94:0/3141480247 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c023040 con 0x55d40182b5d0 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.957+0000 7f72927fc640 1 -- 172.21.15.94:0/3141480247 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f729c027c00 con 0x55d40182b5d0 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a409ca10 con 0x55d40182b5d0 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72927fc640 1 -- 172.21.15.94:0/3141480247 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f729c00b480 con 0x55d40182b5d0 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72b3681740 10 client.26935 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d4018b9500 con 0x55d40182b5d0 2024-09-19T04:30:45.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72927fc640 1 -- 172.21.15.94:0/3141480247 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72a409ca10 con 0x55d40182b5d0 2024-09-19T04:30:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72927fc640 1 -- 172.21.15.94:0/3141480247 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f729c03c470 con 0x55d40182b5d0 2024-09-19T04:30:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72b3681740 10 client.26935 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72b3681740 10 client.26935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:30:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72b3681740 20 client.26935 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72b3681740 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d4018b9c50 0x55d4018bc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72b3681740 4 client.26935 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.958+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d40179e860 con 0x55d4018b9c50 2024-09-19T04:30:45.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.991+0000 7f72a3fff640 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d4018b9c50 0x55d4018bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:45.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.992+0000 7f72a3fff640 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d4018b9c50 0x55d4018bc070 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ba2316f1c2ec724 server_cookie=7dfd1429ec7b1b9e in_seq=0 out_seq=0 2024-09-19T04:30:45.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.992+0000 7f72927fc640 10 client.26935 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:30:45.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.992+0000 7f72927fc640 1 -- 172.21.15.94:0/3141480247 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d40179e860 con 0x55d4018b9c50 2024-09-19T04:30:45.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.992+0000 7f72927fc640 10 client.26935 handle_command_reply: tid=0 2024-09-19T04:30:45.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72b3681740 1 client.26935 shutdown 2024-09-19T04:30:45.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 msgr2=0x55d401846fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:45.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72b3681740 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d401846fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f729c00f600 tx=0x7f729c0275e0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 shutdown_connections 2024-09-19T04:30:45.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72b3681740 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55d4018b9c50 0x55d4018bc070 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72b3681740 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d40183eb80 0x55d4018383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72b3681740 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40182cb40 0x55d401847510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72b3681740 1 --2- 172.21.15.94:0/3141480247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d40182b5d0 0x55d401846fd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72927fc640 0 client.26935 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:30:45.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.993+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 >> 172.21.15.94:0/3141480247 conn(0x55d4017f4310 msgr2=0x55d401836b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:45.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.994+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 shutdown_connections 2024-09-19T04:30:45.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.994+0000 7f72b3681740 1 -- 172.21.15.94:0/3141480247 wait complete. 2024-09-19T04:30:45.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.994+0000 7f72b3681740 20 client.26935 trim_cache size 0 max 16384 2024-09-19T04:30:45.996 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 -- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f72980a3430 msgr2=0x7f72980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f72980a3430 0x7f72980a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55d401828ac0 tx=0x7f729c00b410 comp rx=0 tx=0).stop 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 -- 172.21.15.94:0/3358042398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac109120 msgr2=0x7f72ac19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac109120 0x7f72ac19ee10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72a400efd0 tx=0x7f72a400c6a0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 -- 172.21.15.94:0/3358042398 shutdown_connections 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f72980a3430 0x7f72980a58f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac109120 0x7f72ac19ee10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72ac1049a0 0x7f72ac19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 --2- 172.21.15.94:0/3358042398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72ac103ff0 0x7f72ac19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 -- 172.21.15.94:0/3358042398 >> 172.21.15.94:0/3358042398 conn(0x7f72ac0ffab0 msgr2=0x7f72ac100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:45.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 -- 172.21.15.94:0/3358042398 shutdown_connections 2024-09-19T04:30:45.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:45.995+0000 7f72b12a8640 1 -- 172.21.15.94:0/3358042398 wait complete. 2024-09-19T04:30:47.107 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:30:47.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:46 smithi112 ceph-mon[35051]: pgmap v493: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 590 op/s 2024-09-19T04:30:47.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:46 smithi094 ceph-mon[30972]: pgmap v493: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 590 op/s 2024-09-19T04:30:47.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:46 smithi104 ceph-mon[34145]: pgmap v493: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 590 op/s 2024-09-19T04:30:47.426 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:47.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.970+0000 7f5e28d92640 1 -- 172.21.15.94:0/1337593713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24106f00 msgr2=0x7f5e2410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:47.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.970+0000 7f5e28d92640 1 --2- 172.21.15.94:0/1337593713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24106f00 0x7f5e2410ea80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009ca0 tx=0x7f5e1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.970+0000 7f5e28d92640 1 -- 172.21.15.94:0/1337593713 shutdown_connections 2024-09-19T04:30:47.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.970+0000 7f5e28d92640 1 --2- 172.21.15.94:0/1337593713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24106f00 0x7f5e2410ea80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.970+0000 7f5e28d92640 1 --2- 172.21.15.94:0/1337593713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e24102780 0x7f5e241067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.970+0000 7f5e28d92640 1 --2- 172.21.15.94:0/1337593713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e24101dd0 0x7f5e241021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.970+0000 7f5e28d92640 1 -- 172.21.15.94:0/1337593713 >> 172.21.15.94:0/1337593713 conn(0x7f5e240fd8b0 msgr2=0x7f5e240ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.970+0000 7f5e28d92640 1 -- 172.21.15.94:0/1337593713 shutdown_connections 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.970+0000 7f5e28d92640 1 -- 172.21.15.94:0/1337593713 wait complete. 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.971+0000 7f5e28d92640 1 Processor -- start 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.971+0000 7f5e28d92640 1 -- start start 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.971+0000 7f5e28d92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24101dd0 0x7f5e241982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.971+0000 7f5e28d92640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e24102780 0x7f5e24198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.971+0000 7f5e28d92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e24106f00 0x7f5e2419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.971+0000 7f5e28d92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e24111e10 con 0x7f5e24101dd0 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.971+0000 7f5e28d92640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e24111c90 con 0x7f5e24102780 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.971+0000 7f5e28d92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e24111f90 con 0x7f5e24106f00 2024-09-19T04:30:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.971+0000 7f5e2259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24101dd0 0x7f5e241982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e2259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24101dd0 0x7f5e241982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38392/0 (socket says 172.21.15.94:38392) 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e2259c640 1 -- 172.21.15.94:0/4174244084 learned_addr learned my addr 172.21.15.94:0/4174244084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e24102780 0x7f5e24198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e24106f00 0x7f5e2419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e2259c640 1 -- 172.21.15.94:0/4174244084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e24106f00 msgr2=0x7f5e2419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e2259c640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e24106f00 0x7f5e2419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e2259c640 1 -- 172.21.15.94:0/4174244084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e24102780 msgr2=0x7f5e24198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e2259c640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e24102780 0x7f5e24198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e2259c640 1 -- 172.21.15.94:0/4174244084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2419d290 con 0x7f5e24101dd0 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e24102780 0x7f5e24198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e2259c640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24101dd0 0x7f5e241982e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400e9f0 tx=0x7f5e1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e1b7fe640 1 -- 172.21.15.94:0/4174244084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400cd80 con 0x7f5e24101dd0 2024-09-19T04:30:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e1b7fe640 1 -- 172.21.15.94:0/4174244084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e140108e0 con 0x7f5e24101dd0 2024-09-19T04:30:47.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e28d92640 1 -- 172.21.15.94:0/4174244084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2419d580 con 0x7f5e24101dd0 2024-09-19T04:30:47.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e1b7fe640 1 -- 172.21.15.94:0/4174244084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14018920 con 0x7f5e24101dd0 2024-09-19T04:30:47.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.972+0000 7f5e28d92640 1 -- 172.21.15.94:0/4174244084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e241a4e60 con 0x7f5e24101dd0 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.976+0000 7f5e2b16b740 1 Processor -- start 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.976+0000 7f5e1b7fe640 1 -- 172.21.15.94:0/4174244084 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5e14010a90 con 0x7f5e24101dd0 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.976+0000 7f5e2b16b740 1 -- start start 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.976+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a359090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.976+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a3595d0 0x55971a359a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.976+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a366760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.976+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55971a2bf2f0 con 0x55971a364370 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.976+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55971a266cf0 con 0x55971a3595d0 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.976+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55971a2bb2b0 con 0x55971a35ab40 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e22d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a359090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e2259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a3595d0 0x55971a359a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e22d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a359090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:40076/0 (socket says 172.21.15.94:40076) 2024-09-19T04:30:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e22d9d640 1 -- 172.21.15.94:0/3484346467 learned_addr learned my addr 172.21.15.94:0/3484346467 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/3484346467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a366760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e22d9d640 1 -- 172.21.15.94:0/3484346467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a3595d0 msgr2=0x55971a359a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/3484346467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a3595d0 0x55971a359a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e22d9d640 1 -- 172.21.15.94:0/3484346467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 msgr2=0x55971a366760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/3484346467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a366760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e22d9d640 1 -- 172.21.15.94:0/3484346467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971a24b3a0 con 0x55971a35ab40 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e2259c640 1 --2- 172.21.15.94:0/3484346467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a3595d0 0x55971a359a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e1b7fe640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5e100a33b0 0x7f5e100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/3484346467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a359090 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009ca0 tx=0x7f5e1c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12d37fecab9954d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.977+0000 7f5e2259c640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5e100a33b0 0x7f5e100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.978+0000 7f5e1b7fe640 1 -- 172.21.15.94:0/4174244084 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f5e140d0030 con 0x7f5e24101dd0 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.978+0000 7f5e18ff9640 1 -- 172.21.15.94:0/3484346467 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c031d00 con 0x55971a35ab40 2024-09-19T04:30:47.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.978+0000 7f5e18ff9640 1 -- 172.21.15.94:0/3484346467 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e1c036030 con 0x55971a35ab40 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.978+0000 7f5e18ff9640 1 -- 172.21.15.94:0/3484346467 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0042a0 con 0x55971a35ab40 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 -- 172.21.15.94:0/3484346467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 msgr2=0x55971a359090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/3484346467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a359090 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009ca0 tx=0x7f5e1c009a20 comp rx=0 tx=0).stop 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2259c640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5e100a33b0 0x7f5e100a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5e140cd7f0 tx=0x7f5e1409c150 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 -- 172.21.15.94:0/3484346467 shutdown_connections 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/3484346467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a366760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/3484346467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a3595d0 0x55971a359a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/3484346467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a359090 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 -- 172.21.15.94:0/3484346467 >> 172.21.15.94:0/3484346467 conn(0x55971a322310 msgr2=0x55971a355c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 -- 172.21.15.94:0/3484346467 shutdown_connections 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 -- 172.21.15.94:0/3484346467 wait complete. 2024-09-19T04:30:47.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.979+0000 7f5e2b16b740 1 -- start start 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e2b16b740 1 Processor -- start 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a363cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a360fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a364370 0x55971a361520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55971a36b2f0 con 0x55971a3595d0 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55971a27a060 con 0x55971a364370 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55971a36b470 con 0x55971a35ab40 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e2b16b740 10 client.0 _ll_register_callbacks cb 0x559719fcd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e22d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a363cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e22d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a363cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38410/0 (socket says 172.21.15.94:38410) 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e22d9d640 1 -- 172.21.15.94:0/2798232993 learned_addr learned my addr 172.21.15.94:0/2798232993 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a360fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a364370 0x55971a361520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e1bfff640 1 -- 172.21.15.94:0/2798232993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a364370 msgr2=0x55971a361520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:47.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a364370 0x55971a361520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e1bfff640 1 -- 172.21.15.94:0/2798232993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 msgr2=0x55971a363cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:47.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a363cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e1bfff640 1 -- 172.21.15.94:0/2798232993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971a0303c0 con 0x55971a35ab40 2024-09-19T04:30:47.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.980+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a363cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:47.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a360fa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e240ffe60 tx=0x7f5e0c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df962c827a331b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:47.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:30:47.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e09ffb640 1 -- 172.21.15.94:0/2798232993 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c012110 con 0x55971a35ab40 2024-09-19T04:30:47.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e2b16b740 1 -- 172.21.15.94:0/2798232993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e1c036030 con 0x55971a35ab40 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e09ffb640 1 -- 172.21.15.94:0/2798232993 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f5e0c004050 con 0x55971a35ab40 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e09ffb640 1 -- 172.21.15.94:0/2798232993 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5e0c0122b0 con 0x55971a35ab40 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e2b16b740 10 client.26947 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e2b16b740 1 -- 172.21.15.94:0/2798232993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55971a376dd0 con 0x55971a35ab40 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e09ffb640 1 -- 172.21.15.94:0/2798232993 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e1c036030 con 0x55971a35ab40 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e09ffb640 1 -- 172.21.15.94:0/2798232993 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f5e0c010430 con 0x55971a35ab40 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e2b16b740 10 client.26947 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e2b16b740 10 client.26947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.981+0000 7f5e2b16b740 20 client.26947 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.982+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55971a378230 0x55971a37a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:47.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.982+0000 7f5e2b16b740 4 client.26947 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.982+0000 7f5e2b16b740 1 -- 172.21.15.94:0/2798232993 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55971a2cc860 con 0x55971a378230 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.982+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55971a378230 0x55971a37a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.982+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55971a378230 0x55971a37a650 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37210bf2d8ab8917 server_cookie=29faa2b863b82131 in_seq=0 out_seq=0 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.982+0000 7f5e09ffb640 10 client.26947 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e09ffb640 1 -- 172.21.15.94:0/2798232993 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55971a2cc860 con 0x55971a378230 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e09ffb640 10 client.26947 handle_command_reply: tid=0 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 client.26947 shutdown 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 -- 172.21.15.94:0/2798232993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 msgr2=0x55971a360fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a360fa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e240ffe60 tx=0x7f5e0c00f790 comp rx=0 tx=0).stop 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 -- 172.21.15.94:0/2798232993 shutdown_connections 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55971a378230 0x55971a37a650 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e09ffb640 0 client.26947 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:47.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a364370 0x55971a361520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a35ab40 0x55971a360fa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/2798232993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a363cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:47.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 -- 172.21.15.94:0/2798232993 >> 172.21.15.94:0/2798232993 conn(0x55971a322310 msgr2=0x55971a35b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:47.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 -- 172.21.15.94:0/2798232993 shutdown_connections 2024-09-19T04:30:47.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.983+0000 7f5e2b16b740 1 -- 172.21.15.94:0/2798232993 wait complete. 2024-09-19T04:30:47.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.984+0000 7f5e2b16b740 20 client.26947 trim_cache size 0 max 16384 2024-09-19T04:30:48.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.998+0000 7f5e2b16b740 1 Processor -- start 2024-09-19T04:30:48.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e2b16b740 1 -- start start 2024-09-19T04:30:48.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a3595d0 0x55971a374590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:48.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a354c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:48.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a35c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:48.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241adf40 con 0x55971a364370 2024-09-19T04:30:48.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241ab810 con 0x55971a35ab40 2024-09-19T04:30:48.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241a9680 con 0x55971a3595d0 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e22d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a35c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e1bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a354c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e22d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a35c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38422/0 (socket says 172.21.15.94:38422) 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e1bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a354c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59264/0 (socket says 172.21.15.94:59264) 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e22d9d640 1 -- 172.21.15.94:0/4264303792 learned_addr learned my addr 172.21.15.94:0/4264303792 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/4264303792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a3595d0 0x55971a374590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e1bfff640 1 -- 172.21.15.94:0/4264303792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a3595d0 msgr2=0x55971a374590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/4264303792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a3595d0 0x55971a374590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e1bfff640 1 -- 172.21.15.94:0/4264303792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 msgr2=0x55971a35c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/4264303792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a35c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e1bfff640 1 -- 172.21.15.94:0/4264303792 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971a356960 con 0x55971a35ab40 2024-09-19T04:30:48.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/4264303792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a35c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:48.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:47.999+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/4264303792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a354c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0108d0 tx=0x7f5e0c0057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d5fffc81da5f66e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:48.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e0bfff640 1 -- 172.21.15.94:0/4264303792 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c028070 con 0x55971a35ab40 2024-09-19T04:30:48.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e0bfff640 1 -- 172.21.15.94:0/4264303792 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e0c005590 con 0x55971a35ab40 2024-09-19T04:30:48.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e0bfff640 1 -- 172.21.15.94:0/4264303792 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c0035f0 con 0x55971a35ab40 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e2b16b740 1 -- 172.21.15.94:0/4264303792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 msgr2=0x55971a354c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/4264303792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a354c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0108d0 tx=0x7f5e0c0057d0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e2b16b740 1 -- 172.21.15.94:0/4264303792 shutdown_connections 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/4264303792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a364370 0x55971a35c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/4264303792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a354c10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/4264303792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a3595d0 0x55971a374590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.000+0000 7f5e2b16b740 1 -- 172.21.15.94:0/4264303792 >> 172.21.15.94:0/4264303792 conn(0x55971a322310 msgr2=0x55971a354500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 -- 172.21.15.94:0/4264303792 shutdown_connections 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 -- 172.21.15.94:0/4264303792 wait complete. 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 -- start start 2024-09-19T04:30:48.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 Processor -- start 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a36bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a36c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a364370 0x55971a3e0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241a9cd0 con 0x55971a3595d0 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241ab140 con 0x55971a35ab40 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e2b16b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241aa3a0 con 0x55971a364370 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e22d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a36bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e22d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a36bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38434/0 (socket says 172.21.15.94:38434) 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e22d9d640 1 -- 172.21.15.94:0/1389034040 learned_addr learned my addr 172.21.15.94:0/1389034040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.001+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a36c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e2b16b740 10 client.0 _ll_register_callbacks cb 0x7f5e0c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:48.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a364370 0x55971a3e0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e1bfff640 1 -- 172.21.15.94:0/1389034040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a364370 msgr2=0x55971a3e0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a364370 0x55971a3e0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e1bfff640 1 -- 172.21.15.94:0/1389034040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 msgr2=0x55971a36bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a36bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e1bfff640 1 -- 172.21.15.94:0/1389034040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971a3e1450 con 0x55971a35ab40 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a36bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e1bfff640 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a36c1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55971a36b200 tx=0x7f5e0c0104a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2410fcd11e48afb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e09ffb640 1 -- 172.21.15.94:0/1389034040 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c028070 con 0x55971a35ab40 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e2b16b740 1 -- 172.21.15.94:0/1389034040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e0c005590 con 0x55971a35ab40 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e09ffb640 1 -- 172.21.15.94:0/1389034040 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f5e0c0052c0 con 0x55971a35ab40 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e09ffb640 1 -- 172.21.15.94:0/1389034040 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5e0c0035f0 con 0x55971a35ab40 2024-09-19T04:30:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e2b16b740 10 client.26990 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e2b16b740 1 -- 172.21.15.94:0/1389034040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55971a3e26f0 con 0x55971a35ab40 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.002+0000 7f5e09ffb640 1 -- 172.21.15.94:0/1389034040 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e0c005590 con 0x55971a35ab40 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e09ffb640 1 -- 172.21.15.94:0/1389034040 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f5e0c01c690 con 0x55971a35ab40 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e2b16b740 10 client.26990 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e2b16b740 10 client.26990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e2b16b740 20 client.26990 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55971a378230 0x55971a3e4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e2b16b740 4 client.26990 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e2b16b740 1 -- 172.21.15.94:0/1389034040 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55971a2c6d70 con 0x55971a378230 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55971a378230 0x55971a3e4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e22d9d640 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55971a378230 0x55971a3e4db0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6edfed4623272235 server_cookie=fd1b48459fe28bff in_seq=0 out_seq=0 2024-09-19T04:30:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e09ffb640 10 client.26990 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e09ffb640 1 -- 172.21.15.94:0/1389034040 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55971a2c6d70 con 0x55971a378230 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.003+0000 7f5e09ffb640 10 client.26990 handle_command_reply: tid=0 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 client.26990 shutdown 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 -- 172.21.15.94:0/1389034040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 msgr2=0x55971a36c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a36c1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55971a36b200 tx=0x7f5e0c0104a0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 -- 172.21.15.94:0/1389034040 shutdown_connections 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55971a378230 0x55971a3e4db0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e09ffb640 0 client.26990 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55971a364370 0x55971a3e0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55971a35ab40 0x55971a36c1f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 --2- 172.21.15.94:0/1389034040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971a3595d0 0x55971a36bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 -- 172.21.15.94:0/1389034040 >> 172.21.15.94:0/1389034040 conn(0x55971a322310 msgr2=0x55971a363020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 -- 172.21.15.94:0/1389034040 shutdown_connections 2024-09-19T04:30:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 1 -- 172.21.15.94:0/1389034040 wait complete. 2024-09-19T04:30:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.004+0000 7f5e2b16b740 20 client.26990 trim_cache size 0 max 16384 2024-09-19T04:30:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 -- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5e100a33b0 msgr2=0x7f5e100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5e100a33b0 0x7f5e100a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5e140cd7f0 tx=0x7f5e1409c150 comp rx=0 tx=0).stop 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 -- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24101dd0 msgr2=0x7f5e241982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24101dd0 0x7f5e241982e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400e9f0 tx=0x7f5e1400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 -- 172.21.15.94:0/4174244084 shutdown_connections 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5e100a33b0 0x7f5e100a5870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e24106f00 0x7f5e2419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e24102780 0x7f5e24198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 --2- 172.21.15.94:0/4174244084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24101dd0 0x7f5e241982e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 -- 172.21.15.94:0/4174244084 >> 172.21.15.94:0/4174244084 conn(0x7f5e240fd8b0 msgr2=0x7f5e240fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 -- 172.21.15.94:0/4174244084 shutdown_connections 2024-09-19T04:30:48.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:48.005+0000 7f5e28d92640 1 -- 172.21.15.94:0/4174244084 wait complete. 2024-09-19T04:30:49.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:48 smithi112 ceph-mon[35051]: pgmap v494: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.9 MiB/s wr, 702 op/s 2024-09-19T04:30:49.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:48 smithi094 ceph-mon[30972]: pgmap v494: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.9 MiB/s wr, 702 op/s 2024-09-19T04:30:49.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:48 smithi104 ceph-mon[34145]: pgmap v494: 97 pgs: 97 active+clean; 3.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.9 MiB/s wr, 702 op/s 2024-09-19T04:30:49.810 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:30:49.811 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:30:50.113 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:50.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 875.0 s, 431.6 tps, lat 234.083 ms stddev 36.425 2024-09-19T04:30:51.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:50 smithi112 ceph-mon[35051]: pgmap v495: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-09-19T04:30:51.312 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:50 smithi094 ceph-mon[30972]: pgmap v495: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-09-19T04:30:51.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:50 smithi104 ceph-mon[34145]: pgmap v495: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-09-19T04:30:53.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:52 smithi112 ceph-mon[35051]: pgmap v496: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 6.5 MiB/s wr, 576 op/s 2024-09-19T04:30:53.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:52 smithi094 ceph-mon[30972]: pgmap v496: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 6.5 MiB/s wr, 576 op/s 2024-09-19T04:30:53.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:52 smithi104 ceph-mon[34145]: pgmap v496: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 6.5 MiB/s wr, 576 op/s 2024-09-19T04:30:53.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.067+0000 7f384e365640 1 -- 172.21.15.94:0/851105260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3848105f40 msgr2=0x7f38481063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.067+0000 7f384e365640 1 --2- 172.21.15.94:0/851105260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3848105f40 0x7f38481063d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3840009be0 tx=0x7f384002a750 comp rx=0 tx=0).stop 2024-09-19T04:30:53.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.067+0000 7f384e365640 1 -- 172.21.15.94:0/851105260 shutdown_connections 2024-09-19T04:30:53.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.067+0000 7f384e365640 1 --2- 172.21.15.94:0/851105260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3848105f40 0x7f38481063d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.067+0000 7f384e365640 1 --2- 172.21.15.94:0/851105260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3848101450 0x7f3848105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.067+0000 7f384e365640 1 --2- 172.21.15.94:0/851105260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3848100b30 0x7f3848100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.067+0000 7f384e365640 1 -- 172.21.15.94:0/851105260 >> 172.21.15.94:0/851105260 conn(0x7f38480fc7a0 msgr2=0x7f38480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.067+0000 7f384e365640 1 -- 172.21.15.94:0/851105260 shutdown_connections 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.067+0000 7f384e365640 1 -- 172.21.15.94:0/851105260 wait complete. 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.068+0000 7f384e365640 1 Processor -- start 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.068+0000 7f384e365640 1 -- start start 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.068+0000 7f384e365640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3848100b30 0x7f384819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.068+0000 7f384e365640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3848101450 0x7f384819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.068+0000 7f384e365640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3848105f40 0x7f38481a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f384e365640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3848118410 con 0x7f3848101450 2024-09-19T04:30:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f384e365640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3848118290 con 0x7f3848100b30 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f384e365640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3848118590 con 0x7f3848105f40 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f38477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3848101450 0x7f384819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f38477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3848101450 0x7f384819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33014/0 (socket says 172.21.15.94:33014) 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f38477fe640 1 -- 172.21.15.94:0/1916691403 learned_addr learned my addr 172.21.15.94:0/1916691403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f3847fff640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3848100b30 0x7f384819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f384c902640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3848105f40 0x7f38481a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f38477fe640 1 -- 172.21.15.94:0/1916691403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3848105f40 msgr2=0x7f38481a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f38477fe640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3848105f40 0x7f38481a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f38477fe640 1 -- 172.21.15.94:0/1916691403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3848100b30 msgr2=0x7f384819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f38477fe640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3848100b30 0x7f384819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.069+0000 7f38477fe640 1 -- 172.21.15.94:0/1916691403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38481a3700 con 0x7f3848101450 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.070+0000 7f38477fe640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3848101450 0x7f384819ec90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f383000e9f0 tx=0x7f383000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.070+0000 7f38457fa640 1 -- 172.21.15.94:0/1916691403 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383000cd80 con 0x7f3848101450 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.070+0000 7f38457fa640 1 -- 172.21.15.94:0/1916691403 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38300035f0 con 0x7f3848101450 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.070+0000 7f38457fa640 1 -- 172.21.15.94:0/1916691403 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830010690 con 0x7f3848101450 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.071+0000 7f384e365640 1 -- 172.21.15.94:0/1916691403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38481a39f0 con 0x7f3848101450 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.071+0000 7f384e365640 1 -- 172.21.15.94:0/1916691403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38481ab2d0 con 0x7f3848101450 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.076+0000 7f385073e740 1 Processor -- start 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.076+0000 7f38457fa640 1 -- 172.21.15.94:0/1916691403 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f38300037a0 con 0x7f3848101450 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.077+0000 7f385073e740 1 -- start start 2024-09-19T04:30:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.078+0000 7f38457fa640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f38340a3430 0x7f38340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f385073e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a98f5a00 0x5585a98f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f385073e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f6320 0x5585a9911810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f385073e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a9914140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f385073e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5585a985b280 con 0x5585a9911d50 2024-09-19T04:30:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f385073e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5585a9802b10 con 0x5585a98f6320 2024-09-19T04:30:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f385073e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5585a9857240 con 0x5585a98f5a00 2024-09-19T04:30:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f38457fa640 1 -- 172.21.15.94:0/1916691403 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f38300cfed0 con 0x7f3848101450 2024-09-19T04:30:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f384c902640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f38340a3430 0x7f38340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f384c902640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a9914140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f38477fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f6320 0x5585a9911810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f384c902640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a9914140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33020/0 (socket says 172.21.15.94:33020) 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.079+0000 7f384c902640 1 -- 172.21.15.94:0/1589198113 learned_addr learned my addr 172.21.15.94:0/1589198113 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.080+0000 7f38477fe640 1 -- 172.21.15.94:0/1589198113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a98f5a00 msgr2=0x5585a98f5de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.080+0000 7f38477fe640 1 --2- 172.21.15.94:0/1589198113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a98f5a00 0x5585a98f5de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.080+0000 7f38477fe640 1 -- 172.21.15.94:0/1589198113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 msgr2=0x5585a9914140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.080+0000 7f38477fe640 1 --2- 172.21.15.94:0/1589198113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a9914140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.080+0000 7f38477fe640 1 -- 172.21.15.94:0/1589198113 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a980f1f0 con 0x5585a98f6320 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.080+0000 7f38477fe640 1 --2- 172.21.15.94:0/1589198113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f6320 0x5585a9911810 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f38300cd5e0 tx=0x7f3830098540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8b7cce4b332285 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.080+0000 7f383e7fc640 1 -- 172.21.15.94:0/1589198113 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300b5070 con 0x5585a98f6320 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.080+0000 7f383e7fc640 1 -- 172.21.15.94:0/1589198113 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f383009c170 con 0x5585a98f6320 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.081+0000 7f383e7fc640 1 -- 172.21.15.94:0/1589198113 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300bc0d0 con 0x5585a98f6320 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f384c902640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f38340a3430 0x7f38340a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f384000b230 tx=0x7f3840005bd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f385073e740 1 -- 172.21.15.94:0/1589198113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f6320 msgr2=0x5585a9911810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f385073e740 1 --2- 172.21.15.94:0/1589198113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f6320 0x5585a9911810 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f38300cd5e0 tx=0x7f3830098540 comp rx=0 tx=0).stop 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f385073e740 1 -- 172.21.15.94:0/1589198113 shutdown_connections 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f385073e740 1 --2- 172.21.15.94:0/1589198113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a9914140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f385073e740 1 --2- 172.21.15.94:0/1589198113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f6320 0x5585a9911810 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f385073e740 1 --2- 172.21.15.94:0/1589198113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a98f5a00 0x5585a98f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f385073e740 1 -- 172.21.15.94:0/1589198113 >> 172.21.15.94:0/1589198113 conn(0x5585a98f20f0 msgr2=0x5585a98f24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f385073e740 1 -- 172.21.15.94:0/1589198113 shutdown_connections 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.082+0000 7f385073e740 1 -- 172.21.15.94:0/1589198113 wait complete. 2024-09-19T04:30:53.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.083+0000 7f385073e740 1 -- start start 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.083+0000 7f385073e740 1 Processor -- start 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.084+0000 7f385073e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a98f5a00 0x5585a990ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.084+0000 7f385073e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 0x5585a990f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.084+0000 7f385073e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a990f760 0x5585a991dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.084+0000 7f385073e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5585a99149c0 con 0x5585a98f5a00 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.084+0000 7f385073e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5585a9815f10 con 0x5585a990f760 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.084+0000 7f384c902640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a990f760 0x5585a991dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.084+0000 7f385073e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5585a9914b40 con 0x5585a9911d50 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f384c902640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a990f760 0x5585a991dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:55176/0 (socket says 172.21.15.94:55176) 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f384c902640 1 -- 172.21.15.94:0/1345676202 learned_addr learned my addr 172.21.15.94:0/1345676202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f385073e740 10 client.0 _ll_register_callbacks cb 0x5585a97dea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:53.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f3847fff640 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a98f5a00 0x5585a990ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f38477fe640 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 0x5585a990f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f384c902640 1 -- 172.21.15.94:0/1345676202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 msgr2=0x5585a990f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f384c902640 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 0x5585a990f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f384c902640 1 -- 172.21.15.94:0/1345676202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a98f5a00 msgr2=0x5585a990ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f384c902640 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a98f5a00 0x5585a990ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.085+0000 7f384c902640 1 -- 172.21.15.94:0/1345676202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a95cc6a0 con 0x5585a990f760 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.086+0000 7f384c902640 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a990f760 0x5585a991dec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f384002ac70 tx=0x7f384002a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5253b9f4a484f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.086+0000 7f38277fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.086+0000 7f38277fe640 1 -- 172.21.15.94:0/1345676202 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840051060 con 0x5585a990f760 2024-09-19T04:30:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.086+0000 7f38277fe640 1 -- 172.21.15.94:0/1345676202 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f3840031da0 con 0x5585a990f760 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.086+0000 7f385073e740 1 -- 172.21.15.94:0/1345676202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383009c170 con 0x5585a990f760 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.086+0000 7f38277fe640 1 -- 172.21.15.94:0/1345676202 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3840036070 con 0x5585a990f760 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.086+0000 7f385073e740 10 client.27002 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.086+0000 7f385073e740 1 -- 172.21.15.94:0/1345676202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5585a9920b80 con 0x5585a990f760 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.087+0000 7f38277fe640 1 -- 172.21.15.94:0/1345676202 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f383009c170 con 0x5585a990f760 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.087+0000 7f38277fe640 1 -- 172.21.15.94:0/1345676202 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f3840005710 con 0x5585a990f760 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.087+0000 7f385073e740 10 client.27002 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.087+0000 7f385073e740 10 client.27002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.087+0000 7f385073e740 20 client.27002 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.087+0000 7f385073e740 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5585a9922040 0x5585a9924430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.087+0000 7f385073e740 4 client.27002 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:53.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.087+0000 7f385073e740 1 -- 172.21.15.94:0/1345676202 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585a98687f0 con 0x5585a9922040 2024-09-19T04:30:53.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.087+0000 7f3847fff640 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5585a9922040 0x5585a9924430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.088+0000 7f3847fff640 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5585a9922040 0x5585a9924430 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b83d5e46a91a96 server_cookie=ab80a59867efa4b1 in_seq=0 out_seq=0 2024-09-19T04:30:53.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.088+0000 7f38277fe640 10 client.27002 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:53.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.088+0000 7f38277fe640 1 -- 172.21.15.94:0/1345676202 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585a98687f0 con 0x5585a9922040 2024-09-19T04:30:53.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.088+0000 7f38277fe640 10 client.27002 handle_command_reply: tid=0 2024-09-19T04:30:53.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 client.27002 shutdown 2024-09-19T04:30:53.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 -- 172.21.15.94:0/1345676202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a990f760 msgr2=0x5585a991dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a990f760 0x5585a991dec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f384002ac70 tx=0x7f384002a750 comp rx=0 tx=0).stop 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 -- 172.21.15.94:0/1345676202 shutdown_connections 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5585a9922040 0x5585a9924430 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a990f760 0x5585a991dec0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 0x5585a990f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 --2- 172.21.15.94:0/1345676202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a98f5a00 0x5585a990ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 -- 172.21.15.94:0/1345676202 >> 172.21.15.94:0/1345676202 conn(0x5585a98f20f0 msgr2=0x5585a98f1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f38277fe640 0 client.27002 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 -- 172.21.15.94:0/1345676202 shutdown_connections 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 1 -- 172.21.15.94:0/1345676202 wait complete. 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.089+0000 7f385073e740 20 client.27002 trim_cache size 0 max 16384 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.116+0000 7f385073e740 1 Processor -- start 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.116+0000 7f385073e740 1 -- start start 2024-09-19T04:30:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.116+0000 7f385073e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 0x5585a991dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.116+0000 7f385073e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a990f760 0x5585a9929d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.116+0000 7f385073e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a992a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.116+0000 7f385073e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38481ae000 con 0x5585a9911d50 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.116+0000 7f385073e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38481ab8d0 con 0x5585a98f5a00 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.116+0000 7f385073e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f384810ebf0 con 0x5585a990f760 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f384c902640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a992a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f384c902640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a992a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33034/0 (socket says 172.21.15.94:33034) 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f384c902640 1 -- 172.21.15.94:0/3588848845 learned_addr learned my addr 172.21.15.94:0/3588848845 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f38477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a990f760 0x5585a9929d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f3847fff640 1 --2- 172.21.15.94:0/3588848845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 0x5585a991dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f384c902640 1 -- 172.21.15.94:0/3588848845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a990f760 msgr2=0x5585a9929d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f384c902640 1 --2- 172.21.15.94:0/3588848845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a990f760 0x5585a9929d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f384c902640 1 -- 172.21.15.94:0/3588848845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 msgr2=0x5585a991dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f384c902640 1 --2- 172.21.15.94:0/3588848845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 0x5585a991dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f384c902640 1 -- 172.21.15.94:0/3588848845 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a9911650 con 0x5585a9911d50 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f384c902640 1 --2- 172.21.15.94:0/3588848845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a992a240 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3840009b30 tx=0x7f3840005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17da6267257bc28a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f383dffb640 1 -- 172.21.15.94:0/3588848845 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840051040 con 0x5585a9911d50 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f383dffb640 1 -- 172.21.15.94:0/3588848845 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3840037040 con 0x5585a9911d50 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.117+0000 7f383dffb640 1 -- 172.21.15.94:0/3588848845 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840054400 con 0x5585a9911d50 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.118+0000 7f385073e740 1 -- 172.21.15.94:0/3588848845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 msgr2=0x5585a992a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.118+0000 7f385073e740 1 --2- 172.21.15.94:0/3588848845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a992a240 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3840009b30 tx=0x7f3840005e30 comp rx=0 tx=0).stop 2024-09-19T04:30:53.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.118+0000 7f385073e740 1 -- 172.21.15.94:0/3588848845 shutdown_connections 2024-09-19T04:30:53.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.118+0000 7f385073e740 1 --2- 172.21.15.94:0/3588848845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a9911d50 0x5585a992a240 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.118+0000 7f385073e740 1 --2- 172.21.15.94:0/3588848845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a990f760 0x5585a9929d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.118+0000 7f385073e740 1 --2- 172.21.15.94:0/3588848845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 0x5585a991dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.119+0000 7f385073e740 1 -- 172.21.15.94:0/3588848845 >> 172.21.15.94:0/3588848845 conn(0x5585a98f20f0 msgr2=0x5585a990d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.119+0000 7f385073e740 1 -- 172.21.15.94:0/3588848845 shutdown_connections 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.119+0000 7f385073e740 1 -- 172.21.15.94:0/3588848845 wait complete. 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.119+0000 7f385073e740 1 -- start start 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.119+0000 7f385073e740 1 Processor -- start 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f385073e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 0x5585a991a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f385073e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a990f760 0x5585a9915750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f385073e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 0x5585a9915c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f385073e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f384810f240 con 0x5585a990f760 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f385073e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38481106b0 con 0x5585a98f5a00 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f385073e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f384810f910 con 0x5585a9911d50 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f38477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a990f760 0x5585a9915750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f385073e740 10 client.0 _ll_register_callbacks cb 0x7f3840009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:53.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f38477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a990f760 0x5585a9915750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33042/0 (socket says 172.21.15.94:33042) 2024-09-19T04:30:53.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f38477fe640 1 -- 172.21.15.94:0/1094004085 learned_addr learned my addr 172.21.15.94:0/1094004085 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:53.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f384c902640 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 0x5585a9915c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f3847fff640 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 0x5585a991a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f384c902640 1 -- 172.21.15.94:0/1094004085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 msgr2=0x5585a991a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f384c902640 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 0x5585a991a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f384c902640 1 -- 172.21.15.94:0/1094004085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a990f760 msgr2=0x5585a9915750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f384c902640 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a990f760 0x5585a9915750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f384c902640 1 -- 172.21.15.94:0/1094004085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a9916380 con 0x5585a9911d50 2024-09-19T04:30:53.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f384c902640 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 0x5585a9915c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3840004af0 tx=0x7f384003c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f5cb3da00d6fa06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.120+0000 7f38277fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:30:53.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f38277fe640 1 -- 172.21.15.94:0/1094004085 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840051070 con 0x5585a9911d50 2024-09-19T04:30:53.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f38277fe640 1 -- 172.21.15.94:0/1094004085 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f384003cd50 con 0x5585a9911d50 2024-09-19T04:30:53.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f385073e740 1 -- 172.21.15.94:0/1094004085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3840037040 con 0x5585a9911d50 2024-09-19T04:30:53.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f38277fe640 1 -- 172.21.15.94:0/1094004085 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f384005d430 con 0x5585a9911d50 2024-09-19T04:30:53.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f385073e740 10 client.26968 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:53.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f385073e740 1 -- 172.21.15.94:0/1094004085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5585a998cbb0 con 0x5585a9911d50 2024-09-19T04:30:53.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f38277fe640 1 -- 172.21.15.94:0/1094004085 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3840037040 con 0x5585a9911d50 2024-09-19T04:30:53.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f38277fe640 1 -- 172.21.15.94:0/1094004085 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f3840067020 con 0x5585a9911d50 2024-09-19T04:30:53.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f385073e740 10 client.26968 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:53.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f385073e740 10 client.26968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:53.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.121+0000 7f385073e740 20 client.26968 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:53.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.122+0000 7f385073e740 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5585a998d1c0 0x5585a998f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.122+0000 7f385073e740 4 client.26968 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:30:53.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.122+0000 7f385073e740 1 -- 172.21.15.94:0/1094004085 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5585a9862d00 con 0x5585a998d1c0 2024-09-19T04:30:53.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.122+0000 7f3847fff640 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5585a998d1c0 0x5585a998f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.122+0000 7f3847fff640 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5585a998d1c0 0x5585a998f5b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=475af6dbac9efdbd server_cookie=e3c79d5bfd3d4c60 in_seq=0 out_seq=0 2024-09-19T04:30:53.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.122+0000 7f38277fe640 10 client.26968 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:53.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f38277fe640 1 -- 172.21.15.94:0/1094004085 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5585a9862d00 con 0x5585a998d1c0 2024-09-19T04:30:53.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f38277fe640 10 client.26968 handle_command_reply: tid=0 2024-09-19T04:30:53.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f385073e740 1 client.26968 shutdown 2024-09-19T04:30:53.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f385073e740 1 -- 172.21.15.94:0/1094004085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 msgr2=0x5585a9915c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f385073e740 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 0x5585a9915c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3840004af0 tx=0x7f384003c750 comp rx=0 tx=0).stop 2024-09-19T04:30:53.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f385073e740 1 -- 172.21.15.94:0/1094004085 shutdown_connections 2024-09-19T04:30:53.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f385073e740 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5585a998d1c0 0x5585a998f5b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f385073e740 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585a9911d50 0x5585a9915c90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f385073e740 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585a990f760 0x5585a9915750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f38277fe640 0 client.26968 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:53.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f385073e740 1 --2- 172.21.15.94:0/1094004085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5585a98f5a00 0x5585a991a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.123+0000 7f385073e740 1 -- 172.21.15.94:0/1094004085 >> 172.21.15.94:0/1094004085 conn(0x5585a98f20f0 msgr2=0x5585a990d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.124+0000 7f385073e740 1 -- 172.21.15.94:0/1094004085 shutdown_connections 2024-09-19T04:30:53.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.124+0000 7f385073e740 1 -- 172.21.15.94:0/1094004085 wait complete. 2024-09-19T04:30:53.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.124+0000 7f385073e740 20 client.26968 trim_cache size 0 max 16384 2024-09-19T04:30:53.659 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:30:53.659 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:30:53.659 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "ebaf86cd-73e3-4873-beae-ee36b26c8b1e", 2024-09-19T04:30:53.660 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:30:53.660 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:30:53.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.124+0000 7f384e365640 1 -- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f38340a3430 msgr2=0x7f38340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.124+0000 7f384e365640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f38340a3430 0x7f38340a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f384000b230 tx=0x7f3840005bd0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.124+0000 7f384e365640 1 -- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3848101450 msgr2=0x7f384819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.124+0000 7f384e365640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3848101450 0x7f384819ec90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f383000e9f0 tx=0x7f383000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.124+0000 7f384e365640 1 -- 172.21.15.94:0/1916691403 shutdown_connections 2024-09-19T04:30:53.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.124+0000 7f384e365640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f38340a3430 0x7f38340a58f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.125+0000 7f384e365640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3848105f40 0x7f38481a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.125+0000 7f384e365640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3848101450 0x7f384819ec90 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.125+0000 7f384e365640 1 --2- 172.21.15.94:0/1916691403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3848100b30 0x7f384819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.125+0000 7f384e365640 1 -- 172.21.15.94:0/1916691403 >> 172.21.15.94:0/1916691403 conn(0x7f38480fc7a0 msgr2=0x7f38480fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.125+0000 7f384e365640 1 -- 172.21.15.94:0/1916691403 shutdown_connections 2024-09-19T04:30:53.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:53.125+0000 7f384e365640 1 -- 172.21.15.94:0/1916691403 wait complete. 2024-09-19T04:30:54.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:53 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:53 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.325 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:30:54.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:30:54.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:54.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:54.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:54.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:54.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:30:54.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:53 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:54.659 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:55.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:54 smithi094 ceph-mon[30972]: pgmap v497: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.1 MiB/s wr, 777 op/s 2024-09-19T04:30:55.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.233+0000 7f43a4971640 1 -- 172.21.15.94:0/2390844750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43a0100c80 msgr2=0x7f43a0110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.233+0000 7f43a4971640 1 --2- 172.21.15.94:0/2390844750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43a0100c80 0x7f43a0110890 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4398009be0 tx=0x7f439802a6c0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.233+0000 7f43a4971640 1 -- 172.21.15.94:0/2390844750 shutdown_connections 2024-09-19T04:30:55.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.233+0000 7f43a4971640 1 --2- 172.21.15.94:0/2390844750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43a0100c80 0x7f43a0110890 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.233+0000 7f43a4971640 1 --2- 172.21.15.94:0/2390844750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a0100100 0x7f43a0100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.233+0000 7f43a4971640 1 --2- 172.21.15.94:0/2390844750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a0102a50 0x7f43a0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.233+0000 7f43a4971640 1 -- 172.21.15.94:0/2390844750 >> 172.21.15.94:0/2390844750 conn(0x7f43a0077230 msgr2=0x7f43a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.234+0000 7f43a4971640 1 -- 172.21.15.94:0/2390844750 shutdown_connections 2024-09-19T04:30:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.234+0000 7f43a4971640 1 -- 172.21.15.94:0/2390844750 wait complete. 2024-09-19T04:30:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.235+0000 7f43a4971640 1 Processor -- start 2024-09-19T04:30:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.235+0000 7f43a4971640 1 -- start start 2024-09-19T04:30:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.235+0000 7f43a4971640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a0100100 0x7f43a019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.235+0000 7f43a4971640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a0100c80 0x7f43a019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.235+0000 7f43a4971640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43a0102a50 0x7f43a019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.235+0000 7f43a4971640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0113fd0 con 0x7f43a0102a50 2024-09-19T04:30:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.235+0000 7f43a4971640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0113e50 con 0x7f43a0100100 2024-09-19T04:30:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.235+0000 7f43a4971640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0114150 con 0x7f43a0100c80 2024-09-19T04:30:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43a0102a50 0x7f43a019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a0100100 0x7f43a019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a0100100 0x7f43a019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:55208/0 (socket says 172.21.15.94:55208) 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439e59c640 1 -- 172.21.15.94:0/757667477 learned_addr learned my addr 172.21.15.94:0/757667477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f4397fff640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a0100c80 0x7f43a019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439e59c640 1 -- 172.21.15.94:0/757667477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a0100c80 msgr2=0x7f43a019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439e59c640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a0100c80 0x7f43a019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439e59c640 1 -- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43a0102a50 msgr2=0x7f43a019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439e59c640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43a0102a50 0x7f43a019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439e59c640 1 -- 172.21.15.94:0/757667477 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a019f490 con 0x7f43a0100100 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439ed9d640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43a0102a50 0x7f43a019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f439e59c640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a0100100 0x7f43a019a4e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f439000e9c0 tx=0x7f439000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f43977fe640 1 -- 172.21.15.94:0/757667477 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439000cde0 con 0x7f43a0100100 2024-09-19T04:30:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.236+0000 7f43a4971640 1 -- 172.21.15.94:0/757667477 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a019f780 con 0x7f43a0100100 2024-09-19T04:30:55.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.237+0000 7f43a4971640 1 -- 172.21.15.94:0/757667477 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a0103420 con 0x7f43a0100100 2024-09-19T04:30:55.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.237+0000 7f43977fe640 1 -- 172.21.15.94:0/757667477 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43900035f0 con 0x7f43a0100100 2024-09-19T04:30:55.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.237+0000 7f43977fe640 1 -- 172.21.15.94:0/757667477 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390010690 con 0x7f43a0100100 2024-09-19T04:30:55.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.239+0000 7f43977fe640 1 -- 172.21.15.94:0/757667477 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4390003fe0 con 0x7f43a0100100 2024-09-19T04:30:55.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.240+0000 7f43a6d4a740 1 Processor -- start 2024-09-19T04:30:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.240+0000 7f43a6d4a740 1 -- start start 2024-09-19T04:30:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f04e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f059140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563e2efb42e0 con 0x563e2f059680 2024-09-19T04:30:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563e2ef5bce0 con 0x563e2f04edc0 2024-09-19T04:30:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563e2efb02a0 con 0x563e2f04e4a0 2024-09-19T04:30:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f439e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f059140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f439e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f059140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:55218/0 (socket says 172.21.15.94:55218) 2024-09-19T04:30:55.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f439ed9d640 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f04e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f43977fe640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f438c0a3430 0x7f438c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.241+0000 7f4397fff640 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f439ed9d640 1 -- 172.21.15.94:0/487271262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 msgr2=0x563e2f059140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f439ed9d640 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f059140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f439ed9d640 1 -- 172.21.15.94:0/487271262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 msgr2=0x563e2f05ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f43977fe640 1 -- 172.21.15.94:0/757667477 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f43900cfeb0 con 0x7f43a0100100 2024-09-19T04:30:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f439ed9d640 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f439e59c640 1 -- 172.21.15.94:0/487271262 learned_addr learned my addr 172.21.15.94:0/487271262 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:55.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f439e59c640 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f059140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-19T04:30:55.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f439ed9d640 1 -- 172.21.15.94:0/487271262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2ef40440 con 0x563e2f04e4a0 2024-09-19T04:30:55.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f4397fff640 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:55.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f439ed9d640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f438c0a3430 0x7f438c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f439ed9d640 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f04e880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f439802abe0 tx=0x7f4398003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29315627a759cb78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:55.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f4394ff9640 1 -- 172.21.15.94:0/487271262 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398036040 con 0x563e2f04e4a0 2024-09-19T04:30:55.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.242+0000 7f4394ff9640 1 -- 172.21.15.94:0/487271262 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4398037040 con 0x563e2f04e4a0 2024-09-19T04:30:55.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.243+0000 7f4394ff9640 1 -- 172.21.15.94:0/487271262 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398049740 con 0x563e2f04e4a0 2024-09-19T04:30:55.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.243+0000 7f439ed9d640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f438c0a3430 0x7f438c0a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x563e2f04f460 tx=0x7f4398048000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:55.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 -- 172.21.15.94:0/487271262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 msgr2=0x563e2f04e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f04e880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f439802abe0 tx=0x7f4398003970 comp rx=0 tx=0).stop 2024-09-19T04:30:55.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 -- 172.21.15.94:0/487271262 shutdown_connections 2024-09-19T04:30:55.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f059140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/487271262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f04e880 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 -- 172.21.15.94:0/487271262 >> 172.21.15.94:0/487271262 conn(0x563e2f04aa30 msgr2=0x563e2f04ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:55.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 -- 172.21.15.94:0/487271262 shutdown_connections 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 -- 172.21.15.94:0/487271262 wait complete. 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 -- start start 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.244+0000 7f43a6d4a740 1 Processor -- start 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f058f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f04edc0 0x563e2f056220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f059680 0x563e2f056760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563e2f05c250 con 0x563e2f04edc0 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f4397fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f04edc0 0x563e2f056220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f439e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f058f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f439ed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f059680 0x563e2f056760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f439e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f058f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:51140/0 (socket says 172.21.15.94:51140) 2024-09-19T04:30:55.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f439e59c640 1 -- 172.21.15.94:0/1133398063 learned_addr learned my addr 172.21.15.94:0/1133398063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:55.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563e2ef6f050 con 0x563e2f059680 2024-09-19T04:30:55.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563e2f05c3d0 con 0x563e2f04e4a0 2024-09-19T04:30:55.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.245+0000 7f43a6d4a740 10 client.0 _ll_register_callbacks cb 0x563e2efaf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:55.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f439e59c640 1 -- 172.21.15.94:0/1133398063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f059680 msgr2=0x563e2f056760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:55.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f439e59c640 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f059680 0x563e2f056760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f439e59c640 1 -- 172.21.15.94:0/1133398063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f04edc0 msgr2=0x563e2f056220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:55.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f439e59c640 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f04edc0 0x563e2f056220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f439e59c640 1 -- 172.21.15.94:0/1133398063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2ed253c0 con 0x563e2f04e4a0 2024-09-19T04:30:55.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f439ed9d640 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f059680 0x563e2f056760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:55.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f439e59c640 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f058f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f43900985b0 tx=0x7f439009bba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=635f3bd2f525f22b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:55.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f4385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f4385ffb640 1 -- 172.21.15.94:0/1133398063 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900983f0 con 0x563e2f04e4a0 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.246+0000 7f4385ffb640 1 -- 172.21.15.94:0/1133398063 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f43900042d0 con 0x563e2f04e4a0 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.247+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4398037040 con 0x563e2f04e4a0 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.247+0000 7f4385ffb640 1 -- 172.21.15.94:0/1133398063 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f43900a0500 con 0x563e2f04e4a0 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.247+0000 7f43a6d4a740 10 client.26980 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.247+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x563e2f067b90 con 0x563e2f04e4a0 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.247+0000 7f4385ffb640 1 -- 172.21.15.94:0/1133398063 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4398037040 con 0x563e2f04e4a0 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.247+0000 7f4385ffb640 1 -- 172.21.15.94:0/1133398063 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f4390097ea0 con 0x563e2f04e4a0 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.247+0000 7f43a6d4a740 10 client.26980 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.247+0000 7f43a6d4a740 10 client.26980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:55.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.248+0000 7f43a6d4a740 20 client.26980 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:55.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.248+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563e2f0690a0 0x563e2f06b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.248+0000 7f43a6d4a740 4 client.26980 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:55.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.248+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e2efc1850 con 0x563e2f0690a0 2024-09-19T04:30:55.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.248+0000 7f4397fff640 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563e2f0690a0 0x563e2f06b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.249+0000 7f4385ffb640 10 client.26980 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:55.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.249+0000 7f4397fff640 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563e2f0690a0 0x563e2f06b540 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3777eef86820502 server_cookie=eec0faa2440d1fdd in_seq=0 out_seq=0 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f4385ffb640 1 -- 172.21.15.94:0/1133398063 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e2efc1850 con 0x563e2f0690a0 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f4385ffb640 10 client.26980 handle_command_reply: tid=0 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 client.26980 shutdown 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 msgr2=0x563e2f058f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f058f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f43900985b0 tx=0x7f439009bba0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 shutdown_connections 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563e2f0690a0 0x563e2f06b540 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f4385ffb640 0 client.26980 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f059680 0x563e2f056760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f04edc0 0x563e2f056220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/1133398063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f058f70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 >> 172.21.15.94:0/1133398063 conn(0x563e2f04aa30 msgr2=0x563e2f050710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:55.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 shutdown_connections 2024-09-19T04:30:55.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.250+0000 7f43a6d4a740 1 -- 172.21.15.94:0/1133398063 wait complete. 2024-09-19T04:30:55.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.251+0000 7f43a6d4a740 20 client.26980 trim_cache size 0 max 16384 2024-09-19T04:30:55.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f43a6d4a740 1 Processor -- start 2024-09-19T04:30:55.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f43a6d4a740 1 -- start start 2024-09-19T04:30:55.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04e4a0 0x563e2f04ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04edc0 0x563e2f070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43800110a0 con 0x563e2f059680 2024-09-19T04:30:55.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f438000e970 con 0x563e2f04e4a0 2024-09-19T04:30:55.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f438000c7e0 con 0x563e2f04edc0 2024-09-19T04:30:55.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f439ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f439ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33088/0 (socket says 172.21.15.94:33088) 2024-09-19T04:30:55.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f439ed9d640 1 -- 172.21.15.94:0/2746855600 learned_addr learned my addr 172.21.15.94:0/2746855600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:55.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f4397fff640 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04edc0 0x563e2f070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.276+0000 7f439e59c640 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04e4a0 0x563e2f04ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f439ed9d640 1 -- 172.21.15.94:0/2746855600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04edc0 msgr2=0x563e2f070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:55.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f439ed9d640 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04edc0 0x563e2f070e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f439ed9d640 1 -- 172.21.15.94:0/2746855600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04e4a0 msgr2=0x563e2f04ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:55.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f439ed9d640 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04e4a0 0x563e2f04ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f439ed9d640 1 -- 172.21.15.94:0/2746855600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2f0551d0 con 0x563e2f059680 2024-09-19T04:30:55.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f439e59c640 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04e4a0 0x563e2f04ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:55.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f4397fff640 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04edc0 0x563e2f070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:55.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f439ed9d640 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f071350 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4398004af0 tx=0x7f439804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9627e4d751a4d25c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:55.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f4387fff640 1 -- 172.21.15.94:0/2746855600 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398036040 con 0x563e2f059680 2024-09-19T04:30:55.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f4387fff640 1 -- 172.21.15.94:0/2746855600 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4398049a60 con 0x563e2f059680 2024-09-19T04:30:55.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.277+0000 7f4387fff640 1 -- 172.21.15.94:0/2746855600 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43980044e0 con 0x563e2f059680 2024-09-19T04:30:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.278+0000 7f43a6d4a740 1 -- 172.21.15.94:0/2746855600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 msgr2=0x563e2f071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.278+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f071350 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4398004af0 tx=0x7f439804f000 comp rx=0 tx=0).stop 2024-09-19T04:30:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.278+0000 7f43a6d4a740 1 -- 172.21.15.94:0/2746855600 shutdown_connections 2024-09-19T04:30:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.278+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f071350 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.278+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04edc0 0x563e2f070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.278+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/2746855600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04e4a0 0x563e2f04ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.278+0000 7f43a6d4a740 1 -- 172.21.15.94:0/2746855600 >> 172.21.15.94:0/2746855600 conn(0x563e2f04aa30 msgr2=0x563e2f049b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.278+0000 7f43a6d4a740 1 -- 172.21.15.94:0/2746855600 shutdown_connections 2024-09-19T04:30:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.279+0000 7f43a6d4a740 1 -- 172.21.15.94:0/2746855600 wait complete. 2024-09-19T04:30:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.279+0000 7f43a6d4a740 1 -- start start 2024-09-19T04:30:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.279+0000 7f43a6d4a740 1 Processor -- start 2024-09-19T04:30:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.279+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f05f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.279+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f05f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.279+0000 7f43a6d4a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.279+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f438000ce30 con 0x563e2f059680 2024-09-19T04:30:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.279+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f438000e2a0 con 0x563e2f04edc0 2024-09-19T04:30:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.279+0000 7f43a6d4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f438000d500 con 0x563e2f04e4a0 2024-09-19T04:30:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f43a6d4a740 10 client.0 _ll_register_callbacks cb 0x7f43900046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f05f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f05f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:51160/0 (socket says 172.21.15.94:51160) 2024-09-19T04:30:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439e59c640 1 -- 172.21.15.94:0/274017581 learned_addr learned my addr 172.21.15.94:0/274017581 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:30:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439ed9d640 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f4397fff640 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f05f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439ed9d640 1 -- 172.21.15.94:0/274017581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 msgr2=0x563e2f05f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439ed9d640 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f05f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439ed9d640 1 -- 172.21.15.94:0/274017581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 msgr2=0x563e2f05f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439ed9d640 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f05f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439ed9d640 1 -- 172.21.15.94:0/274017581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2f0592e0 con 0x563e2f059680 2024-09-19T04:30:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f439ed9d640 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05fec0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4398009370 tx=0x7f4398003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f38c0c8d1105fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.280+0000 7f4385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:30:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.281+0000 7f4385ffb640 1 -- 172.21.15.94:0/274017581 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398064060 con 0x563e2f059680 2024-09-19T04:30:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.281+0000 7f43a6d4a740 1 -- 172.21.15.94:0/274017581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4398049a60 con 0x563e2f059680 2024-09-19T04:30:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.281+0000 7f4385ffb640 1 -- 172.21.15.94:0/274017581 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8585+0+0 (secure 0 0 0) 0x7f4398049640 con 0x563e2f059680 2024-09-19T04:30:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.281+0000 7f4385ffb640 1 -- 172.21.15.94:0/274017581 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4398036070 con 0x563e2f059680 2024-09-19T04:30:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.281+0000 7f43a6d4a740 10 client.17406 fetch_fsmap learned FSMap version 54 2024-09-19T04:30:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.281+0000 7f43a6d4a740 1 -- 172.21.15.94:0/274017581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x563e2f0d3eb0 con 0x563e2f059680 2024-09-19T04:30:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.282+0000 7f4385ffb640 1 -- 172.21.15.94:0/274017581 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4398049a60 con 0x563e2f059680 2024-09-19T04:30:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.282+0000 7f4385ffb640 1 -- 172.21.15.94:0/274017581 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f439803c030 con 0x563e2f059680 2024-09-19T04:30:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.282+0000 7f43a6d4a740 10 client.17406 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:30:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.282+0000 7f43a6d4a740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.282+0000 7f43a6d4a740 20 client.17406 populate_metadata read hostname 'smithi094' 2024-09-19T04:30:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.282+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563e2f0d4080 0x563e2f0d6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.282+0000 7f43a6d4a740 4 client.17406 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:30:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.282+0000 7f43a6d4a740 1 -- 172.21.15.94:0/274017581 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e2efbbd60 con 0x563e2f0d4080 2024-09-19T04:30:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.282+0000 7f439e59c640 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563e2f0d4080 0x563e2f0d6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.283+0000 7f4385ffb640 10 client.17406 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.283+0000 7f439e59c640 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563e2f0d4080 0x563e2f0d6470 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74074a7dc53bc1af server_cookie=bc20d843302bdc2d in_seq=0 out_seq=0 2024-09-19T04:30:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.283+0000 7f4385ffb640 1 -- 172.21.15.94:0/274017581 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e2efbbd60 con 0x563e2f0d4080 2024-09-19T04:30:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.283+0000 7f4385ffb640 10 client.17406 handle_command_reply: tid=0 2024-09-19T04:30:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.283+0000 7f43a6d4a740 1 client.17406 shutdown 2024-09-19T04:30:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.283+0000 7f43a6d4a740 1 -- 172.21.15.94:0/274017581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 msgr2=0x563e2f05fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.283+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05fec0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4398009370 tx=0x7f4398003a00 comp rx=0 tx=0).stop 2024-09-19T04:30:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.283+0000 7f43a6d4a740 1 -- 172.21.15.94:0/274017581 shutdown_connections 2024-09-19T04:30:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.283+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x563e2f0d4080 0x563e2f0d6470 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.284+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e2f059680 0x563e2f05fec0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.284+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e2f04edc0 0x563e2f05f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.284+0000 7f43a6d4a740 1 --2- 172.21.15.94:0/274017581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f04e4a0 0x563e2f05f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.284+0000 7f43a6d4a740 1 -- 172.21.15.94:0/274017581 >> 172.21.15.94:0/274017581 conn(0x563e2f04aa30 msgr2=0x563e2f057a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.284+0000 7f4385ffb640 0 client.17406 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:30:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.284+0000 7f43a6d4a740 1 -- 172.21.15.94:0/274017581 shutdown_connections 2024-09-19T04:30:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.284+0000 7f43a6d4a740 1 -- 172.21.15.94:0/274017581 wait complete. 2024-09-19T04:30:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.284+0000 7f43a6d4a740 20 client.17406 trim_cache size 0 max 16384 2024-09-19T04:30:55.297 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:30:55.297 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:30:55.298 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:30:55.298 INFO:teuthology.orchestra.run.smithi094.stdout: "ebaf86cd-73e3-4873-beae-ee36b26c8b1e": { 2024-09-19T04:30:55.298 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:30:55.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "ebaf86cd-73e3-4873-beae-ee36b26c8b1e", 2024-09-19T04:30:55.298 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:30:55.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:30:55.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:30:55.298 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:30:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 -- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f438c0a3430 msgr2=0x7f438c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f438c0a3430 0x7f438c0a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x563e2f04f460 tx=0x7f4398048000 comp rx=0 tx=0).stop 2024-09-19T04:30:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 -- 172.21.15.94:0/757667477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a0100100 msgr2=0x7f43a019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a0100100 0x7f43a019a4e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f439000e9c0 tx=0x7f439000ee80 comp rx=0 tx=0).stop 2024-09-19T04:30:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 -- 172.21.15.94:0/757667477 shutdown_connections 2024-09-19T04:30:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f438c0a3430 0x7f438c0a58f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43a0102a50 0x7f43a019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a0100c80 0x7f43a019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 --2- 172.21.15.94:0/757667477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a0100100 0x7f43a019a4e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 -- 172.21.15.94:0/757667477 >> 172.21.15.94:0/757667477 conn(0x7f43a0077230 msgr2=0x7f43a00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 -- 172.21.15.94:0/757667477 shutdown_connections 2024-09-19T04:30:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:30:55.285+0000 7f43a4971640 1 -- 172.21.15.94:0/757667477 wait complete. 2024-09-19T04:30:55.302 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:54 smithi112 ceph-mon[35051]: pgmap v497: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.1 MiB/s wr, 777 op/s 2024-09-19T04:30:55.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:54 smithi104 ceph-mon[34145]: pgmap v497: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.1 MiB/s wr, 777 op/s 2024-09-19T04:30:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 880.0 s, 387.4 tps, lat 255.975 ms stddev 60.655 2024-09-19T04:30:56.829 INFO:tasks.cephfs.filesystem:scrub status for tag:ebaf86cd-73e3-4873-beae-ee36b26c8b1e - {'path': '/', 'tag': 'ebaf86cd-73e3-4873-beae-ee36b26c8b1e', 'options': 'recursive,force'} 2024-09-19T04:30:57.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:56 smithi094 ceph-mon[30972]: pgmap v498: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 5.9 MiB/s wr, 508 op/s 2024-09-19T04:30:57.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:56 smithi112 ceph-mon[35051]: pgmap v498: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 5.9 MiB/s wr, 508 op/s 2024-09-19T04:30:57.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:56 smithi104 ceph-mon[34145]: pgmap v498: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 5.9 MiB/s wr, 508 op/s 2024-09-19T04:30:59.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:30:59 smithi112 ceph-mon[35051]: pgmap v499: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 7.5 MiB/s wr, 646 op/s 2024-09-19T04:30:59.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:30:59 smithi094 ceph-mon[30972]: pgmap v499: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 7.5 MiB/s wr, 646 op/s 2024-09-19T04:30:59.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:30:59 smithi104 ceph-mon[34145]: pgmap v499: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 7.5 MiB/s wr, 646 op/s 2024-09-19T04:31:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 885.0 s, 419.0 tps, lat 240.031 ms stddev 45.448 2024-09-19T04:31:01.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:01 smithi112 ceph-mon[35051]: pgmap v500: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.8 MiB/s wr, 650 op/s 2024-09-19T04:31:01.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:01 smithi094 ceph-mon[30972]: pgmap v500: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.8 MiB/s wr, 650 op/s 2024-09-19T04:31:01.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:01 smithi104 ceph-mon[34145]: pgmap v500: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.8 MiB/s wr, 650 op/s 2024-09-19T04:31:02.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:02 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:31:02.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:02 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:31:02.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:02 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:31:02.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:02 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:31:02.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:02 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:31:02.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:02 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:31:02.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:02 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:31:02.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:02 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:31:02.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:02 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:31:03.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:03 smithi112 ceph-mon[35051]: pgmap v501: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 5.5 MiB/s wr, 455 op/s 2024-09-19T04:31:03.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:03 smithi094 ceph-mon[30972]: pgmap v501: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 5.5 MiB/s wr, 455 op/s 2024-09-19T04:31:03.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:03 smithi104 ceph-mon[34145]: pgmap v501: 97 pgs: 97 active+clean; 3.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 5.5 MiB/s wr, 455 op/s 2024-09-19T04:31:05.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:05 smithi112 ceph-mon[35051]: pgmap v502: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.6 MiB/s wr, 768 op/s 2024-09-19T04:31:05.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:05 smithi094 ceph-mon[30972]: pgmap v502: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.6 MiB/s wr, 768 op/s 2024-09-19T04:31:05.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:05 smithi104 ceph-mon[34145]: pgmap v502: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.6 MiB/s wr, 768 op/s 2024-09-19T04:31:05.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 890.0 s, 408.0 tps, lat 245.943 ms stddev 38.690 2024-09-19T04:31:07.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:07 smithi112 ceph-mon[35051]: pgmap v503: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 6.0 MiB/s wr, 567 op/s 2024-09-19T04:31:07.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:07 smithi094 ceph-mon[30972]: pgmap v503: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 6.0 MiB/s wr, 567 op/s 2024-09-19T04:31:07.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:07 smithi104 ceph-mon[34145]: pgmap v503: 97 pgs: 97 active+clean; 3.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 6.0 MiB/s wr, 567 op/s 2024-09-19T04:31:08.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:08 smithi112 ceph-mon[35051]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T04:31:08.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:08 smithi094 ceph-mon[30972]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T04:31:08.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:08 smithi104 ceph-mon[34145]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T04:31:09.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:09 smithi112 ceph-mon[35051]: pgmap v505: 97 pgs: 97 active+clean; 3.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 7.4 MiB/s wr, 660 op/s 2024-09-19T04:31:09.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:09 smithi094 ceph-mon[30972]: pgmap v505: 97 pgs: 97 active+clean; 3.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 7.4 MiB/s wr, 660 op/s 2024-09-19T04:31:09.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:09 smithi104 ceph-mon[34145]: pgmap v505: 97 pgs: 97 active+clean; 3.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 7.4 MiB/s wr, 660 op/s 2024-09-19T04:31:10.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 895.0 s, 410.6 tps, lat 241.814 ms stddev 40.026 2024-09-19T04:31:11.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:11 smithi094 ceph-mon[30972]: pgmap v506: 97 pgs: 97 active+clean; 3.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.8 MiB/s wr, 636 op/s 2024-09-19T04:31:11.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:11 smithi094 ceph-mon[30972]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T04:31:11.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:11 smithi104 ceph-mon[34145]: pgmap v506: 97 pgs: 97 active+clean; 3.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.8 MiB/s wr, 636 op/s 2024-09-19T04:31:11.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:11 smithi104 ceph-mon[34145]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T04:31:11.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:11 smithi112 ceph-mon[35051]: pgmap v506: 97 pgs: 97 active+clean; 3.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.8 MiB/s wr, 636 op/s 2024-09-19T04:31:11.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:11 smithi112 ceph-mon[35051]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T04:31:13.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:13 smithi094 ceph-mon[30972]: pgmap v508: 97 pgs: 97 active+clean; 3.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-09-19T04:31:13.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:13 smithi104 ceph-mon[34145]: pgmap v508: 97 pgs: 97 active+clean; 3.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-09-19T04:31:13.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:13 smithi112 ceph-mon[35051]: pgmap v508: 97 pgs: 97 active+clean; 3.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-09-19T04:31:15.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:15 smithi094 ceph-mon[30972]: pgmap v509: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.9 MiB/s wr, 702 op/s 2024-09-19T04:31:15.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:15 smithi104 ceph-mon[34145]: pgmap v509: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.9 MiB/s wr, 702 op/s 2024-09-19T04:31:15.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:15 smithi112 ceph-mon[35051]: pgmap v509: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.9 MiB/s wr, 702 op/s 2024-09-19T04:31:15.602 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 900.0 s, 414.2 tps, lat 241.903 ms stddev 24.680 2024-09-19T04:31:15.854 INFO:teuthology.orchestra.run.smithi094.stdout:transaction type: 2024-09-19T04:31:15.854 INFO:teuthology.orchestra.run.smithi094.stdout:scaling factor: 32 2024-09-19T04:31:15.854 INFO:teuthology.orchestra.run.smithi094.stdout:query mode: simple 2024-09-19T04:31:15.855 INFO:teuthology.orchestra.run.smithi094.stdout:number of clients: 100 2024-09-19T04:31:15.855 INFO:teuthology.orchestra.run.smithi094.stdout:number of threads: 4 2024-09-19T04:31:15.855 INFO:teuthology.orchestra.run.smithi094.stdout:duration: 900 s 2024-09-19T04:31:15.855 INFO:teuthology.orchestra.run.smithi094.stdout:number of transactions actually processed: 982461 2024-09-19T04:31:15.855 INFO:teuthology.orchestra.run.smithi094.stdout:latency average = 91.584 ms 2024-09-19T04:31:15.855 INFO:teuthology.orchestra.run.smithi094.stdout:latency stddev = 102.798 ms 2024-09-19T04:31:15.855 INFO:teuthology.orchestra.run.smithi094.stdout:tps = 1091.317367 (including connections establishing) 2024-09-19T04:31:15.855 INFO:teuthology.orchestra.run.smithi094.stdout:tps = 1091.329940 (excluding connections establishing) 2024-09-19T04:31:15.858 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-19T04:31:17.336 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:17 smithi112 ceph-mon[35051]: pgmap v510: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.3 MiB/s wr, 651 op/s 2024-09-19T04:31:17.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:17 smithi094 ceph-mon[30972]: pgmap v510: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.3 MiB/s wr, 651 op/s 2024-09-19T04:31:17.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:17 smithi104 ceph-mon[34145]: pgmap v510: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.3 MiB/s wr, 651 op/s 2024-09-19T04:31:19.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:19 smithi094 ceph-mon[30972]: pgmap v511: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 577 op/s 2024-09-19T04:31:19.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:19 smithi104 ceph-mon[34145]: pgmap v511: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 577 op/s 2024-09-19T04:31:19.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:19 smithi112 ceph-mon[35051]: pgmap v511: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 577 op/s 2024-09-19T04:31:21.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:21 smithi094 ceph-mon[30972]: pgmap v512: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 6.4 MiB/s wr, 604 op/s 2024-09-19T04:31:21.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:21 smithi104 ceph-mon[34145]: pgmap v512: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 6.4 MiB/s wr, 604 op/s 2024-09-19T04:31:21.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:21 smithi112 ceph-mon[35051]: pgmap v512: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 6.4 MiB/s wr, 604 op/s 2024-09-19T04:31:23.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:23 smithi094 ceph-mon[30972]: pgmap v513: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 5.8 MiB/s wr, 553 op/s 2024-09-19T04:31:23.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:23 smithi104 ceph-mon[34145]: pgmap v513: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 5.8 MiB/s wr, 553 op/s 2024-09-19T04:31:23.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:23 smithi112 ceph-mon[35051]: pgmap v513: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 5.8 MiB/s wr, 553 op/s 2024-09-19T04:31:25.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:25 smithi094 ceph-mon[30972]: pgmap v514: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 5.9 MiB/s wr, 568 op/s 2024-09-19T04:31:25.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:25 smithi104 ceph-mon[34145]: pgmap v514: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 5.9 MiB/s wr, 568 op/s 2024-09-19T04:31:25.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:25 smithi112 ceph-mon[35051]: pgmap v514: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 5.9 MiB/s wr, 568 op/s 2024-09-19T04:31:26.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:31:27.093 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:27.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:27 smithi094 ceph-mon[30972]: pgmap v515: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-09-19T04:31:27.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:27 smithi104 ceph-mon[34145]: pgmap v515: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-09-19T04:31:27.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:27 smithi112 ceph-mon[35051]: pgmap v515: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-09-19T04:31:27.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.617+0000 7fd9a41d6640 1 -- 172.21.15.94:0/3279972731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c103660 msgr2=0x7fd99c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:27.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.617+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/3279972731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c103660 0x7fd99c10bb50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd990009900 tx=0x7fd99002a480 comp rx=0 tx=0).stop 2024-09-19T04:31:27.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.617+0000 7fd9a41d6640 1 -- 172.21.15.94:0/3279972731 shutdown_connections 2024-09-19T04:31:27.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.617+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/3279972731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd99c10c170 0x7fd99c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.617+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/3279972731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c103660 0x7fd99c10bb50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.617+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/3279972731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd99c102d40 0x7fd99c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.617+0000 7fd9a41d6640 1 -- 172.21.15.94:0/3279972731 >> 172.21.15.94:0/3279972731 conn(0x7fd99c0fc7a0 msgr2=0x7fd99c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.617+0000 7fd9a41d6640 1 -- 172.21.15.94:0/3279972731 shutdown_connections 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.617+0000 7fd9a41d6640 1 -- 172.21.15.94:0/3279972731 wait complete. 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a41d6640 1 Processor -- start 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a41d6640 1 -- start start 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a41d6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd99c102d40 0x7fd99c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a41d6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd99c103660 0x7fd99c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a41d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c10c170 0x7fd99c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a41d6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111e20 con 0x7fd99c10c170 2024-09-19T04:31:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a41d6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111ca0 con 0x7fd99c103660 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a41d6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111fa0 con 0x7fd99c102d40 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a2773640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c10c170 0x7fd99c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a2773640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c10c170 0x7fd99c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41380/0 (socket says 172.21.15.94:41380) 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a2773640 1 -- 172.21.15.94:0/377434770 learned_addr learned my addr 172.21.15.94:0/377434770 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd99c102d40 0x7fd99c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a1771640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd99c103660 0x7fd99c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a2773640 1 -- 172.21.15.94:0/377434770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd99c102d40 msgr2=0x7fd99c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a2773640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd99c102d40 0x7fd99c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a2773640 1 -- 172.21.15.94:0/377434770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd99c103660 msgr2=0x7fd99c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a2773640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd99c103660 0x7fd99c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.618+0000 7fd9a2773640 1 -- 172.21.15.94:0/377434770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c1a6800 con 0x7fd99c10c170 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.619+0000 7fd9a2773640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c10c170 0x7fd99c101420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00cad0 tx=0x7fd98c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.619+0000 7fd99affd640 1 -- 172.21.15.94:0/377434770 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00ae20 con 0x7fd99c10c170 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.619+0000 7fd99affd640 1 -- 172.21.15.94:0/377434770 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd98c005020 con 0x7fd99c10c170 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.619+0000 7fd9a41d6640 1 -- 172.21.15.94:0/377434770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c1a6af0 con 0x7fd99c10c170 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.619+0000 7fd99affd640 1 -- 172.21.15.94:0/377434770 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00f6b0 con 0x7fd99c10c170 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.619+0000 7fd9a41d6640 1 -- 172.21.15.94:0/377434770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1a6fb0 con 0x7fd99c10c170 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.621+0000 7fd99affd640 1 -- 172.21.15.94:0/377434770 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd98c0051d0 con 0x7fd99c10c170 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.621+0000 7fd9a65af740 1 Processor -- start 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a65af740 1 -- start start 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd99affd640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd9840a33b0 0x7fd9840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efafc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb07140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 0x5640efb09a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a65af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5640efa622e0 con 0x5640efafcdc0 2024-09-19T04:31:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a65af740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640efa09ce0 con 0x5640efafc4a0 2024-09-19T04:31:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a65af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5640efa5e2a0 con 0x5640efb07680 2024-09-19T04:31:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd99affd640 1 -- 172.21.15.94:0/377434770 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fd98c0cfe90 con 0x7fd99c10c170 2024-09-19T04:31:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1771640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd9840a33b0 0x7fd9840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1f72640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efafc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1f72640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efafc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:59260/0 (socket says 172.21.15.94:59260) 2024-09-19T04:31:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1771640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb07140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1f72640 1 -- 172.21.15.94:0/4230080505 learned_addr learned my addr 172.21.15.94:0/4230080505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a2773640 1 --2- 172.21.15.94:0/4230080505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 0x5640efb09a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1f72640 1 -- 172.21.15.94:0/4230080505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 msgr2=0x5640efb09a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/4230080505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 0x5640efb09a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1f72640 1 -- 172.21.15.94:0/4230080505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 msgr2=0x5640efb07140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/4230080505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb07140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.622+0000 7fd9a1f72640 1 -- 172.21.15.94:0/4230080505 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640ef9ee440 con 0x5640efafc4a0 2024-09-19T04:31:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.623+0000 7fd9a1771640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd9840a33b0 0x7fd9840a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd990031440 tx=0x7fd990032c90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.623+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/4230080505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efafc880 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd99400cc60 tx=0x7fd99400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4988ef0253e46e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.623+0000 7fd983fff640 1 -- 172.21.15.94:0/4230080505 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994014070 con 0x5640efafc4a0 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.623+0000 7fd983fff640 1 -- 172.21.15.94:0/4230080505 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9940047a0 con 0x5640efafc4a0 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.623+0000 7fd983fff640 1 -- 172.21.15.94:0/4230080505 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994010650 con 0x5640efafc4a0 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.623+0000 7fd9a65af740 1 -- 172.21.15.94:0/4230080505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 msgr2=0x5640efafc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.623+0000 7fd9a65af740 1 --2- 172.21.15.94:0/4230080505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efafc880 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd99400cc60 tx=0x7fd99400e720 comp rx=0 tx=0).stop 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 -- 172.21.15.94:0/4230080505 shutdown_connections 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 --2- 172.21.15.94:0/4230080505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 0x5640efb09a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 --2- 172.21.15.94:0/4230080505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb07140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 --2- 172.21.15.94:0/4230080505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efafc880 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 -- 172.21.15.94:0/4230080505 >> 172.21.15.94:0/4230080505 conn(0x5640efaf8a30 msgr2=0x5640efaf8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 -- 172.21.15.94:0/4230080505 shutdown_connections 2024-09-19T04:31:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 -- 172.21.15.94:0/4230080505 wait complete. 2024-09-19T04:31:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 -- start start 2024-09-19T04:31:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 Processor -- start 2024-09-19T04:31:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efb06f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 0x5640efb04720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5640efb0a250 con 0x5640efafcdc0 2024-09-19T04:31:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640efa1d050 con 0x5640efafc4a0 2024-09-19T04:31:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5640efb0a3d0 con 0x5640efb07680 2024-09-19T04:31:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.624+0000 7fd9a65af740 10 client.0 _ll_register_callbacks cb 0x5640efa5d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1771640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1771640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb041e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41404/0 (socket says 172.21.15.94:41404) 2024-09-19T04:31:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1771640 1 -- 172.21.15.94:0/3917418199 learned_addr learned my addr 172.21.15.94:0/3917418199 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a2773640 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 0x5640efb04720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efb06f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1771640 1 -- 172.21.15.94:0/3917418199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 msgr2=0x5640efb04720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1771640 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 0x5640efb04720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1771640 1 -- 172.21.15.94:0/3917418199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 msgr2=0x5640efb06f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1771640 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efb06f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1771640 1 -- 172.21.15.94:0/3917418199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640ef7d33c0 con 0x5640efafcdc0 2024-09-19T04:31:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a1771640 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb041e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd990033000 tx=0x7fd990045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb656899afda29fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9817fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9817fa640 1 -- 172.21.15.94:0/3917418199 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990050020 con 0x5640efafcdc0 2024-09-19T04:31:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9817fa640 1 -- 172.21.15.94:0/3917418199 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fd990003c70 con 0x5640efafcdc0 2024-09-19T04:31:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a65af740 1 -- 172.21.15.94:0/3917418199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9940047a0 con 0x5640efafcdc0 2024-09-19T04:31:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9817fa640 1 -- 172.21.15.94:0/3917418199 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd99005b050 con 0x5640efafcdc0 2024-09-19T04:31:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a65af740 10 client.17418 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.625+0000 7fd9a65af740 1 -- 172.21.15.94:0/3917418199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5640efb15ba0 con 0x5640efafcdc0 2024-09-19T04:31:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.626+0000 7fd9817fa640 1 -- 172.21.15.94:0/3917418199 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9940047a0 con 0x5640efafcdc0 2024-09-19T04:31:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.626+0000 7fd9817fa640 1 -- 172.21.15.94:0/3917418199 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fd99003a040 con 0x5640efafcdc0 2024-09-19T04:31:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.626+0000 7fd9a65af740 10 client.17418 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.626+0000 7fd9a65af740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.626+0000 7fd9a65af740 20 client.17418 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.626+0000 7fd9a65af740 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5640efb170b0 0x5640efb19550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.626+0000 7fd9a65af740 4 client.17418 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.626+0000 7fd9a65af740 1 -- 172.21.15.94:0/3917418199 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640efa6f850 con 0x5640efb170b0 2024-09-19T04:31:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.626+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5640efb170b0 0x5640efb19550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.627+0000 7fd9817fa640 10 client.17418 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.627+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5640efb170b0 0x5640efb19550 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b6077cea76e1f9 server_cookie=c26a75cc2d10c8e1 in_seq=0 out_seq=0 2024-09-19T04:31:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.627+0000 7fd9817fa640 1 -- 172.21.15.94:0/3917418199 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640efa6f850 con 0x5640efb170b0 2024-09-19T04:31:27.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.627+0000 7fd9817fa640 10 client.17418 handle_command_reply: tid=0 2024-09-19T04:31:27.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.627+0000 7fd9a65af740 1 client.17418 shutdown 2024-09-19T04:31:27.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.627+0000 7fd9a65af740 1 -- 172.21.15.94:0/3917418199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 msgr2=0x5640efb041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:27.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.627+0000 7fd9a65af740 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb041e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd990033000 tx=0x7fd990045000 comp rx=0 tx=0).stop 2024-09-19T04:31:27.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9a65af740 1 -- 172.21.15.94:0/3917418199 shutdown_connections 2024-09-19T04:31:27.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9a65af740 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5640efb170b0 0x5640efb19550 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9a65af740 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efb07680 0x5640efb04720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9a65af740 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb041e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9817fa640 0 client.17418 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9a65af740 1 --2- 172.21.15.94:0/3917418199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efb06f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9a65af740 1 -- 172.21.15.94:0/3917418199 >> 172.21.15.94:0/3917418199 conn(0x5640efaf8a30 msgr2=0x5640efafe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9a65af740 1 -- 172.21.15.94:0/3917418199 shutdown_connections 2024-09-19T04:31:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9a65af740 1 -- 172.21.15.94:0/3917418199 wait complete. 2024-09-19T04:31:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.628+0000 7fd9a65af740 20 client.17418 trim_cache size 0 max 16384 2024-09-19T04:31:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.644+0000 7fd9a65af740 1 Processor -- start 2024-09-19T04:31:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.644+0000 7fd9a65af740 1 -- start start 2024-09-19T04:31:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efaf8a30 0x5640efb06150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafc4a0 0x5640efb1ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efafcdc0 0x5640efb1f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a65af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1b2340 con 0x5640efafc4a0 2024-09-19T04:31:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a65af740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1afc10 con 0x5640efaf8a30 2024-09-19T04:31:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a65af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1ada80 con 0x5640efafcdc0 2024-09-19T04:31:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1771640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafc4a0 0x5640efb1ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1771640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafc4a0 0x5640efb1ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41416/0 (socket says 172.21.15.94:41416) 2024-09-19T04:31:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1771640 1 -- 172.21.15.94:0/1824517744 learned_addr learned my addr 172.21.15.94:0/1824517744 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/1824517744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efaf8a30 0x5640efb06150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1771640 1 -- 172.21.15.94:0/1824517744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efafcdc0 msgr2=0x5640efb1f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1771640 1 --2- 172.21.15.94:0/1824517744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efafcdc0 0x5640efb1f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1771640 1 -- 172.21.15.94:0/1824517744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efaf8a30 msgr2=0x5640efb06150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1771640 1 --2- 172.21.15.94:0/1824517744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efaf8a30 0x5640efb06150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1771640 1 -- 172.21.15.94:0/1824517744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640efb072a0 con 0x5640efafc4a0 2024-09-19T04:31:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/1824517744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efaf8a30 0x5640efb06150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:31:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9a1771640 1 --2- 172.21.15.94:0/1824517744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafc4a0 0x5640efb1ee20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd990006010 tx=0x7fd9900319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7890c81af0416cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9837fe640 1 -- 172.21.15.94:0/1824517744 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990050070 con 0x5640efafc4a0 2024-09-19T04:31:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9837fe640 1 -- 172.21.15.94:0/1824517744 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9900039c0 con 0x5640efafc4a0 2024-09-19T04:31:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.645+0000 7fd9837fe640 1 -- 172.21.15.94:0/1824517744 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990035040 con 0x5640efafc4a0 2024-09-19T04:31:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 -- 172.21.15.94:0/1824517744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafc4a0 msgr2=0x5640efb1ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 --2- 172.21.15.94:0/1824517744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafc4a0 0x5640efb1ee20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd990006010 tx=0x7fd9900319f0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 -- 172.21.15.94:0/1824517744 shutdown_connections 2024-09-19T04:31:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 --2- 172.21.15.94:0/1824517744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efafcdc0 0x5640efb1f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 --2- 172.21.15.94:0/1824517744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafc4a0 0x5640efb1ee20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 --2- 172.21.15.94:0/1824517744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efaf8a30 0x5640efb06150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 -- 172.21.15.94:0/1824517744 >> 172.21.15.94:0/1824517744 conn(0x5640efb04880 msgr2=0x5640efb05a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 -- 172.21.15.94:0/1824517744 shutdown_connections 2024-09-19T04:31:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 -- 172.21.15.94:0/1824517744 wait complete. 2024-09-19T04:31:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 -- start start 2024-09-19T04:31:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.646+0000 7fd9a65af740 1 Processor -- start 2024-09-19T04:31:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efaf8a30 0x5640efb0d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efb0da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a65af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb0df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a65af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1ae0d0 con 0x5640efafcdc0 2024-09-19T04:31:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a65af740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1af540 con 0x5640efafc4a0 2024-09-19T04:31:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a65af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1ae7a0 con 0x5640efaf8a30 2024-09-19T04:31:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a65af740 10 client.0 _ll_register_callbacks cb 0x7fd990009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a2773640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb0df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a2773640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb0df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41432/0 (socket says 172.21.15.94:41432) 2024-09-19T04:31:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a2773640 1 -- 172.21.15.94:0/31844842 learned_addr learned my addr 172.21.15.94:0/31844842 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a1771640 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efb0da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efaf8a30 0x5640efb0d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a2773640 1 -- 172.21.15.94:0/31844842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efaf8a30 msgr2=0x5640efb0d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a2773640 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efaf8a30 0x5640efb0d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a2773640 1 -- 172.21.15.94:0/31844842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 msgr2=0x5640efb0da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a2773640 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efb0da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a2773640 1 -- 172.21.15.94:0/31844842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640efb80f00 con 0x5640efafcdc0 2024-09-19T04:31:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efaf8a30 0x5640efb0d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9a2773640 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb0df50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0cd5a0 tx=0x7fd98c004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17411d030ad5956 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9817fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.647+0000 7fd9817fa640 1 -- 172.21.15.94:0/31844842 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0a4d40 con 0x5640efafcdc0 2024-09-19T04:31:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9817fa640 1 -- 172.21.15.94:0/31844842 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fd98c0a0150 con 0x5640efafcdc0 2024-09-19T04:31:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a65af740 1 -- 172.21.15.94:0/31844842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9900039c0 con 0x5640efafcdc0 2024-09-19T04:31:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9817fa640 1 -- 172.21.15.94:0/31844842 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd98c098740 con 0x5640efafcdc0 2024-09-19T04:31:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a65af740 10 client.17424 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a65af740 1 -- 172.21.15.94:0/31844842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5640efb821c0 con 0x5640efafcdc0 2024-09-19T04:31:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9817fa640 1 -- 172.21.15.94:0/31844842 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9900039c0 con 0x5640efafcdc0 2024-09-19T04:31:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9817fa640 1 -- 172.21.15.94:0/31844842 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fd98c004460 con 0x5640efafcdc0 2024-09-19T04:31:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a65af740 10 client.17424 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a65af740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a65af740 20 client.17424 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a65af740 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5640efb07680 0x5640efb84920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a65af740 4 client.17424 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:31:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a65af740 1 -- 172.21.15.94:0/31844842 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640efa69d60 con 0x5640efb07680 2024-09-19T04:31:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.648+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5640efb07680 0x5640efb84920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9817fa640 10 client.17424 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a1f72640 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5640efb07680 0x5640efb84920 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d77ffbacc8ef9b6 server_cookie=3eb41d424b0a4057 in_seq=0 out_seq=0 2024-09-19T04:31:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9817fa640 1 -- 172.21.15.94:0/31844842 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5640efa69d60 con 0x5640efb07680 2024-09-19T04:31:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9817fa640 10 client.17424 handle_command_reply: tid=0 2024-09-19T04:31:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 client.17424 shutdown 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 -- 172.21.15.94:0/31844842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 msgr2=0x5640efb0df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb0df50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0cd5a0 tx=0x7fd98c004b80 comp rx=0 tx=0).stop 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 -- 172.21.15.94:0/31844842 shutdown_connections 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5640efb07680 0x5640efb84920 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640efafcdc0 0x5640efb0df50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640efafc4a0 0x5640efb0da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 --2- 172.21.15.94:0/31844842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640efaf8a30 0x5640efb0d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 -- 172.21.15.94:0/31844842 >> 172.21.15.94:0/31844842 conn(0x5640efb04880 msgr2=0x5640efb06260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9817fa640 0 client.17424 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:27.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 -- 172.21.15.94:0/31844842 shutdown_connections 2024-09-19T04:31:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 1 -- 172.21.15.94:0/31844842 wait complete. 2024-09-19T04:31:27.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.649+0000 7fd9a65af740 20 client.17424 trim_cache size 0 max 16384 2024-09-19T04:31:27.676 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:31:27.676 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:31:27.676 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:31:27.676 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:31:27.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 -- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd9840a33b0 msgr2=0x7fd9840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:27.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd9840a33b0 0x7fd9840a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd990031440 tx=0x7fd990032c90 comp rx=0 tx=0).stop 2024-09-19T04:31:27.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 -- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c10c170 msgr2=0x7fd99c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:27.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c10c170 0x7fd99c101420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00cad0 tx=0x7fd98c00cf90 comp rx=0 tx=0).stop 2024-09-19T04:31:27.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 -- 172.21.15.94:0/377434770 shutdown_connections 2024-09-19T04:31:27.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fd9840a33b0 0x7fd9840a5870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd99c10c170 0x7fd99c101420 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd99c103660 0x7fd99c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 --2- 172.21.15.94:0/377434770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd99c102d40 0x7fd99c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 -- 172.21.15.94:0/377434770 >> 172.21.15.94:0/377434770 conn(0x7fd99c0fc7a0 msgr2=0x7fd99c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 -- 172.21.15.94:0/377434770 shutdown_connections 2024-09-19T04:31:27.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:27.650+0000 7fd9a41d6640 1 -- 172.21.15.94:0/377434770 wait complete. 2024-09-19T04:31:28.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:31:28.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:31:28.649 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:29.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 -- 172.21.15.94:0/363838562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288106a60 msgr2=0x7fb288106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 --2- 172.21.15.94:0/363838562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288106a60 0x7fb288106e40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb278009930 tx=0x7fb27802a420 comp rx=0 tx=0).stop 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 -- 172.21.15.94:0/363838562 shutdown_connections 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 --2- 172.21.15.94:0/363838562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288101400 0x7fb28810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 --2- 172.21.15.94:0/363838562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb288100a60 0x7fb288100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 --2- 172.21.15.94:0/363838562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288106a60 0x7fb288106e40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 -- 172.21.15.94:0/363838562 >> 172.21.15.94:0/363838562 conn(0x7fb2880fc780 msgr2=0x7fb2880feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 -- 172.21.15.94:0/363838562 shutdown_connections 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 -- 172.21.15.94:0/363838562 wait complete. 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.113+0000 7fb28df00640 1 Processor -- start 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb28df00640 1 -- start start 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb28df00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288100a60 0x7fb2881991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:29.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb28df00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb288101400 0x7fb288199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb28df00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288106a60 0x7fb2881a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb28df00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb288110d30 con 0x7fb288100a60 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb28df00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb288110bb0 con 0x7fb288101400 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb28df00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb288110eb0 con 0x7fb288106a60 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288100a60 0x7fb2881991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288100a60 0x7fb2881991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41448/0 (socket says 172.21.15.94:41448) 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2877fe640 1 -- 172.21.15.94:0/2242227163 learned_addr learned my addr 172.21.15.94:0/2242227163 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb27ffff640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb288101400 0x7fb288199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2877fe640 1 -- 172.21.15.94:0/2242227163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288106a60 msgr2=0x7fb2881a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb287fff640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288106a60 0x7fb2881a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2877fe640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288106a60 0x7fb2881a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2877fe640 1 -- 172.21.15.94:0/2242227163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb288101400 msgr2=0x7fb288199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2877fe640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb288101400 0x7fb288199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2877fe640 1 -- 172.21.15.94:0/2242227163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2881a47a0 con 0x7fb288100a60 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb27ffff640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb288101400 0x7fb288199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2877fe640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288100a60 0x7fb2881991e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb278005940 tx=0x7fb2780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.114+0000 7fb2857fa640 1 -- 172.21.15.94:0/2242227163 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27803f030 con 0x7fb288100a60 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.115+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2881a4970 con 0x7fb288100a60 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.115+0000 7fb2857fa640 1 -- 172.21.15.94:0/2242227163 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb278003cb0 con 0x7fb288100a60 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.115+0000 7fb2857fa640 1 -- 172.21.15.94:0/2242227163 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278035070 con 0x7fb288100a60 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.115+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2881a4e50 con 0x7fb288100a60 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.116+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb288100ec0 con 0x7fb288100a60 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.117+0000 7fb2857fa640 1 -- 172.21.15.94:0/2242227163 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fb278042030 con 0x7fb288100a60 2024-09-19T04:31:29.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.117+0000 7fb2857fa640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb2740a33b0 0x7fb2740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:29.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.117+0000 7fb27ffff640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb2740a33b0 0x7fb2740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:29.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.118+0000 7fb2857fa640 1 -- 172.21.15.94:0/2242227163 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fb2780f03a0 con 0x7fb288100a60 2024-09-19T04:31:29.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.118+0000 7fb27ffff640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb2740a33b0 0x7fb2740a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb28806b320 tx=0x7fb270005d90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:29.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.122+0000 7fb2857fa640 1 -- 172.21.15.94:0/2242227163 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb2780bcd40 con 0x7fb288100a60 2024-09-19T04:31:29.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.322+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb28802d7c0 con 0x7fb288100a60 2024-09-19T04:31:29.324 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:29 smithi094 ceph-mon[30972]: pgmap v516: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-19T04:31:29.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.322+0000 7fb2857fa640 1 -- 172.21.15.94:0/2242227163 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11915 (secure 0 0 0) 0x7fb2780b8070 con 0x7fb288100a60 2024-09-19T04:31:29.324 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:31:29.325 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":54,"btime":"2024-09-19T04:29:32:831739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:29:32.764738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:31:29.325 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 54 2024-09-19T04:31:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb2740a33b0 msgr2=0x7fb2740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb2740a33b0 0x7fb2740a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb28806b320 tx=0x7fb270005d90 comp rx=0 tx=0).stop 2024-09-19T04:31:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288100a60 msgr2=0x7fb2881991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288100a60 0x7fb2881991e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb278005940 tx=0x7fb2780042f0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 shutdown_connections 2024-09-19T04:31:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fb2740a33b0 0x7fb2740a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288106a60 0x7fb2881a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb288101400 0x7fb288199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 --2- 172.21.15.94:0/2242227163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb288100a60 0x7fb2881991e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 >> 172.21.15.94:0/2242227163 conn(0x7fb2880fc780 msgr2=0x7fb2880fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 shutdown_connections 2024-09-19T04:31:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:29.325+0000 7fb28df00640 1 -- 172.21.15.94:0/2242227163 wait complete. 2024-09-19T04:31:29.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:29 smithi104 ceph-mon[34145]: pgmap v516: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-19T04:31:29.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:29 smithi112 ceph-mon[35051]: pgmap v516: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-19T04:31:29.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:31:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:30.331 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:30 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2242227163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:31:30.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:30 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2242227163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:31:30.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:30 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2242227163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:31:30.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 -- 172.21.15.94:0/2020816459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4105ba0 msgr2=0x7f76c4105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:30.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/2020816459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4105ba0 0x7f76c4105f80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009900 tx=0x7f76c002a480 comp rx=0 tx=0).stop 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 -- 172.21.15.94:0/2020816459 shutdown_connections 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/2020816459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76c4102110 0x7f76c410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/2020816459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c4101790 0x7f76c4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/2020816459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4105ba0 0x7f76c4105f80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 -- 172.21.15.94:0/2020816459 >> 172.21.15.94:0/2020816459 conn(0x7f76c4077210 msgr2=0x7f76c4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 -- 172.21.15.94:0/2020816459 shutdown_connections 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 -- 172.21.15.94:0/2020816459 wait complete. 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 Processor -- start 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.676+0000 7f76cbdf4640 1 -- start start 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76cbdf4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76c4101790 0x7f76c41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76cbdf4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4102110 0x7f76c41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76cbdf4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c4105ba0 0x7f76c419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76cbdf4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4110d50 con 0x7f76c4102110 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76cbdf4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4110bd0 con 0x7f76c4105ba0 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76cbdf4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4110ed0 con 0x7f76c4101790 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c938f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4102110 0x7f76c41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c938f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4102110 0x7f76c41987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41458/0 (socket says 172.21.15.94:41458) 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c938f640 1 -- 172.21.15.94:0/618609756 learned_addr learned my addr 172.21.15.94:0/618609756 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c9b90640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76c4101790 0x7f76c41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76ca391640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c4105ba0 0x7f76c419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c938f640 1 -- 172.21.15.94:0/618609756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76c4101790 msgr2=0x7f76c41982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c938f640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76c4101790 0x7f76c41982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c938f640 1 -- 172.21.15.94:0/618609756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c4105ba0 msgr2=0x7f76c419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c938f640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c4105ba0 0x7f76c419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c938f640 1 -- 172.21.15.94:0/618609756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c419d250 con 0x7f76c4102110 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76ca391640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c4105ba0 0x7f76c419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76c938f640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4102110 0x7f76c41987e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f76b400c920 tx=0x7f76b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76baffd640 1 -- 172.21.15.94:0/618609756 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4007c70 con 0x7f76c4102110 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76cbdf4640 1 -- 172.21.15.94:0/618609756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c419d540 con 0x7f76c4102110 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.677+0000 7f76baffd640 1 -- 172.21.15.94:0/618609756 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76b4007e10 con 0x7f76c4102110 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.678+0000 7f76baffd640 1 -- 172.21.15.94:0/618609756 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4018820 con 0x7f76c4102110 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.678+0000 7f76cbdf4640 1 -- 172.21.15.94:0/618609756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c41a4e20 con 0x7f76c4102110 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76baffd640 1 -- 172.21.15.94:0/618609756 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f76b40189c0 con 0x7f76c4102110 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ce1cd740 1 Processor -- start 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ce1cd740 1 -- start start 2024-09-19T04:31:30.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da53090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da53a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 0x55cf2da64b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cf2d9b92f0 con 0x55cf2da54b40 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cf2d960cf0 con 0x55cf2da535d0 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cf2d9b52b0 con 0x55cf2da62780 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ca391640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da53090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ca391640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da53090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41466/0 (socket says 172.21.15.94:41466) 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.680+0000 7f76ca391640 1 -- 172.21.15.94:0/3467074614 learned_addr learned my addr 172.21.15.94:0/3467074614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76c938f640 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 0x55cf2da64b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76baffd640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76b00a33b0 0x7f76b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76c9b90640 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da53a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76ca391640 1 -- 172.21.15.94:0/3467074614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 msgr2=0x55cf2da64b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76ca391640 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 0x55cf2da64b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76ca391640 1 -- 172.21.15.94:0/3467074614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 msgr2=0x55cf2da53a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76ca391640 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da53a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76ca391640 1 -- 172.21.15.94:0/3467074614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2d9453a0 con 0x55cf2da54b40 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76c938f640 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 0x55cf2da64b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76ca391640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76b00a33b0 0x7f76b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76baffd640 1 -- 172.21.15.94:0/618609756 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f76b40cfe90 con 0x7f76c4102110 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76ca391640 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da53090 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f76bc010620 tx=0x7f76bc010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dda55bc9afd4261 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76affff640 1 -- 172.21.15.94:0/3467074614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc019070 con 0x55cf2da54b40 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76affff640 1 -- 172.21.15.94:0/3467074614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76bc022070 con 0x55cf2da54b40 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76ca391640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76b00a33b0 0x7f76b00a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f76bc021000 tx=0x7f76bc01f000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.681+0000 7f76affff640 1 -- 172.21.15.94:0/3467074614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc026c90 con 0x55cf2da54b40 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 -- 172.21.15.94:0/3467074614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 msgr2=0x55cf2da53090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da53090 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f76bc010620 tx=0x7f76bc010ae0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 -- 172.21.15.94:0/3467074614 shutdown_connections 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 0x55cf2da64b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da53a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/3467074614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da53090 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 -- 172.21.15.94:0/3467074614 >> 172.21.15.94:0/3467074614 conn(0x55cf2da1c310 msgr2=0x55cf2da4fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 -- 172.21.15.94:0/3467074614 shutdown_connections 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 -- 172.21.15.94:0/3467074614 wait complete. 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 -- start start 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.682+0000 7f76ce1cd740 1 Processor -- start 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da62160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da54b40 0x55cf2da5f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da62780 0x55cf2da5f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cf2da65480 con 0x55cf2da62780 2024-09-19T04:31:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cf2d974060 con 0x55cf2da535d0 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cf2da65600 con 0x55cf2da54b40 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ce1cd740 10 client.0 _ll_register_callbacks cb 0x55cf2d6c78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ca391640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da62780 0x55cf2da5f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ca391640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da62780 0x55cf2da5f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41480/0 (socket says 172.21.15.94:41480) 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ca391640 1 -- 172.21.15.94:0/969826448 learned_addr learned my addr 172.21.15.94:0/969826448 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76c9b90640 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da62160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76c938f640 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da54b40 0x55cf2da5f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ca391640 1 -- 172.21.15.94:0/969826448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da54b40 msgr2=0x55cf2da5f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ca391640 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da54b40 0x55cf2da5f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ca391640 1 -- 172.21.15.94:0/969826448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 msgr2=0x55cf2da62160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ca391640 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da62160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ca391640 1 -- 172.21.15.94:0/969826448 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2d72a3c0 con 0x55cf2da62780 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ca391640 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da62780 0x55cf2da5f9e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f76bc026650 tx=0x7f76bc026680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc981f89dc209120 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ad7fa640 1 -- 172.21.15.94:0/969826448 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc033060 con 0x55cf2da62780 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ce1cd740 1 -- 172.21.15.94:0/969826448 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76bc022070 con 0x55cf2da62780 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.683+0000 7f76ad7fa640 1 -- 172.21.15.94:0/969826448 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f76bc0138b0 con 0x55cf2da62780 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ad7fa640 1 -- 172.21.15.94:0/969826448 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f76bc019070 con 0x55cf2da62780 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ce1cd740 10 client.17442 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ce1cd740 1 -- 172.21.15.94:0/969826448 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55cf2da70f00 con 0x55cf2da62780 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ad7fa640 1 -- 172.21.15.94:0/969826448 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76bc022070 con 0x55cf2da62780 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ad7fa640 1 -- 172.21.15.94:0/969826448 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f76bc011560 con 0x55cf2da62780 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ce1cd740 10 client.17442 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ce1cd740 10 client.17442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:31:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ce1cd740 20 client.17442 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:30.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55cf2da72250 0x55cf2da74640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ce1cd740 4 client.17442 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:30.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.684+0000 7f76ce1cd740 1 -- 172.21.15.94:0/969826448 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf2d9c6860 con 0x55cf2da72250 2024-09-19T04:31:30.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.685+0000 7f76c9b90640 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55cf2da72250 0x55cf2da74640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.685+0000 7f76ad7fa640 10 client.17442 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:31:30.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.685+0000 7f76c9b90640 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55cf2da72250 0x55cf2da74640 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3d15e13bdac857c server_cookie=99d49295be778ab2 in_seq=0 out_seq=0 2024-09-19T04:31:30.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ad7fa640 1 -- 172.21.15.94:0/969826448 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf2d9c6860 con 0x55cf2da72250 2024-09-19T04:31:30.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ad7fa640 10 client.17442 handle_command_reply: tid=0 2024-09-19T04:31:30.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 client.17442 shutdown 2024-09-19T04:31:30.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 -- 172.21.15.94:0/969826448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da62780 msgr2=0x55cf2da5f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:30.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da62780 0x55cf2da5f9e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f76bc026650 tx=0x7f76bc026680 comp rx=0 tx=0).stop 2024-09-19T04:31:30.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 -- 172.21.15.94:0/969826448 shutdown_connections 2024-09-19T04:31:30.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55cf2da72250 0x55cf2da74640 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da62780 0x55cf2da5f9e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da54b40 0x55cf2da5f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ad7fa640 0 client.17442 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:31:30.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/969826448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da62160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 -- 172.21.15.94:0/969826448 >> 172.21.15.94:0/969826448 conn(0x55cf2da1c310 msgr2=0x55cf2da55960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:30.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 -- 172.21.15.94:0/969826448 shutdown_connections 2024-09-19T04:31:30.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 1 -- 172.21.15.94:0/969826448 wait complete. 2024-09-19T04:31:30.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.686+0000 7f76ce1cd740 20 client.17442 trim_cache size 0 max 16384 2024-09-19T04:31:30.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76ce1cd740 1 Processor -- start 2024-09-19T04:31:30.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76ce1cd740 1 -- start start 2024-09-19T04:31:30.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da6e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da61380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 0x55cf2da79d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41adec0 con 0x55cf2da54b40 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41ab790 con 0x55cf2da535d0 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41a9600 con 0x55cf2da62780 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76c938f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da61380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76c938f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da61380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41492/0 (socket says 172.21.15.94:41492) 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76c938f640 1 -- 172.21.15.94:0/291692983 learned_addr learned my addr 172.21.15.94:0/291692983 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76c9b90640 1 --2- 172.21.15.94:0/291692983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da6e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76ca391640 1 --2- 172.21.15.94:0/291692983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 0x55cf2da79d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76c938f640 1 -- 172.21.15.94:0/291692983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 msgr2=0x55cf2da79d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76c938f640 1 --2- 172.21.15.94:0/291692983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 0x55cf2da79d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76c938f640 1 -- 172.21.15.94:0/291692983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 msgr2=0x55cf2da6e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:30.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76c938f640 1 --2- 172.21.15.94:0/291692983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da6e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.699+0000 7f76c938f640 1 -- 172.21.15.94:0/291692983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2da5d960 con 0x55cf2da54b40 2024-09-19T04:31:30.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76c938f640 1 --2- 172.21.15.94:0/291692983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da61380 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f76b409be30 tx=0x7f76b40d5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45150135b7b2076c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:30.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76af7fe640 1 -- 172.21.15.94:0/291692983 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b400fb00 con 0x55cf2da54b40 2024-09-19T04:31:30.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76af7fe640 1 -- 172.21.15.94:0/291692983 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76b409c830 con 0x55cf2da54b40 2024-09-19T04:31:30.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76af7fe640 1 -- 172.21.15.94:0/291692983 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b40987a0 con 0x55cf2da54b40 2024-09-19T04:31:30.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76ce1cd740 1 -- 172.21.15.94:0/291692983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 msgr2=0x55cf2da61380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/291692983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da61380 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f76b409be30 tx=0x7f76b40d5040 comp rx=0 tx=0).stop 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76ce1cd740 1 -- 172.21.15.94:0/291692983 shutdown_connections 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/291692983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da62780 0x55cf2da79d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/291692983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da54b40 0x55cf2da61380 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.700+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/291692983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da535d0 0x55cf2da6e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 -- 172.21.15.94:0/291692983 >> 172.21.15.94:0/291692983 conn(0x55cf2da1c310 msgr2=0x55cf2da60c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 -- 172.21.15.94:0/291692983 shutdown_connections 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 -- 172.21.15.94:0/291692983 wait complete. 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 -- start start 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 Processor -- start 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da535d0 0x55cf2da6ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da54b40 0x55cf2da66050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da62780 0x55cf2da66590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41a9c50 con 0x55cf2da535d0 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41ab0c0 con 0x55cf2da62780 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41aa320 con 0x55cf2da54b40 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ce1cd740 10 client.0 _ll_register_callbacks cb 0x7f76bc010480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76c9b90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da535d0 0x55cf2da6ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76c9b90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da535d0 0x55cf2da6ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41496/0 (socket says 172.21.15.94:41496) 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76c9b90640 1 -- 172.21.15.94:0/4011743021 learned_addr learned my addr 172.21.15.94:0/4011743021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.701+0000 7f76ca391640 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da62780 0x55cf2da66590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76c9b90640 1 -- 172.21.15.94:0/4011743021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da54b40 msgr2=0x55cf2da66050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76c9b90640 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da54b40 0x55cf2da66050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76c9b90640 1 -- 172.21.15.94:0/4011743021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da62780 msgr2=0x55cf2da66590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:30.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76c9b90640 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da62780 0x55cf2da66590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76c9b90640 1 -- 172.21.15.94:0/4011743021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2da66ad0 con 0x55cf2da535d0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76c9b90640 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da535d0 0x55cf2da6ad20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f76c0000c00 tx=0x7f76c00041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415852e695f533c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ad7fa640 1 -- 172.21.15.94:0/4011743021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0035040 con 0x55cf2da535d0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ad7fa640 1 -- 172.21.15.94:0/4011743021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f76c0031630 con 0x55cf2da535d0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ce1cd740 1 -- 172.21.15.94:0/4011743021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b409c830 con 0x55cf2da535d0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ad7fa640 1 -- 172.21.15.94:0/4011743021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f76c0054050 con 0x55cf2da535d0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ce1cd740 10 client.17448 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ce1cd740 1 -- 172.21.15.94:0/4011743021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55cf2dadca50 con 0x55cf2da535d0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ad7fa640 1 -- 172.21.15.94:0/4011743021 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76b409c830 con 0x55cf2da535d0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ad7fa640 1 -- 172.21.15.94:0/4011743021 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f76c0004a20 con 0x55cf2da535d0 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ce1cd740 10 client.17448 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:30.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ce1cd740 10 client.17448 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.702+0000 7f76ce1cd740 20 client.17448 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55cf2dadd1a0 0x55cf2dadf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 4 client.17448 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 -- 172.21.15.94:0/4011743021 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf2d9c0d70 con 0x55cf2dadd1a0 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76c938f640 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55cf2dadd1a0 0x55cf2dadf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76c938f640 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55cf2dadd1a0 0x55cf2dadf5e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e7eb44406e03ec5 server_cookie=c7a611af311106dd in_seq=0 out_seq=0 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ad7fa640 10 client.17448 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ad7fa640 1 -- 172.21.15.94:0/4011743021 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf2d9c0d70 con 0x55cf2dadd1a0 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ad7fa640 10 client.17448 handle_command_reply: tid=0 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 client.17448 shutdown 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 -- 172.21.15.94:0/4011743021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da535d0 msgr2=0x55cf2da6ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da535d0 0x55cf2da6ad20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f76c0000c00 tx=0x7f76c00041e0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 -- 172.21.15.94:0/4011743021 shutdown_connections 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x55cf2dadd1a0 0x55cf2dadf5e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cf2da62780 0x55cf2da66590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf2da54b40 0x55cf2da66050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 --2- 172.21.15.94:0/4011743021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf2da535d0 0x55cf2da6ad20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 -- 172.21.15.94:0/4011743021 >> 172.21.15.94:0/4011743021 conn(0x55cf2da1c310 msgr2=0x55cf2da55930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ad7fa640 0 client.17448 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.703+0000 7f76ce1cd740 1 -- 172.21.15.94:0/4011743021 shutdown_connections 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76ce1cd740 1 -- 172.21.15.94:0/4011743021 wait complete. 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76ce1cd740 20 client.17448 trim_cache size 0 max 16384 2024-09-19T04:31:30.711 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:31:30.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 -- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76b00a33b0 msgr2=0x7f76b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:30.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76b00a33b0 0x7f76b00a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f76bc021000 tx=0x7f76bc01f000 comp rx=0 tx=0).stop 2024-09-19T04:31:30.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 -- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4102110 msgr2=0x7f76c41987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:30.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4102110 0x7f76c41987e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f76b400c920 tx=0x7f76b400cde0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 -- 172.21.15.94:0/618609756 shutdown_connections 2024-09-19T04:31:30.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f76b00a33b0 0x7f76b00a5870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c4105ba0 0x7f76c419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c4102110 0x7f76c41987e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 --2- 172.21.15.94:0/618609756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76c4101790 0x7f76c41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:30.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 -- 172.21.15.94:0/618609756 >> 172.21.15.94:0/618609756 conn(0x7f76c4077210 msgr2=0x7f76c40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:30.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 -- 172.21.15.94:0/618609756 shutdown_connections 2024-09-19T04:31:30.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:30.704+0000 7f76cbdf4640 1 -- 172.21.15.94:0/618609756 wait complete. 2024-09-19T04:31:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:31 smithi094 ceph-mon[30972]: pgmap v517: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-09-19T04:31:31.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:31 smithi104 ceph-mon[34145]: pgmap v517: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-09-19T04:31:31.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:31:31.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:31 smithi112 ceph-mon[35051]: pgmap v517: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-09-19T04:31:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.129+0000 7f462a614640 1 -- 172.21.15.94:0/3650168552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4624100e20 msgr2=0x7f4624101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.129+0000 7f462a614640 1 --2- 172.21.15.94:0/3650168552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4624100e20 0x7f4624101200 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4614009900 tx=0x7f461402a480 comp rx=0 tx=0).stop 2024-09-19T04:31:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.129+0000 7f462a614640 1 -- 172.21.15.94:0/3650168552 shutdown_connections 2024-09-19T04:31:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.129+0000 7f462a614640 1 --2- 172.21.15.94:0/3650168552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4624102ef0 0x7f462410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.129+0000 7f462a614640 1 --2- 172.21.15.94:0/3650168552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4624102570 0x7f46241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.129+0000 7f462a614640 1 --2- 172.21.15.94:0/3650168552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4624100e20 0x7f4624101200 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.129+0000 7f462a614640 1 -- 172.21.15.94:0/3650168552 >> 172.21.15.94:0/3650168552 conn(0x7f46240fc7d0 msgr2=0x7f46240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.129+0000 7f462a614640 1 -- 172.21.15.94:0/3650168552 shutdown_connections 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.129+0000 7f462a614640 1 -- 172.21.15.94:0/3650168552 wait complete. 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f462a614640 1 Processor -- start 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f462a614640 1 -- start start 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f462a614640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4624100e20 0x7f4624103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f462a614640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4624102570 0x7f4624103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f462a614640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4624102ef0 0x7f46241041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f462a614640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4624110e30 con 0x7f4624102ef0 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f462a614640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4624110cb0 con 0x7f4624102570 2024-09-19T04:31:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f462a614640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4624110fb0 con 0x7f4624100e20 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f46237fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4624102570 0x7f4624103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f46237fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4624102570 0x7f4624103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56902/0 (socket says 172.21.15.94:56902) 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f46237fe640 1 -- 172.21.15.94:0/620466559 learned_addr learned my addr 172.21.15.94:0/620466559 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f4628bb1640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4624102ef0 0x7f46241041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.130+0000 7f4623fff640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4624100e20 0x7f4624103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f46237fe640 1 -- 172.21.15.94:0/620466559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4624100e20 msgr2=0x7f4624103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f46237fe640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4624100e20 0x7f4624103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f46237fe640 1 -- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4624102ef0 msgr2=0x7f46241041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f46237fe640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4624102ef0 0x7f46241041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f46237fe640 1 -- 172.21.15.94:0/620466559 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46241a68f0 con 0x7f4624102570 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f46237fe640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4624102570 0x7f4624103c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f461800e9f0 tx=0x7f461800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f46217fa640 1 -- 172.21.15.94:0/620466559 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461800cd80 con 0x7f4624102570 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f462a614640 1 -- 172.21.15.94:0/620466559 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46241a6be0 con 0x7f4624102570 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f46217fa640 1 -- 172.21.15.94:0/620466559 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46180108e0 con 0x7f4624102570 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f46217fa640 1 -- 172.21.15.94:0/620466559 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618018920 con 0x7f4624102570 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.131+0000 7f462a614640 1 -- 172.21.15.94:0/620466559 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46241a7170 con 0x7f4624102570 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.134+0000 7f462c9ed740 1 Processor -- start 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.134+0000 7f462c9ed740 1 -- start start 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.134+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c43090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.134+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c435d0 0x55c4c9c43a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.134+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c4e370 0x55c4c9c50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.134+0000 7f462c9ed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c4c9ba92f0 con 0x55c4c9c4e370 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.134+0000 7f462c9ed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c4c9b50cf0 con 0x55c4c9c44b40 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.134+0000 7f462c9ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c4c9ba52b0 con 0x55c4c9c435d0 2024-09-19T04:31:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f46217fa640 1 -- 172.21.15.94:0/620466559 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4618010430 con 0x7f4624102570 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4628bb1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c43090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4623fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c435d0 0x55c4c9c43a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4628bb1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c43090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:56904/0 (socket says 172.21.15.94:56904) 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4628bb1640 1 -- 172.21.15.94:0/2391717850 learned_addr learned my addr 172.21.15.94:0/2391717850 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f46237fe640 1 --2- 172.21.15.94:0/2391717850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c4e370 0x55c4c9c50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4628bb1640 1 -- 172.21.15.94:0/2391717850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c435d0 msgr2=0x55c4c9c43a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4628bb1640 1 --2- 172.21.15.94:0/2391717850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c435d0 0x55c4c9c43a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4628bb1640 1 -- 172.21.15.94:0/2391717850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c4e370 msgr2=0x55c4c9c50760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4628bb1640 1 --2- 172.21.15.94:0/2391717850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c4e370 0x55c4c9c50760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4628bb1640 1 -- 172.21.15.94:0/2391717850 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4c9b353a0 con 0x55c4c9c44b40 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4628bb1640 1 --2- 172.21.15.94:0/2391717850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c43090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f461c009f50 tx=0x7f461c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1e7fa73025b898 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f46217fa640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f460c0a33b0 0x7f460c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f460a7fc640 1 -- 172.21.15.94:0/2391717850 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c004d20 con 0x55c4c9c44b40 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f460a7fc640 1 -- 172.21.15.94:0/2391717850 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c004050 con 0x55c4c9c44b40 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f460a7fc640 1 -- 172.21.15.94:0/2391717850 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c005280 con 0x55c4c9c44b40 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f4623fff640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f460c0a33b0 0x7f460c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.135+0000 7f46217fa640 1 -- 172.21.15.94:0/620466559 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f46180d0080 con 0x7f4624102570 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f462c9ed740 1 -- 172.21.15.94:0/2391717850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 msgr2=0x55c4c9c43090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f4623fff640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f460c0a33b0 0x7f460c0a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4614009230 tx=0x7f46140058d0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f462c9ed740 1 --2- 172.21.15.94:0/2391717850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c43090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f461c009f50 tx=0x7f461c0118d0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f462c9ed740 1 -- 172.21.15.94:0/2391717850 shutdown_connections 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f462c9ed740 1 --2- 172.21.15.94:0/2391717850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c4e370 0x55c4c9c50760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f462c9ed740 1 --2- 172.21.15.94:0/2391717850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c435d0 0x55c4c9c43a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f462c9ed740 1 --2- 172.21.15.94:0/2391717850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c43090 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f462c9ed740 1 -- 172.21.15.94:0/2391717850 >> 172.21.15.94:0/2391717850 conn(0x55c4c9c0c310 msgr2=0x55c4c9c3fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f462c9ed740 1 -- 172.21.15.94:0/2391717850 shutdown_connections 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.136+0000 7f462c9ed740 1 -- 172.21.15.94:0/2391717850 wait complete. 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f462c9ed740 1 -- start start 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f462c9ed740 1 Processor -- start 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f462c9ed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c4c9c51070 con 0x55c4c9c435d0 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f462c9ed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c4c9b64060 con 0x55c4c9c44b40 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f462c9ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c4c9c511f0 con 0x55c4c9c4e370 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f462c9ed740 10 client.0 _ll_register_callbacks cb 0x55c4c98b78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f4623fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f4623fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46344/0 (socket says 172.21.15.94:46344) 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f4623fff640 1 -- 172.21.15.94:0/1177547885 learned_addr learned my addr 172.21.15.94:0/1177547885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f46237fe640 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.137+0000 7f4628bb1640 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f4623fff640 1 -- 172.21.15.94:0/1177547885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 msgr2=0x55c4c9c4b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f4623fff640 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c4b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f4623fff640 1 -- 172.21.15.94:0/1177547885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 msgr2=0x55c4c9c4afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f4623fff640 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f4623fff640 1 -- 172.21.15.94:0/1177547885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4c991a3c0 con 0x55c4c9c435d0 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f4623fff640 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4dd00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4614009900 tx=0x7f4614009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0578f9ec7d1a9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f46037fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f46037fe640 1 -- 172.21.15.94:0/1177547885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461404d030 con 0x55c4c9c435d0 2024-09-19T04:31:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f46037fe640 1 -- 172.21.15.94:0/1177547885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f46140035f0 con 0x55c4c9c435d0 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f462c9ed740 1 -- 172.21.15.94:0/1177547885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c004050 con 0x55c4c9c435d0 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f46037fe640 1 -- 172.21.15.94:0/1177547885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4614035040 con 0x55c4c9c435d0 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f462c9ed740 10 client.17460 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f462c9ed740 1 -- 172.21.15.94:0/1177547885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c4c9c5caa0 con 0x55c4c9c435d0 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f46037fe640 1 -- 172.21.15.94:0/1177547885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c004050 con 0x55c4c9c435d0 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.138+0000 7f46037fe640 1 -- 172.21.15.94:0/1177547885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f461403d890 con 0x55c4c9c435d0 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.139+0000 7f462c9ed740 10 client.17460 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.139+0000 7f462c9ed740 10 client.17460 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.139+0000 7f462c9ed740 20 client.17460 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.139+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55c4c9c5df60 0x55c4c9c60390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.139+0000 7f462c9ed740 4 client.17460 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.139+0000 7f462c9ed740 1 -- 172.21.15.94:0/1177547885 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4c9bb6860 con 0x55c4c9c5df60 2024-09-19T04:31:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.139+0000 7f46237fe640 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55c4c9c5df60 0x55c4c9c60390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.139+0000 7f46237fe640 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55c4c9c5df60 0x55c4c9c60390 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54b00b5110af15e1 server_cookie=d01618c875152964 in_seq=0 out_seq=0 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.139+0000 7f46037fe640 10 client.17460 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f46037fe640 1 -- 172.21.15.94:0/1177547885 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4c9bb6860 con 0x55c4c9c5df60 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f46037fe640 10 client.17460 handle_command_reply: tid=0 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 client.17460 shutdown 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 -- 172.21.15.94:0/1177547885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c435d0 msgr2=0x55c4c9c4dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4dd00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4614009900 tx=0x7f4614009260 comp rx=0 tx=0).stop 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 -- 172.21.15.94:0/1177547885 shutdown_connections 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55c4c9c5df60 0x55c4c9c60390 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c4b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1177547885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4dd00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 -- 172.21.15.94:0/1177547885 >> 172.21.15.94:0/1177547885 conn(0x55c4c9c0c310 msgr2=0x55c4c9c45960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:32.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f46037fe640 0 client.17460 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:31:32.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 -- 172.21.15.94:0/1177547885 shutdown_connections 2024-09-19T04:31:32.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.140+0000 7f462c9ed740 1 -- 172.21.15.94:0/1177547885 wait complete. 2024-09-19T04:31:32.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.141+0000 7f462c9ed740 20 client.17460 trim_cache size 0 max 16384 2024-09-19T04:31:32.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.154+0000 7f462c9ed740 1 Processor -- start 2024-09-19T04:31:32.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.154+0000 7f462c9ed740 1 -- start start 2024-09-19T04:31:32.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.154+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.154+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.154+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c46120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.154+0000 7f462c9ed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46241b2340 con 0x55c4c9c44b40 2024-09-19T04:31:32.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.154+0000 7f462c9ed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46241afc10 con 0x55c4c9c435d0 2024-09-19T04:31:32.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.154+0000 7f462c9ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46241ada80 con 0x55c4c9c4e370 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f46237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f46237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46360/0 (socket says 172.21.15.94:46360) 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f46237fe640 1 -- 172.21.15.94:0/3479963284 learned_addr learned my addr 172.21.15.94:0/3479963284 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f4623fff640 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f4628bb1640 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c46120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f46237fe640 1 -- 172.21.15.94:0/3479963284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 msgr2=0x55c4c9c46120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f46237fe640 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c46120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f46237fe640 1 -- 172.21.15.94:0/3479963284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 msgr2=0x55c4c9c4a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f46237fe640 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f46237fe640 1 -- 172.21.15.94:0/3479963284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4c9c49f60 con 0x55c4c9c44b40 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f4623fff640 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f46237fe640 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4cf20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f461800c6a0 tx=0x7f46180d2590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cdf8855fb6df87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f4609ffb640 1 -- 172.21.15.94:0/3479963284 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461809bd30 con 0x55c4c9c44b40 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f4609ffb640 1 -- 172.21.15.94:0/3479963284 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4618010de0 con 0x55c4c9c44b40 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.155+0000 7f4609ffb640 1 -- 172.21.15.94:0/3479963284 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180a4f90 con 0x55c4c9c44b40 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 -- 172.21.15.94:0/3479963284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 msgr2=0x55c4c9c4cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4cf20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f461800c6a0 tx=0x7f46180d2590 comp rx=0 tx=0).stop 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 -- 172.21.15.94:0/3479963284 shutdown_connections 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c46120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c4cf20 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 --2- 172.21.15.94:0/3479963284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c4a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 -- 172.21.15.94:0/3479963284 >> 172.21.15.94:0/3479963284 conn(0x55c4c9c0c310 msgr2=0x55c4c9c4c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 -- 172.21.15.94:0/3479963284 shutdown_connections 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 -- 172.21.15.94:0/3479963284 wait complete. 2024-09-19T04:31:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.156+0000 7f462c9ed740 1 -- start start 2024-09-19T04:31:32.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f462c9ed740 1 Processor -- start 2024-09-19T04:31:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c56d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c52070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f462c9ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c52800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f462c9ed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46241ae0d0 con 0x55c4c9c44b40 2024-09-19T04:31:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f462c9ed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46241af540 con 0x55c4c9c435d0 2024-09-19T04:31:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f462c9ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46241ae7a0 con 0x55c4c9c4e370 2024-09-19T04:31:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f462c9ed740 10 client.0 _ll_register_callbacks cb 0x7f4614009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f46237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c52070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f4623fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c56d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f46237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c52070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46362/0 (socket says 172.21.15.94:46362) 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f46237fe640 1 -- 172.21.15.94:0/1227818191 learned_addr learned my addr 172.21.15.94:0/1227818191 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f4628bb1640 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c52800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f46237fe640 1 -- 172.21.15.94:0/1227818191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 msgr2=0x55c4c9c52800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f46237fe640 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c52800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f46237fe640 1 -- 172.21.15.94:0/1227818191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 msgr2=0x55c4c9c56d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f46237fe640 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c56d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f46237fe640 1 -- 172.21.15.94:0/1227818191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4c9cc7c80 con 0x55c4c9c44b40 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f4628bb1640 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c52800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.157+0000 7f4623fff640 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c56d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f46237fe640 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c52070 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f46180cd840 tx=0x7f46180d39c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30b869da73b4d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f46037fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f46037fe640 1 -- 172.21.15.94:0/1227818191 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461809c3d0 con 0x55c4c9c44b40 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f46037fe640 1 -- 172.21.15.94:0/1227818191 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f46180cda10 con 0x55c4c9c44b40 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f462c9ed740 1 -- 172.21.15.94:0/1227818191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c4c9bb0d70 con 0x55c4c9c44b40 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f46037fe640 1 -- 172.21.15.94:0/1227818191 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f46180a4f90 con 0x55c4c9c44b40 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f462c9ed740 10 client.17466 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f462c9ed740 1 -- 172.21.15.94:0/1227818191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c4c9cc8e00 con 0x55c4c9c44b40 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f46037fe640 1 -- 172.21.15.94:0/1227818191 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c4c9bb0d70 con 0x55c4c9c44b40 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f46037fe640 1 -- 172.21.15.94:0/1227818191 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f46180b5120 con 0x55c4c9c44b40 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f462c9ed740 10 client.17466 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f462c9ed740 10 client.17466 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f462c9ed740 20 client.17466 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55c4c9cc9480 0x55c4c9ccb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:32.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f462c9ed740 4 client.17466 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.158+0000 7f462c9ed740 1 -- 172.21.15.94:0/1227818191 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4c9b2d170 con 0x55c4c9cc9480 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.159+0000 7f4623fff640 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55c4c9cc9480 0x55c4c9ccb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.159+0000 7f46037fe640 10 client.17466 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.159+0000 7f4623fff640 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55c4c9cc9480 0x55c4c9ccb8a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9b2bf02256d507d server_cookie=6386c35fbaf00670 in_seq=0 out_seq=0 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.159+0000 7f46037fe640 1 -- 172.21.15.94:0/1227818191 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4c9b2d170 con 0x55c4c9cc9480 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.159+0000 7f46037fe640 10 client.17466 handle_command_reply: tid=0 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.159+0000 7f462c9ed740 1 client.17466 shutdown 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.159+0000 7f462c9ed740 1 -- 172.21.15.94:0/1227818191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 msgr2=0x55c4c9c52070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.159+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c52070 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f46180cd840 tx=0x7f46180d39c0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.159+0000 7f462c9ed740 1 -- 172.21.15.94:0/1227818191 shutdown_connections 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55c4c9cc9480 0x55c4c9ccb8a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c4c9c4e370 0x55c4c9c52800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4c9c44b40 0x55c4c9c52070 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f46037fe640 0 client.17466 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:31:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462c9ed740 1 --2- 172.21.15.94:0/1227818191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4c9c435d0 0x55c4c9c56d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462c9ed740 1 -- 172.21.15.94:0/1227818191 >> 172.21.15.94:0/1227818191 conn(0x55c4c9c0c310 msgr2=0x55c4c9c4d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462c9ed740 1 -- 172.21.15.94:0/1227818191 shutdown_connections 2024-09-19T04:31:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462c9ed740 1 -- 172.21.15.94:0/1227818191 wait complete. 2024-09-19T04:31:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462c9ed740 20 client.17466 trim_cache size 0 max 16384 2024-09-19T04:31:32.168 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:31:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 -- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f460c0a33b0 msgr2=0x7f460c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f460c0a33b0 0x7f460c0a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4614009230 tx=0x7f46140058d0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 -- 172.21.15.94:0/620466559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4624102570 msgr2=0x7f4624103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4624102570 0x7f4624103c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f461800e9f0 tx=0x7f461800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 -- 172.21.15.94:0/620466559 shutdown_connections 2024-09-19T04:31:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f460c0a33b0 0x7f460c0a5870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4624102ef0 0x7f46241041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4624102570 0x7f4624103c90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 --2- 172.21.15.94:0/620466559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4624100e20 0x7f4624103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:32.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 -- 172.21.15.94:0/620466559 >> 172.21.15.94:0/620466559 conn(0x7f46240fc7d0 msgr2=0x7f462406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:32.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 -- 172.21.15.94:0/620466559 shutdown_connections 2024-09-19T04:31:32.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:32.160+0000 7f462a614640 1 -- 172.21.15.94:0/620466559 wait complete. 2024-09-19T04:31:32.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:32 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:31:32.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:32 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:31:32.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:32 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:31:32.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:31:33.118 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:33.239 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:33 smithi094 ceph-mon[30972]: pgmap v518: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 666 KiB/s wr, 68 op/s 2024-09-19T04:31:33.239 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:33 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:31:33.239 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:33 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:31:33.239 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:33 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:31:33.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:33 smithi104 ceph-mon[34145]: pgmap v518: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 666 KiB/s wr, 68 op/s 2024-09-19T04:31:33.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:33 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:31:33.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:33 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:31:33.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:33 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:31:33.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:33 smithi112 ceph-mon[35051]: pgmap v518: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 666 KiB/s wr, 68 op/s 2024-09-19T04:31:33.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:33 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:31:33.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:33 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:31:33.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:33 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/4236241521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48102ef0 msgr2=0x7f0f4810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/4236241521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48102ef0 0x7f0f4810da40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009900 tx=0x7f0f4002a480 comp rx=0 tx=0).stop 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/4236241521 shutdown_connections 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/4236241521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48102ef0 0x7f0f4810da40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/4236241521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f48102570 0x7f0f481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/4236241521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f48100e20 0x7f0f48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/4236241521 >> 172.21.15.94:0/4236241521 conn(0x7f0f480fc7d0 msgr2=0x7f0f480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/4236241521 shutdown_connections 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/4236241521 wait complete. 2024-09-19T04:31:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 Processor -- start 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.588+0000 7f0f4dcb9640 1 -- start start 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f4dcb9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48100e20 0x7f0f48197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f4dcb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f48102570 0x7f0f48197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f4dcb9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f48102ef0 0x7f0f4819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f4dcb9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48110e30 con 0x7f0f48100e20 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f4dcb9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48110cb0 con 0x7f0f48102ef0 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f4dcb9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48110fb0 con 0x7f0f48102570 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48100e20 0x7f0f48197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48100e20 0x7f0f48197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46378/0 (socket says 172.21.15.94:46378) 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f477fe640 1 -- 172.21.15.94:0/2800144204 learned_addr learned my addr 172.21.15.94:0/2800144204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f48102570 0x7f0f48197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f47fff640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f48102ef0 0x7f0f4819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f477fe640 1 -- 172.21.15.94:0/2800144204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f48102570 msgr2=0x7f0f48197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f477fe640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f48102570 0x7f0f48197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f477fe640 1 -- 172.21.15.94:0/2800144204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f48102ef0 msgr2=0x7f0f4819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f477fe640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f48102ef0 0x7f0f4819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f477fe640 1 -- 172.21.15.94:0/2800144204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4819c2e0 con 0x7f0f48100e20 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f47fff640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f48102ef0 0x7f0f4819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f477fe640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48100e20 0x7f0f48197330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009970 tx=0x7f0f38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.589+0000 7f0f44ff9640 1 -- 172.21.15.94:0/2800144204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800d020 con 0x7f0f48100e20 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.590+0000 7f0f44ff9640 1 -- 172.21.15.94:0/2800144204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f3800d1c0 con 0x7f0f48100e20 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.590+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/2800144204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4819c5d0 con 0x7f0f48100e20 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.590+0000 7f0f44ff9640 1 -- 172.21.15.94:0/2800144204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800cc50 con 0x7f0f48100e20 2024-09-19T04:31:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.590+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/2800144204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f48069f70 con 0x7f0f48100e20 2024-09-19T04:31:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f44ff9640 1 -- 172.21.15.94:0/2800144204 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0f3800c460 con 0x7f0f48100e20 2024-09-19T04:31:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f50092740 1 Processor -- start 2024-09-19T04:31:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f50092740 1 -- start start 2024-09-19T04:31:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f50092740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f50092740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 0x56539588ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f50092740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 0x565395899760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f50092740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5653957f22f0 con 0x56539588db40 2024-09-19T04:31:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f50092740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565395799cf0 con 0x565395897370 2024-09-19T04:31:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f50092740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5653957ee2b0 con 0x56539588c5d0 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f44ff9640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f0f300a3430 0x7f0f300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46392/0 (socket says 172.21.15.94:46392) 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f47fff640 1 -- 172.21.15.94:0/179207023 learned_addr learned my addr 172.21.15.94:0/179207023 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.592+0000 7f0f47fff640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f0f300a3430 0x7f0f300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/179207023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 0x565395899760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f44ff9640 1 -- 172.21.15.94:0/2800144204 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f0f380d1100 con 0x7f0f48100e20 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f477fe640 1 --2- 172.21.15.94:0/179207023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 0x56539588ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f47fff640 1 -- 172.21.15.94:0/179207023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 msgr2=0x56539588ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f47fff640 1 --2- 172.21.15.94:0/179207023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 0x56539588ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f47fff640 1 -- 172.21.15.94:0/179207023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 msgr2=0x565395899760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f47fff640 1 --2- 172.21.15.94:0/179207023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 0x565395899760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f47fff640 1 -- 172.21.15.94:0/179207023 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56539577e3a0 con 0x56539588db40 2024-09-19T04:31:33.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f47fff640 1 --2- 172.21.15.94:0/179207023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588c090 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0f400057c0 tx=0x7f0f400318b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ffbc6867586065 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:33.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f47fff640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f0f300a3430 0x7f0f300a58f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f40033000 tx=0x7f0f40046000 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:33.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f2dffb640 1 -- 172.21.15.94:0/179207023 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40035040 con 0x56539588db40 2024-09-19T04:31:33.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f2dffb640 1 -- 172.21.15.94:0/179207023 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f40031ca0 con 0x56539588db40 2024-09-19T04:31:33.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.593+0000 7f0f2dffb640 1 -- 172.21.15.94:0/179207023 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f400328f0 con 0x56539588db40 2024-09-19T04:31:33.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 -- 172.21.15.94:0/179207023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 msgr2=0x56539588c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:33.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 --2- 172.21.15.94:0/179207023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588c090 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0f400057c0 tx=0x7f0f400318b0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 -- 172.21.15.94:0/179207023 shutdown_connections 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 --2- 172.21.15.94:0/179207023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 0x565395899760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 --2- 172.21.15.94:0/179207023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 0x56539588ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 --2- 172.21.15.94:0/179207023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588c090 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 -- 172.21.15.94:0/179207023 >> 172.21.15.94:0/179207023 conn(0x565395855310 msgr2=0x565395888c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 -- 172.21.15.94:0/179207023 shutdown_connections 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 -- 172.21.15.94:0/179207023 wait complete. 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 -- start start 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.594+0000 7f0f50092740 1 Processor -- start 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f50092740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 0x565395896da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f50092740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x565395894050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f50092740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 0x5653958947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f50092740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56539589a010 con 0x56539588db40 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f50092740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5653957ad060 con 0x565395897370 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f50092740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56539589a190 con 0x56539588c5d0 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f50092740 10 client.0 _ll_register_callbacks cb 0x5653955008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x565395894050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x565395894050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46404/0 (socket says 172.21.15.94:46404) 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f46ffd640 1 -- 172.21.15.94:0/3860047119 learned_addr learned my addr 172.21.15.94:0/3860047119 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f477fe640 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 0x565395896da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f47fff640 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 0x5653958947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f46ffd640 1 -- 172.21.15.94:0/3860047119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 msgr2=0x565395896da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 0x565395896da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f46ffd640 1 -- 172.21.15.94:0/3860047119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 msgr2=0x5653958947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 0x5653958947e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f46ffd640 1 -- 172.21.15.94:0/3860047119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653955633c0 con 0x56539588db40 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f47fff640 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 0x5653958947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x565395894050 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c012980 tx=0x7f0f3c012e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51447889877a8e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f26ffd640 1 -- 172.21.15.94:0/3860047119 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00f980 con 0x56539588db40 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f26ffd640 1 -- 172.21.15.94:0/3860047119 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f0f3c00fb20 con 0x56539588db40 2024-09-19T04:31:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.595+0000 7f0f50092740 1 -- 172.21.15.94:0/3860047119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f40031ca0 con 0x56539588db40 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f26ffd640 1 -- 172.21.15.94:0/3860047119 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c00fd20 con 0x56539588db40 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f50092740 10 client.17478 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f50092740 1 -- 172.21.15.94:0/3860047119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5653958a5ba0 con 0x56539588db40 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f26ffd640 1 -- 172.21.15.94:0/3860047119 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f40031ca0 con 0x56539588db40 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f26ffd640 1 -- 172.21.15.94:0/3860047119 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f0f3c004630 con 0x56539588db40 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f50092740 10 client.17478 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f50092740 10 client.17478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f50092740 20 client.17478 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f50092740 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5653958a70b0 0x5653958a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f50092740 4 client.17478 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f50092740 1 -- 172.21.15.94:0/3860047119 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653957ff860 con 0x5653958a70b0 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.596+0000 7f0f477fe640 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5653958a70b0 0x5653958a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f477fe640 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5653958a70b0 0x5653958a94f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c08798af0c9052 server_cookie=54f7b855d2f0a89d in_seq=0 out_seq=0 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f26ffd640 10 client.17478 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f26ffd640 1 -- 172.21.15.94:0/3860047119 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653957ff860 con 0x5653958a70b0 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f26ffd640 10 client.17478 handle_command_reply: tid=0 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f50092740 1 client.17478 shutdown 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f50092740 1 -- 172.21.15.94:0/3860047119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 msgr2=0x565395894050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f50092740 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x565395894050 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c012980 tx=0x7f0f3c012e40 comp rx=0 tx=0).stop 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f50092740 1 -- 172.21.15.94:0/3860047119 shutdown_connections 2024-09-19T04:31:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f50092740 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5653958a70b0 0x5653958a94f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f50092740 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565395897370 0x5653958947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f50092740 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x565395894050 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f26ffd640 0 client.17478 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:33.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f50092740 1 --2- 172.21.15.94:0/3860047119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539588c5d0 0x565395896da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.597+0000 7f0f50092740 1 -- 172.21.15.94:0/3860047119 >> 172.21.15.94:0/3860047119 conn(0x565395855310 msgr2=0x56539588e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:33.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.598+0000 7f0f50092740 1 -- 172.21.15.94:0/3860047119 shutdown_connections 2024-09-19T04:31:33.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.598+0000 7f0f50092740 1 -- 172.21.15.94:0/3860047119 wait complete. 2024-09-19T04:31:33.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.598+0000 7f0f50092740 20 client.17478 trim_cache size 0 max 16384 2024-09-19T04:31:33.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f50092740 1 Processor -- start 2024-09-19T04:31:33.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f50092740 1 -- start start 2024-09-19T04:31:33.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f50092740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 0x56539588f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f50092740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f50092740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565395897370 0x5653958aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f50092740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481adf00 con 0x56539588db40 2024-09-19T04:31:33.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f50092740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ab7d0 con 0x56539588c5d0 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f50092740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481a9640 con 0x565395897370 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46414/0 (socket says 172.21.15.94:46414) 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f46ffd640 1 -- 172.21.15.94:0/4195915781 learned_addr learned my addr 172.21.15.94:0/4195915781 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f477fe640 1 --2- 172.21.15.94:0/4195915781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 0x56539588f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f46ffd640 1 -- 172.21.15.94:0/4195915781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565395897370 msgr2=0x5653958aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/4195915781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565395897370 0x5653958aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f46ffd640 1 -- 172.21.15.94:0/4195915781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 msgr2=0x56539588f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/4195915781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 0x56539588f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f46ffd640 1 -- 172.21.15.94:0/4195915781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565395889960 con 0x56539588db40 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.611+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/4195915781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588f660 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009a00 tx=0x7f0f3c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33354c7693897594 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f2d7fa640 1 -- 172.21.15.94:0/4195915781 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c028030 con 0x56539588db40 2024-09-19T04:31:33.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f2d7fa640 1 -- 172.21.15.94:0/4195915781 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f3c0106c0 con 0x56539588db40 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f2d7fa640 1 -- 172.21.15.94:0/4195915781 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c003c10 con 0x56539588db40 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f50092740 1 -- 172.21.15.94:0/4195915781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 msgr2=0x56539588f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f50092740 1 --2- 172.21.15.94:0/4195915781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588f660 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009a00 tx=0x7f0f3c00fc60 comp rx=0 tx=0).stop 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f50092740 1 -- 172.21.15.94:0/4195915781 shutdown_connections 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f50092740 1 --2- 172.21.15.94:0/4195915781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565395897370 0x5653958aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f50092740 1 --2- 172.21.15.94:0/4195915781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x56539588f660 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f50092740 1 --2- 172.21.15.94:0/4195915781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 0x56539588f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f50092740 1 -- 172.21.15.94:0/4195915781 >> 172.21.15.94:0/4195915781 conn(0x565395855310 msgr2=0x5653958931a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f50092740 1 -- 172.21.15.94:0/4195915781 shutdown_connections 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.612+0000 7f0f50092740 1 -- 172.21.15.94:0/4195915781 wait complete. 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f50092740 1 -- start start 2024-09-19T04:31:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f50092740 1 Processor -- start 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f50092740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 0x565395890fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f50092740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x5653958914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f50092740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565395897370 0x565395891a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f50092740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481a9c90 con 0x56539588db40 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f50092740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ab100 con 0x56539588c5d0 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f50092740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481aa360 con 0x565395897370 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f50092740 10 client.0 _ll_register_callbacks cb 0x7f0f3c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x5653958914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 0x565395890fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x5653958914f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46426/0 (socket says 172.21.15.94:46426) 2024-09-19T04:31:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f46ffd640 1 -- 172.21.15.94:0/1252644807 learned_addr learned my addr 172.21.15.94:0/1252644807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f47fff640 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565395897370 0x565395891a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.613+0000 7f0f46ffd640 1 -- 172.21.15.94:0/1252644807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565395897370 msgr2=0x565395891a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565395897370 0x565395891a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f46ffd640 1 -- 172.21.15.94:0/1252644807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 msgr2=0x565395890fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 0x565395890fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f46ffd640 1 -- 172.21.15.94:0/1252644807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565395910d90 con 0x56539588db40 2024-09-19T04:31:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f477fe640 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 0x565395890fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f46ffd640 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x5653958914f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c012e80 tx=0x7f0f3c004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc4e719f32e70dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f26ffd640 1 -- 172.21.15.94:0/1252644807 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c031070 con 0x56539588db40 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f26ffd640 1 -- 172.21.15.94:0/1252644807 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f0f3c002030 con 0x56539588db40 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f50092740 1 -- 172.21.15.94:0/1252644807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3c0106c0 con 0x56539588db40 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f26ffd640 1 -- 172.21.15.94:0/1252644807 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c03d050 con 0x56539588db40 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f50092740 10 client.17484 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f50092740 1 -- 172.21.15.94:0/1252644807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x565395911f70 con 0x56539588db40 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f26ffd640 1 -- 172.21.15.94:0/1252644807 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f3c0106c0 con 0x56539588db40 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f26ffd640 1 -- 172.21.15.94:0/1252644807 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f0f3c024b30 con 0x56539588db40 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f50092740 10 client.17484 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f50092740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.614+0000 7f0f50092740 20 client.17484 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f50092740 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5653959125e0 0x5653959149d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f50092740 4 client.17484 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f50092740 1 -- 172.21.15.94:0/1252644807 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653957f9d70 con 0x5653959125e0 2024-09-19T04:31:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f477fe640 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5653959125e0 0x5653959149d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f477fe640 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5653959125e0 0x5653959149d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c916900eac406e72 server_cookie=3cf219bf812931ab in_seq=0 out_seq=0 2024-09-19T04:31:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f26ffd640 10 client.17484 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f26ffd640 1 -- 172.21.15.94:0/1252644807 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653957f9d70 con 0x5653959125e0 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f26ffd640 10 client.17484 handle_command_reply: tid=0 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f50092740 1 client.17484 shutdown 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f50092740 1 -- 172.21.15.94:0/1252644807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 msgr2=0x5653958914f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.615+0000 7f0f50092740 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x5653958914f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c012e80 tx=0x7f0f3c004c30 comp rx=0 tx=0).stop 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f50092740 1 -- 172.21.15.94:0/1252644807 shutdown_connections 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f50092740 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5653959125e0 0x5653959149d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f26ffd640 0 client.17484 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f50092740 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565395897370 0x565395891a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f50092740 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56539588db40 0x5653958914f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f50092740 1 --2- 172.21.15.94:0/1252644807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56539588c5d0 0x565395890fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f50092740 1 -- 172.21.15.94:0/1252644807 >> 172.21.15.94:0/1252644807 conn(0x565395855310 msgr2=0x565395887500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f50092740 1 -- 172.21.15.94:0/1252644807 shutdown_connections 2024-09-19T04:31:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f50092740 1 -- 172.21.15.94:0/1252644807 wait complete. 2024-09-19T04:31:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f50092740 20 client.17484 trim_cache size 0 max 16384 2024-09-19T04:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f0f300a3430 msgr2=0x7f0f300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f0f300a3430 0x7f0f300a58f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f40033000 tx=0x7f0f40046000 comp rx=0 tx=0).stop 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48100e20 msgr2=0x7f0f48197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48100e20 0x7f0f48197330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009970 tx=0x7f0f38009e30 comp rx=0 tx=0).stop 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/2800144204 shutdown_connections 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f0f300a3430 0x7f0f300a58f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f48102ef0 0x7f0f4819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f48102570 0x7f0f48197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 --2- 172.21.15.94:0/2800144204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48100e20 0x7f0f48197330 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.616+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/2800144204 >> 172.21.15.94:0/2800144204 conn(0x7f0f480fc7d0 msgr2=0x7f0f480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.617+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/2800144204 shutdown_connections 2024-09-19T04:31:33.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:33.617+0000 7f0f4dcb9640 1 -- 172.21.15.94:0/2800144204 wait complete. 2024-09-19T04:31:35.307 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:31:35.307 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:31:35.334 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:35 smithi094 ceph-mon[30972]: pgmap v519: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 714 KiB/s wr, 73 op/s 2024-09-19T04:31:35.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:35 smithi104 ceph-mon[34145]: pgmap v519: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 714 KiB/s wr, 73 op/s 2024-09-19T04:31:35.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:35 smithi112 ceph-mon[35051]: pgmap v519: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 714 KiB/s wr, 73 op/s 2024-09-19T04:31:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:36.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 -- 172.21.15.94:0/2208019129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808100310 msgr2=0x7f48081006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:36.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 --2- 172.21.15.94:0/2208019129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808100310 0x7f48081006f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4804009900 tx=0x7f480402a410 comp rx=0 tx=0).stop 2024-09-19T04:31:36.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 -- 172.21.15.94:0/2208019129 shutdown_connections 2024-09-19T04:31:36.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 --2- 172.21.15.94:0/2208019129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4808106f30 0x7f480810eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 --2- 172.21.15.94:0/2208019129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4808100cc0 0x7f4808106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 --2- 172.21.15.94:0/2208019129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808100310 0x7f48081006f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 -- 172.21.15.94:0/2208019129 >> 172.21.15.94:0/2208019129 conn(0x7f4808077210 msgr2=0x7f48080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 -- 172.21.15.94:0/2208019129 shutdown_connections 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 -- 172.21.15.94:0/2208019129 wait complete. 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 Processor -- start 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.048+0000 7f480fe58640 1 -- start start 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480fe58640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4808100310 0x7f48081983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480fe58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4808100cc0 0x7f48081988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480fe58640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808106f30 0x7f480819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480fe58640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4808111e60 con 0x7f4808106f30 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480fe58640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4808111ce0 con 0x7f4808100310 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480fe58640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4808111fe0 con 0x7f4808100cc0 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480e3f5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808106f30 0x7f480819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480dbf4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4808100310 0x7f48081983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480dbf4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4808100310 0x7f48081983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.94:57000/0 (socket says 172.21.15.94:57000) 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480e3f5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808106f30 0x7f480819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46432/0 (socket says 172.21.15.94:46432) 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480dbf4640 1 -- 172.21.15.94:0/100420209 learned_addr learned my addr 172.21.15.94:0/100420209 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480d3f3640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4808100cc0 0x7f48081988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480e3f5640 1 -- 172.21.15.94:0/100420209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4808100cc0 msgr2=0x7f48081988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480e3f5640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4808100cc0 0x7f48081988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480e3f5640 1 -- 172.21.15.94:0/100420209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4808100310 msgr2=0x7f48081983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480e3f5640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4808100310 0x7f48081983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480e3f5640 1 -- 172.21.15.94:0/100420209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480819d350 con 0x7f4808106f30 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480e3f5640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808106f30 0x7f480819cc70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f480000be20 tx=0x7f480000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f47feffd640 1 -- 172.21.15.94:0/100420209 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800019070 con 0x7f4808106f30 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f47feffd640 1 -- 172.21.15.94:0/100420209 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48000148f0 con 0x7f4808106f30 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f480fe58640 1 -- 172.21.15.94:0/100420209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480819d640 con 0x7f4808106f30 2024-09-19T04:31:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.049+0000 7f47feffd640 1 -- 172.21.15.94:0/100420209 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48000127f0 con 0x7f4808106f30 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.050+0000 7f480fe58640 1 -- 172.21.15.94:0/100420209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48081a4f20 con 0x7f4808106f30 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.052+0000 7f47feffd640 1 -- 172.21.15.94:0/100420209 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f480001c070 con 0x7f4808106f30 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.052+0000 7f4812231740 1 Processor -- start 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.052+0000 7f4812231740 1 -- start start 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.052+0000 7f4812231740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb467e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f4812231740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb476c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f4812231740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb479540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f4812231740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5586eb3cd280 con 0x5586eb477150 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f4812231740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5586eb374b10 con 0x5586eb467a40 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f4812231740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5586eb3c9240 con 0x5586eb468360 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb479540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb479540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46434/0 (socket says 172.21.15.94:46434) 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 -- 172.21.15.94:0/3762364676 learned_addr learned my addr 172.21.15.94:0/3762364676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480dbf4640 1 --2- 172.21.15.94:0/3762364676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb476c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 --2- 172.21.15.94:0/3762364676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb467e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f47feffd640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f47f40a33b0 0x7f47f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480e3f5640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f47f40a33b0 0x7f47f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 -- 172.21.15.94:0/3762364676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 msgr2=0x5586eb476c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 --2- 172.21.15.94:0/3762364676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb476c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 -- 172.21.15.94:0/3762364676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 msgr2=0x5586eb467e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 --2- 172.21.15.94:0/3762364676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb467e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 -- 172.21.15.94:0/3762364676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586eb3811f0 con 0x5586eb477150 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f47feffd640 1 -- 172.21.15.94:0/100420209 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f48000cf4b0 con 0x7f4808106f30 2024-09-19T04:31:36.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480d3f3640 1 --2- 172.21.15.94:0/3762364676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb479540 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f47f80127c0 tx=0x7f47f800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddb0a2b5786ec01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f47f3fff640 1 -- 172.21.15.94:0/3762364676 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8012e00 con 0x5586eb477150 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f47f3fff640 1 -- 172.21.15.94:0/3762364676 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47f8010c20 con 0x5586eb477150 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f47f3fff640 1 -- 172.21.15.94:0/3762364676 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f801ea90 con 0x5586eb477150 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.053+0000 7f480e3f5640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f47f40a33b0 0x7f47f40a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f480009b840 tx=0x7f4800086040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.054+0000 7f4812231740 1 -- 172.21.15.94:0/3762364676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 msgr2=0x5586eb479540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.054+0000 7f4812231740 1 --2- 172.21.15.94:0/3762364676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb479540 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f47f80127c0 tx=0x7f47f800b7c0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.054+0000 7f4812231740 1 -- 172.21.15.94:0/3762364676 shutdown_connections 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.054+0000 7f4812231740 1 --2- 172.21.15.94:0/3762364676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb479540 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.054+0000 7f4812231740 1 --2- 172.21.15.94:0/3762364676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb476c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.054+0000 7f4812231740 1 --2- 172.21.15.94:0/3762364676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb467e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.054+0000 7f4812231740 1 -- 172.21.15.94:0/3762364676 >> 172.21.15.94:0/3762364676 conn(0x5586eb4640f0 msgr2=0x5586eb4644d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.054+0000 7f4812231740 1 -- 172.21.15.94:0/3762364676 shutdown_connections 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.054+0000 7f4812231740 1 -- 172.21.15.94:0/3762364676 wait complete. 2024-09-19T04:31:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f4812231740 1 -- start start 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f4812231740 1 Processor -- start 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f4812231740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb467a40 0x5586eb474090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f4812231740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb4745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f4812231740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb477150 0x5586eb474b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f4812231740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5586eb479d20 con 0x5586eb467a40 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f4812231740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5586eb387f10 con 0x5586eb477150 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f4812231740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5586eb479ea0 con 0x5586eb468360 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f4812231740 10 client.0 _ll_register_callbacks cb 0x5586eb350a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f480d3f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb467a40 0x5586eb474090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f480d3f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb467a40 0x5586eb474090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46442/0 (socket says 172.21.15.94:46442) 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f480d3f3640 1 -- 172.21.15.94:0/4058984537 learned_addr learned my addr 172.21.15.94:0/4058984537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f480dbf4640 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb4745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f480d3f3640 1 -- 172.21.15.94:0/4058984537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 msgr2=0x5586eb4745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f480d3f3640 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb4745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f480d3f3640 1 -- 172.21.15.94:0/4058984537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb477150 msgr2=0x5586eb474b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.055+0000 7f480d3f3640 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb477150 0x5586eb474b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f480d3f3640 1 -- 172.21.15.94:0/4058984537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586eb13e6a0 con 0x5586eb467a40 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f480d3f3640 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb467a40 0x5586eb474090 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005d00 tx=0x7f47f800b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85312f9f2b987520 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f47f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f47f17fa640 1 -- 172.21.15.94:0/4058984537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f802c060 con 0x5586eb467a40 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f47f17fa640 1 -- 172.21.15.94:0/4058984537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f47f8005050 con 0x5586eb467a40 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f4812231740 1 -- 172.21.15.94:0/4058984537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47f8010c20 con 0x5586eb467a40 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f47f17fa640 1 -- 172.21.15.94:0/4058984537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f47f8005250 con 0x5586eb467a40 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f4812231740 10 client.17496 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f4812231740 1 -- 172.21.15.94:0/4058984537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5586eb4859b0 con 0x5586eb467a40 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f47f17fa640 1 -- 172.21.15.94:0/4058984537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47f8010c20 con 0x5586eb467a40 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f47f17fa640 1 -- 172.21.15.94:0/4058984537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f47f80046c0 con 0x5586eb467a40 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f4812231740 10 client.17496 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f4812231740 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f4812231740 20 client.17496 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f4812231740 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5586eb486d00 0x5586eb4890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f4812231740 4 client.17496 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.056+0000 7f4812231740 1 -- 172.21.15.94:0/4058984537 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586eb3da7f0 con 0x5586eb486d00 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.057+0000 7f480dbf4640 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5586eb486d00 0x5586eb4890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.057+0000 7f47f17fa640 10 client.17496 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.057+0000 7f480dbf4640 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5586eb486d00 0x5586eb4890f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=401b815e1b27c787 server_cookie=592db1e7a9408723 in_seq=0 out_seq=0 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f47f17fa640 1 -- 172.21.15.94:0/4058984537 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586eb3da7f0 con 0x5586eb486d00 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f47f17fa640 10 client.17496 handle_command_reply: tid=0 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 client.17496 shutdown 2024-09-19T04:31:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 -- 172.21.15.94:0/4058984537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb467a40 msgr2=0x5586eb474090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb467a40 0x5586eb474090 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005d00 tx=0x7f47f800b400 comp rx=0 tx=0).stop 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 -- 172.21.15.94:0/4058984537 shutdown_connections 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5586eb486d00 0x5586eb4890f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb477150 0x5586eb474b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f47f17fa640 0 client.17496 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb4745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 --2- 172.21.15.94:0/4058984537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb467a40 0x5586eb474090 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 -- 172.21.15.94:0/4058984537 >> 172.21.15.94:0/4058984537 conn(0x5586eb4640f0 msgr2=0x5586eb469e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 -- 172.21.15.94:0/4058984537 shutdown_connections 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 1 -- 172.21.15.94:0/4058984537 wait complete. 2024-09-19T04:31:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.058+0000 7f4812231740 20 client.17496 trim_cache size 0 max 16384 2024-09-19T04:31:36.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.075+0000 7f4812231740 1 Processor -- start 2024-09-19T04:31:36.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.075+0000 7f4812231740 1 -- start start 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f4812231740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb472f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f4812231740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb48e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f4812231740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb48ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f4812231740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48081ae030 con 0x5586eb477150 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f4812231740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f48081ab900 con 0x5586eb467a40 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f4812231740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48081a9770 con 0x5586eb468360 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb48ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb48ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46456/0 (socket says 172.21.15.94:46456) 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 -- 172.21.15.94:0/2888652871 learned_addr learned my addr 172.21.15.94:0/2888652871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 --2- 172.21.15.94:0/2888652871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb472f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480dbf4640 1 --2- 172.21.15.94:0/2888652871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb48e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 -- 172.21.15.94:0/2888652871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 msgr2=0x5586eb48e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:36.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 --2- 172.21.15.94:0/2888652871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb48e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 -- 172.21.15.94:0/2888652871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 msgr2=0x5586eb472f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 --2- 172.21.15.94:0/2888652871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb472f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 -- 172.21.15.94:0/2888652871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586eb476960 con 0x5586eb477150 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f480d3f3640 1 --2- 172.21.15.94:0/2888652871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb48ec90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f47f800be30 tx=0x7f47f8004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f67c5092dfe481 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f47f37fe640 1 -- 172.21.15.94:0/2888652871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f802c040 con 0x5586eb477150 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f47f37fe640 1 -- 172.21.15.94:0/2888652871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47f8016040 con 0x5586eb477150 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.076+0000 7f47f37fe640 1 -- 172.21.15.94:0/2888652871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8010bb0 con 0x5586eb477150 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.077+0000 7f4812231740 1 -- 172.21.15.94:0/2888652871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 msgr2=0x5586eb48ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.077+0000 7f4812231740 1 --2- 172.21.15.94:0/2888652871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb48ec90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f47f800be30 tx=0x7f47f8004f20 comp rx=0 tx=0).stop 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.077+0000 7f4812231740 1 -- 172.21.15.94:0/2888652871 shutdown_connections 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.077+0000 7f4812231740 1 --2- 172.21.15.94:0/2888652871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb48ec90 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.077+0000 7f4812231740 1 --2- 172.21.15.94:0/2888652871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb48e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.077+0000 7f4812231740 1 --2- 172.21.15.94:0/2888652871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb472f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.077+0000 7f4812231740 1 -- 172.21.15.94:0/2888652871 >> 172.21.15.94:0/2888652871 conn(0x5586eb4640f0 msgr2=0x5586eb472830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.077+0000 7f4812231740 1 -- 172.21.15.94:0/2888652871 shutdown_connections 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.077+0000 7f4812231740 1 -- 172.21.15.94:0/2888652871 wait complete. 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f4812231740 1 -- start start 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f4812231740 1 Processor -- start 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f4812231740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb47f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f4812231740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb47acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f4812231740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb47b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f4812231740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48081a9dc0 con 0x5586eb477150 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f4812231740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f48081ab230 con 0x5586eb467a40 2024-09-19T04:31:36.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f4812231740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48081aa490 con 0x5586eb468360 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f4812231740 10 client.0 _ll_register_callbacks cb 0x7f47f800be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f480d3f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb47b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f480dbf4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb47acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f480d3f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb47b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46470/0 (socket says 172.21.15.94:46470) 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f480d3f3640 1 -- 172.21.15.94:0/3796602542 learned_addr learned my addr 172.21.15.94:0/3796602542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.078+0000 7f480d3f3640 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb47f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.079+0000 7f480d3f3640 1 -- 172.21.15.94:0/3796602542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 msgr2=0x5586eb47acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.079+0000 7f480d3f3640 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb47acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.079+0000 7f480d3f3640 1 -- 172.21.15.94:0/3796602542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 msgr2=0x5586eb47f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.079+0000 7f480d3f3640 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb47f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.079+0000 7f480d3f3640 1 -- 172.21.15.94:0/3796602542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586eb47baf0 con 0x5586eb477150 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.079+0000 7f480d3f3640 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb47b2c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005210 tx=0x7f47f8005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746f44c9985ebe6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.079+0000 7f47f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:36.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f47f17fa640 1 -- 172.21.15.94:0/3796602542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f802c040 con 0x5586eb477150 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f47f17fa640 1 -- 172.21.15.94:0/3796602542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f47f80037f0 con 0x5586eb477150 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f4812231740 1 -- 172.21.15.94:0/3796602542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47f8016040 con 0x5586eb477150 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f47f17fa640 1 -- 172.21.15.94:0/3796602542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f47f80039f0 con 0x5586eb477150 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f4812231740 10 client.17502 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f4812231740 1 -- 172.21.15.94:0/3796602542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5586eb4f1870 con 0x5586eb477150 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f47f17fa640 1 -- 172.21.15.94:0/3796602542 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47f8016040 con 0x5586eb477150 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f47f17fa640 1 -- 172.21.15.94:0/3796602542 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f47f8004640 con 0x5586eb477150 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f4812231740 10 client.17502 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f4812231740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f4812231740 20 client.17502 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f4812231740 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5586eb4f1e10 0x5586eb4f4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f4812231740 4 client.17502 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:31:36.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.080+0000 7f4812231740 1 -- 172.21.15.94:0/3796602542 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5586eb3d4d00 con 0x5586eb4f1e10 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.081+0000 7f480dbf4640 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5586eb4f1e10 0x5586eb4f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.081+0000 7f480dbf4640 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5586eb4f1e10 0x5586eb4f4230 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f2462565efdebe server_cookie=d101ed0e66e05892 in_seq=0 out_seq=0 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.081+0000 7f47f17fa640 10 client.17502 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f47f17fa640 1 -- 172.21.15.94:0/3796602542 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5586eb3d4d00 con 0x5586eb4f1e10 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f47f17fa640 10 client.17502 handle_command_reply: tid=0 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 client.17502 shutdown 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 -- 172.21.15.94:0/3796602542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 msgr2=0x5586eb47b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb47b2c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005210 tx=0x7f47f8005720 comp rx=0 tx=0).stop 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 -- 172.21.15.94:0/3796602542 shutdown_connections 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5586eb4f1e10 0x5586eb4f4230 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586eb477150 0x5586eb47b2c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586eb468360 0x5586eb47acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 --2- 172.21.15.94:0/3796602542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5586eb467a40 0x5586eb47f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f47f17fa640 0 client.17502 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:36.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 -- 172.21.15.94:0/3796602542 >> 172.21.15.94:0/3796602542 conn(0x5586eb4640f0 msgr2=0x5586eb473050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:36.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 -- 172.21.15.94:0/3796602542 shutdown_connections 2024-09-19T04:31:36.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 1 -- 172.21.15.94:0/3796602542 wait complete. 2024-09-19T04:31:36.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.082+0000 7f4812231740 20 client.17502 trim_cache size 0 max 16384 2024-09-19T04:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "bc9415f0-e5e9-4e38-9ca1-62562be55caf", 2024-09-19T04:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:31:36.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 -- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f47f40a33b0 msgr2=0x7f47f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:36.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f47f40a33b0 0x7f47f40a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f480009b840 tx=0x7f4800086040 comp rx=0 tx=0).stop 2024-09-19T04:31:36.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 -- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808106f30 msgr2=0x7f480819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:36.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808106f30 0x7f480819cc70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f480000be20 tx=0x7f480000bf10 comp rx=0 tx=0).stop 2024-09-19T04:31:36.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 -- 172.21.15.94:0/100420209 shutdown_connections 2024-09-19T04:31:36.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f47f40a33b0 0x7f47f40a5870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4808106f30 0x7f480819cc70 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4808100cc0 0x7f48081988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 --2- 172.21.15.94:0/100420209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4808100310 0x7f48081983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:36.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 -- 172.21.15.94:0/100420209 >> 172.21.15.94:0/100420209 conn(0x7f4808077210 msgr2=0x7f48081051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:36.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 -- 172.21.15.94:0/100420209 shutdown_connections 2024-09-19T04:31:36.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:36.083+0000 7f480fe58640 1 -- 172.21.15.94:0/100420209 wait complete. 2024-09-19T04:31:36.733 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:31:37.019 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: pgmap v520: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:31:37.341 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:31:37.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:37 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: pgmap v520: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2024-09-19T04:31:37.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:31:37.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:31:37.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:31:37.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:31:37.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:31:37.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:31:37.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.355 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:37 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.463+0000 7efc77773640 1 -- 172.21.15.94:0/4198430338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70100a80 msgr2=0x7efc70100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:37.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.463+0000 7efc77773640 1 --2- 172.21.15.94:0/4198430338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70100a80 0x7efc70100ee0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009930 tx=0x7efc6c02a420 comp rx=0 tx=0).stop 2024-09-19T04:31:37.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 -- 172.21.15.94:0/4198430338 shutdown_connections 2024-09-19T04:31:37.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 --2- 172.21.15.94:0/4198430338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc70101420 0x7efc7010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 --2- 172.21.15.94:0/4198430338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70100a80 0x7efc70100ee0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 --2- 172.21.15.94:0/4198430338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc70106a80 0x7efc70106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 -- 172.21.15.94:0/4198430338 >> 172.21.15.94:0/4198430338 conn(0x7efc700fc7a0 msgr2=0x7efc700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 -- 172.21.15.94:0/4198430338 shutdown_connections 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 -- 172.21.15.94:0/4198430338 wait complete. 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 Processor -- start 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 -- start start 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc70100a80 0x7efc701973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70101420 0x7efc70197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc70106a80 0x7efc7019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc70110dd0 con 0x7efc70101420 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc70110c50 con 0x7efc70106a80 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.464+0000 7efc77773640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc70110f50 con 0x7efc70100a80 2024-09-19T04:31:37.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc74d0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70101420 0x7efc70197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc74d0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70101420 0x7efc70197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46496/0 (socket says 172.21.15.94:46496) 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc74d0e640 1 -- 172.21.15.94:0/819398095 learned_addr learned my addr 172.21.15.94:0/819398095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc75d10640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc70106a80 0x7efc7019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc7550f640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc70100a80 0x7efc701973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc74d0e640 1 -- 172.21.15.94:0/819398095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc70100a80 msgr2=0x7efc701973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc74d0e640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc70100a80 0x7efc701973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc74d0e640 1 -- 172.21.15.94:0/819398095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc70106a80 msgr2=0x7efc7019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc74d0e640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc70106a80 0x7efc7019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc74d0e640 1 -- 172.21.15.94:0/819398095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7019c3a0 con 0x7efc70101420 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc7550f640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc70100a80 0x7efc701973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:31:37.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc74d0e640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70101420 0x7efc70197930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efc6c005940 tx=0x7efc6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc627fc640 1 -- 172.21.15.94:0/819398095 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c03f030 con 0x7efc70101420 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc627fc640 1 -- 172.21.15.94:0/819398095 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc6c003cb0 con 0x7efc70101420 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc77773640 1 -- 172.21.15.94:0/819398095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7019c630 con 0x7efc70101420 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc627fc640 1 -- 172.21.15.94:0/819398095 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c035070 con 0x7efc70101420 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.465+0000 7efc77773640 1 -- 172.21.15.94:0/819398095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701a3f70 con 0x7efc70101420 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.467+0000 7efc627fc640 1 -- 172.21.15.94:0/819398095 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7efc6c042030 con 0x7efc70101420 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc79b4c740 1 Processor -- start 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc79b4c740 1 -- start start 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aa5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8ab0680 0x5628d8ab2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc79b4c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5628d8a0b2e0 con 0x5628d8aa5dc0 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc79b4c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5628d89b2ce0 con 0x5628d8aa54a0 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc79b4c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5628d8a072a0 con 0x5628d8ab0680 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc627fc640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc5c0a33b0 0x7efc5c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc75d10640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc5c0a33b0 0x7efc5c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.468+0000 7efc7550f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc7550f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46512/0 (socket says 172.21.15.94:46512) 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc627fc640 1 -- 172.21.15.94:0/819398095 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7efc6c0f02f0 con 0x7efc70101420 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc7550f640 1 -- 172.21.15.94:0/2685835524 learned_addr learned my addr 172.21.15.94:0/2685835524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc74d0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8ab0680 0x5628d8ab2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc75d10640 1 --2- 172.21.15.94:0/2685835524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aa5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc7550f640 1 -- 172.21.15.94:0/2685835524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8ab0680 msgr2=0x5628d8ab2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc7550f640 1 --2- 172.21.15.94:0/2685835524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8ab0680 0x5628d8ab2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc7550f640 1 -- 172.21.15.94:0/2685835524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 msgr2=0x5628d8aa5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc7550f640 1 --2- 172.21.15.94:0/2685835524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aa5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc7550f640 1 -- 172.21.15.94:0/2685835524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628d8997440 con 0x5628d8aa5dc0 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc7550f640 1 --2- 172.21.15.94:0/2685835524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab0140 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efc6400aa10 tx=0x7efc6400ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac24a944853f9f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc75d10640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc5c0a33b0 0x7efc5c0a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc68009fd0 tx=0x7efc68009ee0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc5b7fe640 1 -- 172.21.15.94:0/2685835524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64015070 con 0x5628d8aa5dc0 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc5b7fe640 1 -- 172.21.15.94:0/2685835524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc6400cc00 con 0x5628d8aa5dc0 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.469+0000 7efc5b7fe640 1 -- 172.21.15.94:0/2685835524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64018560 con 0x5628d8aa5dc0 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 -- 172.21.15.94:0/2685835524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8aa5dc0 msgr2=0x5628d8ab0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:37.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2685835524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab0140 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efc6400aa10 tx=0x7efc6400ae10 comp rx=0 tx=0).stop 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 -- 172.21.15.94:0/2685835524 shutdown_connections 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2685835524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8ab0680 0x5628d8ab2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2685835524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab0140 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2685835524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aa5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 -- 172.21.15.94:0/2685835524 >> 172.21.15.94:0/2685835524 conn(0x5628d8aa1a30 msgr2=0x5628d8aa1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 -- 172.21.15.94:0/2685835524 shutdown_connections 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 -- 172.21.15.94:0/2685835524 wait complete. 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 -- start start 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.470+0000 7efc79b4c740 1 Processor -- start 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 0x5628d8aad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc79b4c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5628d8ab5460 con 0x5628d8ab0680 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc79b4c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5628d89c6050 con 0x5628d8aa54a0 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc79b4c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5628d8ab55e0 con 0x5628d8aa5dc0 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc79b4c740 10 client.0 _ll_register_callbacks cb 0x5628d8a06160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc75d10640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc75d10640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46522/0 (socket says 172.21.15.94:46522) 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc75d10640 1 -- 172.21.15.94:0/868670694 learned_addr learned my addr 172.21.15.94:0/868670694 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc74d0e640 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 0x5628d8aad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc7550f640 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc75d10640 1 -- 172.21.15.94:0/868670694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 msgr2=0x5628d8aad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:37.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc75d10640 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 0x5628d8aad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc75d10640 1 -- 172.21.15.94:0/868670694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 msgr2=0x5628d8aaff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc75d10640 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc75d10640 1 -- 172.21.15.94:0/868670694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628d877c3c0 con 0x5628d8ab0680 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc7550f640 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc75d10640 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aad760 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efc68009c10 tx=0x7efc6801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b874bce20e6909b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc58ff9640 1 -- 172.21.15.94:0/868670694 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68014040 con 0x5628d8ab0680 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc58ff9640 1 -- 172.21.15.94:0/868670694 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7efc68026620 con 0x5628d8ab0680 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.471+0000 7efc79b4c740 1 -- 172.21.15.94:0/868670694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc6400cc00 con 0x5628d8ab0680 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc58ff9640 1 -- 172.21.15.94:0/868670694 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7efc68026820 con 0x5628d8ab0680 2024-09-19T04:31:37.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc79b4c740 10 client.17514 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc79b4c740 1 -- 172.21.15.94:0/868670694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5628d8ac0e20 con 0x5628d8ab0680 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc58ff9640 1 -- 172.21.15.94:0/868670694 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc6400cc00 con 0x5628d8ab0680 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc58ff9640 1 -- 172.21.15.94:0/868670694 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7efc68025940 con 0x5628d8ab0680 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc79b4c740 10 client.17514 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc79b4c740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc79b4c740 20 client.17514 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc79b4c740 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5628d8ac22e0 0x5628d8ac47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc79b4c740 4 client.17514 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc79b4c740 1 -- 172.21.15.94:0/868670694 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628d8a18850 con 0x5628d8ac22e0 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.472+0000 7efc7550f640 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5628d8ac22e0 0x5628d8ac47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.473+0000 7efc58ff9640 10 client.17514 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.473+0000 7efc7550f640 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5628d8ac22e0 0x5628d8ac47a0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb4735f0b3330a05 server_cookie=18908bb9114bbed in_seq=0 out_seq=0 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.473+0000 7efc58ff9640 1 -- 172.21.15.94:0/868670694 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628d8a18850 con 0x5628d8ac22e0 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.473+0000 7efc58ff9640 10 client.17514 handle_command_reply: tid=0 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.473+0000 7efc79b4c740 1 client.17514 shutdown 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 -- 172.21.15.94:0/868670694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 msgr2=0x5628d8aad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aad760 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efc68009c10 tx=0x7efc6801a000 comp rx=0 tx=0).stop 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 -- 172.21.15.94:0/868670694 shutdown_connections 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5628d8ac22e0 0x5628d8ac47a0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aad760 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 0x5628d8aad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc58ff9640 0 client.17514 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:37.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 --2- 172.21.15.94:0/868670694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 -- 172.21.15.94:0/868670694 >> 172.21.15.94:0/868670694 conn(0x5628d8aa1a30 msgr2=0x5628d8aa7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:37.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 -- 172.21.15.94:0/868670694 shutdown_connections 2024-09-19T04:31:37.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 1 -- 172.21.15.94:0/868670694 wait complete. 2024-09-19T04:31:37.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.474+0000 7efc79b4c740 20 client.17514 trim_cache size 0 max 16384 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc79b4c740 1 Processor -- start 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc79b4c740 1 -- start start 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 0x5628d8aca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc79b4c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc701adf10 con 0x5628d8ab0680 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc79b4c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc701ab7e0 con 0x5628d8aa54a0 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc79b4c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a9650 con 0x5628d8aa5dc0 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc75d10640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc75d10640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46526/0 (socket says 172.21.15.94:46526) 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc75d10640 1 -- 172.21.15.94:0/2335648717 learned_addr learned my addr 172.21.15.94:0/2335648717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc7550f640 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc75d10640 1 -- 172.21.15.94:0/2335648717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 msgr2=0x5628d8aca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc74d0e640 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 0x5628d8aca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc75d10640 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 0x5628d8aca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc75d10640 1 -- 172.21.15.94:0/2335648717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 msgr2=0x5628d8aaf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc75d10640 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc75d10640 1 -- 172.21.15.94:0/2335648717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628d8aa1860 con 0x5628d8ab0680 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.491+0000 7efc7550f640 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc75d10640 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aca600 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efc6800bec0 tx=0x7efc68004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d14c909caebf69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc5affd640 1 -- 172.21.15.94:0/2335648717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6803b070 con 0x5628d8ab0680 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc5affd640 1 -- 172.21.15.94:0/2335648717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc68020030 con 0x5628d8ab0680 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc5affd640 1 -- 172.21.15.94:0/2335648717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6801ae60 con 0x5628d8ab0680 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc79b4c740 1 -- 172.21.15.94:0/2335648717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 msgr2=0x5628d8aca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aca600 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efc6800bec0 tx=0x7efc68004b50 comp rx=0 tx=0).stop 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc79b4c740 1 -- 172.21.15.94:0/2335648717 shutdown_connections 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8aca600 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa5dc0 0x5628d8aca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2335648717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa54a0 0x5628d8aaf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc79b4c740 1 -- 172.21.15.94:0/2335648717 >> 172.21.15.94:0/2335648717 conn(0x5628d8aa1a30 msgr2=0x5628d8aaea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc79b4c740 1 -- 172.21.15.94:0/2335648717 shutdown_connections 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.492+0000 7efc79b4c740 1 -- 172.21.15.94:0/2335648717 wait complete. 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc79b4c740 1 -- start start 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc79b4c740 1 Processor -- start 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa54a0 0x5628d8abaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc79b4c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8ab8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc79b4c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a9ca0 con 0x5628d8ab0680 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc79b4c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc701ab110 con 0x5628d8aa5dc0 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc79b4c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc701aa370 con 0x5628d8aa54a0 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc79b4c740 10 client.0 _ll_register_callbacks cb 0x7efc680099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc75d10640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8ab8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc75d10640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8ab8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46538/0 (socket says 172.21.15.94:46538) 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc75d10640 1 -- 172.21.15.94:0/2215220525 learned_addr learned my addr 172.21.15.94:0/2215220525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.493+0000 7efc74d0e640 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc7550f640 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa54a0 0x5628d8abaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc75d10640 1 -- 172.21.15.94:0/2215220525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa54a0 msgr2=0x5628d8abaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc75d10640 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa54a0 0x5628d8abaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc75d10640 1 -- 172.21.15.94:0/2215220525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa5dc0 msgr2=0x5628d8ab8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc75d10640 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc75d10640 1 -- 172.21.15.94:0/2215220525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628d8ab8f70 con 0x5628d8ab0680 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc7550f640 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa54a0 0x5628d8abaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:31:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc75d10640 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8ab8770 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efc68025e30 tx=0x7efc68008fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978a74d8a7d338b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc58ff9640 1 -- 172.21.15.94:0/2215220525 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6803b070 con 0x5628d8ab0680 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc58ff9640 1 -- 172.21.15.94:0/2215220525 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7efc68026dd0 con 0x5628d8ab0680 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc79b4c740 1 -- 172.21.15.94:0/2215220525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc68020030 con 0x5628d8ab0680 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc58ff9640 1 -- 172.21.15.94:0/2215220525 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7efc6801ae60 con 0x5628d8ab0680 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc79b4c740 10 client.17520 fetch_fsmap learned FSMap version 54 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc79b4c740 1 -- 172.21.15.94:0/2215220525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5628d8b2d020 con 0x5628d8ab0680 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc58ff9640 1 -- 172.21.15.94:0/2215220525 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc68020030 con 0x5628d8ab0680 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc58ff9640 1 -- 172.21.15.94:0/2215220525 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7efc68036770 con 0x5628d8ab0680 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc79b4c740 10 client.17520 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc79b4c740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.494+0000 7efc79b4c740 20 client.17520 populate_metadata read hostname 'smithi094' 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5628d8b2d5d0 0x5628d8b2f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 4 client.17520 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 -- 172.21.15.94:0/2215220525 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5628d8aadaf0 con 0x5628d8b2d5d0 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc7550f640 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5628d8b2d5d0 0x5628d8b2f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc58ff9640 10 client.17520 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc7550f640 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5628d8b2d5d0 0x5628d8b2f9c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b549df173625fb server_cookie=1e93b1e56aba3ddb in_seq=0 out_seq=0 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc58ff9640 1 -- 172.21.15.94:0/2215220525 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x5628d8aadaf0 con 0x5628d8b2d5d0 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc58ff9640 10 client.17520 handle_command_reply: tid=0 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 client.17520 shutdown 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 -- 172.21.15.94:0/2215220525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 msgr2=0x5628d8ab8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8ab8770 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efc68025e30 tx=0x7efc68008fa0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 -- 172.21.15.94:0/2215220525 shutdown_connections 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5628d8b2d5d0 0x5628d8b2f9c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628d8ab0680 0x5628d8ab8770 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc58ff9640 0 client.17520 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628d8aa5dc0 0x5628d8ab8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 --2- 172.21.15.94:0/2215220525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d8aa54a0 0x5628d8abaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.495+0000 7efc79b4c740 1 -- 172.21.15.94:0/2215220525 >> 172.21.15.94:0/2215220525 conn(0x5628d8aa1a30 msgr2=0x5628d8aa0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc79b4c740 1 -- 172.21.15.94:0/2215220525 shutdown_connections 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc79b4c740 1 -- 172.21.15.94:0/2215220525 wait complete. 2024-09-19T04:31:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc79b4c740 20 client.17520 trim_cache size 0 max 16384 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (0 inodes in the stack)", 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout: "bc9415f0-e5e9-4e38-9ca1-62562be55caf": { 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "bc9415f0-e5e9-4e38-9ca1-62562be55caf", 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:31:37.510 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:31:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 -- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc5c0a33b0 msgr2=0x7efc5c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc5c0a33b0 0x7efc5c0a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc68009fd0 tx=0x7efc68009ee0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 -- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70101420 msgr2=0x7efc70197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70101420 0x7efc70197930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efc6c005940 tx=0x7efc6c0042f0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 -- 172.21.15.94:0/819398095 shutdown_connections 2024-09-19T04:31:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7efc5c0a33b0 0x7efc5c0a5870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc70106a80 0x7efc7019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc70101420 0x7efc70197930 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 --2- 172.21.15.94:0/819398095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc70100a80 0x7efc701973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 -- 172.21.15.94:0/819398095 >> 172.21.15.94:0/819398095 conn(0x7efc700fc7a0 msgr2=0x7efc700fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 -- 172.21.15.94:0/819398095 shutdown_connections 2024-09-19T04:31:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:31:37.496+0000 7efc77773640 1 -- 172.21.15.94:0/819398095 wait complete. 2024-09-19T04:31:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: pgmap v520: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2024-09-19T04:31:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:31:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:31:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:31:37.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:31:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:31:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:31:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:37.536 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:37 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:38.218 INFO:tasks.cephfs.filesystem:scrub status for tag:bc9415f0-e5e9-4e38-9ca1-62562be55caf - {'path': '/', 'tag': 'bc9415f0-e5e9-4e38-9ca1-62562be55caf', 'options': 'recursive,force'} 2024-09-19T04:31:38.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:38 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:38.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:38 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:38.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:38 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:39.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:39 smithi094 ceph-mon[30972]: pgmap v521: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2024-09-19T04:31:39.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:39 smithi104 ceph-mon[34145]: pgmap v521: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2024-09-19T04:31:39.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:39 smithi112 ceph-mon[35051]: pgmap v521: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2024-09-19T04:31:41.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:41 smithi094 ceph-mon[30972]: pgmap v522: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 12 op/s 2024-09-19T04:31:41.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:41 smithi104 ceph-mon[34145]: pgmap v522: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 12 op/s 2024-09-19T04:31:41.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:41 smithi112 ceph-mon[35051]: pgmap v522: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 12 op/s 2024-09-19T04:31:42.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:42 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:31:42.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:42 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:31:42.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:42 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:31:42.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:42 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:31:42.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:42 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:31:42.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:42 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:31:42.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:42 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:31:42.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:42 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:31:42.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:42 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:31:43.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:43 smithi104 ceph-mon[34145]: pgmap v523: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 67 KiB/s wr, 9 op/s 2024-09-19T04:31:43.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:43 smithi112 ceph-mon[35051]: pgmap v523: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 67 KiB/s wr, 9 op/s 2024-09-19T04:31:43.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:43 smithi094 ceph-mon[30972]: pgmap v523: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 67 KiB/s wr, 9 op/s 2024-09-19T04:31:45.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:45 smithi104 ceph-mon[34145]: pgmap v524: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 168 KiB/s wr, 19 op/s 2024-09-19T04:31:45.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:45 smithi112 ceph-mon[35051]: pgmap v524: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 168 KiB/s wr, 19 op/s 2024-09-19T04:31:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:45 smithi094 ceph-mon[30972]: pgmap v524: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 168 KiB/s wr, 19 op/s 2024-09-19T04:31:47.354 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:47 smithi104 ceph-mon[34145]: pgmap v525: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 120 KiB/s wr, 14 op/s 2024-09-19T04:31:47.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:47 smithi112 ceph-mon[35051]: pgmap v525: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 120 KiB/s wr, 14 op/s 2024-09-19T04:31:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:47 smithi094 ceph-mon[30972]: pgmap v525: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 120 KiB/s wr, 14 op/s 2024-09-19T04:31:49.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:49 smithi112 ceph-mon[35051]: pgmap v526: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 133 KiB/s wr, 17 op/s 2024-09-19T04:31:49.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:49 smithi094 ceph-mon[30972]: pgmap v526: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 133 KiB/s wr, 17 op/s 2024-09-19T04:31:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:49 smithi104 ceph-mon[34145]: pgmap v526: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 133 KiB/s wr, 17 op/s 2024-09-19T04:31:51.393 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:51 smithi094 ceph-mon[30972]: pgmap v527: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 137 KiB/s wr, 17 op/s 2024-09-19T04:31:51.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:51 smithi112 ceph-mon[35051]: pgmap v527: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 137 KiB/s wr, 17 op/s 2024-09-19T04:31:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:51 smithi104 ceph-mon[34145]: pgmap v527: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 137 KiB/s wr, 17 op/s 2024-09-19T04:31:53.406 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:53 smithi104 ceph-mon[34145]: pgmap v528: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 118 KiB/s wr, 14 op/s 2024-09-19T04:31:53.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:53 smithi112 ceph-mon[35051]: pgmap v528: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 118 KiB/s wr, 14 op/s 2024-09-19T04:31:53.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:53 smithi094 ceph-mon[30972]: pgmap v528: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 118 KiB/s wr, 14 op/s 2024-09-19T04:31:55.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:55 smithi112 ceph-mon[35051]: pgmap v529: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 144 KiB/s wr, 19 op/s 2024-09-19T04:31:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:55 smithi094 ceph-mon[30972]: pgmap v529: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 144 KiB/s wr, 19 op/s 2024-09-19T04:31:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:55 smithi104 ceph-mon[34145]: pgmap v529: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 144 KiB/s wr, 19 op/s 2024-09-19T04:31:57.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:57 smithi112 ceph-mon[35051]: pgmap v530: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-09-19T04:31:57.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:57 smithi094 ceph-mon[30972]: pgmap v530: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-09-19T04:31:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:57 smithi104 ceph-mon[34145]: pgmap v530: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-09-19T04:31:59.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:31:59 smithi112 ceph-mon[35051]: pgmap v531: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-09-19T04:31:59.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:31:59 smithi094 ceph-mon[30972]: pgmap v531: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-09-19T04:31:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:31:59 smithi104 ceph-mon[34145]: pgmap v531: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-09-19T04:32:01.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:01 smithi112 ceph-mon[35051]: pgmap v532: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-09-19T04:32:01.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:01 smithi094 ceph-mon[30972]: pgmap v532: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-09-19T04:32:01.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:01 smithi104 ceph-mon[34145]: pgmap v532: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-09-19T04:32:03.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:03 smithi112 ceph-mon[35051]: pgmap v533: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 9 op/s 2024-09-19T04:32:03.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:03 smithi112 ceph-mon[35051]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T04:32:03.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:03 smithi094 ceph-mon[30972]: pgmap v533: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 9 op/s 2024-09-19T04:32:03.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:03 smithi094 ceph-mon[30972]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T04:32:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:03 smithi104 ceph-mon[34145]: pgmap v533: 97 pgs: 97 active+clean; 3.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 9 op/s 2024-09-19T04:32:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:03 smithi104 ceph-mon[34145]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T04:32:05.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:05 smithi112 ceph-mon[35051]: pgmap v535: 97 pgs: 97 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 184 KiB/s wr, 36 op/s 2024-09-19T04:32:05.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:05 smithi094 ceph-mon[30972]: pgmap v535: 97 pgs: 97 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 184 KiB/s wr, 36 op/s 2024-09-19T04:32:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:05 smithi104 ceph-mon[34145]: pgmap v535: 97 pgs: 97 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 184 KiB/s wr, 36 op/s 2024-09-19T04:32:07.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:06 smithi112 ceph-mon[35051]: pgmap v536: 97 pgs: 97 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 184 KiB/s wr, 36 op/s 2024-09-19T04:32:07.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:06 smithi112 ceph-mon[35051]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T04:32:07.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:06 smithi094 ceph-mon[30972]: pgmap v536: 97 pgs: 97 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 184 KiB/s wr, 36 op/s 2024-09-19T04:32:07.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:06 smithi094 ceph-mon[30972]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T04:32:07.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:06 smithi104 ceph-mon[34145]: pgmap v536: 97 pgs: 97 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 184 KiB/s wr, 36 op/s 2024-09-19T04:32:07.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:06 smithi104 ceph-mon[34145]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T04:32:08.219 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:32:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:08.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:08 smithi094 ceph-mon[30972]: pgmap v538: 97 pgs: 97 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 299 KiB/s wr, 46 op/s 2024-09-19T04:32:09.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:08 smithi112 ceph-mon[35051]: pgmap v538: 97 pgs: 97 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 299 KiB/s wr, 46 op/s 2024-09-19T04:32:09.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:08 smithi104 ceph-mon[34145]: pgmap v538: 97 pgs: 97 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 299 KiB/s wr, 46 op/s 2024-09-19T04:32:09.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.308+0000 7f52550ab640 1 -- 172.21.15.94:0/1728307007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525011b140 msgr2=0x7f525011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:09.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.308+0000 7f52550ab640 1 --2- 172.21.15.94:0/1728307007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525011b140 0x7f525011d530 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5248009ca0 tx=0x7f524802a8f0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.312+0000 7f52550ab640 1 -- 172.21.15.94:0/1728307007 shutdown_connections 2024-09-19T04:32:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.312+0000 7f52550ab640 1 --2- 172.21.15.94:0/1728307007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525011b140 0x7f525011d530 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.312+0000 7f52550ab640 1 --2- 172.21.15.94:0/1728307007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5250109830 0x7f5250109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.312+0000 7f52550ab640 1 --2- 172.21.15.94:0/1728307007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5250109450 0x7f5250110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.312+0000 7f52550ab640 1 -- 172.21.15.94:0/1728307007 >> 172.21.15.94:0/1728307007 conn(0x7f525006e140 msgr2=0x7f525006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:09.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.313+0000 7f52550ab640 1 -- 172.21.15.94:0/1728307007 shutdown_connections 2024-09-19T04:32:09.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.313+0000 7f52550ab640 1 -- 172.21.15.94:0/1728307007 wait complete. 2024-09-19T04:32:09.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.315+0000 7f52550ab640 1 Processor -- start 2024-09-19T04:32:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.318+0000 7f52550ab640 1 -- start start 2024-09-19T04:32:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.319+0000 7f52550ab640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5250109450 0x7f52501a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.319+0000 7f52550ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5250109830 0x7f52501a7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.319+0000 7f52550ab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525011b140 0x7f52501ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.319+0000 7f52550ab640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5250120c50 con 0x7f525011b140 2024-09-19T04:32:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.319+0000 7f52550ab640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5250120ad0 con 0x7f5250109450 2024-09-19T04:32:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.319+0000 7f524e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5250109830 0x7f52501a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.319+0000 7f524e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5250109830 0x7f52501a7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.94:48072/0 (socket says 172.21.15.94:48072) 2024-09-19T04:32:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.319+0000 7f524e59c640 1 -- 172.21.15.94:0/735591791 learned_addr learned my addr 172.21.15.94:0/735591791 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.320+0000 7f52550ab640 1 -- 172.21.15.94:0/735591791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5250120dd0 con 0x7f5250109830 2024-09-19T04:32:09.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.321+0000 7f524f59e640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525011b140 0x7f52501ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.322+0000 7f524ed9d640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5250109450 0x7f52501a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.323+0000 7f524ed9d640 1 -- 172.21.15.94:0/735591791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5250109830 msgr2=0x7f52501a7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:09.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.323+0000 7f524ed9d640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5250109830 0x7f52501a7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.323+0000 7f524ed9d640 1 -- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525011b140 msgr2=0x7f52501ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:09.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.323+0000 7f524ed9d640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525011b140 0x7f52501ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.323+0000 7f524ed9d640 1 -- 172.21.15.94:0/735591791 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52501ac080 con 0x7f5250109450 2024-09-19T04:32:09.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.325+0000 7f524ed9d640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5250109450 0x7f52501a70d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f524000e9f0 tx=0x7f524000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:09.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.326+0000 7f5237fff640 1 -- 172.21.15.94:0/735591791 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524000cc40 con 0x7f5250109450 2024-09-19T04:32:09.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.326+0000 7f52550ab640 1 -- 172.21.15.94:0/735591791 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52501ac370 con 0x7f5250109450 2024-09-19T04:32:09.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.326+0000 7f52550ab640 1 -- 172.21.15.94:0/735591791 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52501b3c50 con 0x7f5250109450 2024-09-19T04:32:09.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.329+0000 7f5257484740 1 Processor -- start 2024-09-19T04:32:09.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.329+0000 7f5237fff640 1 -- 172.21.15.94:0/735591791 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f524000cde0 con 0x7f5250109450 2024-09-19T04:32:09.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.329+0000 7f5237fff640 1 -- 172.21.15.94:0/735591791 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52400187c0 con 0x7f5250109450 2024-09-19T04:32:09.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.330+0000 7f5257484740 1 -- start start 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.331+0000 7f5257484740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d29754a0 0x5630d2975880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.331+0000 7f5257484740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d2975dc0 0x5630d2980140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.331+0000 7f5257484740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2982a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.332+0000 7f5257484740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5630d28db2e0 con 0x5630d2980680 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.332+0000 7f5257484740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630d2882ce0 con 0x5630d2975dc0 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.332+0000 7f524e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2982a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.332+0000 7f524e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2982a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45086/0 (socket says 172.21.15.94:45086) 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.332+0000 7f524e59c640 1 -- 172.21.15.94:0/4245731827 learned_addr learned my addr 172.21.15.94:0/4245731827 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.332+0000 7f5257484740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630d28d72a0 con 0x5630d29754a0 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.333+0000 7f524e59c640 1 -- 172.21.15.94:0/4245731827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d29754a0 msgr2=0x5630d2975880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.333+0000 7f524e59c640 1 --2- 172.21.15.94:0/4245731827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d29754a0 0x5630d2975880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.333+0000 7f524e59c640 1 -- 172.21.15.94:0/4245731827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d2975dc0 msgr2=0x5630d2980140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:32:09.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.333+0000 7f524e59c640 1 --2- 172.21.15.94:0/4245731827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d2975dc0 0x5630d2980140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.333+0000 7f524e59c640 1 -- 172.21.15.94:0/4245731827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d2867440 con 0x5630d2980680 2024-09-19T04:32:09.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.334+0000 7f524e59c640 1 --2- 172.21.15.94:0/4245731827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2982a70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5244009d40 tx=0x7f5244011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5e51132ac2e88a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:09.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.335+0000 7f52357fa640 1 -- 172.21.15.94:0/4245731827 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244005220 con 0x5630d2980680 2024-09-19T04:32:09.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.335+0000 7f52357fa640 1 -- 172.21.15.94:0/4245731827 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5244004c50 con 0x5630d2980680 2024-09-19T04:32:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.336+0000 7f5237fff640 1 -- 172.21.15.94:0/735591791 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5240018a20 con 0x7f5250109450 2024-09-19T04:32:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.337+0000 7f5257484740 1 -- 172.21.15.94:0/4245731827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 msgr2=0x5630d2982a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:09.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.337+0000 7f5257484740 1 --2- 172.21.15.94:0/4245731827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2982a70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5244009d40 tx=0x7f5244011810 comp rx=0 tx=0).stop 2024-09-19T04:32:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.337+0000 7f5237fff640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52380a35c0 0x7f52380a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.338+0000 7f5237fff640 1 -- 172.21.15.94:0/735591791 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f52400d0350 con 0x7f5250109450 2024-09-19T04:32:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.339+0000 7f524f59e640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52380a35c0 0x7f52380a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.339+0000 7f5257484740 1 -- 172.21.15.94:0/4245731827 shutdown_connections 2024-09-19T04:32:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.339+0000 7f5257484740 1 --2- 172.21.15.94:0/4245731827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2982a70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.339+0000 7f5257484740 1 --2- 172.21.15.94:0/4245731827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d2975dc0 0x5630d2980140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.339+0000 7f5257484740 1 --2- 172.21.15.94:0/4245731827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d29754a0 0x5630d2975880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.339+0000 7f5257484740 1 -- 172.21.15.94:0/4245731827 >> 172.21.15.94:0/4245731827 conn(0x5630d2971a30 msgr2=0x5630d2971e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.339+0000 7f5257484740 1 -- 172.21.15.94:0/4245731827 shutdown_connections 2024-09-19T04:32:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.339+0000 7f5257484740 1 -- 172.21.15.94:0/4245731827 wait complete. 2024-09-19T04:32:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.340+0000 7f5257484740 1 -- start start 2024-09-19T04:32:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.340+0000 7f5257484740 1 Processor -- start 2024-09-19T04:32:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.340+0000 7f5257484740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 0x5630d297ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.340+0000 7f5257484740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d297d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.340+0000 7f5257484740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d2980680 0x5630d297d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.340+0000 7f5257484740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5630d29832a0 con 0x5630d2975dc0 2024-09-19T04:32:09.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.340+0000 7f5257484740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630d2896050 con 0x5630d29754a0 2024-09-19T04:32:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.340+0000 7f5257484740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630d2983420 con 0x5630d2980680 2024-09-19T04:32:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.340+0000 7f5257484740 10 client.0 _ll_register_callbacks cb 0x5630d28d6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.341+0000 7f524e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d297d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.341+0000 7f524e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d297d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45098/0 (socket says 172.21.15.94:45098) 2024-09-19T04:32:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.341+0000 7f524e59c640 1 -- 172.21.15.94:0/1632006488 learned_addr learned my addr 172.21.15.94:0/1632006488 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.341+0000 7f524ed9d640 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 0x5630d297ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.341+0000 7f524e59c640 1 -- 172.21.15.94:0/1632006488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d2980680 msgr2=0x5630d297d790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:32:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.341+0000 7f524e59c640 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d2980680 0x5630d297d790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.341+0000 7f524e59c640 1 -- 172.21.15.94:0/1632006488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 msgr2=0x5630d297ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:09.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.341+0000 7f524e59c640 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 0x5630d297ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.341+0000 7f524e59c640 1 -- 172.21.15.94:0/1632006488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d264c3c0 con 0x5630d2975dc0 2024-09-19T04:32:09.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.343+0000 7f524e59c640 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d297d250 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f52440057c0 tx=0x7f5244009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c60a4a588319d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:09.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.343+0000 7f5232ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:09.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.343+0000 7f5232ff5640 1 -- 172.21.15.94:0/1632006488 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244029020 con 0x5630d2975dc0 2024-09-19T04:32:09.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.344+0000 7f5257484740 1 -- 172.21.15.94:0/1632006488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5244004c50 con 0x5630d2975dc0 2024-09-19T04:32:09.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.344+0000 7f5232ff5640 1 -- 172.21.15.94:0/1632006488 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f5244005000 con 0x5630d2975dc0 2024-09-19T04:32:09.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.344+0000 7f5232ff5640 1 -- 172.21.15.94:0/1632006488 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5244034050 con 0x5630d2975dc0 2024-09-19T04:32:09.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.344+0000 7f5257484740 10 client.17532 fetch_fsmap learned FSMap version 54 2024-09-19T04:32:09.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.344+0000 7f5257484740 1 -- 172.21.15.94:0/1632006488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5630d298ed00 con 0x5630d2975dc0 2024-09-19T04:32:09.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.345+0000 7f5232ff5640 1 -- 172.21.15.94:0/1632006488 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5244004c50 con 0x5630d2975dc0 2024-09-19T04:32:09.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.345+0000 7f5232ff5640 1 -- 172.21.15.94:0/1632006488 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f5244022bd0 con 0x5630d2975dc0 2024-09-19T04:32:09.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.346+0000 7f5257484740 10 client.17532 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:32:09.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.346+0000 7f5257484740 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:09.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.346+0000 7f5257484740 20 client.17532 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:09.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.346+0000 7f5257484740 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d2990470 0x5630d29928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.346+0000 7f5257484740 4 client.17532 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:09.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.346+0000 7f5257484740 1 -- 172.21.15.94:0/1632006488 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630d28e8850 con 0x5630d2990470 2024-09-19T04:32:09.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.347+0000 7f524f59e640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52380a35c0 0x7f52380a5a80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5248006010 tx=0x7f52480097a0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:09.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.351+0000 7f524ed9d640 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d2990470 0x5630d29928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.362+0000 7f524ed9d640 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d2990470 0x5630d29928d0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c0aa095eb7c421 server_cookie=a93cf9d01f7860e9 in_seq=0 out_seq=0 2024-09-19T04:32:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.362+0000 7f5232ff5640 10 client.17532 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.363+0000 7f5232ff5640 1 -- 172.21.15.94:0/1632006488 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630d28e8850 con 0x5630d2990470 2024-09-19T04:32:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.363+0000 7f5232ff5640 10 client.17532 handle_command_reply: tid=0 2024-09-19T04:32:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.363+0000 7f5257484740 1 client.17532 shutdown 2024-09-19T04:32:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.366+0000 7f5257484740 1 -- 172.21.15.94:0/1632006488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 msgr2=0x5630d297d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.366+0000 7f5257484740 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d297d250 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f52440057c0 tx=0x7f5244009f60 comp rx=0 tx=0).stop 2024-09-19T04:32:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.367+0000 7f5257484740 1 -- 172.21.15.94:0/1632006488 shutdown_connections 2024-09-19T04:32:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.367+0000 7f5257484740 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d2990470 0x5630d29928d0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.367+0000 7f5257484740 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d2980680 0x5630d297d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.367+0000 7f5257484740 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d297d250 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.367+0000 7f5257484740 1 --2- 172.21.15.94:0/1632006488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 0x5630d297ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.367+0000 7f5257484740 1 -- 172.21.15.94:0/1632006488 >> 172.21.15.94:0/1632006488 conn(0x5630d2971a30 msgr2=0x5630d2977760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:09.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.367+0000 7f5232ff5640 0 client.17532 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:09.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.372+0000 7f5257484740 1 -- 172.21.15.94:0/1632006488 shutdown_connections 2024-09-19T04:32:09.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.372+0000 7f5257484740 1 -- 172.21.15.94:0/1632006488 wait complete. 2024-09-19T04:32:09.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.373+0000 7f5257484740 20 client.17532 trim_cache size 0 max 16384 2024-09-19T04:32:09.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.414+0000 7f5257484740 1 Processor -- start 2024-09-19T04:32:09.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.414+0000 7f5257484740 1 -- start start 2024-09-19T04:32:09.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.414+0000 7f5257484740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 0x5630d298c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.415+0000 7f5257484740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d29981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.415+0000 7f5257484740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d2980680 0x5630d29986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.415+0000 7f5257484740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f523c011260 con 0x5630d2975dc0 2024-09-19T04:32:09.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.415+0000 7f5257484740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f523c00eb30 con 0x5630d29754a0 2024-09-19T04:32:09.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.415+0000 7f5257484740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f523c00c9a0 con 0x5630d2980680 2024-09-19T04:32:09.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.415+0000 7f524e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d29981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.415+0000 7f524e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d29981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45110/0 (socket says 172.21.15.94:45110) 2024-09-19T04:32:09.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.415+0000 7f524e59c640 1 -- 172.21.15.94:0/1602833496 learned_addr learned my addr 172.21.15.94:0/1602833496 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:09.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.415+0000 7f524f59e640 1 --2- 172.21.15.94:0/1602833496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d2980680 0x5630d29986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.416+0000 7f524ed9d640 1 --2- 172.21.15.94:0/1602833496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 0x5630d298c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.416+0000 7f524e59c640 1 -- 172.21.15.94:0/1602833496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d2980680 msgr2=0x5630d29986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:09.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.416+0000 7f524e59c640 1 --2- 172.21.15.94:0/1602833496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d2980680 0x5630d29986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.416+0000 7f524e59c640 1 -- 172.21.15.94:0/1602833496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 msgr2=0x5630d298c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:09.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.416+0000 7f524e59c640 1 --2- 172.21.15.94:0/1602833496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 0x5630d298c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.416+0000 7f524e59c640 1 -- 172.21.15.94:0/1602833496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d297c200 con 0x5630d2975dc0 2024-09-19T04:32:09.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.420+0000 7f524e59c640 1 --2- 172.21.15.94:0/1602833496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d29981a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5244001e50 tx=0x7f5244011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365d80f310045a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:09.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.420+0000 7f5234ff9640 1 -- 172.21.15.94:0/1602833496 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244029030 con 0x5630d2975dc0 2024-09-19T04:32:09.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.420+0000 7f5234ff9640 1 -- 172.21.15.94:0/1602833496 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5244030b10 con 0x5630d2975dc0 2024-09-19T04:32:09.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.420+0000 7f5234ff9640 1 -- 172.21.15.94:0/1602833496 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244026040 con 0x5630d2975dc0 2024-09-19T04:32:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.422+0000 7f5257484740 1 -- 172.21.15.94:0/1602833496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 msgr2=0x5630d29981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:09.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.422+0000 7f5257484740 1 --2- 172.21.15.94:0/1602833496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d29981a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5244001e50 tx=0x7f5244011810 comp rx=0 tx=0).stop 2024-09-19T04:32:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.426+0000 7f5257484740 1 -- 172.21.15.94:0/1602833496 shutdown_connections 2024-09-19T04:32:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.426+0000 7f5257484740 1 --2- 172.21.15.94:0/1602833496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d2980680 0x5630d29986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.426+0000 7f5257484740 1 --2- 172.21.15.94:0/1602833496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2975dc0 0x5630d29981a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.426+0000 7f5257484740 1 --2- 172.21.15.94:0/1602833496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d29754a0 0x5630d298c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.426+0000 7f5257484740 1 -- 172.21.15.94:0/1602833496 >> 172.21.15.94:0/1602833496 conn(0x5630d2971a30 msgr2=0x5630d2978870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:09.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.427+0000 7f5257484740 1 -- 172.21.15.94:0/1602833496 shutdown_connections 2024-09-19T04:32:09.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.428+0000 7f5257484740 1 -- 172.21.15.94:0/1602833496 wait complete. 2024-09-19T04:32:09.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.428+0000 7f5257484740 1 -- start start 2024-09-19T04:32:09.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.428+0000 7f5257484740 1 Processor -- start 2024-09-19T04:32:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.429+0000 7f5257484740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d29754a0 0x5630d29837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.429+0000 7f5257484740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d2975dc0 0x5630d2983d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.429+0000 7f5257484740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2a0a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.429+0000 7f5257484740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f523c00cff0 con 0x5630d2980680 2024-09-19T04:32:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.429+0000 7f5257484740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f523c00e460 con 0x5630d2975dc0 2024-09-19T04:32:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.429+0000 7f5257484740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f523c00d6c0 con 0x5630d29754a0 2024-09-19T04:32:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.430+0000 7f5257484740 10 client.0 _ll_register_callbacks cb 0x7f5244009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:09.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.431+0000 7f524f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2a0a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.431+0000 7f524f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2a0a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45120/0 (socket says 172.21.15.94:45120) 2024-09-19T04:32:09.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.431+0000 7f524f59e640 1 -- 172.21.15.94:0/3562376577 learned_addr learned my addr 172.21.15.94:0/3562376577 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.434+0000 7f524e59c640 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d2975dc0 0x5630d2983d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.434+0000 7f524ed9d640 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d29754a0 0x5630d29837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.434+0000 7f524f59e640 1 -- 172.21.15.94:0/3562376577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d29754a0 msgr2=0x5630d29837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.434+0000 7f524f59e640 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d29754a0 0x5630d29837f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.434+0000 7f524f59e640 1 -- 172.21.15.94:0/3562376577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d2975dc0 msgr2=0x5630d2983d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.434+0000 7f524f59e640 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d2975dc0 0x5630d2983d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.434+0000 7f524f59e640 1 -- 172.21.15.94:0/3562376577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d29845d0 con 0x5630d2980680 2024-09-19T04:32:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.441+0000 7f524f59e640 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2a0a170 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5248009ca0 tx=0x7f524802a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54bf86a48181c09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.441+0000 7f5232ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:09.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.446+0000 7f5232ff5640 1 -- 172.21.15.94:0/3562376577 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248031b60 con 0x5630d2980680 2024-09-19T04:32:09.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.446+0000 7f5232ff5640 1 -- 172.21.15.94:0/3562376577 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f5248031d00 con 0x5630d2980680 2024-09-19T04:32:09.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.447+0000 7f5257484740 1 -- 172.21.15.94:0/3562376577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5244030b10 con 0x5630d2980680 2024-09-19T04:32:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.451+0000 7f5232ff5640 1 -- 172.21.15.94:0/3562376577 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5248003970 con 0x5630d2980680 2024-09-19T04:32:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5257484740 10 client.17538 fetch_fsmap learned FSMap version 54 2024-09-19T04:32:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5257484740 1 -- 172.21.15.94:0/3562376577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5630d2a0b790 con 0x5630d2980680 2024-09-19T04:32:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5232ff5640 1 -- 172.21.15.94:0/3562376577 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5244030b10 con 0x5630d2980680 2024-09-19T04:32:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5232ff5640 1 -- 172.21.15.94:0/3562376577 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f5248004590 con 0x5630d2980680 2024-09-19T04:32:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5257484740 10 client.17538 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:32:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5257484740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5257484740 20 client.17538 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5257484740 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d2a0bf60 0x5630d2a0e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5257484740 4 client.17538 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:32:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.452+0000 7f5257484740 1 -- 172.21.15.94:0/3562376577 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630d28e2d60 con 0x5630d2a0bf60 2024-09-19T04:32:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.453+0000 7f524ed9d640 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d2a0bf60 0x5630d2a0e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.455+0000 7f524ed9d640 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d2a0bf60 0x5630d2a0e380 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3046d666b43121f2 server_cookie=42f6c9c7a498b574 in_seq=0 out_seq=0 2024-09-19T04:32:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.455+0000 7f5232ff5640 10 client.17538 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.456+0000 7f5232ff5640 1 -- 172.21.15.94:0/3562376577 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5630d28e2d60 con 0x5630d2a0bf60 2024-09-19T04:32:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.456+0000 7f5232ff5640 10 client.17538 handle_command_reply: tid=0 2024-09-19T04:32:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.456+0000 7f5257484740 1 client.17538 shutdown 2024-09-19T04:32:09.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.461+0000 7f5257484740 1 -- 172.21.15.94:0/3562376577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 msgr2=0x5630d2a0a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:09.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.461+0000 7f5257484740 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2a0a170 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5248009ca0 tx=0x7f524802a8f0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.461+0000 7f5257484740 1 -- 172.21.15.94:0/3562376577 shutdown_connections 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.461+0000 7f5257484740 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d2a0bf60 0x5630d2a0e380 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.461+0000 7f5257484740 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d2980680 0x5630d2a0a170 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.461+0000 7f5257484740 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d2975dc0 0x5630d2983d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.462+0000 7f5257484740 1 --2- 172.21.15.94:0/3562376577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d29754a0 0x5630d29837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.462+0000 7f5257484740 1 -- 172.21.15.94:0/3562376577 >> 172.21.15.94:0/3562376577 conn(0x5630d2971a30 msgr2=0x5630d2970350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.462+0000 7f5232ff5640 0 client.17538 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.462+0000 7f5257484740 1 -- 172.21.15.94:0/3562376577 shutdown_connections 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.462+0000 7f5257484740 1 -- 172.21.15.94:0/3562376577 wait complete. 2024-09-19T04:32:09.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.462+0000 7f5257484740 20 client.17538 trim_cache size 0 max 16384 2024-09-19T04:32:09.465 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:32:09.465 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:32:09.465 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:32:09.465 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 -- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52380a35c0 msgr2=0x7f52380a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52380a35c0 0x7f52380a5a80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5248006010 tx=0x7f52480097a0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 -- 172.21.15.94:0/735591791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5250109450 msgr2=0x7f52501a70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5250109450 0x7f52501a70d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f524000e9f0 tx=0x7f524000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 -- 172.21.15.94:0/735591791 shutdown_connections 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52380a35c0 0x7f52380a5a80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525011b140 0x7f52501ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5250109830 0x7f52501a7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 --2- 172.21.15.94:0/735591791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5250109450 0x7f52501a70d0 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f524000e9f0 tx=0x7f524000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:32:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 -- 172.21.15.94:0/735591791 >> 172.21.15.94:0/735591791 conn(0x7f525006e140 msgr2=0x7f525011c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:09.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.463+0000 7f52550ab640 1 -- 172.21.15.94:0/735591791 shutdown_connections 2024-09-19T04:32:09.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:09.464+0000 7f52550ab640 1 -- 172.21.15.94:0/735591791 wait complete. 2024-09-19T04:32:10.860 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:32:10.861 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:32:11.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:10 smithi112 ceph-mon[35051]: pgmap v539: 97 pgs: 97 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 302 KiB/s wr, 49 op/s 2024-09-19T04:32:11.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:10 smithi094 ceph-mon[30972]: pgmap v539: 97 pgs: 97 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 302 KiB/s wr, 49 op/s 2024-09-19T04:32:11.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:10 smithi104 ceph-mon[34145]: pgmap v539: 97 pgs: 97 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 302 KiB/s wr, 49 op/s 2024-09-19T04:32:11.435 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:12.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.016+0000 7f41745ff640 1 -- 172.21.15.94:0/3569365780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c102ef0 msgr2=0x7f416c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:12.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.016+0000 7f41745ff640 1 --2- 172.21.15.94:0/3569365780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c102ef0 0x7f416c10da40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4164009ca0 tx=0x7f416402a950 comp rx=0 tx=0).stop 2024-09-19T04:32:12.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.016+0000 7f41745ff640 1 -- 172.21.15.94:0/3569365780 shutdown_connections 2024-09-19T04:32:12.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.016+0000 7f41745ff640 1 --2- 172.21.15.94:0/3569365780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c102ef0 0x7f416c10da40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.016+0000 7f41745ff640 1 --2- 172.21.15.94:0/3569365780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f416c102570 0x7f416c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.016+0000 7f41745ff640 1 --2- 172.21.15.94:0/3569365780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f416c100e20 0x7f416c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.016+0000 7f41745ff640 1 -- 172.21.15.94:0/3569365780 >> 172.21.15.94:0/3569365780 conn(0x7f416c0fc7d0 msgr2=0x7f416c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.016+0000 7f41745ff640 1 -- 172.21.15.94:0/3569365780 shutdown_connections 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.016+0000 7f41745ff640 1 -- 172.21.15.94:0/3569365780 wait complete. 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f41745ff640 1 Processor -- start 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f41745ff640 1 -- start start 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f41745ff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f416c100e20 0x7f416c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f41745ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c102570 0x7f416c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f41745ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f416c102ef0 0x7f416c1a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f41745ff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f416c110e30 con 0x7f416c102570 2024-09-19T04:32:12.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f41745ff640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f416c110cb0 con 0x7f416c100e20 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f41745ff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f416c110fb0 con 0x7f416c102ef0 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f4171b9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c102570 0x7f416c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f4171b9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c102570 0x7f416c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34710/0 (socket says 172.21.15.94:34710) 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f4171b9a640 1 -- 172.21.15.94:0/4213384660 learned_addr learned my addr 172.21.15.94:0/4213384660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.017+0000 7f417239b640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f416c100e20 0x7f416c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f4172b9c640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f416c102ef0 0x7f416c1a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f417239b640 1 -- 172.21.15.94:0/4213384660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f416c102ef0 msgr2=0x7f416c1a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f417239b640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f416c102ef0 0x7f416c1a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f417239b640 1 -- 172.21.15.94:0/4213384660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c102570 msgr2=0x7f416c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f417239b640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c102570 0x7f416c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f417239b640 1 -- 172.21.15.94:0/4213384660 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1a4880 con 0x7f416c100e20 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f417239b640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f416c100e20 0x7f416c1992c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f416800e9f0 tx=0x7f416800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f41637fe640 1 -- 172.21.15.94:0/4213384660 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416800ccc0 con 0x7f416c100e20 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f41637fe640 1 -- 172.21.15.94:0/4213384660 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41680108e0 con 0x7f416c100e20 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f41637fe640 1 -- 172.21.15.94:0/4213384660 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168018880 con 0x7f416c100e20 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1a4a50 con 0x7f416c100e20 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.018+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a4f90 con 0x7f416c100e20 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.019+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c101200 con 0x7f416c100e20 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.020+0000 7f41637fe640 1 -- 172.21.15.94:0/4213384660 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4168010a90 con 0x7f416c100e20 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.021+0000 7f41637fe640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f41580a3430 0x7f41580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.021+0000 7f4171b9a640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f41580a3430 0x7f41580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.021+0000 7f41637fe640 1 -- 172.21.15.94:0/4213384660 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f41680d01b0 con 0x7f416c100e20 2024-09-19T04:32:12.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.022+0000 7f4171b9a640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f41580a3430 0x7f41580a58f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f416c06aac0 tx=0x7f415c005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:12.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.024+0000 7f41637fe640 1 -- 172.21.15.94:0/4213384660 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f416809cb50 con 0x7f416c100e20 2024-09-19T04:32:12.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.247+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f416c109090 con 0x7f416c100e20 2024-09-19T04:32:12.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.248+0000 7f41637fe640 1 -- 172.21.15.94:0/4213384660 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11915 (secure 0 0 0) 0x7f416809c450 con 0x7f416c100e20 2024-09-19T04:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":54,"btime":"2024-09-19T04:29:32:831739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:29:32.764738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:32:12.250 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 54 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f41580a3430 msgr2=0x7f41580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f41580a3430 0x7f41580a58f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f416c06aac0 tx=0x7f415c005f90 comp rx=0 tx=0).stop 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f416c100e20 msgr2=0x7f416c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f416c100e20 0x7f416c1992c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f416800e9f0 tx=0x7f416800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 shutdown_connections 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f41580a3430 0x7f41580a58f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f416c102ef0 0x7f416c1a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c102570 0x7f416c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 --2- 172.21.15.94:0/4213384660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f416c100e20 0x7f416c1992c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 >> 172.21.15.94:0/4213384660 conn(0x7f416c0fc7d0 msgr2=0x7f416c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 shutdown_connections 2024-09-19T04:32:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:12.251+0000 7f41745ff640 1 -- 172.21.15.94:0/4213384660 wait complete. 2024-09-19T04:32:12.944 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:12 smithi094 ceph-mon[30972]: pgmap v540: 97 pgs: 97 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 254 KiB/s wr, 41 op/s 2024-09-19T04:32:12.944 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:12 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4213384660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:32:12.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:32:13.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:12 smithi112 ceph-mon[35051]: pgmap v540: 97 pgs: 97 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 254 KiB/s wr, 41 op/s 2024-09-19T04:32:13.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:12 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4213384660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:32:13.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:12 smithi104 ceph-mon[34145]: pgmap v540: 97 pgs: 97 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 254 KiB/s wr, 41 op/s 2024-09-19T04:32:13.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:12 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4213384660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:32:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:13.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.761+0000 7f58d9423640 1 -- 172.21.15.94:0/325443944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4103740 msgr2=0x7f58d4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:13.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.761+0000 7f58d9423640 1 --2- 172.21.15.94:0/325443944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4103740 0x7f58d4107b80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009930 tx=0x7f58bc02a420 comp rx=0 tx=0).stop 2024-09-19T04:32:13.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.762+0000 7f58d9423640 1 -- 172.21.15.94:0/325443944 shutdown_connections 2024-09-19T04:32:13.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.762+0000 7f58d9423640 1 --2- 172.21.15.94:0/325443944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d4108290 0x7f58d410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.762+0000 7f58d9423640 1 --2- 172.21.15.94:0/325443944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4103740 0x7f58d4107b80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.762+0000 7f58d9423640 1 --2- 172.21.15.94:0/325443944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58d4102d90 0x7f58d4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.762+0000 7f58d9423640 1 -- 172.21.15.94:0/325443944 >> 172.21.15.94:0/325443944 conn(0x7f58d40fc7d0 msgr2=0x7f58d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.762+0000 7f58d9423640 1 -- 172.21.15.94:0/325443944 shutdown_connections 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.762+0000 7f58d9423640 1 -- 172.21.15.94:0/325443944 wait complete. 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.762+0000 7f58d9423640 1 Processor -- start 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d9423640 1 -- start start 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d9423640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58d4102d90 0x7f58d4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d9423640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d4103740 0x7f58d4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d9423640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4108290 0x7f58d419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d9423640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4111f40 con 0x7f58d4108290 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d9423640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4111dc0 con 0x7f58d4102d90 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d9423640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41120c0 con 0x7f58d4103740 2024-09-19T04:32:13.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4108290 0x7f58d419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4108290 0x7f58d419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34728/0 (socket says 172.21.15.94:34728) 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d37fe640 1 -- 172.21.15.94:0/777591914 learned_addr learned my addr 172.21.15.94:0/777591914 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58d4102d90 0x7f58d4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d4103740 0x7f58d4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d37fe640 1 -- 172.21.15.94:0/777591914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d4103740 msgr2=0x7f58d4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d37fe640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d4103740 0x7f58d4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d37fe640 1 -- 172.21.15.94:0/777591914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58d4102d90 msgr2=0x7f58d4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d37fe640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58d4102d90 0x7f58d4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d37fe640 1 -- 172.21.15.94:0/777591914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d419d500 con 0x7f58d4108290 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.763+0000 7f58d37fe640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4108290 0x7f58d419ce20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00eaa0 tx=0x7f58cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.764+0000 7f58d0ff9640 1 -- 172.21.15.94:0/777591914 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00cdb0 con 0x7f58d4108290 2024-09-19T04:32:13.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.764+0000 7f58d9423640 1 -- 172.21.15.94:0/777591914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d419d7f0 con 0x7f58d4108290 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.764+0000 7f58d0ff9640 1 -- 172.21.15.94:0/777591914 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58cc005020 con 0x7f58d4108290 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.764+0000 7f58d0ff9640 1 -- 172.21.15.94:0/777591914 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0116d0 con 0x7f58d4108290 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.764+0000 7f58d9423640 1 -- 172.21.15.94:0/777591914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d41a50d0 con 0x7f58d4108290 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.766+0000 7f58db7fc740 1 Processor -- start 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58d0ff9640 1 -- 172.21.15.94:0/777591914 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f58cc0051d0 con 0x7f58d4108290 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58db7fc740 1 -- start start 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a49a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 0x558774a56760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58db7fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5587749af2f0 con 0x558774a495d0 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58db7fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558774956cf0 con 0x558774a4ab40 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58db7fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587749ab2b0 con 0x558774a54370 2024-09-19T04:32:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58d2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a49a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58d2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a49a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34740/0 (socket says 172.21.15.94:34740) 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58d2ffd640 1 -- 172.21.15.94:0/3403586582 learned_addr learned my addr 172.21.15.94:0/3403586582 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/3403586582 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/3403586582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 0x558774a56760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58d2ffd640 1 -- 172.21.15.94:0/3403586582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 msgr2=0x558774a56760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/3403586582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 0x558774a56760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58d2ffd640 1 -- 172.21.15.94:0/3403586582 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 msgr2=0x558774a49090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/3403586582 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a49090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.767+0000 7f58d2ffd640 1 -- 172.21.15.94:0/3403586582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877493b3a0 con 0x558774a495d0 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.768+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/3403586582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a49a10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f58c400bd10 tx=0x7f58c40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c310e6e40cd9b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.768+0000 7f58c97fa640 1 -- 172.21.15.94:0/3403586582 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4004030 con 0x558774a495d0 2024-09-19T04:32:13.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.768+0000 7f58c97fa640 1 -- 172.21.15.94:0/3403586582 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58c40035f0 con 0x558774a495d0 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.768+0000 7f58c97fa640 1 -- 172.21.15.94:0/3403586582 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4011750 con 0x558774a495d0 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.768+0000 7f58d0ff9640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f58c00a33b0 0x7f58c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.768+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f58c00a33b0 0x7f58c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.768+0000 7f58d0ff9640 1 -- 172.21.15.94:0/777591914 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f58cc0cfdf0 con 0x7f58d4108290 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 -- 172.21.15.94:0/3403586582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 msgr2=0x558774a49a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f58c00a33b0 0x7f58c00a5870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58bc004770 tx=0x7f58bc005fd0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 --2- 172.21.15.94:0/3403586582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a49a10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f58c400bd10 tx=0x7f58c40107a0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 -- 172.21.15.94:0/3403586582 shutdown_connections 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 --2- 172.21.15.94:0/3403586582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 0x558774a56760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 --2- 172.21.15.94:0/3403586582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a49a10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 --2- 172.21.15.94:0/3403586582 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 -- 172.21.15.94:0/3403586582 >> 172.21.15.94:0/3403586582 conn(0x558774a12310 msgr2=0x558774a45c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 -- 172.21.15.94:0/3403586582 shutdown_connections 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 -- 172.21.15.94:0/3403586582 wait complete. 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 -- start start 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 Processor -- start 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 0x558774a53d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a50fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558774a57070 con 0x558774a54370 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55877496a060 con 0x558774a4ab40 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558774a571f0 con 0x558774a495d0 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.769+0000 7f58db7fc740 10 client.0 _ll_register_callbacks cb 0x5587746bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34748/0 (socket says 172.21.15.94:34748) 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d37fe640 1 -- 172.21.15.94:0/1382949226 learned_addr learned my addr 172.21.15.94:0/1382949226 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:13.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a50fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 0x558774a53d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d37fe640 1 -- 172.21.15.94:0/1382949226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 msgr2=0x558774a53d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d37fe640 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 0x558774a53d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d37fe640 1 -- 172.21.15.94:0/1382949226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 msgr2=0x558774a50fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d37fe640 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a50fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d37fe640 1 -- 172.21.15.94:0/1382949226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587747203c0 con 0x558774a54370 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 0x558774a53d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58d37fe640 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a515d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0985a0 tx=0x7f58cc09c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9be529b354c913b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58ba7fc640 1 -- 172.21.15.94:0/1382949226 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc01f030 con 0x558774a54370 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58ba7fc640 1 -- 172.21.15.94:0/1382949226 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f58cc0987b0 con 0x558774a54370 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58db7fc740 1 -- 172.21.15.94:0/1382949226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c40035f0 con 0x558774a54370 2024-09-19T04:32:13.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58ba7fc640 1 -- 172.21.15.94:0/1382949226 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f58cc0055d0 con 0x558774a54370 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58db7fc740 10 client.17556 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.770+0000 7f58db7fc740 1 -- 172.21.15.94:0/1382949226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558774a62ad0 con 0x558774a54370 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58ba7fc640 1 -- 172.21.15.94:0/1382949226 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58c40035f0 con 0x558774a54370 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58ba7fc640 1 -- 172.21.15.94:0/1382949226 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f58cc00f040 con 0x558774a54370 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58db7fc740 10 client.17556 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58db7fc740 10 client.17556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58db7fc740 20 client.17556 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58db7fc740 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558774a63f90 0x558774a66430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58db7fc740 4 client.17556 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58db7fc740 1 -- 172.21.15.94:0/1382949226 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587749bc860 con 0x558774a63f90 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558774a63f90 0x558774a66430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58ba7fc640 10 client.17556 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:32:13.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.771+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558774a63f90 0x558774a66430 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f697bb4aeccda33 server_cookie=f6ce6e95fa4688a8 in_seq=0 out_seq=0 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58ba7fc640 1 -- 172.21.15.94:0/1382949226 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587749bc860 con 0x558774a63f90 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58ba7fc640 10 client.17556 handle_command_reply: tid=0 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 client.17556 shutdown 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 -- 172.21.15.94:0/1382949226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 msgr2=0x558774a515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a515d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0985a0 tx=0x7f58cc09c280 comp rx=0 tx=0).stop 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 -- 172.21.15.94:0/1382949226 shutdown_connections 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558774a63f90 0x558774a66430 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a515d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a50fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58ba7fc640 0 client.17556 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 --2- 172.21.15.94:0/1382949226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 0x558774a53d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 -- 172.21.15.94:0/1382949226 >> 172.21.15.94:0/1382949226 conn(0x558774a12310 msgr2=0x558774a4b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:13.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 -- 172.21.15.94:0/1382949226 shutdown_connections 2024-09-19T04:32:13.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 1 -- 172.21.15.94:0/1382949226 wait complete. 2024-09-19T04:32:13.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.772+0000 7f58db7fc740 20 client.17556 trim_cache size 0 max 16384 2024-09-19T04:32:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.785+0000 7f58db7fc740 1 Processor -- start 2024-09-19T04:32:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.785+0000 7f58db7fc740 1 -- start start 2024-09-19T04:32:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.785+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 0x558774a52f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.785+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a4c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.785+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a4c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.785+0000 7f58db7fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41b02b0 con 0x558774a54370 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.785+0000 7f58db7fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41adb80 con 0x558774a4ab40 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.785+0000 7f58db7fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41ab9f0 con 0x558774a495d0 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a4c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a4c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34764/0 (socket says 172.21.15.94:34764) 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d37fe640 1 -- 172.21.15.94:0/4155485229 learned_addr learned my addr 172.21.15.94:0/4155485229 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 0x558774a52f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a4c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d37fe640 1 -- 172.21.15.94:0/4155485229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 msgr2=0x558774a52f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d37fe640 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 0x558774a52f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d37fe640 1 -- 172.21.15.94:0/4155485229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 msgr2=0x558774a4c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d37fe640 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a4c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d37fe640 1 -- 172.21.15.94:0/4155485229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558774a53950 con 0x558774a54370 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a4c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58d37fe640 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a4c660 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0054c0 tx=0x7f58cc09bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd2787db2827063 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58c8ff9640 1 -- 172.21.15.94:0/4155485229 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc01f070 con 0x558774a54370 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58c8ff9640 1 -- 172.21.15.94:0/4155485229 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58cc0a00c0 con 0x558774a54370 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.786+0000 7f58c8ff9640 1 -- 172.21.15.94:0/4155485229 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc098210 con 0x558774a54370 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 -- 172.21.15.94:0/4155485229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 msgr2=0x558774a4c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a4c660 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0054c0 tx=0x7f58cc09bd00 comp rx=0 tx=0).stop 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 -- 172.21.15.94:0/4155485229 shutdown_connections 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a54370 0x558774a4c660 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a4c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 --2- 172.21.15.94:0/4155485229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a495d0 0x558774a52f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 -- 172.21.15.94:0/4155485229 >> 172.21.15.94:0/4155485229 conn(0x558774a12310 msgr2=0x558774a52840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 -- 172.21.15.94:0/4155485229 shutdown_connections 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 -- 172.21.15.94:0/4155485229 wait complete. 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.787+0000 7f58db7fc740 1 -- start start 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58db7fc740 1 Processor -- start 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a58060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58db7fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 0x558774a587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58db7fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41ac040 con 0x558774a495d0 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58db7fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41ad4b0 con 0x558774a4ab40 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58db7fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41ac710 con 0x558774a54370 2024-09-19T04:32:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58db7fc740 10 client.0 _ll_register_callbacks cb 0x7f58cc0057e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:13.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58d2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58d2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a5cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34772/0 (socket says 172.21.15.94:34772) 2024-09-19T04:32:13.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58d2ffd640 1 -- 172.21.15.94:0/146953321 learned_addr learned my addr 172.21.15.94:0/146953321 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:13.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58d37fe640 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 0x558774a587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a58060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58d2ffd640 1 -- 172.21.15.94:0/146953321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 msgr2=0x558774a587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:13.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 0x558774a587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58d2ffd640 1 -- 172.21.15.94:0/146953321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 msgr2=0x558774a58060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:13.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a58060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.788+0000 7f58d2ffd640 1 -- 172.21.15.94:0/146953321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558774acdd20 con 0x558774a495d0 2024-09-19T04:32:13.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58d2ffd640 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a5cd30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f58c4011010 tx=0x7f58c40051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab58460db39ba57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:13.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:13.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58ba7fc640 1 -- 172.21.15.94:0/146953321 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c40035f0 con 0x558774a495d0 2024-09-19T04:32:13.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58ba7fc640 1 -- 172.21.15.94:0/146953321 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f58c4003790 con 0x558774a495d0 2024-09-19T04:32:13.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58db7fc740 1 -- 172.21.15.94:0/146953321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58cc0a00c0 con 0x558774a495d0 2024-09-19T04:32:13.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58ba7fc640 1 -- 172.21.15.94:0/146953321 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f58c4003990 con 0x558774a495d0 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58db7fc740 10 client.17562 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58db7fc740 1 -- 172.21.15.94:0/146953321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558774acee80 con 0x558774a495d0 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58ba7fc640 1 -- 172.21.15.94:0/146953321 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58cc0a00c0 con 0x558774a495d0 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58ba7fc640 1 -- 172.21.15.94:0/146953321 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f58c4011a10 con 0x558774a495d0 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58db7fc740 10 client.17562 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58db7fc740 10 client.17562 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58db7fc740 20 client.17562 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58db7fc740 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558774acf2d0 0x558774ad16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58db7fc740 4 client.17562 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.789+0000 7f58db7fc740 1 -- 172.21.15.94:0/146953321 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587749b6d70 con 0x558774acf2d0 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558774acf2d0 0x558774ad16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58ca7fc640 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558774acf2d0 0x558774ad16c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4caf1af44569fbc9 server_cookie=c2f1b7290a9c454e in_seq=0 out_seq=0 2024-09-19T04:32:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58ba7fc640 10 client.17562 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58ba7fc640 1 -- 172.21.15.94:0/146953321 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587749b6d70 con 0x558774acf2d0 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58ba7fc640 10 client.17562 handle_command_reply: tid=0 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 client.17562 shutdown 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 -- 172.21.15.94:0/146953321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 msgr2=0x558774a5cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a5cd30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f58c4011010 tx=0x7f58c40051a0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 -- 172.21.15.94:0/146953321 shutdown_connections 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x558774acf2d0 0x558774ad16c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558774a54370 0x558774a587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558774a4ab40 0x558774a58060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 --2- 172.21.15.94:0/146953321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558774a495d0 0x558774a5cd30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58ba7fc640 0 client.17562 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:32:13.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 -- 172.21.15.94:0/146953321 >> 172.21.15.94:0/146953321 conn(0x558774a12310 msgr2=0x558774a53060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:13.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 -- 172.21.15.94:0/146953321 shutdown_connections 2024-09-19T04:32:13.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 1 -- 172.21.15.94:0/146953321 wait complete. 2024-09-19T04:32:13.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.790+0000 7f58db7fc740 20 client.17562 trim_cache size 0 max 16384 2024-09-19T04:32:13.804 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 -- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f58c00a33b0 msgr2=0x7f58c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f58c00a33b0 0x7f58c00a5870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58bc004770 tx=0x7f58bc005fd0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 -- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4108290 msgr2=0x7f58d419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4108290 0x7f58d419ce20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00eaa0 tx=0x7f58cc00ef60 comp rx=0 tx=0).stop 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 -- 172.21.15.94:0/777591914 shutdown_connections 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f58c00a33b0 0x7f58c00a5870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d4108290 0x7f58d419ce20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d4103740 0x7f58d4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 --2- 172.21.15.94:0/777591914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58d4102d90 0x7f58d4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:13.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 -- 172.21.15.94:0/777591914 >> 172.21.15.94:0/777591914 conn(0x7f58d40fc7d0 msgr2=0x7f58d4105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:13.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 -- 172.21.15.94:0/777591914 shutdown_connections 2024-09-19T04:32:13.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:13.791+0000 7f58d9423640 1 -- 172.21.15.94:0/777591914 wait complete. 2024-09-19T04:32:14.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:13 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:14.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:13 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:14.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:13 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:14.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:32:14.774 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:15.011 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:14 smithi094 ceph-mon[30972]: pgmap v541: 97 pgs: 97 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-09-19T04:32:15.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:14 smithi112 ceph-mon[35051]: pgmap v541: 97 pgs: 97 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-09-19T04:32:15.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:14 smithi104 ceph-mon[34145]: pgmap v541: 97 pgs: 97 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-09-19T04:32:15.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.213+0000 7f515bf84640 1 -- 172.21.15.94:0/2391745460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154100a60 msgr2=0x7f5154100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.213+0000 7f515bf84640 1 --2- 172.21.15.94:0/2391745460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154100a60 0x7f5154100ec0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5144009930 tx=0x7f514402a420 comp rx=0 tx=0).stop 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.213+0000 7f515bf84640 1 -- 172.21.15.94:0/2391745460 shutdown_connections 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.213+0000 7f515bf84640 1 --2- 172.21.15.94:0/2391745460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5154101400 0x7f515410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.213+0000 7f515bf84640 1 --2- 172.21.15.94:0/2391745460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154100a60 0x7f5154100ec0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.213+0000 7f515bf84640 1 --2- 172.21.15.94:0/2391745460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5154106a60 0x7f5154106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.213+0000 7f515bf84640 1 -- 172.21.15.94:0/2391745460 >> 172.21.15.94:0/2391745460 conn(0x7f51540fc780 msgr2=0x7f51540feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.213+0000 7f515bf84640 1 -- 172.21.15.94:0/2391745460 shutdown_connections 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.213+0000 7f515bf84640 1 -- 172.21.15.94:0/2391745460 wait complete. 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515bf84640 1 Processor -- start 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515bf84640 1 -- start start 2024-09-19T04:32:15.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515bf84640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5154100a60 0x7f51541972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515bf84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5154101400 0x7f5154197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515bf84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154106a60 0x7f515419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515bf84640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5154110d10 con 0x7f5154106a60 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515bf84640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5154110b90 con 0x7f5154100a60 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515bf84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5154110e90 con 0x7f5154101400 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515a521640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154106a60 0x7f515419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515a521640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154106a60 0x7f515419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34802/0 (socket says 172.21.15.94:34802) 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515a521640 1 -- 172.21.15.94:0/2478948497 learned_addr learned my addr 172.21.15.94:0/2478948497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515951f640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5154101400 0x7f5154197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f5159d20640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5154100a60 0x7f51541972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515a521640 1 -- 172.21.15.94:0/2478948497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5154101400 msgr2=0x7f5154197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:15.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515a521640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5154101400 0x7f5154197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515a521640 1 -- 172.21.15.94:0/2478948497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5154100a60 msgr2=0x7f51541972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515a521640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5154100a60 0x7f51541972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.214+0000 7f515a521640 1 -- 172.21.15.94:0/2478948497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515419c280 con 0x7f5154106a60 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.215+0000 7f515a521640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154106a60 0x7f515419bba0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f514c00ebb0 tx=0x7f514c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.215+0000 7f514affd640 1 -- 172.21.15.94:0/2478948497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c007ca0 con 0x7f5154106a60 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.215+0000 7f514affd640 1 -- 172.21.15.94:0/2478948497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f514c007e40 con 0x7f5154106a60 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.215+0000 7f515bf84640 1 -- 172.21.15.94:0/2478948497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515419c510 con 0x7f5154106a60 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.215+0000 7f514affd640 1 -- 172.21.15.94:0/2478948497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c0057d0 con 0x7f5154106a60 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.215+0000 7f515bf84640 1 -- 172.21.15.94:0/2478948497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515406a790 con 0x7f5154106a60 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.217+0000 7f514affd640 1 -- 172.21.15.94:0/2478948497 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f514c004a40 con 0x7f5154106a60 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.217+0000 7f515e35d740 1 Processor -- start 2024-09-19T04:32:15.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515e35d740 1 -- start start 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515e35d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db7844c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515e35d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db7844ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515e35d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 0x55db78459760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515e35d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db783b22f0 con 0x55db7844db40 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515e35d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db78359cf0 con 0x55db78457370 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515e35d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db783ae2b0 con 0x55db7844c5d0 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db7844c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db7844c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34804/0 (socket says 172.21.15.94:34804) 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 -- 172.21.15.94:0/2530186041 learned_addr learned my addr 172.21.15.94:0/2530186041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f5159d20640 1 --2- 172.21.15.94:0/2530186041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db7844ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f514affd640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51400a3430 0x7f51400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 --2- 172.21.15.94:0/2530186041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 0x55db78459760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 -- 172.21.15.94:0/2530186041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 msgr2=0x55db7844ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 --2- 172.21.15.94:0/2530186041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db7844ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 -- 172.21.15.94:0/2530186041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 msgr2=0x55db78459760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 --2- 172.21.15.94:0/2530186041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 0x55db78459760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 -- 172.21.15.94:0/2530186041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db7833e3a0 con 0x55db7844db40 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515a521640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51400a3430 0x7f51400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f514affd640 1 -- 172.21.15.94:0/2478948497 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f514c0d11d0 con 0x7f5154106a60 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.218+0000 7f515951f640 1 --2- 172.21.15.94:0/2530186041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db7844c090 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5144031fd0 tx=0x7f5144005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1122bc423a60844e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f513ffff640 1 -- 172.21.15.94:0/2530186041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144044030 con 0x55db7844db40 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f513ffff640 1 -- 172.21.15.94:0/2530186041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51440316e0 con 0x55db7844db40 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f513ffff640 1 -- 172.21.15.94:0/2530186041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144035040 con 0x55db7844db40 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f515e35d740 1 -- 172.21.15.94:0/2530186041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 msgr2=0x55db7844c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:15.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f515a521640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51400a3430 0x7f51400a58f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f514c0ce990 tx=0x7f514c070350 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:15.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f515e35d740 1 --2- 172.21.15.94:0/2530186041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db7844c090 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5144031fd0 tx=0x7f5144005fd0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:32:15.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f515e35d740 1 -- 172.21.15.94:0/2530186041 shutdown_connections 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f515e35d740 1 --2- 172.21.15.94:0/2530186041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 0x55db78459760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f515e35d740 1 --2- 172.21.15.94:0/2530186041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db7844ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f515e35d740 1 --2- 172.21.15.94:0/2530186041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db7844c090 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.219+0000 7f515e35d740 1 -- 172.21.15.94:0/2530186041 >> 172.21.15.94:0/2530186041 conn(0x55db78415310 msgr2=0x55db78448c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 -- 172.21.15.94:0/2530186041 shutdown_connections 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 -- 172.21.15.94:0/2530186041 wait complete. 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 -- start start 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 Processor -- start 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db78456df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 0x55db784540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db784547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db7845a010 con 0x55db78457370 2024-09-19T04:32:15.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db7836d060 con 0x55db7844db40 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db7845a190 con 0x55db7844c5d0 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.220+0000 7f515e35d740 10 client.0 _ll_register_callbacks cb 0x55db780c08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db784547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db784547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34820/0 (socket says 172.21.15.94:34820) 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 -- 172.21.15.94:0/455201914 learned_addr learned my addr 172.21.15.94:0/455201914 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f5159d20640 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 0x55db784540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db78456df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 -- 172.21.15.94:0/455201914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 msgr2=0x55db78456df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db78456df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 -- 172.21.15.94:0/455201914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 msgr2=0x55db784540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 0x55db784540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 -- 172.21.15.94:0/455201914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db781233c0 con 0x55db78457370 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515951f640 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db784547e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5144031e40 tx=0x7f5144031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac3390c42a050a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f513d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f513d7fa640 1 -- 172.21.15.94:0/455201914 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144035040 con 0x55db78457370 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f513d7fa640 1 -- 172.21.15.94:0/455201914 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f5144004460 con 0x55db78457370 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515e35d740 1 -- 172.21.15.94:0/455201914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51440316e0 con 0x55db78457370 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f513d7fa640 1 -- 172.21.15.94:0/455201914 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f51440035f0 con 0x55db78457370 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515e35d740 10 client.17574 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f515e35d740 1 -- 172.21.15.94:0/455201914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55db78465b40 con 0x55db78457370 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.221+0000 7f513d7fa640 1 -- 172.21.15.94:0/455201914 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51440316e0 con 0x55db78457370 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f513d7fa640 1 -- 172.21.15.94:0/455201914 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f514404a6d0 con 0x55db78457370 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f515e35d740 10 client.17574 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f515e35d740 10 client.17574 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:32:15.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f515e35d740 20 client.17574 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:15.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f515e35d740 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db78466e90 0x55db78469280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f515e35d740 4 client.17574 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:15.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f515e35d740 1 -- 172.21.15.94:0/455201914 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db783bf860 con 0x55db78466e90 2024-09-19T04:32:15.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f5159d20640 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db78466e90 0x55db78469280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f513d7fa640 10 client.17574 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:32:15.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.222+0000 7f5159d20640 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db78466e90 0x55db78469280 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=766d70e8bc6ba013 server_cookie=c2e285d0bff7b829 in_seq=0 out_seq=0 2024-09-19T04:32:15.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f513d7fa640 1 -- 172.21.15.94:0/455201914 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db783bf860 con 0x55db78466e90 2024-09-19T04:32:15.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f513d7fa640 10 client.17574 handle_command_reply: tid=0 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 client.17574 shutdown 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 -- 172.21.15.94:0/455201914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 msgr2=0x55db784547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db784547e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5144031e40 tx=0x7f5144031c60 comp rx=0 tx=0).stop 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 -- 172.21.15.94:0/455201914 shutdown_connections 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db78466e90 0x55db78469280 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db784547e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 0x55db784540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f513d7fa640 0 client.17574 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 --2- 172.21.15.94:0/455201914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db78456df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 -- 172.21.15.94:0/455201914 >> 172.21.15.94:0/455201914 conn(0x55db78415310 msgr2=0x55db7844e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 -- 172.21.15.94:0/455201914 shutdown_connections 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 1 -- 172.21.15.94:0/455201914 wait complete. 2024-09-19T04:32:15.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.223+0000 7f515e35d740 20 client.17574 trim_cache size 0 max 16384 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515e35d740 1 Processor -- start 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515e35d740 1 -- start start 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515e35d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db78453320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515e35d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db784484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515e35d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 0x55db78448a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515e35d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51541adf50 con 0x55db7844db40 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515e35d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f51541ab820 con 0x55db78457370 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515e35d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51541a9690 con 0x55db7844c5d0 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f5159d20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db784484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f5159d20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db784484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34834/0 (socket says 172.21.15.94:34834) 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f5159d20640 1 -- 172.21.15.94:0/769086642 learned_addr learned my addr 172.21.15.94:0/769086642 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515951f640 1 --2- 172.21.15.94:0/769086642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db78453320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f5159d20640 1 -- 172.21.15.94:0/769086642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 msgr2=0x55db78453320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f5159d20640 1 --2- 172.21.15.94:0/769086642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db78453320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f5159d20640 1 -- 172.21.15.94:0/769086642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 msgr2=0x55db78448a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f515951f640 1 --2- 172.21.15.94:0/769086642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 0x55db78448a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f5159d20640 1 --2- 172.21.15.94:0/769086642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 0x55db78448a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.236+0000 7f5159d20640 1 -- 172.21.15.94:0/769086642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db7844f120 con 0x55db7844db40 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f5159d20640 1 --2- 172.21.15.94:0/769086642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db784484f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f515000ace0 tx=0x7f515000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f631041c3f4daed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f513f7fe640 1 -- 172.21.15.94:0/769086642 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150005220 con 0x55db7844db40 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f513f7fe640 1 -- 172.21.15.94:0/769086642 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5150004c50 con 0x55db7844db40 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f513f7fe640 1 -- 172.21.15.94:0/769086642 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515001e820 con 0x55db7844db40 2024-09-19T04:32:15.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f515e35d740 1 -- 172.21.15.94:0/769086642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 msgr2=0x55db784484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f515e35d740 1 --2- 172.21.15.94:0/769086642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db784484f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f515000ace0 tx=0x7f515000a2d0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f515e35d740 1 -- 172.21.15.94:0/769086642 shutdown_connections 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f515e35d740 1 --2- 172.21.15.94:0/769086642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db78457370 0x55db78448a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f515e35d740 1 --2- 172.21.15.94:0/769086642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db7844db40 0x55db784484f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f515e35d740 1 --2- 172.21.15.94:0/769086642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db78453320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.237+0000 7f515e35d740 1 -- 172.21.15.94:0/769086642 >> 172.21.15.94:0/769086642 conn(0x55db78415310 msgr2=0x55db78456120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 -- 172.21.15.94:0/769086642 shutdown_connections 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 -- 172.21.15.94:0/769086642 wait complete. 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 -- start start 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 Processor -- start 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db7846a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 0x55db7846a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db7846ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51541a9ce0 con 0x55db78457370 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f51541ab150 con 0x55db7844db40 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51541aa3b0 con 0x55db7844c5d0 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515e35d740 10 client.0 _ll_register_callbacks cb 0x7f515000a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515951f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db7846ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515951f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db7846ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34848/0 (socket says 172.21.15.94:34848) 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.238+0000 7f515951f640 1 -- 172.21.15.94:0/842540667 learned_addr learned my addr 172.21.15.94:0/842540667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515951f640 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db7846a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f5159d20640 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 0x55db7846a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515951f640 1 -- 172.21.15.94:0/842540667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 msgr2=0x55db7846a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515951f640 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db7846a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515951f640 1 -- 172.21.15.94:0/842540667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 msgr2=0x55db7846a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515951f640 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 0x55db7846a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515951f640 1 -- 172.21.15.94:0/842540667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db784d0ac0 con 0x55db78457370 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f5159d20640 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 0x55db7846a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515951f640 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db7846ae00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f51440095e0 tx=0x7f5144031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24de011042ecfb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f513d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f513d7fa640 1 -- 172.21.15.94:0/842540667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144044020 con 0x55db78457370 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f513d7fa640 1 -- 172.21.15.94:0/842540667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f51440318e0 con 0x55db78457370 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515e35d740 1 -- 172.21.15.94:0/842540667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5150004c50 con 0x55db78457370 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f513d7fa640 1 -- 172.21.15.94:0/842540667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5144035040 con 0x55db78457370 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515e35d740 10 client.17580 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.239+0000 7f515e35d740 1 -- 172.21.15.94:0/842540667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55db784d1dc0 con 0x55db78457370 2024-09-19T04:32:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f513d7fa640 1 -- 172.21.15.94:0/842540667 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5150004c50 con 0x55db78457370 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f513d7fa640 1 -- 172.21.15.94:0/842540667 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f514404a450 con 0x55db78457370 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f515e35d740 10 client.17580 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f515e35d740 10 client.17580 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f515e35d740 20 client.17580 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f515e35d740 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db784d24e0 0x55db784d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f515e35d740 4 client.17580 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f515e35d740 1 -- 172.21.15.94:0/842540667 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db783b9d70 con 0x55db784d24e0 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f5159d20640 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db784d24e0 0x55db784d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f513d7fa640 10 client.17580 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f5159d20640 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db784d24e0 0x55db784d4900 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11264c938e41f061 server_cookie=b2ceef5b944fc78d in_seq=0 out_seq=0 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f513d7fa640 1 -- 172.21.15.94:0/842540667 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db783b9d70 con 0x55db784d24e0 2024-09-19T04:32:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f513d7fa640 10 client.17580 handle_command_reply: tid=0 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.240+0000 7f515e35d740 1 client.17580 shutdown 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 -- 172.21.15.94:0/842540667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 msgr2=0x55db7846ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db7846ae00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f51440095e0 tx=0x7f5144031c60 comp rx=0 tx=0).stop 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 -- 172.21.15.94:0/842540667 shutdown_connections 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x55db784d24e0 0x55db784d4900 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db78457370 0x55db7846ae00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db7844db40 0x55db7846a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 --2- 172.21.15.94:0/842540667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db7844c5d0 0x55db7846a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 -- 172.21.15.94:0/842540667 >> 172.21.15.94:0/842540667 conn(0x55db78415310 msgr2=0x55db7844e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f513d7fa640 0 client.17580 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 -- 172.21.15.94:0/842540667 shutdown_connections 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 1 -- 172.21.15.94:0/842540667 wait complete. 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515e35d740 20 client.17580 trim_cache size 0 max 16384 2024-09-19T04:32:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 -- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51400a3430 msgr2=0x7f51400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51400a3430 0x7f51400a58f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f514c0ce990 tx=0x7f514c070350 comp rx=0 tx=0).stop 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 -- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154106a60 msgr2=0x7f515419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154106a60 0x7f515419bba0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f514c00ebb0 tx=0x7f514c007590 comp rx=0 tx=0).stop 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 -- 172.21.15.94:0/2478948497 shutdown_connections 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f51400a3430 0x7f51400a58f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5154106a60 0x7f515419bba0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5154101400 0x7f5154197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 --2- 172.21.15.94:0/2478948497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5154100a60 0x7f51541972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.241+0000 7f515bf84640 1 -- 172.21.15.94:0/2478948497 >> 172.21.15.94:0/2478948497 conn(0x7f51540fc780 msgr2=0x7f51540fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.242+0000 7f515bf84640 1 -- 172.21.15.94:0/2478948497 shutdown_connections 2024-09-19T04:32:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:15.242+0000 7f515bf84640 1 -- 172.21.15.94:0/2478948497 wait complete. 2024-09-19T04:32:16.507 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:32:16.758 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:17.025 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:16 smithi094 ceph-mon[30972]: pgmap v542: 97 pgs: 97 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-09-19T04:32:17.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:16 smithi112 ceph-mon[35051]: pgmap v542: 97 pgs: 97 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-09-19T04:32:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:16 smithi104 ceph-mon[34145]: pgmap v542: 97 pgs: 97 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-09-19T04:32:17.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.228+0000 7f54653b6640 1 -- 172.21.15.94:0/548460227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5460102d30 msgr2=0x7f5460103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:17.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.228+0000 7f54653b6640 1 --2- 172.21.15.94:0/548460227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5460102d30 0x7f5460103110 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5450009930 tx=0x7f545002a420 comp rx=0 tx=0).stop 2024-09-19T04:32:17.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 -- 172.21.15.94:0/548460227 shutdown_connections 2024-09-19T04:32:17.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 --2- 172.21.15.94:0/548460227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54601082a0 0x7f546010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 --2- 172.21.15.94:0/548460227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54601036e0 0x7f5460107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 --2- 172.21.15.94:0/548460227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5460102d30 0x7f5460103110 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 -- 172.21.15.94:0/548460227 >> 172.21.15.94:0/548460227 conn(0x7f54600fc7d0 msgr2=0x7f54600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 -- 172.21.15.94:0/548460227 shutdown_connections 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 -- 172.21.15.94:0/548460227 wait complete. 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 Processor -- start 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 -- start start 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5460102d30 0x7f5460198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54601036e0 0x7f54601989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.229+0000 7f54653b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601082a0 0x7f546019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f54653b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5460111f50 con 0x7f54601082a0 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f54653b6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5460111dd0 con 0x7f5460102d30 2024-09-19T04:32:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f54653b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54601120d0 con 0x7f54601036e0 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601082a0 0x7f546019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601082a0 0x7f546019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34862/0 (socket says 172.21.15.94:34862) 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5460102d30 0x7f5460198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545f7fe640 1 -- 172.21.15.94:0/4294675516 learned_addr learned my addr 172.21.15.94:0/4294675516 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f54567fc640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54601036e0 0x7f54601989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545f7fe640 1 -- 172.21.15.94:0/4294675516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54601036e0 msgr2=0x7f54601989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545f7fe640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54601036e0 0x7f54601989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545f7fe640 1 -- 172.21.15.94:0/4294675516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5460102d30 msgr2=0x7f5460198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545f7fe640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5460102d30 0x7f5460198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545f7fe640 1 -- 172.21.15.94:0/4294675516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546019d420 con 0x7f54601082a0 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545f7fe640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601082a0 0x7f546019cd40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f545800cad0 tx=0x7f545800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545cff9640 1 -- 172.21.15.94:0/4294675516 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545800ad60 con 0x7f54601082a0 2024-09-19T04:32:17.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545cff9640 1 -- 172.21.15.94:0/4294675516 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5458005020 con 0x7f54601082a0 2024-09-19T04:32:17.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f54653b6640 1 -- 172.21.15.94:0/4294675516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546019d710 con 0x7f54601082a0 2024-09-19T04:32:17.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f545cff9640 1 -- 172.21.15.94:0/4294675516 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545800f6b0 con 0x7f54601082a0 2024-09-19T04:32:17.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.230+0000 7f54653b6640 1 -- 172.21.15.94:0/4294675516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54601a4ff0 con 0x7f54601082a0 2024-09-19T04:32:17.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.233+0000 7f545cff9640 1 -- 172.21.15.94:0/4294675516 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f54580051d0 con 0x7f54601082a0 2024-09-19T04:32:17.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.233+0000 7f546778f740 1 Processor -- start 2024-09-19T04:32:17.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.233+0000 7f546778f740 1 -- start start 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f546778f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f545cff9640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f544c0a33b0 0x7f544c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f546778f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f546778f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f546778f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558838b9d2f0 con 0x558838c375d0 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f546778f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558838b44cf0 con 0x558838c42370 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f546778f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558838b992b0 con 0x558838c38b40 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f545effd640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f544c0a33b0 0x7f544c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f54567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f54567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c37a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34866/0 (socket says 172.21.15.94:34866) 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f54567fc640 1 -- 172.21.15.94:0/40294981 learned_addr learned my addr 172.21.15.94:0/40294981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f545cff9640 1 -- 172.21.15.94:0/4294675516 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f54580cfcf0 con 0x7f54601082a0 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f54567fc640 1 --2- 172.21.15.94:0/40294981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f545f7fe640 1 --2- 172.21.15.94:0/40294981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f54567fc640 1 -- 172.21.15.94:0/40294981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 msgr2=0x558838c37090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f54567fc640 1 --2- 172.21.15.94:0/40294981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c37090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f54567fc640 1 -- 172.21.15.94:0/40294981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 msgr2=0x558838c44760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f54567fc640 1 --2- 172.21.15.94:0/40294981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c44760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f54567fc640 1 -- 172.21.15.94:0/40294981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558838b293a0 con 0x558838c375d0 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.234+0000 7f545effd640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f544c0a33b0 0x7f544c0a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5450006010 tx=0x7f5450000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:17.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.235+0000 7f54567fc640 1 --2- 172.21.15.94:0/40294981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c37a10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5448011780 tx=0x7f544800a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdc02fd0b2367c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.235+0000 7f54557fa640 1 -- 172.21.15.94:0/40294981 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448005020 con 0x558838c375d0 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.235+0000 7f54557fa640 1 -- 172.21.15.94:0/40294981 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54480047a0 con 0x558838c375d0 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.235+0000 7f54557fa640 1 -- 172.21.15.94:0/40294981 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800fa20 con 0x558838c375d0 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.235+0000 7f546778f740 1 -- 172.21.15.94:0/40294981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 msgr2=0x558838c37a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.235+0000 7f546778f740 1 --2- 172.21.15.94:0/40294981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c37a10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5448011780 tx=0x7f544800a700 comp rx=0 tx=0).stop 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 -- 172.21.15.94:0/40294981 shutdown_connections 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 --2- 172.21.15.94:0/40294981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c44760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 --2- 172.21.15.94:0/40294981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c37a10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 --2- 172.21.15.94:0/40294981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c37090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 -- 172.21.15.94:0/40294981 >> 172.21.15.94:0/40294981 conn(0x558838c00310 msgr2=0x558838c33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 -- 172.21.15.94:0/40294981 shutdown_connections 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 -- 172.21.15.94:0/40294981 wait complete. 2024-09-19T04:32:17.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 -- start start 2024-09-19T04:32:17.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 Processor -- start 2024-09-19T04:32:17.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c3f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c3f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558838c45070 con 0x558838c375d0 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558838b58060 con 0x558838c42370 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.236+0000 7f546778f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558838c451f0 con 0x558838c38b40 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f546778f740 10 client.0 _ll_register_callbacks cb 0x5588388ab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34870/0 (socket says 172.21.15.94:34870) 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545effd640 1 -- 172.21.15.94:0/4014657108 learned_addr learned my addr 172.21.15.94:0/4014657108 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f54567fc640 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c3f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545f7fe640 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c3f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545effd640 1 -- 172.21.15.94:0/4014657108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 msgr2=0x558838c3f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545effd640 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c3f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545effd640 1 -- 172.21.15.94:0/4014657108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 msgr2=0x558838c3f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545effd640 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c3f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545effd640 1 -- 172.21.15.94:0/4014657108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883890e3c0 con 0x558838c375d0 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f545effd640 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41de0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5450033000 tx=0x7f5450009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351cb6bd8bd0850e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f54467fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f54467fc640 1 -- 172.21.15.94:0/4014657108 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545004c030 con 0x558838c375d0 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f54467fc640 1 -- 172.21.15.94:0/4014657108 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f5450005020 con 0x558838c375d0 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f546778f740 1 -- 172.21.15.94:0/4014657108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54480047a0 con 0x558838c375d0 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f54467fc640 1 -- 172.21.15.94:0/4014657108 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5450058050 con 0x558838c375d0 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f546778f740 10 client.17592 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:17.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.237+0000 7f546778f740 1 -- 172.21.15.94:0/4014657108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558838c50ba0 con 0x558838c375d0 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.238+0000 7f54467fc640 1 -- 172.21.15.94:0/4014657108 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54480047a0 con 0x558838c375d0 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.238+0000 7f54467fc640 1 -- 172.21.15.94:0/4014657108 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f54500047a0 con 0x558838c375d0 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.238+0000 7f546778f740 10 client.17592 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.238+0000 7f546778f740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.238+0000 7f546778f740 20 client.17592 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.238+0000 7f546778f740 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558838c52070 0x558838c54510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.238+0000 7f546778f740 4 client.17592 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.238+0000 7f546778f740 1 -- 172.21.15.94:0/4014657108 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558838baa860 con 0x558838c52070 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.238+0000 7f54567fc640 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558838c52070 0x558838c54510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.239+0000 7f54467fc640 10 client.17592 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.239+0000 7f54567fc640 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558838c52070 0x558838c54510 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=647d071006c2e9f2 server_cookie=14681c662ca0be11 in_seq=0 out_seq=0 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.239+0000 7f54467fc640 1 -- 172.21.15.94:0/4014657108 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558838baa860 con 0x558838c52070 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.239+0000 7f54467fc640 10 client.17592 handle_command_reply: tid=0 2024-09-19T04:32:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 client.17592 shutdown 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 -- 172.21.15.94:0/4014657108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 msgr2=0x558838c41de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41de0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5450033000 tx=0x7f5450009260 comp rx=0 tx=0).stop 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 -- 172.21.15.94:0/4014657108 shutdown_connections 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558838c52070 0x558838c54510 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c3f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c3f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 --2- 172.21.15.94:0/4014657108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41de0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f54467fc640 0 client.17592 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 -- 172.21.15.94:0/4014657108 >> 172.21.15.94:0/4014657108 conn(0x558838c00310 msgr2=0x558838c39960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 -- 172.21.15.94:0/4014657108 shutdown_connections 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 1 -- 172.21.15.94:0/4014657108 wait complete. 2024-09-19T04:32:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.240+0000 7f546778f740 20 client.17592 trim_cache size 0 max 16384 2024-09-19T04:32:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f546778f740 1 Processor -- start 2024-09-19T04:32:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f546778f740 1 -- start start 2024-09-19T04:32:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f546778f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f546778f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c41540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f546778f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f546778f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54601b0180 con 0x558838c375d0 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f546778f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f54601ada50 con 0x558838c42370 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f546778f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54601ab8c0 con 0x558838c38b40 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f545effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f545effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34872/0 (socket says 172.21.15.94:34872) 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f545effd640 1 -- 172.21.15.94:0/1176987973 learned_addr learned my addr 172.21.15.94:0/1176987973 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f54567fc640 1 --2- 172.21.15.94:0/1176987973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c41540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f545effd640 1 -- 172.21.15.94:0/1176987973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 msgr2=0x558838c41540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f545f7fe640 1 --2- 172.21.15.94:0/1176987973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f545effd640 1 --2- 172.21.15.94:0/1176987973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c41540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f545effd640 1 -- 172.21.15.94:0/1176987973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 msgr2=0x558838c3a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f545effd640 1 --2- 172.21.15.94:0/1176987973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c3a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.253+0000 7f545effd640 1 -- 172.21.15.94:0/1176987973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558838c32bc0 con 0x558838c375d0 2024-09-19T04:32:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f545effd640 1 --2- 172.21.15.94:0/1176987973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41000 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5450004bf0 tx=0x7f54500314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1060d7ceca6da14a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f5454ff9640 1 -- 172.21.15.94:0/1176987973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450061070 con 0x558838c375d0 2024-09-19T04:32:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f5454ff9640 1 -- 172.21.15.94:0/1176987973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54500054b0 con 0x558838c375d0 2024-09-19T04:32:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f5454ff9640 1 -- 172.21.15.94:0/1176987973 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545004c040 con 0x558838c375d0 2024-09-19T04:32:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f546778f740 1 -- 172.21.15.94:0/1176987973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 msgr2=0x558838c41000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f546778f740 1 --2- 172.21.15.94:0/1176987973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41000 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5450004bf0 tx=0x7f54500314a0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f546778f740 1 -- 172.21.15.94:0/1176987973 shutdown_connections 2024-09-19T04:32:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f546778f740 1 --2- 172.21.15.94:0/1176987973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c42370 0x558838c3a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f546778f740 1 --2- 172.21.15.94:0/1176987973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c38b40 0x558838c41540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f546778f740 1 --2- 172.21.15.94:0/1176987973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c375d0 0x558838c41000 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f546778f740 1 -- 172.21.15.94:0/1176987973 >> 172.21.15.94:0/1176987973 conn(0x558838c00310 msgr2=0x558838c408f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.254+0000 7f546778f740 1 -- 172.21.15.94:0/1176987973 shutdown_connections 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 1 -- 172.21.15.94:0/1176987973 wait complete. 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 1 -- start start 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 1 Processor -- start 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c375d0 0x558838c46260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c38b40 0x558838c467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c42370 0x558838c46ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54601abf10 con 0x558838c38b40 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f54601ad380 con 0x558838c375d0 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54601ac5e0 con 0x558838c42370 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f546778f740 10 client.0 _ll_register_callbacks cb 0x7f5450040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f54567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c38b40 0x558838c467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f54567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c38b40 0x558838c467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34888/0 (socket says 172.21.15.94:34888) 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f54567fc640 1 -- 172.21.15.94:0/2152403218 learned_addr learned my addr 172.21.15.94:0/2152403218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f545effd640 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c375d0 0x558838c46260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f54567fc640 1 -- 172.21.15.94:0/2152403218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c42370 msgr2=0x558838c46ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f54567fc640 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c42370 0x558838c46ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f54567fc640 1 -- 172.21.15.94:0/2152403218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c375d0 msgr2=0x558838c46260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.255+0000 7f54567fc640 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c375d0 0x558838c46260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f54567fc640 1 -- 172.21.15.94:0/2152403218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558838c44ad0 con 0x558838c38b40 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f545effd640 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c375d0 0x558838c46260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f54567fc640 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c38b40 0x558838c467a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x558838c34ac0 tx=0x7f5448005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8023029a7475dd7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f54467fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f54467fc640 1 -- 172.21.15.94:0/2152403218 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544802e070 con 0x558838c38b40 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f54467fc640 1 -- 172.21.15.94:0/2152403218 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f5448004d70 con 0x558838c38b40 2024-09-19T04:32:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f546778f740 1 -- 172.21.15.94:0/2152403218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54500054b0 con 0x558838c38b40 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f54467fc640 1 -- 172.21.15.94:0/2152403218 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f544803a050 con 0x558838c38b40 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f546778f740 10 client.17598 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f546778f740 1 -- 172.21.15.94:0/2152403218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558838c3fb40 con 0x558838c38b40 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f54467fc640 1 -- 172.21.15.94:0/2152403218 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54500054b0 con 0x558838c38b40 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f54467fc640 1 -- 172.21.15.94:0/2152403218 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f544800fa80 con 0x558838c38b40 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f546778f740 10 client.17598 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f546778f740 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f546778f740 20 client.17598 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f546778f740 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558838cbd380 0x558838cbf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f546778f740 4 client.17598 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.256+0000 7f546778f740 1 -- 172.21.15.94:0/2152403218 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558838ba4d70 con 0x558838cbd380 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f545effd640 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558838cbd380 0x558838cbf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f545effd640 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558838cbd380 0x558838cbf7a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e43b0e8c33cd72 server_cookie=7da96dbc8622f721 in_seq=0 out_seq=0 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f54467fc640 10 client.17598 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f54467fc640 1 -- 172.21.15.94:0/2152403218 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558838ba4d70 con 0x558838cbd380 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f54467fc640 10 client.17598 handle_command_reply: tid=0 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f546778f740 1 client.17598 shutdown 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f546778f740 1 -- 172.21.15.94:0/2152403218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c38b40 msgr2=0x558838c467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f546778f740 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c38b40 0x558838c467a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x558838c34ac0 tx=0x7f5448005c90 comp rx=0 tx=0).stop 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f546778f740 1 -- 172.21.15.94:0/2152403218 shutdown_connections 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f546778f740 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x558838cbd380 0x558838cbf7a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f54467fc640 0 client.17598 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f546778f740 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558838c42370 0x558838c46ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f546778f740 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558838c38b40 0x558838c467a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f546778f740 1 --2- 172.21.15.94:0/2152403218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558838c375d0 0x558838c46260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.257+0000 7f546778f740 1 -- 172.21.15.94:0/2152403218 >> 172.21.15.94:0/2152403218 conn(0x558838c00310 msgr2=0x558838c41110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f546778f740 1 -- 172.21.15.94:0/2152403218 shutdown_connections 2024-09-19T04:32:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f546778f740 1 -- 172.21.15.94:0/2152403218 wait complete. 2024-09-19T04:32:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f546778f740 20 client.17598 trim_cache size 0 max 16384 2024-09-19T04:32:17.266 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:32:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 -- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f544c0a33b0 msgr2=0x7f544c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f544c0a33b0 0x7f544c0a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5450006010 tx=0x7f5450000bc0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 -- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601082a0 msgr2=0x7f546019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601082a0 0x7f546019cd40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f545800cad0 tx=0x7f545800ced0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 -- 172.21.15.94:0/4294675516 shutdown_connections 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f544c0a33b0 0x7f544c0a5870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601082a0 0x7f546019cd40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54601036e0 0x7f54601989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 --2- 172.21.15.94:0/4294675516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5460102d30 0x7f5460198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 -- 172.21.15.94:0/4294675516 >> 172.21.15.94:0/4294675516 conn(0x7f54600fc7d0 msgr2=0x7f5460105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 -- 172.21.15.94:0/4294675516 shutdown_connections 2024-09-19T04:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:17.258+0000 7f54653b6640 1 -- 172.21.15.94:0/4294675516 wait complete. 2024-09-19T04:32:19.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:18 smithi112 ceph-mon[35051]: pgmap v543: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.7 MiB/s wr, 613 op/s 2024-09-19T04:32:19.057 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:32:19.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:32:19.082 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:18 smithi094 ceph-mon[30972]: pgmap v543: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.7 MiB/s wr, 613 op/s 2024-09-19T04:32:19.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:18 smithi104 ceph-mon[34145]: pgmap v543: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.7 MiB/s wr, 613 op/s 2024-09-19T04:32:19.307 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.798+0000 7f530bb7f640 1 -- 172.21.15.94:0/1827502806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304106f10 msgr2=0x7f530410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.798+0000 7f530bb7f640 1 --2- 172.21.15.94:0/1827502806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304106f10 0x7f530410ea90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009ca0 tx=0x7f52fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.798+0000 7f530bb7f640 1 -- 172.21.15.94:0/1827502806 shutdown_connections 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.798+0000 7f530bb7f640 1 --2- 172.21.15.94:0/1827502806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304106f10 0x7f530410ea90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.798+0000 7f530bb7f640 1 --2- 172.21.15.94:0/1827502806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5304102790 0x7f53041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.798+0000 7f530bb7f640 1 --2- 172.21.15.94:0/1827502806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5304101de0 0x7f53041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.798+0000 7f530bb7f640 1 -- 172.21.15.94:0/1827502806 >> 172.21.15.94:0/1827502806 conn(0x7f53040fd8c0 msgr2=0x7f53040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.798+0000 7f530bb7f640 1 -- 172.21.15.94:0/1827502806 shutdown_connections 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.798+0000 7f530bb7f640 1 -- 172.21.15.94:0/1827502806 wait complete. 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530bb7f640 1 Processor -- start 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530bb7f640 1 -- start start 2024-09-19T04:32:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530bb7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5304101de0 0x7f53040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530bb7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304102790 0x7f5304074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530bb7f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5304106f10 0x7f5304074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530bb7f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5304111e20 con 0x7f5304102790 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530bb7f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5304111ca0 con 0x7f5304106f10 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530bb7f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5304111fa0 con 0x7f5304101de0 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530911a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304102790 0x7f5304074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530911a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304102790 0x7f5304074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34908/0 (socket says 172.21.15.94:34908) 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530911a640 1 -- 172.21.15.94:0/4260401581 learned_addr learned my addr 172.21.15.94:0/4260401581 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530991b640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5304101de0 0x7f53040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530a11c640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5304106f10 0x7f5304074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530911a640 1 -- 172.21.15.94:0/4260401581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5304101de0 msgr2=0x7f53040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530911a640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5304101de0 0x7f53040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530911a640 1 -- 172.21.15.94:0/4260401581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5304106f10 msgr2=0x7f5304074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530911a640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5304106f10 0x7f5304074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530911a640 1 -- 172.21.15.94:0/4260401581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53041a67e0 con 0x7f5304102790 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.799+0000 7f530a11c640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5304106f10 0x7f5304074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.800+0000 7f530911a640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304102790 0x7f5304074a30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c890 tx=0x7f52f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.800+0000 7f52faffd640 1 -- 172.21.15.94:0/4260401581 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4004220 con 0x7f5304102790 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.800+0000 7f530bb7f640 1 -- 172.21.15.94:0/4260401581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53041a6ad0 con 0x7f5304102790 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.800+0000 7f52faffd640 1 -- 172.21.15.94:0/4260401581 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52f40043c0 con 0x7f5304102790 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.800+0000 7f52faffd640 1 -- 172.21.15.94:0/4260401581 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4004a00 con 0x7f5304102790 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.800+0000 7f530bb7f640 1 -- 172.21.15.94:0/4260401581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53041a7010 con 0x7f5304102790 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f52faffd640 1 -- 172.21.15.94:0/4260401581 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f52f4004ba0 con 0x7f5304102790 2024-09-19T04:32:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f530df58740 1 Processor -- start 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f530df58740 1 -- start start 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f530df58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 0x559690acef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f530df58740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ad9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f530df58740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 0x559690adc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f530df58740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559690a34280 con 0x559690acf480 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f530df58740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5596909dbb10 con 0x559690ad9d40 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f530df58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559690a30240 con 0x559690aceb60 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.803+0000 7f530991b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ad9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530991b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ad9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34914/0 (socket says 172.21.15.94:34914) 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530991b640 1 -- 172.21.15.94:0/1249631394 learned_addr learned my addr 172.21.15.94:0/1249631394 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530911a640 1 --2- 172.21.15.94:0/1249631394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 0x559690adc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530a11c640 1 --2- 172.21.15.94:0/1249631394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 0x559690acef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f52faffd640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52f00a33b0 0x7f52f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530991b640 1 -- 172.21.15.94:0/1249631394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 msgr2=0x559690acef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530991b640 1 --2- 172.21.15.94:0/1249631394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 0x559690acef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530991b640 1 -- 172.21.15.94:0/1249631394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 msgr2=0x559690adc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530991b640 1 --2- 172.21.15.94:0/1249631394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 0x559690adc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530991b640 1 -- 172.21.15.94:0/1249631394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596909e81f0 con 0x559690acf480 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530a11c640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52f00a33b0 0x7f52f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f52faffd640 1 -- 172.21.15.94:0/4260401581 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f52f40cf9f0 con 0x7f5304102790 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f530991b640 1 --2- 172.21.15.94:0/1249631394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ad9800 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f530000b760 tx=0x7f530000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf698229b245e07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f52effff640 1 -- 172.21.15.94:0/1249631394 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300003dd0 con 0x559690acf480 2024-09-19T04:32:19.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f52effff640 1 -- 172.21.15.94:0/1249631394 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5300015070 con 0x559690acf480 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.804+0000 7f52effff640 1 -- 172.21.15.94:0/1249631394 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300004d90 con 0x559690acf480 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 -- 172.21.15.94:0/1249631394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 msgr2=0x559690ad9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530a11c640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52f00a33b0 0x7f52f00a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009ca0 tx=0x7f52fc0095c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 --2- 172.21.15.94:0/1249631394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ad9800 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f530000b760 tx=0x7f530000bc20 comp rx=0 tx=0).stop 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 -- 172.21.15.94:0/1249631394 shutdown_connections 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 --2- 172.21.15.94:0/1249631394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 0x559690adc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 --2- 172.21.15.94:0/1249631394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ad9800 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 --2- 172.21.15.94:0/1249631394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 0x559690acef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 -- 172.21.15.94:0/1249631394 >> 172.21.15.94:0/1249631394 conn(0x559690acb0f0 msgr2=0x559690acb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 -- 172.21.15.94:0/1249631394 shutdown_connections 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 -- 172.21.15.94:0/1249631394 wait complete. 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.805+0000 7f530df58740 1 -- start start 2024-09-19T04:32:19.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530df58740 1 Processor -- start 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530df58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 0x559690ad6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530df58740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690acf480 0x559690ad70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530df58740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690ad9d40 0x559690ad7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530df58740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559690adc910 con 0x559690ad9d40 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530df58740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5596909eef10 con 0x559690acf480 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530df58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559690adca90 con 0x559690aceb60 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530df58740 10 client.0 _ll_register_callbacks cb 0x5596909b7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530a11c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690ad9d40 0x559690ad7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530a11c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690ad9d40 0x559690ad7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34918/0 (socket says 172.21.15.94:34918) 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530a11c640 1 -- 172.21.15.94:0/1794281248 learned_addr learned my addr 172.21.15.94:0/1794281248 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530991b640 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 0x559690ad6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530a11c640 1 -- 172.21.15.94:0/1794281248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 msgr2=0x559690ad6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530911a640 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690acf480 0x559690ad70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530a11c640 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 0x559690ad6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530a11c640 1 -- 172.21.15.94:0/1794281248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690acf480 msgr2=0x559690ad70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530a11c640 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690acf480 0x559690ad70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530a11c640 1 -- 172.21.15.94:0/1794281248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596907a56a0 con 0x559690ad9d40 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.806+0000 7f530911a640 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690acf480 0x559690ad70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530a11c640 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690ad9d40 0x559690ad7600 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0317d0 tx=0x7f52fc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28eadfe204a09c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f52ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f52ed7fa640 1 -- 172.21.15.94:0/1794281248 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc004450 con 0x559690ad9d40 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f52ed7fa640 1 -- 172.21.15.94:0/1794281248 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f52fc0045f0 con 0x559690ad9d40 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530df58740 1 -- 172.21.15.94:0/1794281248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5300015070 con 0x559690ad9d40 2024-09-19T04:32:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f52ed7fa640 1 -- 172.21.15.94:0/1794281248 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f52fc0047f0 con 0x559690ad9d40 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530df58740 10 client.17610 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530df58740 1 -- 172.21.15.94:0/1794281248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559690ae84b0 con 0x559690ad9d40 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f52ed7fa640 1 -- 172.21.15.94:0/1794281248 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5300015070 con 0x559690ad9d40 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f52ed7fa640 1 -- 172.21.15.94:0/1794281248 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f52fc059570 con 0x559690ad9d40 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530df58740 10 client.17610 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530df58740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530df58740 20 client.17610 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530df58740 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559690ae9910 0x559690aebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530df58740 4 client.17610 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.807+0000 7f530df58740 1 -- 172.21.15.94:0/1794281248 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559690a417f0 con 0x559690ae9910 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.808+0000 7f530991b640 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559690ae9910 0x559690aebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.808+0000 7f530991b640 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559690ae9910 0x559690aebdd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1426e1123869dd2c server_cookie=ccd6bf1c19e1f4b2 in_seq=0 out_seq=0 2024-09-19T04:32:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.808+0000 7f52ed7fa640 10 client.17610 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f52ed7fa640 1 -- 172.21.15.94:0/1794281248 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559690a417f0 con 0x559690ae9910 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f52ed7fa640 10 client.17610 handle_command_reply: tid=0 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 client.17610 shutdown 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 -- 172.21.15.94:0/1794281248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690ad9d40 msgr2=0x559690ad7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690ad9d40 0x559690ad7600 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0317d0 tx=0x7f52fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 -- 172.21.15.94:0/1794281248 shutdown_connections 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559690ae9910 0x559690aebdd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690ad9d40 0x559690ad7600 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690acf480 0x559690ad70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 --2- 172.21.15.94:0/1794281248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690aceb60 0x559690ad6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f52ed7fa640 0 client.17610 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 -- 172.21.15.94:0/1794281248 >> 172.21.15.94:0/1794281248 conn(0x559690acb0f0 msgr2=0x559690ad0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:19.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 -- 172.21.15.94:0/1794281248 shutdown_connections 2024-09-19T04:32:19.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 1 -- 172.21.15.94:0/1794281248 wait complete. 2024-09-19T04:32:19.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.809+0000 7f530df58740 20 client.17610 trim_cache size 0 max 16384 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530df58740 1 Processor -- start 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530df58740 1 -- start start 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530df58740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690aceb60 0x559690ad15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530df58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690acf480 0x559690af16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530df58740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 0x559690af1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530df58740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b22c0 con 0x559690aceb60 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530df58740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53041afb90 con 0x559690ad9d40 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530df58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53041ada00 con 0x559690acf480 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530991b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690aceb60 0x559690ad15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530991b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690aceb60 0x559690ad15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34926/0 (socket says 172.21.15.94:34926) 2024-09-19T04:32:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530991b640 1 -- 172.21.15.94:0/1558041721 learned_addr learned my addr 172.21.15.94:0/1558041721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530911a640 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690acf480 0x559690af16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530a11c640 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 0x559690af1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530991b640 1 -- 172.21.15.94:0/1558041721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690acf480 msgr2=0x559690af16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530991b640 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690acf480 0x559690af16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530991b640 1 -- 172.21.15.94:0/1558041721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 msgr2=0x559690af1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530991b640 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 0x559690af1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.827+0000 7f530991b640 1 -- 172.21.15.94:0/1558041721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559690acaf20 con 0x559690aceb60 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.828+0000 7f530911a640 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690acf480 0x559690af16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.828+0000 7f530991b640 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690aceb60 0x559690ad15c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f53000044d0 tx=0x7f5300005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e59a50f059dfdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.828+0000 7f52ef7fe640 1 -- 172.21.15.94:0/1558041721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300003e90 con 0x559690aceb60 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.828+0000 7f52ef7fe640 1 -- 172.21.15.94:0/1558041721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5300015030 con 0x559690aceb60 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.828+0000 7f52ef7fe640 1 -- 172.21.15.94:0/1558041721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300019680 con 0x559690aceb60 2024-09-19T04:32:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 -- 172.21.15.94:0/1558041721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690aceb60 msgr2=0x559690ad15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690aceb60 0x559690ad15c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f53000044d0 tx=0x7f5300005f70 comp rx=0 tx=0).stop 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 -- 172.21.15.94:0/1558041721 shutdown_connections 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690ad9d40 0x559690af1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690acf480 0x559690af16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 --2- 172.21.15.94:0/1558041721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690aceb60 0x559690ad15c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 -- 172.21.15.94:0/1558041721 >> 172.21.15.94:0/1558041721 conn(0x559690acb0f0 msgr2=0x559690ad8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 -- 172.21.15.94:0/1558041721 shutdown_connections 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 -- 172.21.15.94:0/1558041721 wait complete. 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 -- start start 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.829+0000 7f530df58740 1 Processor -- start 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530df58740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690aceb60 0x559690ae1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530df58740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ae2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530df58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690ad9d40 0x559690adccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530df58740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53041ae050 con 0x559690acf480 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530df58740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53041af4c0 con 0x559690aceb60 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530df58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53041ae720 con 0x559690ad9d40 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530911a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ae2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530df58740 10 client.0 _ll_register_callbacks cb 0x7f52fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530911a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ae2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34942/0 (socket says 172.21.15.94:34942) 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530911a640 1 -- 172.21.15.94:0/2171180183 learned_addr learned my addr 172.21.15.94:0/2171180183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530a11c640 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690ad9d40 0x559690adccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530991b640 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690aceb60 0x559690ae1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530911a640 1 -- 172.21.15.94:0/2171180183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690ad9d40 msgr2=0x559690adccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530911a640 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690ad9d40 0x559690adccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530911a640 1 -- 172.21.15.94:0/2171180183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690aceb60 msgr2=0x559690ae1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:19.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530911a640 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690aceb60 0x559690ae1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530911a640 1 -- 172.21.15.94:0/2171180183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559690add210 con 0x559690acf480 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530911a640 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ae2260 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f52f4005fd0 tx=0x7f52f400a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0497d4e706329a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f52ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f52ed7fa640 1 -- 172.21.15.94:0/2171180183 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f401d070 con 0x559690acf480 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f52ed7fa640 1 -- 172.21.15.94:0/2171180183 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f52f4098300 con 0x559690acf480 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.830+0000 7f530df58740 1 -- 172.21.15.94:0/2171180183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5300015030 con 0x559690acf480 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f52ed7fa640 1 -- 172.21.15.94:0/2171180183 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f52f409c390 con 0x559690acf480 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f530df58740 10 client.17616 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f530df58740 1 -- 172.21.15.94:0/2171180183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559690b54e20 con 0x559690acf480 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f52ed7fa640 1 -- 172.21.15.94:0/2171180183 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5300015030 con 0x559690acf480 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f52ed7fa640 1 -- 172.21.15.94:0/2171180183 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f52f40979e0 con 0x559690acf480 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f530df58740 10 client.17616 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:19.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f530df58740 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f530df58740 20 client.17616 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f530df58740 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559690b55550 0x559690b57970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f530df58740 4 client.17616 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f530df58740 1 -- 172.21.15.94:0/2171180183 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559690a3bd00 con 0x559690b55550 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.831+0000 7f530991b640 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559690b55550 0x559690b57970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.832+0000 7f530991b640 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559690b55550 0x559690b57970 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be38cd6c0d8cb72d server_cookie=c5d02dedb5b12a8c in_seq=0 out_seq=0 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.832+0000 7f52ed7fa640 10 client.17616 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f52ed7fa640 1 -- 172.21.15.94:0/2171180183 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559690a3bd00 con 0x559690b55550 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f52ed7fa640 10 client.17616 handle_command_reply: tid=0 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 client.17616 shutdown 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 -- 172.21.15.94:0/2171180183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 msgr2=0x559690ae2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ae2260 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f52f4005fd0 tx=0x7f52f400a5d0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 -- 172.21.15.94:0/2171180183 shutdown_connections 2024-09-19T04:32:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x559690b55550 0x559690b57970 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559690ad9d40 0x559690adccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559690acf480 0x559690ae2260 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 --2- 172.21.15.94:0/2171180183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690aceb60 0x559690ae1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 -- 172.21.15.94:0/2171180183 >> 172.21.15.94:0/2171180183 conn(0x559690acb0f0 msgr2=0x559690ad6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f52ed7fa640 0 client.17616 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 -- 172.21.15.94:0/2171180183 shutdown_connections 2024-09-19T04:32:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 1 -- 172.21.15.94:0/2171180183 wait complete. 2024-09-19T04:32:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.833+0000 7f530df58740 20 client.17616 trim_cache size 0 max 16384 2024-09-19T04:32:19.848 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:32:19.848 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-09-19T04:32:19.848 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "11cad015-caac-4f24-ab93-9f8098b3af01", 2024-09-19T04:32:19.848 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-09-19T04:32:19.848 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 -- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52f00a33b0 msgr2=0x7f52f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52f00a33b0 0x7f52f00a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009ca0 tx=0x7f52fc0095c0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 -- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304102790 msgr2=0x7f5304074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304102790 0x7f5304074a30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c890 tx=0x7f52f400cd50 comp rx=0 tx=0).stop 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 -- 172.21.15.94:0/4260401581 shutdown_connections 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f52f00a33b0 0x7f52f00a5870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5304106f10 0x7f5304074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5304102790 0x7f5304074a30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 --2- 172.21.15.94:0/4260401581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5304101de0 0x7f53040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 -- 172.21.15.94:0/4260401581 >> 172.21.15.94:0/4260401581 conn(0x7f53040fd8c0 msgr2=0x7f53040fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 -- 172.21.15.94:0/4260401581 shutdown_connections 2024-09-19T04:32:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:19.834+0000 7f530bb7f640 1 -- 172.21.15.94:0/4260401581 wait complete. 2024-09-19T04:32:20.517 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:32:20.806 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: pgmap v544: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 6.1 MiB/s wr, 652 op/s 2024-09-19T04:32:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub queued for path: / 2024-09-19T04:32:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:32:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:32:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:32:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:32:21.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:32:21.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: active paths [/] 2024-09-19T04:32:21.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.036 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:20 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: pgmap v544: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 6.1 MiB/s wr, 652 op/s 2024-09-19T04:32:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub queued for path: / 2024-09-19T04:32:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:32:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:32:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:32:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:32:21.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:32:21.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T04:32:21.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:20 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: pgmap v544: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 6.1 MiB/s wr, 652 op/s 2024-09-19T04:32:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub queued for path: / 2024-09-19T04:32:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:32:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:32:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:32:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:32:21.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:32:21.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: active paths [/] 2024-09-19T04:32:21.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:20 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:21.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.324+0000 7f1e88462640 1 -- 172.21.15.94:0/1469769026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e8010c590 msgr2=0x7f1e8010c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:21.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.324+0000 7f1e88462640 1 --2- 172.21.15.94:0/1469769026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e8010c590 0x7f1e8010c9f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009900 tx=0x7f1e7c02a410 comp rx=0 tx=0).stop 2024-09-19T04:32:21.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.324+0000 7f1e88462640 1 -- 172.21.15.94:0/1469769026 shutdown_connections 2024-09-19T04:32:21.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.324+0000 7f1e88462640 1 --2- 172.21.15.94:0/1469769026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8010cf30 0x7f1e80119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.324+0000 7f1e88462640 1 --2- 172.21.15.94:0/1469769026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e8010c590 0x7f1e8010c9f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.324+0000 7f1e88462640 1 --2- 172.21.15.94:0/1469769026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e80112640 0x7f1e80112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.324+0000 7f1e88462640 1 -- 172.21.15.94:0/1469769026 >> 172.21.15.94:0/1469769026 conn(0x7f1e8006cdc0 msgr2=0x7f1e8006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:21.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.324+0000 7f1e88462640 1 -- 172.21.15.94:0/1469769026 shutdown_connections 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.324+0000 7f1e88462640 1 -- 172.21.15.94:0/1469769026 wait complete. 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e88462640 1 Processor -- start 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e88462640 1 -- start start 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e88462640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8010c590 0x7f1e801a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e88462640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8010cf30 0x7f1e801a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e88462640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e80112640 0x7f1e801a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e88462640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8011c810 con 0x7f1e80112640 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e88462640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8011c690 con 0x7f1e8010cf30 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e88462640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8011c990 con 0x7f1e8010c590 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e869ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e80112640 0x7f1e801a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e869ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e80112640 0x7f1e801a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34962/0 (socket says 172.21.15.94:34962) 2024-09-19T04:32:21.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e869ff640 1 -- 172.21.15.94:0/282075471 learned_addr learned my addr 172.21.15.94:0/282075471 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e861fe640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8010c590 0x7f1e801a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.325+0000 7f1e859fd640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8010cf30 0x7f1e801a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e869ff640 1 -- 172.21.15.94:0/282075471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8010c590 msgr2=0x7f1e801a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e869ff640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8010c590 0x7f1e801a8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e869ff640 1 -- 172.21.15.94:0/282075471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8010cf30 msgr2=0x7f1e801a8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e869ff640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8010cf30 0x7f1e801a8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e869ff640 1 -- 172.21.15.94:0/282075471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e801a3390 con 0x7f1e80112640 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e869ff640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e80112640 0x7f1e801a2b20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800eb10 tx=0x7f1e7800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e777fe640 1 -- 172.21.15.94:0/282075471 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7800ce20 con 0x7f1e80112640 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e88462640 1 -- 172.21.15.94:0/282075471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e801a3680 con 0x7f1e80112640 2024-09-19T04:32:21.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e777fe640 1 -- 172.21.15.94:0/282075471 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e78005020 con 0x7f1e80112640 2024-09-19T04:32:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e777fe640 1 -- 172.21.15.94:0/282075471 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78010690 con 0x7f1e80112640 2024-09-19T04:32:21.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.326+0000 7f1e88462640 1 -- 172.21.15.94:0/282075471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e801af800 con 0x7f1e80112640 2024-09-19T04:32:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.328+0000 7f1e777fe640 1 -- 172.21.15.94:0/282075471 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1e780051d0 con 0x7f1e80112640 2024-09-19T04:32:21.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.328+0000 7f1e8a83b740 1 Processor -- start 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e8a83b740 1 -- start start 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e9716a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716adc0 0x564e97175140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97175680 0x564e97177a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564e970d02e0 con 0x564e97175680 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564e97077ce0 con 0x564e9716a4a0 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e970cc2a0 con 0x564e9716adc0 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e859fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97175680 0x564e97177a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e861fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716adc0 0x564e97175140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e859fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97175680 0x564e97177a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34968/0 (socket says 172.21.15.94:34968) 2024-09-19T04:32:21.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e859fd640 1 -- 172.21.15.94:0/3542093617 learned_addr learned my addr 172.21.15.94:0/3542093617 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e777fe640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e6c0a33b0 0x7f1e6c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e859fd640 1 --2- 172.21.15.94:0/3542093617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e9716a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.329+0000 7f1e777fe640 1 -- 172.21.15.94:0/282075471 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f1e780d03a0 con 0x7f1e80112640 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e861fe640 1 -- 172.21.15.94:0/3542093617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 msgr2=0x564e9716a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e869ff640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e6c0a33b0 0x7f1e6c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e861fe640 1 --2- 172.21.15.94:0/3542093617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e9716a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e861fe640 1 -- 172.21.15.94:0/3542093617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97175680 msgr2=0x564e97177a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e861fe640 1 --2- 172.21.15.94:0/3542093617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97175680 0x564e97177a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e861fe640 1 -- 172.21.15.94:0/3542093617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e9705c440 con 0x564e9716adc0 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e861fe640 1 --2- 172.21.15.94:0/3542093617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716adc0 0x564e97175140 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000abf0 tx=0x7f1e7000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e274e71b481117a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:21.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e869ff640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e6c0a33b0 0x7f1e6c0a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1e780cdb10 tx=0x7f1e78009a70 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e74dea640 1 -- 172.21.15.94:0/3542093617 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70005220 con 0x564e9716adc0 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e74dea640 1 -- 172.21.15.94:0/3542093617 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e70004c50 con 0x564e9716adc0 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.330+0000 7f1e74dea640 1 -- 172.21.15.94:0/3542093617 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700187a0 con 0x564e9716adc0 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 -- 172.21.15.94:0/3542093617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716adc0 msgr2=0x564e97175140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/3542093617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716adc0 0x564e97175140 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000abf0 tx=0x7f1e7000f790 comp rx=0 tx=0).stop 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 -- 172.21.15.94:0/3542093617 shutdown_connections 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/3542093617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97175680 0x564e97177a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/3542093617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716adc0 0x564e97175140 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/3542093617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e9716a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 -- 172.21.15.94:0/3542093617 >> 172.21.15.94:0/3542093617 conn(0x564e97166a30 msgr2=0x564e97166e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 -- 172.21.15.94:0/3542093617 shutdown_connections 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 -- 172.21.15.94:0/3542093617 wait complete. 2024-09-19T04:32:21.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 -- start start 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.331+0000 7f1e8a83b740 1 Processor -- start 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e97174e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e97172120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 0x564e97172660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564e97178250 con 0x564e9716adc0 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564e9708b050 con 0x564e9716a4a0 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e971783d0 con 0x564e97175680 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e8a83b740 10 client.0 _ll_register_callbacks cb 0x564e970cb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e861fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e97172120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e861fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e97172120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34978/0 (socket says 172.21.15.94:34978) 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e861fe640 1 -- 172.21.15.94:0/1397994159 learned_addr learned my addr 172.21.15.94:0/1397994159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e859fd640 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 0x564e97172660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e861fe640 1 -- 172.21.15.94:0/1397994159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 msgr2=0x564e97172660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:21.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e861fe640 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 0x564e97172660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e861fe640 1 -- 172.21.15.94:0/1397994159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 msgr2=0x564e97174e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e861fe640 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e97174e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e861fe640 1 -- 172.21.15.94:0/1397994159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e96e413c0 con 0x564e9716adc0 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e859fd640 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 0x564e97172660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e861fe640 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e97172120 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1e70005e40 tx=0x7f1e70005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9947b83d0204e167 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.332+0000 7f1e69ffb640 1 -- 172.21.15.94:0/1397994159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70027070 con 0x564e9716adc0 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e69ffb640 1 -- 172.21.15.94:0/1397994159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f1e70005010 con 0x564e9716adc0 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1397994159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e70004c50 con 0x564e9716adc0 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e69ffb640 1 -- 172.21.15.94:0/1397994159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1e70033050 con 0x564e9716adc0 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e8a83b740 10 client.17628 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1397994159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564e97183b20 con 0x564e9716adc0 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e69ffb640 1 -- 172.21.15.94:0/1397994159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e70004c50 con 0x564e9716adc0 2024-09-19T04:32:21.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e69ffb640 1 -- 172.21.15.94:0/1397994159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f1e70023430 con 0x564e9716adc0 2024-09-19T04:32:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e8a83b740 10 client.17628 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e8a83b740 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e8a83b740 20 client.17628 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564e97184ed0 0x564e971872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e8a83b740 4 client.17628 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.333+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1397994159 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e970dd850 con 0x564e97184ed0 2024-09-19T04:32:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.334+0000 7f1e859fd640 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564e97184ed0 0x564e971872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.334+0000 7f1e69ffb640 10 client.17628 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.334+0000 7f1e859fd640 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564e97184ed0 0x564e971872c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33bab98780eab007 server_cookie=6a275dc3545d987f in_seq=0 out_seq=0 2024-09-19T04:32:21.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e69ffb640 1 -- 172.21.15.94:0/1397994159 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e970dd850 con 0x564e97184ed0 2024-09-19T04:32:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e69ffb640 10 client.17628 handle_command_reply: tid=0 2024-09-19T04:32:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 client.17628 shutdown 2024-09-19T04:32:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1397994159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 msgr2=0x564e97172120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e97172120 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1e70005e40 tx=0x7f1e70005e70 comp rx=0 tx=0).stop 2024-09-19T04:32:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1397994159 shutdown_connections 2024-09-19T04:32:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564e97184ed0 0x564e971872c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 0x564e97172660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e97172120 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e69ffb640 0 client.17628 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:21.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1397994159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e97174e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1397994159 >> 172.21.15.94:0/1397994159 conn(0x564e97166a30 msgr2=0x564e9716c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:21.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1397994159 shutdown_connections 2024-09-19T04:32:21.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1397994159 wait complete. 2024-09-19T04:32:21.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.335+0000 7f1e8a83b740 20 client.17628 trim_cache size 0 max 16384 2024-09-19T04:32:21.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.352+0000 7f1e8a83b740 1 Processor -- start 2024-09-19T04:32:21.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.352+0000 7f1e8a83b740 1 -- start start 2024-09-19T04:32:21.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.352+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716a4a0 0x564e9716d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.352+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e971709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.352+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e97175680 0x564e97174090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.352+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801baad0 con 0x564e9716adc0 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.352+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801b83a0 con 0x564e97175680 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.352+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801b6210 con 0x564e9716a4a0 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e861fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e971709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e861fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e971709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34990/0 (socket says 172.21.15.94:34990) 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e861fe640 1 -- 172.21.15.94:0/1393141923 learned_addr learned my addr 172.21.15.94:0/1393141923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e859fd640 1 --2- 172.21.15.94:0/1393141923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716a4a0 0x564e9716d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e859fd640 1 --2- 172.21.15.94:0/1393141923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e97175680 0x564e97174090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e861fe640 1 -- 172.21.15.94:0/1393141923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716a4a0 msgr2=0x564e9716d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:21.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e861fe640 1 --2- 172.21.15.94:0/1393141923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716a4a0 0x564e9716d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e861fe640 1 -- 172.21.15.94:0/1393141923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e97175680 msgr2=0x564e97174090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:21.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e861fe640 1 --2- 172.21.15.94:0/1393141923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e97175680 0x564e97174090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e861fe640 1 -- 172.21.15.94:0/1393141923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e971710d0 con 0x564e9716adc0 2024-09-19T04:32:21.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e861fe640 1 --2- 172.21.15.94:0/1393141923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e971709f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1e70000be0 tx=0x7f1e7000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5401c509b0586fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:21.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e6bfff640 1 -- 172.21.15.94:0/1393141923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7003c070 con 0x564e9716adc0 2024-09-19T04:32:21.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e6bfff640 1 -- 172.21.15.94:0/1393141923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e70004c30 con 0x564e9716adc0 2024-09-19T04:32:21.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.353+0000 7f1e6bfff640 1 -- 172.21.15.94:0/1393141923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70027070 con 0x564e9716adc0 2024-09-19T04:32:21.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.354+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1393141923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 msgr2=0x564e971709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:21.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.354+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1393141923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e971709f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1e70000be0 tx=0x7f1e7000a8f0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.354+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1393141923 shutdown_connections 2024-09-19T04:32:21.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.354+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1393141923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e97175680 0x564e97174090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.354+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1393141923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e971709f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.354+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/1393141923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e9716a4a0 0x564e9716d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.354+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1393141923 >> 172.21.15.94:0/1393141923 conn(0x564e97166a30 msgr2=0x564e97173980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:21.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.354+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1393141923 shutdown_connections 2024-09-19T04:32:21.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.354+0000 7f1e8a83b740 1 -- 172.21.15.94:0/1393141923 wait complete. 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e8a83b740 1 -- start start 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e8a83b740 1 Processor -- start 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e9716f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e9716f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e8a83b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 0x564e9716fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801b6860 con 0x564e9716adc0 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801b7cd0 con 0x564e9716a4a0 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e8a83b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801b6f30 con 0x564e97175680 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e8a83b740 10 client.0 _ll_register_callbacks cb 0x7f1e780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e861fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e9716f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e861fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e9716f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35002/0 (socket says 172.21.15.94:35002) 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e861fe640 1 -- 172.21.15.94:0/28666036 learned_addr learned my addr 172.21.15.94:0/28666036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:21.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e859fd640 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 0x564e9716fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.355+0000 7f1e859fd640 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e9716f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e861fe640 1 -- 172.21.15.94:0/28666036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 msgr2=0x564e9716fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e861fe640 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 0x564e9716fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e861fe640 1 -- 172.21.15.94:0/28666036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 msgr2=0x564e9716f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e861fe640 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e9716f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e861fe640 1 -- 172.21.15.94:0/28666036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e971751e0 con 0x564e9716adc0 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e861fe640 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e9716f780 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1e70023ad0 tx=0x7f1e700236e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7270047c6761be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e69ffb640 1 -- 172.21.15.94:0/28666036 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7003c070 con 0x564e9716adc0 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e69ffb640 1 -- 172.21.15.94:0/28666036 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f1e70005210 con 0x564e9716adc0 2024-09-19T04:32:21.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e8a83b740 1 -- 172.21.15.94:0/28666036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e70004c30 con 0x564e9716adc0 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e69ffb640 1 -- 172.21.15.94:0/28666036 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1e70027040 con 0x564e9716adc0 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e8a83b740 10 client.17634 fetch_fsmap learned FSMap version 55 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.356+0000 7f1e8a83b740 1 -- 172.21.15.94:0/28666036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564e971efda0 con 0x564e9716adc0 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e69ffb640 1 -- 172.21.15.94:0/28666036 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e70004c30 con 0x564e9716adc0 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e69ffb640 1 -- 172.21.15.94:0/28666036 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 55) ==== 8354+0+0 (secure 0 0 0) 0x7f1e70048ca0 con 0x564e9716adc0 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e8a83b740 10 client.17634 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e8a83b740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e8a83b740 20 client.17634 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564e97184ed0 0x564e971f2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e8a83b740 4 client.17634 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e8a83b740 1 -- 172.21.15.94:0/28666036 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e970d7d60 con 0x564e97184ed0 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e859fd640 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564e97184ed0 0x564e971f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:21.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e69ffb640 10 client.17634 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.357+0000 7f1e859fd640 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564e97184ed0 0x564e971f2450 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be65cd5fa1884e3e server_cookie=96c3885aa34e1fd9 in_seq=0 out_seq=0 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e69ffb640 1 -- 172.21.15.94:0/28666036 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x564e970d7d60 con 0x564e97184ed0 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e69ffb640 10 client.17634 handle_command_reply: tid=0 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 client.17634 shutdown 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 -- 172.21.15.94:0/28666036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 msgr2=0x564e9716f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e9716f780 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1e70023ad0 tx=0x7f1e700236e0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 -- 172.21.15.94:0/28666036 shutdown_connections 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x564e97184ed0 0x564e971f2450 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e97175680 0x564e9716fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e9716adc0 0x564e9716f780 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e69ffb640 0 client.17634 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 --2- 172.21.15.94:0/28666036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e9716a4a0 0x564e9716f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 -- 172.21.15.94:0/28666036 >> 172.21.15.94:0/28666036 conn(0x564e97166a30 msgr2=0x564e971741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 -- 172.21.15.94:0/28666036 shutdown_connections 2024-09-19T04:32:21.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 1 -- 172.21.15.94:0/28666036 wait complete. 2024-09-19T04:32:21.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.358+0000 7f1e8a83b740 20 client.17634 trim_cache size 0 max 16384 2024-09-19T04:32:21.366 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:32:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (0 inodes in the stack)", 2024-09-19T04:32:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-09-19T04:32:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "11cad015-caac-4f24-ab93-9f8098b3af01": { 2024-09-19T04:32:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-09-19T04:32:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "11cad015-caac-4f24-ab93-9f8098b3af01", 2024-09-19T04:32:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-09-19T04:32:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:32:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:32:21.367 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 -- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e6c0a33b0 msgr2=0x7f1e6c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e6c0a33b0 0x7f1e6c0a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1e780cdb10 tx=0x7f1e78009a70 comp rx=0 tx=0).stop 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 -- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e80112640 msgr2=0x7f1e801a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e80112640 0x7f1e801a2b20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800eb10 tx=0x7f1e7800ef10 comp rx=0 tx=0).stop 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 -- 172.21.15.94:0/282075471 shutdown_connections 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f1e6c0a33b0 0x7f1e6c0a5870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e80112640 0x7f1e801a2b20 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8010cf30 0x7f1e801a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 --2- 172.21.15.94:0/282075471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8010c590 0x7f1e801a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 -- 172.21.15.94:0/282075471 >> 172.21.15.94:0/282075471 conn(0x7f1e8006cdc0 msgr2=0x7f1e8010af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 -- 172.21.15.94:0/282075471 shutdown_connections 2024-09-19T04:32:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:21.359+0000 7f1e88462640 1 -- 172.21.15.94:0/282075471 wait complete. 2024-09-19T04:32:22.032 INFO:tasks.cephfs.filesystem:scrub status for tag:11cad015-caac-4f24-ab93-9f8098b3af01 - {'path': '/', 'tag': '11cad015-caac-4f24-ab93-9f8098b3af01', 'options': 'recursive,force'} 2024-09-19T04:32:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:22 smithi112 ceph-mon[35051]: pgmap v545: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-09-19T04:32:23.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:22 smithi094 ceph-mon[30972]: pgmap v545: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-09-19T04:32:23.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:22 smithi104 ceph-mon[34145]: pgmap v545: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-09-19T04:32:24.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:23 smithi112 ceph-mon[35051]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:24.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:23 smithi094 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:24.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:23 smithi104 ceph-mon[34145]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:25.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:24 smithi112 ceph-mon[35051]: pgmap v546: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 6.1 MiB/s wr, 657 op/s 2024-09-19T04:32:25.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:24 smithi094 ceph-mon[30972]: pgmap v546: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 6.1 MiB/s wr, 657 op/s 2024-09-19T04:32:25.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:24 smithi104 ceph-mon[34145]: pgmap v546: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 6.1 MiB/s wr, 657 op/s 2024-09-19T04:32:27.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:26 smithi112 ceph-mon[35051]: pgmap v547: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 952 KiB/s wr, 102 op/s 2024-09-19T04:32:27.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:26 smithi094 ceph-mon[30972]: pgmap v547: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 952 KiB/s wr, 102 op/s 2024-09-19T04:32:27.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:26 smithi104 ceph-mon[34145]: pgmap v547: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 952 KiB/s wr, 102 op/s 2024-09-19T04:32:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:27 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:32:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:27 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:32:28.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:27 smithi094 ceph-mon[30972]: scrub summary: idle 2024-09-19T04:32:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:27 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:32:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:27 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:32:28.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:27 smithi104 ceph-mon[34145]: scrub summary: idle 2024-09-19T04:32:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:27 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:32:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:27 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:32:28.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:27 smithi112 ceph-mon[35051]: scrub summary: idle 2024-09-19T04:32:29.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:28 smithi094 ceph-mon[30972]: pgmap v548: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 952 KiB/s wr, 104 op/s 2024-09-19T04:32:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:28 smithi104 ceph-mon[34145]: pgmap v548: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 952 KiB/s wr, 104 op/s 2024-09-19T04:32:29.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:28 smithi112 ceph-mon[35051]: pgmap v548: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 952 KiB/s wr, 104 op/s 2024-09-19T04:32:31.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:30 smithi094 ceph-mon[30972]: pgmap v549: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 510 KiB/s wr, 59 op/s 2024-09-19T04:32:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:30 smithi104 ceph-mon[34145]: pgmap v549: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 510 KiB/s wr, 59 op/s 2024-09-19T04:32:31.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:30 smithi112 ceph-mon[35051]: pgmap v549: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 510 KiB/s wr, 59 op/s 2024-09-19T04:32:33.094 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:32 smithi094 ceph-mon[30972]: pgmap v550: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 55 KiB/s wr, 12 op/s 2024-09-19T04:32:33.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:32 smithi104 ceph-mon[34145]: pgmap v550: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 55 KiB/s wr, 12 op/s 2024-09-19T04:32:33.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:32 smithi112 ceph-mon[35051]: pgmap v550: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 55 KiB/s wr, 12 op/s 2024-09-19T04:32:34.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:33 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:32:34.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:33 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:32:34.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:33 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:32:34.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:33 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:32:34.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:33 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:32:34.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:33 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:32:34.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:33 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:32:34.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:33 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:32:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:33 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:32:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:33 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:32:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:33 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:32:34.285 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:33 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:32:35.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:35 smithi112 ceph-mon[35051]: pgmap v551: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 110 KiB/s wr, 20 op/s 2024-09-19T04:32:35.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:35 smithi094 ceph-mon[30972]: pgmap v551: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 110 KiB/s wr, 20 op/s 2024-09-19T04:32:35.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:35 smithi104 ceph-mon[34145]: pgmap v551: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 110 KiB/s wr, 20 op/s 2024-09-19T04:32:37.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:37 smithi112 ceph-mon[35051]: pgmap v552: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 12 op/s 2024-09-19T04:32:37.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:37 smithi094 ceph-mon[30972]: pgmap v552: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 12 op/s 2024-09-19T04:32:37.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:37 smithi104 ceph-mon[34145]: pgmap v552: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 12 op/s 2024-09-19T04:32:37.895 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-19T04:32:38.199 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/: 2024-09-19T04:32:38.199 INFO:teuthology.orchestra.run.smithi094.stdout:total 2 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1636263690 Sep 19 04:15 . 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1636263690 Sep 19 04:15 .. 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 data 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data: 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:total 46 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 . 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1636263690 Sep 19 04:15 .. 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 594916917 Sep 19 04:16 base 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 19 04:15 current_logfiles 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 Sep 19 04:15 global 2024-09-19T04:32:38.200 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 Sep 19 04:15 log 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_commit_ts 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_dynshmem 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 19 04:15 pg_hba.conf 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 19 04:15 pg_ident.conf 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:32 pg_logical 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 pg_multixact 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_notify 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_replslot 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_serial 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_snapshots 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Sep 19 04:32 pg_stat 2024-09-19T04:32:38.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Sep 19 04:32 pg_stat_tmp 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 19 04:32 pg_subtrans 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_tblspc 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 pg_twophase 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 04:15 PG_VERSION 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1040187392 Sep 19 04:32 pg_wal 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 245760 Sep 19 04:15 pg_xact 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 19 04:15 postgresql.auto.conf 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 19 04:15 postgresql.conf 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 19 04:15 postmaster.opts 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base: 2024-09-19T04:32:38.202 INFO:teuthology.orchestra.run.smithi094.stdout:total 3 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 594916917 Sep 19 04:16 . 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 1 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 13493 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 578990639 Sep 19 04:21 13494 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:16 pgsql_tmp 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:total 7778 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 . 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 594916917 Sep 19 04:16 .. 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 112 2024-09-19T04:32:38.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 113 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 1247 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1247_fsm 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1247_vm 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 04:15 1249 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1249_fsm 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1249_vm 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 04:15 1255 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1255_fsm 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1255_vm 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 04:15 1259 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1259_fsm 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1259_vm 2024-09-19T04:32:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 04:15 13344 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13344_fsm 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13344_vm 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13346 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13348 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13349_fsm 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349_vm 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13351 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13353 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13354_fsm 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354_vm 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13356 2024-09-19T04:32:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13358 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13359_fsm 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359_vm 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13361 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13363 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1417 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1418 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 174 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 175 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2187 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2224 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2328 2024-09-19T04:32:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2336 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2337 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2579 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2600 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2600_fsm 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2600_vm 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2601_fsm 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601_vm 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2602 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2602_fsm 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2602_vm 2024-09-19T04:32:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2603 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2603_fsm 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2603_vm 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2604 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2605 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2605_fsm 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2605_vm 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2606_fsm 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606_vm 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2607 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2607_fsm 2024-09-19T04:32:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2607_vm 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 04:15 2608 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2608_fsm 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2608_vm 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 04:15 2609 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2609_fsm 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2609_vm 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2610 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2610_fsm 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2610_vm 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2611 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2612_fsm 2024-09-19T04:32:38.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612_vm 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2613 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2615_fsm 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615_vm 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2616 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2616_fsm 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2616_vm 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 04:15 2617 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2617_fsm 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2617_vm 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 04:15 2618 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2618_fsm 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2618_vm 2024-09-19T04:32:38.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 04:15 2619 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2619_fsm 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2619_vm 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2620 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2650 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2651 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2652 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2653 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2654 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2655 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2656 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2657 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 04:15 2658 2024-09-19T04:32:38.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 2659 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2660 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2661 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2662 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2663 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2664 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2665 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2666 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2667 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2668 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2669 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2670 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 04:15 2673 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 04:15 2674 2024-09-19T04:32:38.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 04:15 2675 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2678 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2679 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2680 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2681 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2682 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2683 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2684 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2685 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2686 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2687 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2688 2024-09-19T04:32:38.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2689 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 04:15 2690 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 04:15 2691 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2692 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2693 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2696 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2699 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2701 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2702 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2703 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2704 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2753 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2753_fsm 2024-09-19T04:32:38.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2753_vm 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2754 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2755 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2756 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2757 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2830 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2831 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2832 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2833 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2834 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2835 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2836 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2837 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 04:15 2838 2024-09-19T04:32:38.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2838_fsm 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2838_vm 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2839 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840_fsm 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2840_vm 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2841 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2995 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2996 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3079_fsm 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079_vm 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3080 2024-09-19T04:32:38.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3081 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3085 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3118 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3119 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 04:15 3164 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3256 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3257 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3258 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3350 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3351 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3379 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3380 2024-09-19T04:32:38.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3381 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3394 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3394_fsm 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3394_vm 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3395 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3429 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3430 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3431 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3433 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3439 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3440 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3455 2024-09-19T04:32:38.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 04:15 3456 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3456_fsm 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3456_vm 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3466 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3467 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3468 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3501 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3502 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3503 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3534 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3541_fsm 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541_vm 2024-09-19T04:32:38.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3542 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3574 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3575 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3576 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3596 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3597 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3598 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3599 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3600_fsm 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600_vm 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601 2024-09-19T04:32:38.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3601_fsm 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601_vm 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3602_fsm 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602_vm 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603_fsm 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3603_vm 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3604 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3605 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3606 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3607 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3608 2024-09-19T04:32:38.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 3609 2024-09-19T04:32:38.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3712 2024-09-19T04:32:38.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764 2024-09-19T04:32:38.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3764_fsm 2024-09-19T04:32:38.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764_vm 2024-09-19T04:32:38.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3766 2024-09-19T04:32:38.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3767 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3997 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4143 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4144 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4145 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4146 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4147 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4148 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4149 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4150 2024-09-19T04:32:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4151 2024-09-19T04:32:38.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4152 2024-09-19T04:32:38.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4153 2024-09-19T04:32:38.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4154 2024-09-19T04:32:38.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4155 2024-09-19T04:32:38.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4156 2024-09-19T04:32:38.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4157 2024-09-19T04:32:38.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4158 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4159 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4160 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4161 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4162 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4163 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4164 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4165 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4166 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4167 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4168 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4169 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4170 2024-09-19T04:32:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4171 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4172 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4173 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4174 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 5002 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 548 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 549 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6102 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6104 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6106 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6110 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6111 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6112 2024-09-19T04:32:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6113 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6117 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 826 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 827 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 828 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 04:15 pg_filenode.map 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 04:15 PG_VERSION 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:total 7778 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 19 04:15 . 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 594916917 Sep 19 04:16 .. 2024-09-19T04:32:38.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 112 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 113 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 1247 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1247_fsm 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1247_vm 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 04:15 1249 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1249_fsm 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1249_vm 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 04:15 1255 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1255_fsm 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1255_vm 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 04:15 1259 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1259_fsm 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1259_vm 2024-09-19T04:32:38.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 04:15 13344 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13344_fsm 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13344_vm 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13346 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13348 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13349_fsm 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349_vm 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13351 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13353 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13354_fsm 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354_vm 2024-09-19T04:32:38.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13356 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13358 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13359_fsm 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359_vm 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13361 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13363 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1417 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1418 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 174 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 175 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2187 2024-09-19T04:32:38.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2224 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2328 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2336 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2337 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2579 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2600 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2600_fsm 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2600_vm 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2601_fsm 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601_vm 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2602 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2602_fsm 2024-09-19T04:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2602_vm 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2603 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2603_fsm 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2603_vm 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2604 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2605 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2605_fsm 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2605_vm 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2606_fsm 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2606_vm 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2607 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2607_fsm 2024-09-19T04:32:38.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2607_vm 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 04:15 2608 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2608_fsm 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2608_vm 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 04:15 2609 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2609_fsm 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2609_vm 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2610 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2610_fsm 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2610_vm 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2611 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2612_fsm 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612_vm 2024-09-19T04:32:38.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2613 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2615_fsm 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615_vm 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2616 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2616_fsm 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2616_vm 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 04:15 2617 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2617_fsm 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2617_vm 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 04:15 2618 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2618_fsm 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2618_vm 2024-09-19T04:32:38.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 04:15 2619 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2619_fsm 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2619_vm 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2620 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2650 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2651 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2652 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2653 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2654 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2655 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2656 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2657 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 04:15 2658 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:15 2659 2024-09-19T04:32:38.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2660 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2661 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2662 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2663 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2664 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2665 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2666 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2667 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2668 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2669 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2670 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 04:15 2673 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 04:15 2674 2024-09-19T04:32:38.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 04:15 2675 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2678 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2679 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2680 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2681 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2682 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2683 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2684 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2685 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2686 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2687 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2688 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2689 2024-09-19T04:32:38.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 04:15 2690 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 04:15 2691 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2692 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2693 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2696 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2699 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2701 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2702 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2703 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2704 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2753 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2753_fsm 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2753_vm 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2754 2024-09-19T04:32:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2755 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2756 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2757 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2830 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2831 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2832 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2833 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2834 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2835 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2836 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2837 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 04:15 2838 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2838_fsm 2024-09-19T04:32:38.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2838_vm 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2839 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840_fsm 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2840_vm 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2841 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2995 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2996 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3079_fsm 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079_vm 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3080 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3081 2024-09-19T04:32:38.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3085 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3118 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3119 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 04:15 3164 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3256 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3257 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3258 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3350 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3351 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3379 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3380 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3381 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3394 2024-09-19T04:32:38.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3394_fsm 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3394_vm 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3395 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3429 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3430 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3431 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3433 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3439 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3440 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3455 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 04:15 3456 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3456_fsm 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3456_vm 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3466 2024-09-19T04:32:38.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3467 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3468 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3501 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3502 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3503 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3534 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3541_fsm 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541_vm 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3542 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3574 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3575 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3576 2024-09-19T04:32:38.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3596 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3597 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3598 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3599 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3600_fsm 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600_vm 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3601_fsm 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601_vm 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3602_fsm 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602_vm 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603 2024-09-19T04:32:38.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603_fsm 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3603_vm 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3604 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3605 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3606 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3607 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3608 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 3609 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3712 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3764_fsm 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764_vm 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3766 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3767 2024-09-19T04:32:38.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3997 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4143 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4144 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4145 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4146 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4147 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4148 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4149 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4150 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4151 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4152 2024-09-19T04:32:38.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4153 2024-09-19T04:32:38.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4154 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4155 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4156 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4157 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4158 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4159 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4160 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4161 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4162 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4163 2024-09-19T04:32:38.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4164 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4165 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4166 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4167 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4168 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4169 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4170 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4171 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4172 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4173 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4174 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 5002 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 548 2024-09-19T04:32:38.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 549 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6102 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6104 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6106 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6110 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6111 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6112 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6113 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6117 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 826 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 827 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 828 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 04:15 pg_filenode.map 2024-09-19T04:32:38.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 04:15 PG_VERSION 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:total 565423 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 578990639 Sep 19 04:21 . 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 594916917 Sep 19 04:16 .. 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 112 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 113 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:16 1247 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1247_fsm 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:16 1247_vm 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 04:16 1249 2024-09-19T04:32:38.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:16 1249_fsm 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:16 1249_vm 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 04:15 1255 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1255_fsm 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1255_vm 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 04:32 1259 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1259_fsm 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:16 1259_vm 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 04:15 13344 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13344_fsm 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13344_vm 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13346 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13348 2024-09-19T04:32:38.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13349_fsm 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13349_vm 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13351 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13353 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13354_fsm 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13354_vm 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13356 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13358 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 13359_fsm 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13359_vm 2024-09-19T04:32:38.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 13361 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 13363 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1417 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 1418 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442384384 Sep 19 04:32 16396 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 19 04:31 16396_fsm 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:23 16396_vm 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1957888 Sep 19 04:31 16397 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:32 16397_fsm 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:32 16397_vm 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2850816 Sep 19 04:31 16399 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:32 16399_fsm 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:32 16399_vm 2024-09-19T04:32:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:31 16400 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 04:31 16402 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 19 04:32 16404 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51421184 Sep 19 04:32 16406 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:32 16406_fsm 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:29 16406_vm 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 174 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 175 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2187 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2224 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2328 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2336 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2337 2024-09-19T04:32:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:16 2579 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2600 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2600_fsm 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2600_vm 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2601_fsm 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2601_vm 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2602 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2602_fsm 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2602_vm 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2603 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2603_fsm 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2603_vm 2024-09-19T04:32:38.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2604 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2605 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2605_fsm 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2605_vm 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:16 2606 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2606_fsm 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:16 2606_vm 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2607 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2607_fsm 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2607_vm 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 04:16 2608 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2608_fsm 2024-09-19T04:32:38.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:16 2608_vm 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 04:15 2609 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2609_fsm 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2609_vm 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:16 2610 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2610_fsm 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:16 2610_vm 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2611 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2612_fsm 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2612_vm 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2613 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2615_fsm 2024-09-19T04:32:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2615_vm 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2616 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2616_fsm 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2616_vm 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 04:15 2617 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2617_fsm 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2617_vm 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 04:15 2618 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2618_fsm 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2618_vm 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 19 04:32 2619 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:16 2619_fsm 2024-09-19T04:32:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:16 2619_vm 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2620 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2650 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2651 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2652 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2653 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2654 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2655 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2656 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2657 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 04:16 2658 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 04:16 2659 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2660 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2661 2024-09-19T04:32:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:16 2662 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:16 2663 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:16 2664 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:16 2665 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:16 2666 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:16 2667 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2668 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2669 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2670 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 04:16 2673 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 04:16 2674 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 04:15 2675 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:16 2678 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:16 2679 2024-09-19T04:32:38.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2680 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2681 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2682 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2683 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2684 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2685 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2686 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2687 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 2688 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 04:15 2689 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 04:15 2690 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 04:15 2691 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2692 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2693 2024-09-19T04:32:38.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:18 2696 2024-09-19T04:32:38.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2699 2024-09-19T04:32:38.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2701 2024-09-19T04:32:38.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2702 2024-09-19T04:32:38.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:16 2703 2024-09-19T04:32:38.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:16 2704 2024-09-19T04:32:38.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2753 2024-09-19T04:32:38.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2753_fsm 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2753_vm 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2754 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2755 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2756 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 2757 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2830 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2831 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2832 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2833 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2834 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2835 2024-09-19T04:32:38.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2836 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2837 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 04:15 2838 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2838_fsm 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2838_vm 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2839 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2840_fsm 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2840_vm 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2841 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2995 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2996 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3079_fsm 2024-09-19T04:32:38.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3079_vm 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3080 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3081 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:15 3085 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3118 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3119 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 04:15 3164 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3256 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3257 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3258 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3350 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3351 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3379 2024-09-19T04:32:38.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3380 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3381 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3394 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3394_fsm 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3394_vm 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3395 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3429 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3430 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3431 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3433 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3439 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3440 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 04:16 3455 2024-09-19T04:32:38.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 04:15 3456 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3456_fsm 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3456_vm 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3466 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3467 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3468 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3501 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3502 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3503 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3534 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3541_fsm 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3541_vm 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3542 2024-09-19T04:32:38.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3574 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3575 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3576 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3596 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3597 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3598 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3599 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3600_fsm 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3600_vm 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3601_fsm 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3601_vm 2024-09-19T04:32:38.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3602_fsm 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3602_vm 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3603_fsm 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3603_vm 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3604 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3605 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3606 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3607 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3608 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 04:15 3609 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3712 2024-09-19T04:32:38.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 3764_fsm 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3764_vm 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3766 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 3767 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3997 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4143 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4144 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4145 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4146 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4147 2024-09-19T04:32:38.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4148 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4149 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4150 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4151 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4152 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4153 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4154 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4155 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4156 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4157 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4158 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4159 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4160 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4161 2024-09-19T04:32:38.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4162 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4163 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4164 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4165 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4166 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4167 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4168 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4169 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4170 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4171 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4172 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4173 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4174 2024-09-19T04:32:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 5002 2024-09-19T04:32:38.744 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 548 2024-09-19T04:32:38.744 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 549 2024-09-19T04:32:38.744 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6102 2024-09-19T04:32:38.744 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6104 2024-09-19T04:32:38.744 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6106 2024-09-19T04:32:38.744 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6110 2024-09-19T04:32:38.768 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6111 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6112 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6113 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6117 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 826 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 827 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 828 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 04:15 pg_filenode.map 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 19 04:15 pg_internal.init 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 04:15 PG_VERSION 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-19T04:32:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:16 . 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 594916917 Sep 19 04:16 .. 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/global: 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:total 561 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 Sep 19 04:15 . 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1213 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1213_fsm 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1213_vm 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1214 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1214_fsm 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1214_vm 2024-09-19T04:32:38.770 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 1232 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 1233 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1260 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1260_fsm 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1260_vm 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1261 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1261_fsm 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1261_vm 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1262 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 1262_fsm 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 1262_vm 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2396 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 04:15 2396_fsm 2024-09-19T04:32:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2396_vm 2024-09-19T04:32:38.772 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2397 2024-09-19T04:32:38.772 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2671 2024-09-19T04:32:38.772 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2672 2024-09-19T04:32:38.772 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2676 2024-09-19T04:32:38.772 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2677 2024-09-19T04:32:38.772 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2694 2024-09-19T04:32:38.783 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2695 2024-09-19T04:32:38.783 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2697 2024-09-19T04:32:38.783 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 2698 2024-09-19T04:32:38.783 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2846 2024-09-19T04:32:38.783 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2847 2024-09-19T04:32:38.783 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2964 2024-09-19T04:32:38.783 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2965 2024-09-19T04:32:38.783 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 2966 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 2967 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 3592 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 3593 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4060 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4061 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4175 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4176 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4177 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4178 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4181 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4182 2024-09-19T04:32:38.784 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4183 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4184 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 4185 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 4186 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6000 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6001 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6002 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 6100 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6114 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 6115 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:32 pg_control 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 04:15 pg_filenode.map 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 19 04:15 pg_internal.init 2024-09-19T04:32:38.785 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/log: 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:total 3 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 Sep 19 04:15 . 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 Sep 19 04:32 postgresql-Thu.log 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.786 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.787 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.787 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.787 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-19T04:32:38.787 INFO:teuthology.orchestra.run.smithi094.stdout:total 3 2024-09-19T04:32:38.827 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:32 . 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 mappings 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:32 replorigin_checkpoint 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 snapshots 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:32 .. 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-19T04:32:38.828 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 04:32 .. 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:total 2 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 . 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 members 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 offsets 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:total 9 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 . 2024-09-19T04:32:38.829 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 .. 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 0000 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:total 9 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:15 . 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 04:15 .. 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 04:20 0000 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.830 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.831 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:total 16 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Sep 19 04:32 . 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 19 04:32 db_0.stat 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 19 04:32 db_13494.stat 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 19 04:32 global.stat 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Sep 19 04:32 . 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.832 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.833 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-19T04:32:38.833 INFO:teuthology.orchestra.run.smithi094.stdout:total 257 2024-09-19T04:32:38.833 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 19 04:32 . 2024-09-19T04:32:38.833 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.892 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 19 04:32 000E 2024-09-19T04:32:38.892 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.892 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-19T04:32:38.892 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.892 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.892 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.892 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:total 999426 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1040187392 Sep 19 04:32 . 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:32 000000010000000000000074 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:23 000000010000000000000075 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:21 000000010000000000000076 2024-09-19T04:32:38.893 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:21 000000010000000000000077 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:20 000000010000000000000078 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:20 000000010000000000000079 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:20 00000001000000000000007A 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:21 00000001000000000000007B 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:20 00000001000000000000007C 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:22 00000001000000000000007D 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:20 00000001000000000000007E 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:24 00000001000000000000007F 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:21 000000010000000000000080 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:21 000000010000000000000081 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:24 000000010000000000000082 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:20 000000010000000000000083 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:23 000000010000000000000084 2024-09-19T04:32:38.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:21 000000010000000000000085 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:23 000000010000000000000086 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:21 000000010000000000000087 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:20 000000010000000000000088 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:20 000000010000000000000089 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 00000001000000000000008A 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 00000001000000000000008B 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:27 00000001000000000000008C 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:27 00000001000000000000008D 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 00000001000000000000008E 2024-09-19T04:32:38.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 00000001000000000000008F 2024-09-19T04:32:38.896 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 000000010000000000000090 2024-09-19T04:32:38.896 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:26 000000010000000000000091 2024-09-19T04:32:38.896 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:29 000000010000000000000092 2024-09-19T04:32:38.896 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 000000010000000000000093 2024-09-19T04:32:38.896 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:26 000000010000000000000094 2024-09-19T04:32:38.896 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:29 000000010000000000000095 2024-09-19T04:32:38.896 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:26 000000010000000000000096 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:28 000000010000000000000097 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 000000010000000000000098 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:26 000000010000000000000099 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:28 00000001000000000000009A 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 00000001000000000000009B 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 00000001000000000000009C 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 00000001000000000000009D 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:26 00000001000000000000009E 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 00000001000000000000009F 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:30 0000000100000000000000A0 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 0000000100000000000000A1 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:26 0000000100000000000000A2 2024-09-19T04:32:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 0000000100000000000000A3 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 0000000100000000000000A4 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 0000000100000000000000A5 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 0000000100000000000000A6 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:26 0000000100000000000000A7 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:25 0000000100000000000000A8 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:26 0000000100000000000000A9 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:31 0000000100000000000000AA 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:30 0000000100000000000000AB 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:31 0000000100000000000000AC 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:30 0000000100000000000000AD 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:30 0000000100000000000000AE 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:30 0000000100000000000000AF 2024-09-19T04:32:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 04:30 0000000100000000000000B0 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 archive_status 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 04:15 . 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1040187392 Sep 19 04:32 .. 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout:total 241 2024-09-19T04:32:38.899 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 245760 Sep 19 04:15 . 2024-09-19T04:32:38.900 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1636264734 Sep 19 04:32 .. 2024-09-19T04:32:38.900 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 245760 Sep 19 04:32 0000 2024-09-19T04:32:38.902 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T04:32:38.912 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T04:32:38.913 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:32:38.913 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T04:32:38.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T04:32:38.967 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T04:32:39.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T04:32:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.236+0000 7f2e87045640 1 -- 172.21.15.94:0/3122635754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8010c0f0 msgr2=0x7f2e8010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.236+0000 7f2e87045640 1 --2- 172.21.15.94:0/3122635754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8010c0f0 0x7f2e8010e580 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009ca0 tx=0x7f2e7802a950 comp rx=0 tx=0).stop 2024-09-19T04:32:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.236+0000 7f2e87045640 1 -- 172.21.15.94:0/3122635754 shutdown_connections 2024-09-19T04:32:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.236+0000 7f2e87045640 1 --2- 172.21.15.94:0/3122635754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8010c0f0 0x7f2e8010e580 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.236+0000 7f2e87045640 1 --2- 172.21.15.94:0/3122635754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e80103680 0x7f2e8010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.236+0000 7f2e87045640 1 --2- 172.21.15.94:0/3122635754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e80102d60 0x7f2e80103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.236+0000 7f2e87045640 1 -- 172.21.15.94:0/3122635754 >> 172.21.15.94:0/3122635754 conn(0x7f2e800fc6f0 msgr2=0x7f2e800feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.236+0000 7f2e87045640 1 -- 172.21.15.94:0/3122635754 shutdown_connections 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e87045640 1 -- 172.21.15.94:0/3122635754 wait complete. 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e87045640 1 Processor -- start 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e87045640 1 -- start start 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e87045640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e80102d60 0x7f2e80198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e87045640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e80103680 0x7f2e80198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e87045640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8010c0f0 0x7f2e8019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e87045640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80111d50 con 0x7f2e80102d60 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e87045640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80111bd0 con 0x7f2e8010c0f0 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e87045640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80111ed0 con 0x7f2e80103680 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e84de1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e80102d60 0x7f2e80198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e84de1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e80102d60 0x7f2e80198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35094/0 (socket says 172.21.15.94:35094) 2024-09-19T04:32:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.237+0000 7f2e84de1640 1 -- 172.21.15.94:0/1572139613 learned_addr learned my addr 172.21.15.94:0/1572139613 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e84de1640 1 -- 172.21.15.94:0/1572139613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e80103680 msgr2=0x7f2e80198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e84de1640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e80103680 0x7f2e80198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e84de1640 1 -- 172.21.15.94:0/1572139613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8010c0f0 msgr2=0x7f2e8019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e855e2640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8010c0f0 0x7f2e8019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e84de1640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8010c0f0 0x7f2e8019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e84de1640 1 -- 172.21.15.94:0/1572139613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8019d200 con 0x7f2e80102d60 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e84de1640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e80102d60 0x7f2e80198210 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400d950 tx=0x7f2e7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e7dffb640 1 -- 172.21.15.94:0/1572139613 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74017070 con 0x7f2e80102d60 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8019d4f0 con 0x7f2e80102d60 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e7dffb640 1 -- 172.21.15.94:0/1572139613 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e74009c30 con 0x7f2e80102d60 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e7dffb640 1 -- 172.21.15.94:0/1572139613 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7400f620 con 0x7f2e80102d60 2024-09-19T04:32:39.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.238+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e801a4d80 con 0x7f2e80102d60 2024-09-19T04:32:39.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.239+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8006ab20 con 0x7f2e80102d60 2024-09-19T04:32:39.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.240+0000 7f2e7dffb640 1 -- 172.21.15.94:0/1572139613 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2e740109a0 con 0x7f2e80102d60 2024-09-19T04:32:39.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.241+0000 7f2e7dffb640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2e700a3460 0x7f2e700a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:39.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.241+0000 7f2e7ffff640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2e700a3460 0x7f2e700a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:39.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.241+0000 7f2e7dffb640 1 -- 172.21.15.94:0/1572139613 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f2e740cfaa0 con 0x7f2e80102d60 2024-09-19T04:32:39.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.241+0000 7f2e7ffff640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2e700a3460 0x7f2e700a5920 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c005d80 tx=0x7f2e6c005c90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.245+0000 7f2e7dffb640 1 -- 172.21.15.94:0/1572139613 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e74009de0 con 0x7f2e80102d60 2024-09-19T04:32:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.434+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 --> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2e80062630 con 0x7f2e700a3460 2024-09-19T04:32:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.442+0000 7f2e7dffb640 1 -- 172.21.15.94:0/1572139613 <== mgr.15159 v2:172.21.15.94:6800/3395191155 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f2e80062630 con 0x7f2e700a3460 2024-09-19T04:32:39.443 INFO:teuthology.orchestra.run.smithi094.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T04:15:12", "first": "2024-09-19T04:16:00", "last": "2024-09-19T04:32:00", "last_pruned": "2024-09-19T04:32:00", "created_count": 17, "pruned_count": 11, "active": true} 2024-09-19T04:32:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2e700a3460 msgr2=0x7f2e700a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2e700a3460 0x7f2e700a5920 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c005d80 tx=0x7f2e6c005c90 comp rx=0 tx=0).stop 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e80102d60 msgr2=0x7f2e80198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e80102d60 0x7f2e80198210 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400d950 tx=0x7f2e7400de10 comp rx=0 tx=0).stop 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 shutdown_connections 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f2e700a3460 0x7f2e700a5920 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8010c0f0 0x7f2e8019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e80103680 0x7f2e80198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 --2- 172.21.15.94:0/1572139613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e80102d60 0x7f2e80198210 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 >> 172.21.15.94:0/1572139613 conn(0x7f2e800fc6f0 msgr2=0x7f2e800fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 shutdown_connections 2024-09-19T04:32:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.444+0000 7f2e87045640 1 -- 172.21.15.94:0/1572139613 wait complete. 2024-09-19T04:32:39.457 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T04:32:39.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:39 smithi112 ceph-mon[35051]: pgmap v553: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 14 op/s 2024-09-19T04:32:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.556+0000 7f32c7d42640 1 -- 172.21.15.94:0/142182515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0103640 msgr2=0x7f32c010bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.556+0000 7f32c7d42640 1 --2- 172.21.15.94:0/142182515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0103640 0x7f32c010bb20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009900 tx=0x7f32b002a480 comp rx=0 tx=0).stop 2024-09-19T04:32:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.556+0000 7f32c7d42640 1 -- 172.21.15.94:0/142182515 shutdown_connections 2024-09-19T04:32:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.556+0000 7f32c7d42640 1 --2- 172.21.15.94:0/142182515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c010c0f0 0x7f32c010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.556+0000 7f32c7d42640 1 --2- 172.21.15.94:0/142182515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0103640 0x7f32c010bb20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.556+0000 7f32c7d42640 1 --2- 172.21.15.94:0/142182515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0102d20 0x7f32c0103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.556+0000 7f32c7d42640 1 -- 172.21.15.94:0/142182515 >> 172.21.15.94:0/142182515 conn(0x7f32c0077990 msgr2=0x7f32c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 -- 172.21.15.94:0/142182515 shutdown_connections 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 -- 172.21.15.94:0/142182515 wait complete. 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 Processor -- start 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 -- start start 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0102d20 0x7f32c01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0103640 0x7f32c01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c010c0f0 0x7f32c019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0111d70 con 0x7f32c0102d20 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0111bf0 con 0x7f32c0103640 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c7d42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0111ef0 con 0x7f32c010c0f0 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c5ade640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0102d20 0x7f32c01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c5ade640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0102d20 0x7f32c01982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35124/0 (socket says 172.21.15.94:35124) 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c5ade640 1 -- 172.21.15.94:0/551102821 learned_addr learned my addr 172.21.15.94:0/551102821 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.557+0000 7f32c52dd640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0103640 0x7f32c01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32c62df640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c010c0f0 0x7f32c019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32c5ade640 1 -- 172.21.15.94:0/551102821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c010c0f0 msgr2=0x7f32c019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32c5ade640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c010c0f0 0x7f32c019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32c5ade640 1 -- 172.21.15.94:0/551102821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0103640 msgr2=0x7f32c01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32c5ade640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0103640 0x7f32c01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32c5ade640 1 -- 172.21.15.94:0/551102821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c019d2a0 con 0x7f32c0102d20 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32c5ade640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0102d20 0x7f32c01982b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f32b800ebc0 tx=0x7f32b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32b6ffd640 1 -- 172.21.15.94:0/551102821 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8010630 con 0x7f32c0102d20 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32b6ffd640 1 -- 172.21.15.94:0/551102821 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b8010c80 con 0x7f32c0102d20 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c019d590 con 0x7f32c0102d20 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32b6ffd640 1 -- 172.21.15.94:0/551102821 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8018b30 con 0x7f32c0102d20 2024-09-19T04:32:39.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.558+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01a4e20 con 0x7f32c0102d20 2024-09-19T04:32:39.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.559+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c0103100 con 0x7f32c0102d20 2024-09-19T04:32:39.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.560+0000 7f32b6ffd640 1 -- 172.21.15.94:0/551102821 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f32b8018cd0 con 0x7f32c0102d20 2024-09-19T04:32:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.561+0000 7f32b6ffd640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f32ac0a3460 0x7f32ac0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.561+0000 7f32c52dd640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f32ac0a3460 0x7f32ac0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.561+0000 7f32b6ffd640 1 -- 172.21.15.94:0/551102821 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f32b80d03f0 con 0x7f32c0102d20 2024-09-19T04:32:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.561+0000 7f32c52dd640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f32ac0a3460 0x7f32ac0a5920 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f32b0031440 tx=0x7f32b0005b50 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:39.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.565+0000 7f32b6ffd640 1 -- 172.21.15.94:0/551102821 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f32b809cd90 con 0x7f32c0102d20 2024-09-19T04:32:39.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:39 smithi094 ceph-mon[30972]: pgmap v553: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 14 op/s 2024-09-19T04:32:39.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:39 smithi104 ceph-mon[34145]: pgmap v553: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 14 op/s 2024-09-19T04:32:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.753+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 --> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f32c0062630 con 0x7f32ac0a3460 2024-09-19T04:32:39.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.760+0000 7f32b6ffd640 1 -- 172.21.15.94:0/551102821 <== mgr.15159 v2:172.21.15.94:6800/3395191155 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f32c0062630 con 0x7f32ac0a3460 2024-09-19T04:32:39.762 INFO:teuthology.orchestra.run.smithi094.stdout:/ 1m 6m3h 2024-09-19T04:32:39.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f32ac0a3460 msgr2=0x7f32ac0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f32ac0a3460 0x7f32ac0a5920 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f32b0031440 tx=0x7f32b0005b50 comp rx=0 tx=0).stop 2024-09-19T04:32:39.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0102d20 msgr2=0x7f32c01982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0102d20 0x7f32c01982b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f32b800ebc0 tx=0x7f32b800c6a0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 shutdown_connections 2024-09-19T04:32:39.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f32ac0a3460 0x7f32ac0a5920 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c010c0f0 0x7f32c019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0103640 0x7f32c01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 --2- 172.21.15.94:0/551102821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c0102d20 0x7f32c01982b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.762+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 >> 172.21.15.94:0/551102821 conn(0x7f32c0077990 msgr2=0x7f32c00fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.763+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 shutdown_connections 2024-09-19T04:32:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:39.763+0000 7f32c7d42640 1 -- 172.21.15.94:0/551102821 wait complete. 2024-09-19T04:32:39.775 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T04:32:39.785 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T04:32:40.534 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:40 smithi112 ceph-mon[35051]: from='client.17640 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:32:40.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:40 smithi094 ceph-mon[30972]: from='client.17640 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:32:40.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:40 smithi104 ceph-mon[34145]: from='client.17640 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:32:41.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:41 smithi112 ceph-mon[35051]: pgmap v554: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 102 KiB/s wr, 14 op/s 2024-09-19T04:32:41.535 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:41 smithi112 ceph-mon[35051]: from='client.17646 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:32:41.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:41 smithi094 ceph-mon[30972]: pgmap v554: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 102 KiB/s wr, 14 op/s 2024-09-19T04:32:41.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:41 smithi094 ceph-mon[30972]: from='client.17646 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:32:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:41 smithi104 ceph-mon[34145]: pgmap v554: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 102 KiB/s wr, 14 op/s 2024-09-19T04:32:41.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:41 smithi104 ceph-mon[34145]: from='client.17646 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:32:43.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:43 smithi094 ceph-mon[30972]: pgmap v555: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 81 KiB/s wr, 11 op/s 2024-09-19T04:32:43.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:43 smithi104 ceph-mon[34145]: pgmap v555: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 81 KiB/s wr, 11 op/s 2024-09-19T04:32:43.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:43 smithi112 ceph-mon[35051]: pgmap v555: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 81 KiB/s wr, 11 op/s 2024-09-19T04:32:44.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:44 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:44 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:44.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:44 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:45.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:45 smithi094 ceph-mon[30972]: pgmap v556: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 103 KiB/s wr, 21 op/s 2024-09-19T04:32:45.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:45 smithi104 ceph-mon[34145]: pgmap v556: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 103 KiB/s wr, 21 op/s 2024-09-19T04:32:45.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:45 smithi112 ceph-mon[35051]: pgmap v556: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 103 KiB/s wr, 21 op/s 2024-09-19T04:32:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:47 smithi094 ceph-mon[30972]: pgmap v557: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-19T04:32:47.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:47 smithi104 ceph-mon[34145]: pgmap v557: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-19T04:32:47.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:47 smithi112 ceph-mon[35051]: pgmap v557: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-19T04:32:49.587 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:49 smithi112 ceph-mon[35051]: pgmap v558: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 16 op/s 2024-09-19T04:32:49.587 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:49 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:49.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:49 smithi094 ceph-mon[30972]: pgmap v558: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 16 op/s 2024-09-19T04:32:49.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:49 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:49 smithi104 ceph-mon[34145]: pgmap v558: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 16 op/s 2024-09-19T04:32:49.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:49 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:51.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:51 smithi094 ceph-mon[30972]: pgmap v559: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 292 KiB/s wr, 21 op/s 2024-09-19T04:32:51.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:51 smithi104 ceph-mon[34145]: pgmap v559: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 292 KiB/s wr, 21 op/s 2024-09-19T04:32:51.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:51 smithi112 ceph-mon[35051]: pgmap v559: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 292 KiB/s wr, 21 op/s 2024-09-19T04:32:52.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:32:52.299 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:52.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.792+0000 7fc98b5c1640 1 -- 172.21.15.94:0/2618182527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984102570 msgr2=0x7fc9841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:52.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.792+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/2618182527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984102570 0x7fc9841029b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc974009900 tx=0x7fc97402a480 comp rx=0 tx=0).stop 2024-09-19T04:32:52.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.792+0000 7fc98b5c1640 1 -- 172.21.15.94:0/2618182527 shutdown_connections 2024-09-19T04:32:52.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.792+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/2618182527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102ef0 0x7fc98410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.792+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/2618182527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984102570 0x7fc9841029b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.792+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/2618182527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984100e20 0x7fc984101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.792+0000 7fc98b5c1640 1 -- 172.21.15.94:0/2618182527 >> 172.21.15.94:0/2618182527 conn(0x7fc9840fc7d0 msgr2=0x7fc9840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.792+0000 7fc98b5c1640 1 -- 172.21.15.94:0/2618182527 shutdown_connections 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.792+0000 7fc98b5c1640 1 -- 172.21.15.94:0/2618182527 wait complete. 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98b5c1640 1 Processor -- start 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98b5c1640 1 -- start start 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98b5c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984100e20 0x7fc984197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98b5c1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984102570 0x7fc9841975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98b5c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102ef0 0x7fc984197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98b5c1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc984110e00 con 0x7fc984100e20 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98b5c1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc984110c80 con 0x7fc984102570 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98b5c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc984110f80 con 0x7fc984102ef0 2024-09-19T04:32:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98935d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984100e20 0x7fc984197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98935d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984100e20 0x7fc984197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51690/0 (socket says 172.21.15.94:51690) 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.793+0000 7fc98935d640 1 -- 172.21.15.94:0/3366358304 learned_addr learned my addr 172.21.15.94:0/3366358304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc988b5c640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984102570 0x7fc9841975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc989b5e640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102ef0 0x7fc984197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc98935d640 1 -- 172.21.15.94:0/3366358304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102ef0 msgr2=0x7fc984197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc98935d640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102ef0 0x7fc984197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc98935d640 1 -- 172.21.15.94:0/3366358304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984102570 msgr2=0x7fc9841975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc98935d640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984102570 0x7fc9841975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc98935d640 1 -- 172.21.15.94:0/3366358304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9841a68d0 con 0x7fc984100e20 2024-09-19T04:32:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc989b5e640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102ef0 0x7fc984197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc98935d640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984100e20 0x7fc984197060 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d950 tx=0x7fc98000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc97a7fc640 1 -- 172.21.15.94:0/3366358304 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980017070 con 0x7fc984100e20 2024-09-19T04:32:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc98b5c1640 1 -- 172.21.15.94:0/3366358304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841a6bc0 con 0x7fc984100e20 2024-09-19T04:32:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc97a7fc640 1 -- 172.21.15.94:0/3366358304 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc980009c30 con 0x7fc984100e20 2024-09-19T04:32:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc97a7fc640 1 -- 172.21.15.94:0/3366358304 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000f620 con 0x7fc984100e20 2024-09-19T04:32:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.794+0000 7fc98b5c1640 1 -- 172.21.15.94:0/3366358304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a7120 con 0x7fc984100e20 2024-09-19T04:32:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.797+0000 7fc98d99a740 1 Processor -- start 2024-09-19T04:32:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.797+0000 7fc98d99a740 1 -- start start 2024-09-19T04:32:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.797+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a504a0 0x55db71a50880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.797+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a50dc0 0x55db71a5b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.797+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a5b680 0x55db71a5da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.797+0000 7fc98d99a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db719b62e0 con 0x55db71a5b680 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.797+0000 7fc98d99a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db7195dce0 con 0x55db71a504a0 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.797+0000 7fc98d99a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db719b22a0 con 0x55db71a50dc0 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc97a7fc640 1 -- 172.21.15.94:0/3366358304 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc9800109a0 con 0x7fc984100e20 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc988b5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a5b680 0x55db71a5da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc988b5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a5b680 0x55db71a5da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51702/0 (socket says 172.21.15.94:51702) 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc988b5c640 1 -- 172.21.15.94:0/3732145215 learned_addr learned my addr 172.21.15.94:0/3732145215 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc989b5e640 1 --2- 172.21.15.94:0/3732145215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a504a0 0x55db71a50880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc98935d640 1 --2- 172.21.15.94:0/3732145215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a50dc0 0x55db71a5b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc988b5c640 1 -- 172.21.15.94:0/3732145215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a50dc0 msgr2=0x55db71a5b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc988b5c640 1 --2- 172.21.15.94:0/3732145215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a50dc0 0x55db71a5b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc988b5c640 1 -- 172.21.15.94:0/3732145215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a504a0 msgr2=0x55db71a50880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc988b5c640 1 --2- 172.21.15.94:0/3732145215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a504a0 0x55db71a50880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc988b5c640 1 -- 172.21.15.94:0/3732145215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db71942440 con 0x55db71a5b680 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc988b5c640 1 --2- 172.21.15.94:0/3732145215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a5b680 0x55db71a5da70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc974009900 tx=0x7fc974031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6a3f6ee92ef398 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc96f7fe640 1 -- 172.21.15.94:0/3732145215 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974040070 con 0x55db71a5b680 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc96f7fe640 1 -- 172.21.15.94:0/3732145215 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc974031630 con 0x55db71a5b680 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.798+0000 7fc96f7fe640 1 -- 172.21.15.94:0/3732145215 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974035040 con 0x55db71a5b680 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc97a7fc640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc9700a3430 0x7fc9700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc989b5e640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc9700a3430 0x7fc9700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc97a7fc640 1 -- 172.21.15.94:0/3366358304 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7fc9800cf980 con 0x7fc984100e20 2024-09-19T04:32:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc98d99a740 1 -- 172.21.15.94:0/3732145215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a5b680 msgr2=0x55db71a5da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc989b5e640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc9700a3430 0x7fc9700a58f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc97c005fd0 tx=0x7fc97c008630 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3732145215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a5b680 0x55db71a5da70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc974009900 tx=0x7fc974031c60 comp rx=0 tx=0).stop 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc98d99a740 1 -- 172.21.15.94:0/3732145215 shutdown_connections 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3732145215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a5b680 0x55db71a5da70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3732145215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a50dc0 0x55db71a5b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3732145215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a504a0 0x55db71a50880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc98d99a740 1 -- 172.21.15.94:0/3732145215 >> 172.21.15.94:0/3732145215 conn(0x55db71a4ca30 msgr2=0x55db71a4ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc98d99a740 1 -- 172.21.15.94:0/3732145215 shutdown_connections 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.799+0000 7fc98d99a740 1 -- 172.21.15.94:0/3732145215 wait complete. 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98d99a740 1 -- start start 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98d99a740 1 Processor -- start 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 0x55db71a5ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 0x55db71a58620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98d99a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db71a5e250 con 0x55db71a50dc0 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98d99a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db71971050 con 0x55db71a5b680 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98d99a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db71a5e3d0 con 0x55db71a504a0 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98d99a740 10 client.0 _ll_register_callbacks cb 0x55db719b1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc988b5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc988b5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51710/0 (socket says 172.21.15.94:51710) 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc988b5c640 1 -- 172.21.15.94:0/2844657189 learned_addr learned my addr 172.21.15.94:0/2844657189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.800+0000 7fc98935d640 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 0x55db71a5ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc989b5e640 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 0x55db71a58620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc988b5c640 1 -- 172.21.15.94:0/2844657189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 msgr2=0x55db71a5ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc988b5c640 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 0x55db71a5ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc988b5c640 1 -- 172.21.15.94:0/2844657189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 msgr2=0x55db71a58620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc988b5c640 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 0x55db71a58620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc988b5c640 1 -- 172.21.15.94:0/2844657189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db717273c0 con 0x55db71a50dc0 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc988b5c640 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a580e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc974005330 tx=0x7fc9740095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec91aacaec952c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc96cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc96cff9640 1 -- 172.21.15.94:0/2844657189 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974040030 con 0x55db71a50dc0 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc96cff9640 1 -- 172.21.15.94:0/2844657189 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7fc9740055e0 con 0x55db71a50dc0 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc98d99a740 1 -- 172.21.15.94:0/2844657189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc974031630 con 0x55db71a50dc0 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc96cff9640 1 -- 172.21.15.94:0/2844657189 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fc974056050 con 0x55db71a50dc0 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc98d99a740 10 client.17658 fetch_fsmap learned FSMap version 57 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.801+0000 7fc98d99a740 1 -- 172.21.15.94:0/2844657189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55db71a69a90 con 0x55db71a50dc0 2024-09-19T04:32:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.802+0000 7fc96cff9640 1 -- 172.21.15.94:0/2844657189 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc974031630 con 0x55db71a50dc0 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.802+0000 7fc96cff9640 1 -- 172.21.15.94:0/2844657189 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fc9740035f0 con 0x55db71a50dc0 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.802+0000 7fc98d99a740 10 client.17658 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.802+0000 7fc98d99a740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.802+0000 7fc98d99a740 20 client.17658 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.802+0000 7fc98d99a740 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55db71a6ade0 0x55db71a6d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.802+0000 7fc98d99a740 4 client.17658 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.802+0000 7fc98d99a740 1 -- 172.21.15.94:0/2844657189 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db719c3850 con 0x55db71a6ade0 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.802+0000 7fc98935d640 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55db71a6ade0 0x55db71a6d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.803+0000 7fc96cff9640 10 client.17658 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.803+0000 7fc98935d640 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55db71a6ade0 0x55db71a6d1d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d2e05ae1c3724f server_cookie=d87abd11e23e4bbf in_seq=0 out_seq=0 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.803+0000 7fc96cff9640 1 -- 172.21.15.94:0/2844657189 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db719c3850 con 0x55db71a6ade0 2024-09-19T04:32:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.803+0000 7fc96cff9640 10 client.17658 handle_command_reply: tid=0 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.803+0000 7fc98d99a740 1 client.17658 shutdown 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 -- 172.21.15.94:0/2844657189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 msgr2=0x55db71a580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a580e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc974005330 tx=0x7fc9740095a0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 -- 172.21.15.94:0/2844657189 shutdown_connections 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55db71a6ade0 0x55db71a6d1d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 0x55db71a58620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a580e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 --2- 172.21.15.94:0/2844657189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 0x55db71a5ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 -- 172.21.15.94:0/2844657189 >> 172.21.15.94:0/2844657189 conn(0x55db71a4ca30 msgr2=0x55db71a52710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc96cff9640 0 client.17658 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 -- 172.21.15.94:0/2844657189 shutdown_connections 2024-09-19T04:32:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 1 -- 172.21.15.94:0/2844657189 wait complete. 2024-09-19T04:32:52.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.804+0000 7fc98d99a740 20 client.17658 trim_cache size 0 max 16384 2024-09-19T04:32:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98d99a740 1 Processor -- start 2024-09-19T04:32:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98d99a740 1 -- start start 2024-09-19T04:32:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a504a0 0x55db71a4ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a50dc0 0x55db71a53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a5b680 0x55db71a55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98d99a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b2390 con 0x55db71a504a0 2024-09-19T04:32:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98d99a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841afbd0 con 0x55db71a50dc0 2024-09-19T04:32:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98d99a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841ada40 con 0x55db71a5b680 2024-09-19T04:32:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98935d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a504a0 0x55db71a4ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98935d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a504a0 0x55db71a4ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51712/0 (socket says 172.21.15.94:51712) 2024-09-19T04:32:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98935d640 1 -- 172.21.15.94:0/537489438 learned_addr learned my addr 172.21.15.94:0/537489438 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc988b5c640 1 --2- 172.21.15.94:0/537489438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a50dc0 0x55db71a53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98935d640 1 -- 172.21.15.94:0/537489438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a5b680 msgr2=0x55db71a55340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98935d640 1 --2- 172.21.15.94:0/537489438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a5b680 0x55db71a55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98935d640 1 -- 172.21.15.94:0/537489438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a50dc0 msgr2=0x55db71a53760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98935d640 1 --2- 172.21.15.94:0/537489438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a50dc0 0x55db71a53760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.821+0000 7fc98935d640 1 -- 172.21.15.94:0/537489438 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db71a57090 con 0x55db71a504a0 2024-09-19T04:32:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98935d640 1 --2- 172.21.15.94:0/537489438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a504a0 0x55db71a4ce10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc9800cd090 tx=0x7fc980086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6d43b57f3cdb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc96effd640 1 -- 172.21.15.94:0/537489438 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980020040 con 0x55db71a504a0 2024-09-19T04:32:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc96effd640 1 -- 172.21.15.94:0/537489438 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc98009ff70 con 0x55db71a504a0 2024-09-19T04:32:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc96effd640 1 -- 172.21.15.94:0/537489438 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9800860b0 con 0x55db71a504a0 2024-09-19T04:32:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98d99a740 1 -- 172.21.15.94:0/537489438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a504a0 msgr2=0x55db71a4ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98d99a740 1 --2- 172.21.15.94:0/537489438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a504a0 0x55db71a4ce10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc9800cd090 tx=0x7fc980086040 comp rx=0 tx=0).stop 2024-09-19T04:32:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98d99a740 1 -- 172.21.15.94:0/537489438 shutdown_connections 2024-09-19T04:32:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98d99a740 1 --2- 172.21.15.94:0/537489438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a5b680 0x55db71a55340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98d99a740 1 --2- 172.21.15.94:0/537489438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a50dc0 0x55db71a53760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98d99a740 1 --2- 172.21.15.94:0/537489438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a504a0 0x55db71a4ce10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98d99a740 1 -- 172.21.15.94:0/537489438 >> 172.21.15.94:0/537489438 conn(0x55db71a4ca30 msgr2=0x55db71a53050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98d99a740 1 -- 172.21.15.94:0/537489438 shutdown_connections 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.822+0000 7fc98d99a740 1 -- 172.21.15.94:0/537489438 wait complete. 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc98d99a740 1 -- start start 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc98d99a740 1 Processor -- start 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 0x55db71a70710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a6ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc98d99a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 0x55db71a6c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc98d99a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841ae090 con 0x55db71a50dc0 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc98d99a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841af500 con 0x55db71a5b680 2024-09-19T04:32:52.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc98d99a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841ae760 con 0x55db71a504a0 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc98d99a740 10 client.0 _ll_register_callbacks cb 0x7fc97402abc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc988b5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a6ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc988b5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a6ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51726/0 (socket says 172.21.15.94:51726) 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.823+0000 7fc988b5c640 1 -- 172.21.15.94:0/3050208718 learned_addr learned my addr 172.21.15.94:0/3050208718 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc989b5e640 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 0x55db71a6c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc98935d640 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 0x55db71a70710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc988b5c640 1 -- 172.21.15.94:0/3050208718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 msgr2=0x55db71a70710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc988b5c640 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 0x55db71a70710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc988b5c640 1 -- 172.21.15.94:0/3050208718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 msgr2=0x55db71a6c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc988b5c640 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 0x55db71a6c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc988b5c640 1 -- 172.21.15.94:0/3050208718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db71ad4710 con 0x55db71a50dc0 2024-09-19T04:32:52.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc98935d640 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 0x55db71a70710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc988b5c640 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a6ba40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc9740039c0 tx=0x7fc974031ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00d3379a52bd5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc96cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc96cff9640 1 -- 172.21.15.94:0/3050208718 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97405e060 con 0x55db71a50dc0 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc98d99a740 1 -- 172.21.15.94:0/3050208718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98009ff70 con 0x55db71a50dc0 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc96cff9640 1 -- 172.21.15.94:0/3050208718 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7fc974003be0 con 0x55db71a50dc0 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc96cff9640 1 -- 172.21.15.94:0/3050208718 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fc974040040 con 0x55db71a50dc0 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc98d99a740 10 client.17664 fetch_fsmap learned FSMap version 57 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc98d99a740 1 -- 172.21.15.94:0/3050208718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55db71ad58c0 con 0x55db71a50dc0 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.824+0000 7fc96cff9640 1 -- 172.21.15.94:0/3050208718 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc98009ff70 con 0x55db71a50dc0 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.825+0000 7fc96cff9640 1 -- 172.21.15.94:0/3050208718 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fc97403c670 con 0x55db71a50dc0 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.825+0000 7fc98d99a740 10 client.17664 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T04:32:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.825+0000 7fc98d99a740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.825+0000 7fc98d99a740 20 client.17664 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.825+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55db71ad5ed0 0x55db71ad82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.825+0000 7fc98d99a740 4 client.17664 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.825+0000 7fc98d99a740 1 -- 172.21.15.94:0/3050208718 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db719bdd60 con 0x55db71ad5ed0 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.825+0000 7fc98935d640 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55db71ad5ed0 0x55db71ad82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98935d640 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55db71ad5ed0 0x55db71ad82f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7db2c444d05a9ad server_cookie=6c3bc0775cc4804f in_seq=0 out_seq=0 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc96cff9640 10 client.17664 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc96cff9640 1 -- 172.21.15.94:0/3050208718 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55db719bdd60 con 0x55db71ad5ed0 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc96cff9640 10 client.17664 handle_command_reply: tid=0 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 client.17664 shutdown 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 -- 172.21.15.94:0/3050208718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 msgr2=0x55db71a6ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a6ba40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc9740039c0 tx=0x7fc974031ce0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 -- 172.21.15.94:0/3050208718 shutdown_connections 2024-09-19T04:32:52.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x55db71ad5ed0 0x55db71ad82f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db71a5b680 0x55db71a6c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db71a50dc0 0x55db71a6ba40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 --2- 172.21.15.94:0/3050208718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db71a504a0 0x55db71a70710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 -- 172.21.15.94:0/3050208718 >> 172.21.15.94:0/3050208718 conn(0x55db71a4ca30 msgr2=0x55db71a53740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:52.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc96cff9640 0 client.17664 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:52.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 -- 172.21.15.94:0/3050208718 shutdown_connections 2024-09-19T04:32:52.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 1 -- 172.21.15.94:0/3050208718 wait complete. 2024-09-19T04:32:52.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.826+0000 7fc98d99a740 20 client.17664 trim_cache size 0 max 16384 2024-09-19T04:32:52.836 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:32:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-09-19T04:32:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-09-19T04:32:52.836 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:32:52.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 -- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc9700a3430 msgr2=0x7fc9700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:52.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc9700a3430 0x7fc9700a58f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc97c005fd0 tx=0x7fc97c008630 comp rx=0 tx=0).stop 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 -- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984100e20 msgr2=0x7fc984197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984100e20 0x7fc984197060 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d950 tx=0x7fc98000de10 comp rx=0 tx=0).stop 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 -- 172.21.15.94:0/3366358304 shutdown_connections 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fc9700a3430 0x7fc9700a58f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102ef0 0x7fc984197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984102570 0x7fc9841975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 --2- 172.21.15.94:0/3366358304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc984100e20 0x7fc984197060 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 -- 172.21.15.94:0/3366358304 >> 172.21.15.94:0/3366358304 conn(0x7fc9840fc7d0 msgr2=0x7fc9840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 -- 172.21.15.94:0/3366358304 shutdown_connections 2024-09-19T04:32:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:52.827+0000 7fc98b5c1640 1 -- 172.21.15.94:0/3366358304 wait complete. 2024-09-19T04:32:53.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:53 smithi094 ceph-mon[30972]: pgmap v560: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 266 KiB/s wr, 19 op/s 2024-09-19T04:32:53.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:53 smithi104 ceph-mon[34145]: pgmap v560: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 266 KiB/s wr, 19 op/s 2024-09-19T04:32:53.717 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:32:53.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:32:53.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:53 smithi112 ceph-mon[35051]: pgmap v560: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 266 KiB/s wr, 19 op/s 2024-09-19T04:32:53.963 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:54.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:54 smithi094 ceph-mon[30972]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:54.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.429+0000 7fe89b3b9640 1 -- 172.21.15.94:0/474827416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894101420 msgr2=0x7fe89410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.429+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/474827416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894101420 0x7fe89410d950 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe884009930 tx=0x7fe88402a420 comp rx=0 tx=0).stop 2024-09-19T04:32:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.429+0000 7fe89b3b9640 1 -- 172.21.15.94:0/474827416 shutdown_connections 2024-09-19T04:32:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.429+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/474827416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894101420 0x7fe89410d950 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.429+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/474827416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe894100a80 0x7fe894100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.429+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/474827416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe894106a80 0x7fe894106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.429+0000 7fe89b3b9640 1 -- 172.21.15.94:0/474827416 >> 172.21.15.94:0/474827416 conn(0x7fe8940fc7a0 msgr2=0x7fe8940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.429+0000 7fe89b3b9640 1 -- 172.21.15.94:0/474827416 shutdown_connections 2024-09-19T04:32:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.429+0000 7fe89b3b9640 1 -- 172.21.15.94:0/474827416 wait complete. 2024-09-19T04:32:54.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe89b3b9640 1 Processor -- start 2024-09-19T04:32:54.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe89b3b9640 1 -- start start 2024-09-19T04:32:54.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe89b3b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894100a80 0x7fe894199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:54.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe89b3b9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe894101420 0x7fe894199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe89b3b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe894106a80 0x7fe8941a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe89b3b9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe894110d30 con 0x7fe894100a80 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe89b3b9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe894110bb0 con 0x7fe894101420 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe89b3b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe894110eb0 con 0x7fe894106a80 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe899155640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894100a80 0x7fe894199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe899155640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894100a80 0x7fe894199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51744/0 (socket says 172.21.15.94:51744) 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe899155640 1 -- 172.21.15.94:0/4028010315 learned_addr learned my addr 172.21.15.94:0/4028010315 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe898954640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe894101420 0x7fe894199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe899956640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe894106a80 0x7fe8941a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe899155640 1 -- 172.21.15.94:0/4028010315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe894106a80 msgr2=0x7fe8941a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe899155640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe894106a80 0x7fe8941a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe899155640 1 -- 172.21.15.94:0/4028010315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe894101420 msgr2=0x7fe894199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe899155640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe894101420 0x7fe894199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.430+0000 7fe899155640 1 -- 172.21.15.94:0/4028010315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8941a46f0 con 0x7fe894100a80 2024-09-19T04:32:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.431+0000 7fe899155640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894100a80 0x7fe894199130 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c920 tx=0x7fe88c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.431+0000 7fe8927fc640 1 -- 172.21.15.94:0/4028010315 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c007c70 con 0x7fe894100a80 2024-09-19T04:32:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.431+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8941a48c0 con 0x7fe894100a80 2024-09-19T04:32:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.431+0000 7fe8927fc640 1 -- 172.21.15.94:0/4028010315 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe88c007e10 con 0x7fe894100a80 2024-09-19T04:32:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.431+0000 7fe8927fc640 1 -- 172.21.15.94:0/4028010315 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c018820 con 0x7fe894100a80 2024-09-19T04:32:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.431+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8941a4e00 con 0x7fe894100a80 2024-09-19T04:32:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.432+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe894100ee0 con 0x7fe894100a80 2024-09-19T04:32:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.433+0000 7fe8927fc640 1 -- 172.21.15.94:0/4028010315 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe88c0189c0 con 0x7fe894100a80 2024-09-19T04:32:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.434+0000 7fe8927fc640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe87c0a33b0 0x7fe87c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.434+0000 7fe898954640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe87c0a33b0 0x7fe87c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.434+0000 7fe8927fc640 1 -- 172.21.15.94:0/4028010315 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7fe88c0cfdb0 con 0x7fe894100a80 2024-09-19T04:32:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.434+0000 7fe898954640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe87c0a33b0 0x7fe87c0a5870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe89406b2e0 tx=0x7fe88800a590 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:54.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.438+0000 7fe8927fc640 1 -- 172.21.15.94:0/4028010315 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe88c014030 con 0x7fe894100a80 2024-09-19T04:32:54.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:54 smithi104 ceph-mon[34145]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:54.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.638+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe894109010 con 0x7fe894100a80 2024-09-19T04:32:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.639+0000 7fe8927fc640 1 -- 172.21.15.94:0/4028010315 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11924 (secure 0 0 0) 0x7fe88c09c750 con 0x7fe894100a80 2024-09-19T04:32:54.640 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 58 2024-09-19T04:32:54.641 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":58,"btime":"2024-09-19T04:32:53:314324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:32:52.773119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:32:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe87c0a33b0 msgr2=0x7fe87c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe87c0a33b0 0x7fe87c0a5870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe89406b2e0 tx=0x7fe88800a590 comp rx=0 tx=0).stop 2024-09-19T04:32:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894100a80 msgr2=0x7fe894199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894100a80 0x7fe894199130 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c920 tx=0x7fe88c00cde0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 shutdown_connections 2024-09-19T04:32:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fe87c0a33b0 0x7fe87c0a5870 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe894106a80 0x7fe8941a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe894101420 0x7fe894199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 --2- 172.21.15.94:0/4028010315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe894100a80 0x7fe894199130 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 >> 172.21.15.94:0/4028010315 conn(0x7fe8940fc7a0 msgr2=0x7fe8940fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 shutdown_connections 2024-09-19T04:32:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:54.641+0000 7fe89b3b9640 1 -- 172.21.15.94:0/4028010315 wait complete. 2024-09-19T04:32:54.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:54 smithi112 ceph-mon[35051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T04:32:55.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:32:55.543 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:55 smithi094 ceph-mon[30972]: pgmap v561: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 283 KiB/s wr, 23 op/s 2024-09-19T04:32:55.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:55 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/4028010315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:32:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:55 smithi104 ceph-mon[34145]: pgmap v561: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 283 KiB/s wr, 23 op/s 2024-09-19T04:32:55.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:55 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/4028010315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:32:55.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:55 smithi112 ceph-mon[35051]: pgmap v561: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 283 KiB/s wr, 23 op/s 2024-09-19T04:32:55.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:55 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/4028010315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:32:56.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.028+0000 7f8358fa9640 1 -- 172.21.15.94:0/1907496599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102570 msgr2=0x7f83541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.028+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1907496599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102570 0x7f83541029b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8348009980 tx=0x7f834802c6b0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.028+0000 7f8358fa9640 1 -- 172.21.15.94:0/1907496599 shutdown_connections 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.028+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1907496599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8354102ef0 0x7f835410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.028+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1907496599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102570 0x7f83541029b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.028+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1907496599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8354100e20 0x7f8354101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.028+0000 7f8358fa9640 1 -- 172.21.15.94:0/1907496599 >> 172.21.15.94:0/1907496599 conn(0x7f83540fc7d0 msgr2=0x7f83540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.028+0000 7f8358fa9640 1 -- 172.21.15.94:0/1907496599 shutdown_connections 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.028+0000 7f8358fa9640 1 -- 172.21.15.94:0/1907496599 wait complete. 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8358fa9640 1 Processor -- start 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8358fa9640 1 -- start start 2024-09-19T04:32:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8358fa9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8354100e20 0x7f8354196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8358fa9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8354102570 0x7f83541967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8358fa9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102ef0 0x7f835419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8358fa9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f835410fd50 con 0x7f8354102ef0 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8358fa9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f835410fbd0 con 0x7f8354102570 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8358fa9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f835410fed0 con 0x7f8354100e20 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8352d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102ef0 0x7f835419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8352d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102ef0 0x7f835419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51758/0 (socket says 172.21.15.94:51758) 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.029+0000 7f8352d9d640 1 -- 172.21.15.94:0/1540329633 learned_addr learned my addr 172.21.15.94:0/1540329633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8351d9b640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8354102570 0x7f83541967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f835259c640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8354100e20 0x7f8354196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8352d9d640 1 -- 172.21.15.94:0/1540329633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8354100e20 msgr2=0x7f8354196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8352d9d640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8354100e20 0x7f8354196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8352d9d640 1 -- 172.21.15.94:0/1540329633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8354102570 msgr2=0x7f83541967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8352d9d640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8354102570 0x7f83541967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8352d9d640 1 -- 172.21.15.94:0/1540329633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835419b220 con 0x7f8354102ef0 2024-09-19T04:32:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8351d9b640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8354102570 0x7f83541967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8352d9d640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102ef0 0x7f835419ab40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f834c00b970 tx=0x7f834c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f83437fe640 1 -- 172.21.15.94:0/1540329633 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c019070 con 0x7f8354102ef0 2024-09-19T04:32:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8358fa9640 1 -- 172.21.15.94:0/1540329633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835419b510 con 0x7f8354102ef0 2024-09-19T04:32:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f83437fe640 1 -- 172.21.15.94:0/1540329633 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f834c014440 con 0x7f8354102ef0 2024-09-19T04:32:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f83437fe640 1 -- 172.21.15.94:0/1540329633 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c0075d0 con 0x7f8354102ef0 2024-09-19T04:32:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.030+0000 7f8358fa9640 1 -- 172.21.15.94:0/1540329633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83541a2df0 con 0x7f8354102ef0 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.033+0000 7f83437fe640 1 -- 172.21.15.94:0/1540329633 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f834c01c070 con 0x7f8354102ef0 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.033+0000 7f835b382740 1 Processor -- start 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f835b382740 1 -- start start 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f835b382740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78377b40 0x557e78376090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f835b382740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e783765d0 0x557e78376a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f835b382740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78381370 0x557e78383760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f835b382740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e782dc2f0 con 0x557e78377b40 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f835b382740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557e78283cf0 con 0x557e78381370 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f835b382740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e782d82b0 con 0x557e783765d0 2024-09-19T04:32:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f83437fe640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f833c0a33b0 0x7f833c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8351d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78377b40 0x557e78376090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8351d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78377b40 0x557e78376090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51764/0 (socket says 172.21.15.94:51764) 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8351d9b640 1 -- 172.21.15.94:0/474680295 learned_addr learned my addr 172.21.15.94:0/474680295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f835259c640 1 --2- 172.21.15.94:0/474680295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e783765d0 0x557e78376a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8352d9d640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f833c0a33b0 0x7f833c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8351d9b640 1 --2- 172.21.15.94:0/474680295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78381370 0x557e78383760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f83437fe640 1 -- 172.21.15.94:0/1540329633 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f834c0cf2c0 con 0x7f8354102ef0 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8351d9b640 1 -- 172.21.15.94:0/474680295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e783765d0 msgr2=0x557e78376a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8351d9b640 1 --2- 172.21.15.94:0/474680295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e783765d0 0x557e78376a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8351d9b640 1 -- 172.21.15.94:0/474680295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78381370 msgr2=0x557e78383760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8351d9b640 1 --2- 172.21.15.94:0/474680295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78381370 0x557e78383760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.034+0000 7f8351d9b640 1 -- 172.21.15.94:0/474680295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e782683a0 con 0x557e78377b40 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.035+0000 7f8351d9b640 1 --2- 172.21.15.94:0/474680295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78377b40 0x557e78376090 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8348034fd0 tx=0x7f83480095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c8ff6846d1123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.035+0000 7f8352d9d640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f833c0a33b0 0x7f833c0a5870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f834c097ad0 tx=0x7f834c00e040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.035+0000 7f8340ff9640 1 -- 172.21.15.94:0/474680295 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348037070 con 0x557e78377b40 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.035+0000 7f8340ff9640 1 -- 172.21.15.94:0/474680295 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8348044020 con 0x557e78377b40 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.035+0000 7f8340ff9640 1 -- 172.21.15.94:0/474680295 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83480054a0 con 0x557e78377b40 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.035+0000 7f835b382740 1 -- 172.21.15.94:0/474680295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78377b40 msgr2=0x557e78376090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.035+0000 7f835b382740 1 --2- 172.21.15.94:0/474680295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78377b40 0x557e78376090 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8348034fd0 tx=0x7f83480095e0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 -- 172.21.15.94:0/474680295 shutdown_connections 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 --2- 172.21.15.94:0/474680295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78381370 0x557e78383760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 --2- 172.21.15.94:0/474680295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e783765d0 0x557e78376a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 --2- 172.21.15.94:0/474680295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78377b40 0x557e78376090 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 -- 172.21.15.94:0/474680295 >> 172.21.15.94:0/474680295 conn(0x557e7833f310 msgr2=0x557e78372c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:56.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 -- 172.21.15.94:0/474680295 shutdown_connections 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 -- 172.21.15.94:0/474680295 wait complete. 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 -- start start 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 Processor -- start 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 0x557e78380d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 0x557e7837e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e7837e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e78388420 con 0x557e78381370 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557e78297060 con 0x557e783765d0 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e783885a0 con 0x557e78377b40 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f835b382740 10 client.0 _ll_register_callbacks cb 0x557e77fea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f8351d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e7837e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f8351d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e7837e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51778/0 (socket says 172.21.15.94:51778) 2024-09-19T04:32:56.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.036+0000 7f8351d9b640 1 -- 172.21.15.94:0/902456879 learned_addr learned my addr 172.21.15.94:0/902456879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8351d9b640 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 0x557e78380d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f835259c640 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 0x557e7837e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8351d9b640 1 -- 172.21.15.94:0/902456879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 msgr2=0x557e7837e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8351d9b640 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 0x557e7837e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8351d9b640 1 -- 172.21.15.94:0/902456879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 msgr2=0x557e78380d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8351d9b640 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 0x557e78380d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8351d9b640 1 -- 172.21.15.94:0/902456879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e7804d3c0 con 0x557e78381370 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f835259c640 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 0x557e7837e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8351d9b640 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e7837e640 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8348005e60 tx=0x7f8348009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bde80648a053ad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8339ffb640 1 -- 172.21.15.94:0/902456879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348037040 con 0x557e78381370 2024-09-19T04:32:56.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f835b382740 1 -- 172.21.15.94:0/902456879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8348044020 con 0x557e78381370 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8339ffb640 1 -- 172.21.15.94:0/902456879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f8348004260 con 0x557e78381370 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f8339ffb640 1 -- 172.21.15.94:0/902456879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f834805a050 con 0x557e78381370 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f835b382740 10 client.17682 fetch_fsmap learned FSMap version 58 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.037+0000 7f835b382740 1 -- 172.21.15.94:0/902456879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557e78393ef0 con 0x557e78381370 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.038+0000 7f8339ffb640 1 -- 172.21.15.94:0/902456879 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8348044020 con 0x557e78381370 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.038+0000 7f8339ffb640 1 -- 172.21.15.94:0/902456879 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 58) ==== 8370+0+0 (secure 0 0 0) 0x7f834803c040 con 0x557e78381370 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.038+0000 7f835b382740 10 client.17682 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.038+0000 7f835b382740 10 client.17682 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.038+0000 7f835b382740 20 client.17682 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.038+0000 7f835b382740 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e78395240 0x557e78397630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.038+0000 7f835b382740 4 client.17682 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.038+0000 7f835b382740 1 -- 172.21.15.94:0/902456879 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e782e9860 con 0x557e78395240 2024-09-19T04:32:56.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.038+0000 7f835259c640 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e78395240 0x557e78397630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.039+0000 7f835259c640 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e78395240 0x557e78397630 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d055c8e59e9ce907 server_cookie=ab26342f85c220fa in_seq=0 out_seq=0 2024-09-19T04:32:56.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.039+0000 7f8339ffb640 10 client.17682 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.039+0000 7f8339ffb640 1 -- 172.21.15.94:0/902456879 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e782e9860 con 0x557e78395240 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.039+0000 7f8339ffb640 10 client.17682 handle_command_reply: tid=0 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.039+0000 7f835b382740 1 client.17682 shutdown 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.039+0000 7f835b382740 1 -- 172.21.15.94:0/902456879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 msgr2=0x557e7837e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.039+0000 7f835b382740 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e7837e640 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8348005e60 tx=0x7f8348009650 comp rx=0 tx=0).stop 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.039+0000 7f835b382740 1 -- 172.21.15.94:0/902456879 shutdown_connections 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.040+0000 7f835b382740 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e78395240 0x557e78397630 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.040+0000 7f835b382740 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e7837e640 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.040+0000 7f835b382740 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 0x557e7837e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.040+0000 7f8339ffb640 0 client.17682 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.040+0000 7f835b382740 1 --2- 172.21.15.94:0/902456879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 0x557e78380d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.040+0000 7f835b382740 1 -- 172.21.15.94:0/902456879 >> 172.21.15.94:0/902456879 conn(0x557e7833f310 msgr2=0x557e78378960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:56.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.040+0000 7f835b382740 1 -- 172.21.15.94:0/902456879 shutdown_connections 2024-09-19T04:32:56.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.040+0000 7f835b382740 1 -- 172.21.15.94:0/902456879 wait complete. 2024-09-19T04:32:56.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.040+0000 7f835b382740 20 client.17682 trim_cache size 0 max 16384 2024-09-19T04:32:56.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.052+0000 7f835b382740 1 Processor -- start 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.052+0000 7f835b382740 1 -- start start 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.052+0000 7f835b382740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e783765d0 0x557e78371c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.052+0000 7f835b382740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78377b40 0x557e78379120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.052+0000 7f835b382740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78381370 0x557e78379660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.052+0000 7f835b382740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83541adf20 con 0x557e783765d0 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.052+0000 7f835b382740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83541ab730 con 0x557e78377b40 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.052+0000 7f835b382740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83541a95a0 con 0x557e78381370 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e783765d0 0x557e78371c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e783765d0 0x557e78371c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51790/0 (socket says 172.21.15.94:51790) 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 -- 172.21.15.94:0/671536343 learned_addr learned my addr 172.21.15.94:0/671536343 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:56.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f835259c640 1 --2- 172.21.15.94:0/671536343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78377b40 0x557e78379120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 --2- 172.21.15.94:0/671536343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78381370 0x557e78379660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 -- 172.21.15.94:0/671536343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78381370 msgr2=0x557e78379660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:56.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 --2- 172.21.15.94:0/671536343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78381370 0x557e78379660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 -- 172.21.15.94:0/671536343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78377b40 msgr2=0x557e78379120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:56.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 --2- 172.21.15.94:0/671536343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78377b40 0x557e78379120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 -- 172.21.15.94:0/671536343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e78372ed0 con 0x557e783765d0 2024-09-19T04:32:56.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f8351d9b640 1 --2- 172.21.15.94:0/671536343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e783765d0 0x557e78371c10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8348003ac0 tx=0x7f8348034e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ada73f4ac7d5806 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:56.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f833bfff640 1 -- 172.21.15.94:0/671536343 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348037040 con 0x557e783765d0 2024-09-19T04:32:56.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f833bfff640 1 -- 172.21.15.94:0/671536343 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8348044040 con 0x557e783765d0 2024-09-19T04:32:56.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.053+0000 7f833bfff640 1 -- 172.21.15.94:0/671536343 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83480044c0 con 0x557e783765d0 2024-09-19T04:32:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 -- 172.21.15.94:0/671536343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e783765d0 msgr2=0x557e78371c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 --2- 172.21.15.94:0/671536343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e783765d0 0x557e78371c10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8348003ac0 tx=0x7f8348034e90 comp rx=0 tx=0).stop 2024-09-19T04:32:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 -- 172.21.15.94:0/671536343 shutdown_connections 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 --2- 172.21.15.94:0/671536343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78381370 0x557e78379660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 --2- 172.21.15.94:0/671536343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e78377b40 0x557e78379120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 --2- 172.21.15.94:0/671536343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e783765d0 0x557e78371c10 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 -- 172.21.15.94:0/671536343 >> 172.21.15.94:0/671536343 conn(0x557e7833f310 msgr2=0x557e78371500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 -- 172.21.15.94:0/671536343 shutdown_connections 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 -- 172.21.15.94:0/671536343 wait complete. 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 -- start start 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 Processor -- start 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 0x557e78398340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 0x557e78398880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e78398dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83541a9bf0 con 0x557e78381370 2024-09-19T04:32:56.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83541ab060 con 0x557e783765d0 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.054+0000 7f835b382740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83541aa2c0 con 0x557e78377b40 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f835b382740 10 client.0 _ll_register_callbacks cb 0x7f83440092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e78398dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e78398dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51796/0 (socket says 172.21.15.94:51796) 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 -- 172.21.15.94:0/623784616 learned_addr learned my addr 172.21.15.94:0/623784616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f835259c640 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 0x557e78398880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 0x557e78398340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 -- 172.21.15.94:0/623784616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 msgr2=0x557e78398880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 0x557e78398880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 -- 172.21.15.94:0/623784616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 msgr2=0x557e78398340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 0x557e78398340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 -- 172.21.15.94:0/623784616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e783feb80 con 0x557e78381370 2024-09-19T04:32:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8351d9b640 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e78398dc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8348009860 tx=0x7f8348000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8817ab4773b964e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8339ffb640 1 -- 172.21.15.94:0/623784616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348037030 con 0x557e78381370 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8339ffb640 1 -- 172.21.15.94:0/623784616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f8348004470 con 0x557e78381370 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f835b382740 1 -- 172.21.15.94:0/623784616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8348044040 con 0x557e78381370 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f8339ffb640 1 -- 172.21.15.94:0/623784616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8348003b40 con 0x557e78381370 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f835b382740 10 client.17688 fetch_fsmap learned FSMap version 58 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.055+0000 7f835b382740 1 -- 172.21.15.94:0/623784616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557e783feb80 con 0x557e78381370 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f8339ffb640 1 -- 172.21.15.94:0/623784616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8348044040 con 0x557e78381370 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f8339ffb640 1 -- 172.21.15.94:0/623784616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 58) ==== 8370+0+0 (secure 0 0 0) 0x7f834806d460 con 0x557e78381370 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f835b382740 10 client.17688 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f835b382740 10 client.17688 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f835b382740 20 client.17688 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f835b382740 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e78400070 0x557e78402460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f835b382740 4 client.17688 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f835b382740 1 -- 172.21.15.94:0/623784616 --> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e782e3d70 con 0x557e78400070 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f835259c640 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e78400070 0x557e78402460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f835259c640 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e78400070 0x557e78402460 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=117395a5b0fd45e4 server_cookie=3fc1985ffeeff072 in_seq=0 out_seq=0 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.056+0000 7f8339ffb640 10 client.17688 ms_handle_connect on v2:172.21.15.112:6836/134813546 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f8339ffb640 1 -- 172.21.15.94:0/623784616 <== mds.2 v2:172.21.15.112:6836/134813546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e782e3d70 con 0x557e78400070 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f8339ffb640 10 client.17688 handle_command_reply: tid=0 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 client.17688 shutdown 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 -- 172.21.15.94:0/623784616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 msgr2=0x557e78398dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e78398dc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8348009860 tx=0x7f8348000bc0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 -- 172.21.15.94:0/623784616 shutdown_connections 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.112:6836/134813546,v1:172.21.15.112:6837/134813546] conn(0x557e78400070 0x557e78402460 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e78381370 0x557e78398dc0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e78377b40 0x557e78398880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 --2- 172.21.15.94:0/623784616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e783765d0 0x557e78398340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 -- 172.21.15.94:0/623784616 >> 172.21.15.94:0/623784616 conn(0x557e7833f310 msgr2=0x557e78371d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f8339ffb640 0 client.17688 ms_handle_reset on v2:172.21.15.112:6836/134813546 2024-09-19T04:32:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 -- 172.21.15.94:0/623784616 shutdown_connections 2024-09-19T04:32:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 1 -- 172.21.15.94:0/623784616 wait complete. 2024-09-19T04:32:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f835b382740 20 client.17688 trim_cache size 0 max 16384 2024-09-19T04:32:56.065 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:32:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f8358fa9640 1 -- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f833c0a33b0 msgr2=0x7f833c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f833c0a33b0 0x7f833c0a5870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f834c097ad0 tx=0x7f834c00e040 comp rx=0 tx=0).stop 2024-09-19T04:32:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f8358fa9640 1 -- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102ef0 msgr2=0x7f835419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:56.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.057+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102ef0 0x7f835419ab40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f834c00b970 tx=0x7f834c00be30 comp rx=0 tx=0).stop 2024-09-19T04:32:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.058+0000 7f8358fa9640 1 -- 172.21.15.94:0/1540329633 shutdown_connections 2024-09-19T04:32:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.058+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f833c0a33b0 0x7f833c0a5870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.058+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8354102ef0 0x7f835419ab40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.058+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8354102570 0x7f83541967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.058+0000 7f8358fa9640 1 --2- 172.21.15.94:0/1540329633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8354100e20 0x7f8354196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.058+0000 7f8358fa9640 1 -- 172.21.15.94:0/1540329633 >> 172.21.15.94:0/1540329633 conn(0x7f83540fc7d0 msgr2=0x7f83540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.058+0000 7f8358fa9640 1 -- 172.21.15.94:0/1540329633 shutdown_connections 2024-09-19T04:32:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:56.058+0000 7f8358fa9640 1 -- 172.21.15.94:0/1540329633 wait complete. 2024-09-19T04:32:56.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:32:57.060 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:57.506 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:57 smithi094 ceph-mon[30972]: pgmap v562: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 13 op/s 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 -- 172.21.15.94:0/2781180309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4102ef0 msgr2=0x7f6ae410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 --2- 172.21.15.94:0/2781180309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4102ef0 0x7f6ae410da40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009ca0 tx=0x7f6adc02a950 comp rx=0 tx=0).stop 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 -- 172.21.15.94:0/2781180309 shutdown_connections 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 --2- 172.21.15.94:0/2781180309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4102ef0 0x7f6ae410da40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 --2- 172.21.15.94:0/2781180309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ae4102570 0x7f6ae41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 --2- 172.21.15.94:0/2781180309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae4100e20 0x7f6ae4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 -- 172.21.15.94:0/2781180309 >> 172.21.15.94:0/2781180309 conn(0x7f6ae40fc7d0 msgr2=0x7f6ae40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 -- 172.21.15.94:0/2781180309 shutdown_connections 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 -- 172.21.15.94:0/2781180309 wait complete. 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.505+0000 7f6aec476640 1 Processor -- start 2024-09-19T04:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aec476640 1 -- start start 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aec476640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4100e20 0x7f6ae40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aec476640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae4102570 0x7f6ae40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aec476640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ae4102ef0 0x7f6ae40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aec476640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4110e30 con 0x7f6ae4100e20 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aec476640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4110cb0 con 0x7f6ae4102570 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aec476640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4110fb0 con 0x7f6ae4102ef0 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aea212640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4100e20 0x7f6ae40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aea212640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4100e20 0x7f6ae40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51810/0 (socket says 172.21.15.94:51810) 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aea212640 1 -- 172.21.15.94:0/2475551570 learned_addr learned my addr 172.21.15.94:0/2475551570 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ae4102ef0 0x7f6ae40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae4102570 0x7f6ae40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aea212640 1 -- 172.21.15.94:0/2475551570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ae4102ef0 msgr2=0x7f6ae40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aea212640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ae4102ef0 0x7f6ae40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aea212640 1 -- 172.21.15.94:0/2475551570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae4102570 msgr2=0x7f6ae40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aea212640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae4102570 0x7f6ae40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aea212640 1 -- 172.21.15.94:0/2475551570 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae41a6840 con 0x7f6ae4100e20 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ae4102ef0 0x7f6ae40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.506+0000 7f6aea212640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4100e20 0x7f6ae40fcc80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c890 tx=0x7f6ae000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.507+0000 7f6adb7fe640 1 -- 172.21.15.94:0/2475551570 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004220 con 0x7f6ae4100e20 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.507+0000 7f6aec476640 1 -- 172.21.15.94:0/2475551570 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae41a6b30 con 0x7f6ae4100e20 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.507+0000 7f6adb7fe640 1 -- 172.21.15.94:0/2475551570 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ae00043c0 con 0x7f6ae4100e20 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.507+0000 7f6adb7fe640 1 -- 172.21.15.94:0/2475551570 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004a00 con 0x7f6ae4100e20 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.507+0000 7f6aec476640 1 -- 172.21.15.94:0/2475551570 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae41a7070 con 0x7f6ae4100e20 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.509+0000 7f6adb7fe640 1 -- 172.21.15.94:0/2475551570 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6ae0004ba0 con 0x7f6ae4100e20 2024-09-19T04:32:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.509+0000 7f6aee84f740 1 Processor -- start 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.509+0000 7f6aee84f740 1 -- start start 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.509+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x56290910e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.509+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910e5d0 0x56290910ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.509+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.509+0000 7f6aee84f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5629090742f0 con 0x562909119370 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.509+0000 7f6aee84f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56290901bcf0 con 0x56290910e5d0 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.509+0000 7f6aee84f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629090702b0 con 0x56290910fb40 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ae9a11640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ae9a11640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51826/0 (socket says 172.21.15.94:51826) 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ae9a11640 1 -- 172.21.15.94:0/504085094 learned_addr learned my addr 172.21.15.94:0/504085094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6aea212640 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910e5d0 0x56290910ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x56290910e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6adb7fe640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6ad00a33b0 0x7f6ad00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ae9a11640 1 -- 172.21.15.94:0/504085094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 msgr2=0x56290910e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x56290910e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ae9a11640 1 -- 172.21.15.94:0/504085094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910e5d0 msgr2=0x56290910ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910e5d0 0x56290910ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ae9a11640 1 -- 172.21.15.94:0/504085094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629090003a0 con 0x562909119370 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x56290910e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6ad00a33b0 0x7f6ad00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6adb7fe640 1 -- 172.21.15.94:0/2475551570 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f6ae00cfad0 con 0x7f6ae4100e20 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911b760 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400bc00 tx=0x7f6ad400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41deb48200bffdbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ad8ff9640 1 -- 172.21.15.94:0/504085094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4015070 con 0x562909119370 2024-09-19T04:32:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ad8ff9640 1 -- 172.21.15.94:0/504085094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad40054d0 con 0x562909119370 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.510+0000 7f6ad8ff9640 1 -- 172.21.15.94:0/504085094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4011440 con 0x562909119370 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 -- 172.21.15.94:0/504085094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 msgr2=0x56290911b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911b760 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400bc00 tx=0x7f6ad400f730 comp rx=0 tx=0).stop 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6ad00a33b0 0x7f6ad00a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009430 tx=0x7f6adc03e040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 -- 172.21.15.94:0/504085094 shutdown_connections 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911b760 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910e5d0 0x56290910ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 --2- 172.21.15.94:0/504085094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x56290910e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 -- 172.21.15.94:0/504085094 >> 172.21.15.94:0/504085094 conn(0x5629090d7310 msgr2=0x56290910ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 -- 172.21.15.94:0/504085094 shutdown_connections 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 -- 172.21.15.94:0/504085094 wait complete. 2024-09-19T04:32:57.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.511+0000 7f6aee84f740 1 -- start start 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aee84f740 1 Processor -- start 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56290910e5d0 0x562909118d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x562909115fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562909119370 0x562909116570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aee84f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56290911c070 con 0x56290910e5d0 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aee84f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56290902f060 con 0x562909119370 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aee84f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56290911c1f0 con 0x56290910fb40 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aee84f740 10 client.0 _ll_register_callbacks cb 0x562908d828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aea212640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56290910e5d0 0x562909118d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aea212640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56290910e5d0 0x562909118d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51834/0 (socket says 172.21.15.94:51834) 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aea212640 1 -- 172.21.15.94:0/664711459 learned_addr learned my addr 172.21.15.94:0/664711459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x562909115fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aea212640 1 -- 172.21.15.94:0/664711459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 msgr2=0x562909115fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aea212640 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x562909115fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aea212640 1 -- 172.21.15.94:0/664711459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562909119370 msgr2=0x562909116570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aea212640 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562909119370 0x562909116570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aea212640 1 -- 172.21.15.94:0/664711459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562908de53c0 con 0x56290910e5d0 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x562909115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aea212640 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56290910e5d0 0x562909118d20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00cd240 tx=0x7f6ae00053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9196c1da3ac9398b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6acdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6acdffb640 1 -- 172.21.15.94:0/664711459 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae001d040 con 0x56290910e5d0 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6acdffb640 1 -- 172.21.15.94:0/664711459 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f6ae00050b0 con 0x56290910e5d0 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.512+0000 7f6aee84f740 1 -- 172.21.15.94:0/664711459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad40054d0 con 0x56290910e5d0 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6acdffb640 1 -- 172.21.15.94:0/664711459 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6ae00b9050 con 0x56290910e5d0 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6aee84f740 10 client.17700 fetch_fsmap learned FSMap version 58 2024-09-19T04:32:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6aee84f740 1 -- 172.21.15.94:0/664711459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562909127b20 con 0x56290910e5d0 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6acdffb640 1 -- 172.21.15.94:0/664711459 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad40054d0 con 0x56290910e5d0 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6acdffb640 1 -- 172.21.15.94:0/664711459 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 58) ==== 8370+0+0 (secure 0 0 0) 0x7f6ae00983e0 con 0x56290910e5d0 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6aee84f740 10 client.17700 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6aee84f740 10 client.17700 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6aee84f740 20 client.17700 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6aee84f740 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x562909129080 0x56290912b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6aee84f740 4 client.17700 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6aee84f740 1 -- 172.21.15.94:0/664711459 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562909081860 con 0x562909129080 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.513+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x562909129080 0x56290912b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.514+0000 7f6acdffb640 10 client.17700 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.514+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x562909129080 0x56290912b540 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3314099004ec2d1 server_cookie=8e9d5ac73ef2705a in_seq=0 out_seq=0 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.514+0000 7f6acdffb640 1 -- 172.21.15.94:0/664711459 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562909081860 con 0x562909129080 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.514+0000 7f6acdffb640 10 client.17700 handle_command_reply: tid=0 2024-09-19T04:32:57.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.514+0000 7f6aee84f740 1 client.17700 shutdown 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 -- 172.21.15.94:0/664711459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56290910e5d0 msgr2=0x562909118d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56290910e5d0 0x562909118d20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00cd240 tx=0x7f6ae00053d0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 -- 172.21.15.94:0/664711459 shutdown_connections 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x562909129080 0x56290912b540 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562909119370 0x562909116570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910fb40 0x562909115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 --2- 172.21.15.94:0/664711459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56290910e5d0 0x562909118d20 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 -- 172.21.15.94:0/664711459 >> 172.21.15.94:0/664711459 conn(0x5629090d7310 msgr2=0x562909110960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6acdffb640 0 client.17700 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 -- 172.21.15.94:0/664711459 shutdown_connections 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 1 -- 172.21.15.94:0/664711459 wait complete. 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.515+0000 7f6aee84f740 20 client.17700 trim_cache size 0 max 16384 2024-09-19T04:32:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.527+0000 7f6aee84f740 1 Processor -- start 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aee84f740 1 -- start start 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 0x56290910de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 0x562909109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x562909111120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aee84f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41b2320 con 0x562909119370 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aee84f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41afb30 con 0x56290910fb40 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aee84f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ad9a0 con 0x56290910e5d0 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aeaa13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x562909111120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aeaa13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x562909111120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51840/0 (socket says 172.21.15.94:51840) 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aeaa13640 1 -- 172.21.15.94:0/3935837334 learned_addr learned my addr 172.21.15.94:0/3935837334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aea212640 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 0x56290910de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 0x562909109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aeaa13640 1 -- 172.21.15.94:0/3935837334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 msgr2=0x56290910de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 0x56290910de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aeaa13640 1 -- 172.21.15.94:0/3935837334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 msgr2=0x562909109c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 0x562909109c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aeaa13640 1 -- 172.21.15.94:0/3935837334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562909114f80 con 0x562909119370 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aea212640 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 0x56290910de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 0x562909109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x562909111120 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6adc02fce0 tx=0x7f6adc02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a6f09a19620c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.528+0000 7f6acffff640 1 -- 172.21.15.94:0/3935837334 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc034040 con 0x562909119370 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6acffff640 1 -- 172.21.15.94:0/3935837334 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6adc003c30 con 0x562909119370 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6acffff640 1 -- 172.21.15.94:0/3935837334 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc030930 con 0x562909119370 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 -- 172.21.15.94:0/3935837334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 msgr2=0x562909111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x562909111120 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6adc02fce0 tx=0x7f6adc02a950 comp rx=0 tx=0).stop 2024-09-19T04:32:57.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 -- 172.21.15.94:0/3935837334 shutdown_connections 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x562909111120 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 0x562909109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 --2- 172.21.15.94:0/3935837334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 0x56290910de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 -- 172.21.15.94:0/3935837334 >> 172.21.15.94:0/3935837334 conn(0x5629090d7310 msgr2=0x562909109500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 -- 172.21.15.94:0/3935837334 shutdown_connections 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 -- 172.21.15.94:0/3935837334 wait complete. 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 -- start start 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.529+0000 7f6aee84f740 1 Processor -- start 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 0x562909121d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 0x56290911d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aee84f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aee84f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41adff0 con 0x562909119370 2024-09-19T04:32:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aee84f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41af460 con 0x56290910fb40 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aee84f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ae6c0 con 0x56290910e5d0 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aee84f740 10 client.0 _ll_register_callbacks cb 0x7f6ae00058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aeaa13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aeaa13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51842/0 (socket says 172.21.15.94:51842) 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aeaa13640 1 -- 172.21.15.94:0/1711644843 learned_addr learned my addr 172.21.15.94:0/1711644843 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6ae9a11640 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 0x56290911d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aea212640 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 0x562909121d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aeaa13640 1 -- 172.21.15.94:0/1711644843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 msgr2=0x562909121d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 0x562909121d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aeaa13640 1 -- 172.21.15.94:0/1711644843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 msgr2=0x56290911d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 0x56290911d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aeaa13640 1 -- 172.21.15.94:0/1711644843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56290911de70 con 0x562909119370 2024-09-19T04:32:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6aeaa13640 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911d640 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6adc006010 tx=0x7f6adc004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7376432ae4902a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.530+0000 7f6acdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6acdffb640 1 -- 172.21.15.94:0/1711644843 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc058060 con 0x562909119370 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6acdffb640 1 -- 172.21.15.94:0/1711644843 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f6adc030500 con 0x562909119370 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aee84f740 1 -- 172.21.15.94:0/1711644843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6adc003c30 con 0x562909119370 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6acdffb640 1 -- 172.21.15.94:0/1711644843 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6adc030700 con 0x562909119370 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aee84f740 10 client.17706 fetch_fsmap learned FSMap version 58 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aee84f740 1 -- 172.21.15.94:0/1711644843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562909193e80 con 0x562909119370 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6acdffb640 1 -- 172.21.15.94:0/1711644843 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6adc003c30 con 0x562909119370 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6acdffb640 1 -- 172.21.15.94:0/1711644843 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 58) ==== 8370+0+0 (secure 0 0 0) 0x7f6adc050b20 con 0x562909119370 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aee84f740 10 client.17706 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aee84f740 10 client.17706 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aee84f740 20 client.17706 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:57.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aee84f740 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x562909194590 0x5629091969b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:57.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aee84f740 4 client.17706 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:57.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aee84f740 1 -- 172.21.15.94:0/1711644843 --> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56290907bd70 con 0x562909194590 2024-09-19T04:32:57.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.531+0000 7f6aea212640 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x562909194590 0x5629091969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:57.553 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6acdffb640 10 client.17706 ms_handle_connect on v2:172.21.15.112:6838/3588924475 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aea212640 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x562909194590 0x5629091969b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11ac7e0f605eb2c3 server_cookie=1ee62d4b37b71bac in_seq=0 out_seq=0 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6acdffb640 1 -- 172.21.15.94:0/1711644843 <== mds.1 v2:172.21.15.112:6838/3588924475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56290907bd70 con 0x562909194590 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6acdffb640 10 client.17706 handle_command_reply: tid=0 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 client.17706 shutdown 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 -- 172.21.15.94:0/1711644843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 msgr2=0x56290911d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911d640 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6adc006010 tx=0x7f6adc004670 comp rx=0 tx=0).stop 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 -- 172.21.15.94:0/1711644843 shutdown_connections 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.112:6838/3588924475,v1:172.21.15.112:6839/3588924475] conn(0x562909194590 0x5629091969b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562909119370 0x56290911d640 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56290910fb40 0x56290911d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 --2- 172.21.15.94:0/1711644843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56290910e5d0 0x562909121d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 -- 172.21.15.94:0/1711644843 >> 172.21.15.94:0/1711644843 conn(0x5629090d7310 msgr2=0x562909109d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6acdffb640 0 client.17706 ms_handle_reset on v2:172.21.15.112:6838/3588924475 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 -- 172.21.15.94:0/1711644843 shutdown_connections 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 1 -- 172.21.15.94:0/1711644843 wait complete. 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.532+0000 7f6aee84f740 20 client.17706 trim_cache size 0 max 16384 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 -- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6ad00a33b0 msgr2=0x7f6ad00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6ad00a33b0 0x7f6ad00a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009430 tx=0x7f6adc03e040 comp rx=0 tx=0).stop 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 -- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4100e20 msgr2=0x7f6ae40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4100e20 0x7f6ae40fcc80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c890 tx=0x7f6ae000cd50 comp rx=0 tx=0).stop 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 -- 172.21.15.94:0/2475551570 shutdown_connections 2024-09-19T04:32:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f6ad00a33b0 0x7f6ad00a5870 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ae4102ef0 0x7f6ae40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae4102570 0x7f6ae40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 --2- 172.21.15.94:0/2475551570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae4100e20 0x7f6ae40fcc80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:57.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 -- 172.21.15.94:0/2475551570 >> 172.21.15.94:0/2475551570 conn(0x7f6ae40fc7d0 msgr2=0x7f6ae406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:57.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 -- 172.21.15.94:0/2475551570 shutdown_connections 2024-09-19T04:32:57.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:57.534+0000 7f6aec476640 1 -- 172.21.15.94:0/2475551570 wait complete. 2024-09-19T04:32:57.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:57 smithi104 ceph-mon[34145]: pgmap v562: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 13 op/s 2024-09-19T04:32:57.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:57 smithi112 ceph-mon[35051]: pgmap v562: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 13 op/s 2024-09-19T04:32:58.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:32:58.545 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:58.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 -- 172.21.15.94:0/2089667487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320106a90 msgr2=0x7f8320106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:58.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 --2- 172.21.15.94:0/2089667487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320106a90 0x7f8320106e70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8310009930 tx=0x7f831002a420 comp rx=0 tx=0).stop 2024-09-19T04:32:58.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 -- 172.21.15.94:0/2089667487 shutdown_connections 2024-09-19T04:32:58.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 --2- 172.21.15.94:0/2089667487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8320101430 0x7f832010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:58.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 --2- 172.21.15.94:0/2089667487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8320100a90 0x7f8320100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:58.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 --2- 172.21.15.94:0/2089667487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320106a90 0x7f8320106e70 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 -- 172.21.15.94:0/2089667487 >> 172.21.15.94:0/2089667487 conn(0x7f83200fc7d0 msgr2=0x7f83200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 -- 172.21.15.94:0/2089667487 shutdown_connections 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 -- 172.21.15.94:0/2089667487 wait complete. 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 Processor -- start 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.986+0000 7f8326772640 1 -- start start 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f8326772640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320100a90 0x7f8320198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f8326772640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8320101430 0x7f8320198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f8326772640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8320106a90 0x7f832019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f8326772640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8320110d40 con 0x7f8320100a90 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f8326772640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8320110bc0 con 0x7f8320106a90 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f8326772640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8320110ec0 con 0x7f8320101430 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320100a90 0x7f8320198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:58.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320100a90 0x7f8320198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51864/0 (socket says 172.21.15.94:51864) 2024-09-19T04:32:58.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831ffff640 1 -- 172.21.15.94:0/513392886 learned_addr learned my addr 172.21.15.94:0/513392886 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:58.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831f7fe640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8320101430 0x7f8320198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:58.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f8324d0f640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8320106a90 0x7f832019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:58.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831ffff640 1 -- 172.21.15.94:0/513392886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8320101430 msgr2=0x7f8320198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:58.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831ffff640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8320101430 0x7f8320198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:58.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831ffff640 1 -- 172.21.15.94:0/513392886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8320106a90 msgr2=0x7f832019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:58.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831ffff640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8320106a90 0x7f832019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:58.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831ffff640 1 -- 172.21.15.94:0/513392886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832019d3e0 con 0x7f8320100a90 2024-09-19T04:32:58.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.987+0000 7f831ffff640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320100a90 0x7f8320198430 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8310004770 tx=0x7f83100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:58.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.988+0000 7f831d7fa640 1 -- 172.21.15.94:0/513392886 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831003f070 con 0x7f8320100a90 2024-09-19T04:32:58.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.988+0000 7f831d7fa640 1 -- 172.21.15.94:0/513392886 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f831003b460 con 0x7f8320100a90 2024-09-19T04:32:58.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.988+0000 7f8326772640 1 -- 172.21.15.94:0/513392886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832019d670 con 0x7f8320100a90 2024-09-19T04:32:58.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.988+0000 7f831d7fa640 1 -- 172.21.15.94:0/513392886 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310035070 con 0x7f8320100a90 2024-09-19T04:32:58.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.988+0000 7f8326772640 1 -- 172.21.15.94:0/513392886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83201a4fb0 con 0x7f8320100a90 2024-09-19T04:32:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f8328b4b740 1 Processor -- start 2024-09-19T04:32:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f8328b4b740 1 -- start start 2024-09-19T04:32:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f831d7fa640 1 -- 172.21.15.94:0/513392886 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f831003b610 con 0x7f8320100a90 2024-09-19T04:32:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 0x5630d500a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d500aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d501bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f8328b4b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5630d4f702f0 con 0x5630d500a5d0 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f8328b4b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630d4f17cf0 con 0x5630d5019780 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f8328b4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630d4f6c2b0 con 0x5630d500bb40 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f831ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d500aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f831ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d500aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51880/0 (socket says 172.21.15.94:51880) 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f831ffff640 1 -- 172.21.15.94:0/710282801 learned_addr learned my addr 172.21.15.94:0/710282801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f8324d0f640 1 --2- 172.21.15.94:0/710282801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 0x5630d500a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f831f7fe640 1 --2- 172.21.15.94:0/710282801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d501bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.991+0000 7f831ffff640 1 -- 172.21.15.94:0/710282801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 msgr2=0x5630d500a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f831ffff640 1 --2- 172.21.15.94:0/710282801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 0x5630d500a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f831ffff640 1 -- 172.21.15.94:0/710282801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 msgr2=0x5630d501bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f831ffff640 1 --2- 172.21.15.94:0/710282801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d501bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f831ffff640 1 -- 172.21.15.94:0/710282801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d4efc3a0 con 0x5630d500a5d0 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f831d7fa640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83080a33b0 0x7f83080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f831ffff640 1 --2- 172.21.15.94:0/710282801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d500aa10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f83100ed750 tx=0x7f83100f8540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ee2141c9de2d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f831d7fa640 1 -- 172.21.15.94:0/513392886 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f83100f00d0 con 0x7f8320100a90 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f8324d0f640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83080a33b0 0x7f83080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f83067fc640 1 -- 172.21.15.94:0/710282801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83100b8610 con 0x5630d500a5d0 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f83067fc640 1 -- 172.21.15.94:0/710282801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83100bca70 con 0x5630d500a5d0 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.992+0000 7f83067fc640 1 -- 172.21.15.94:0/710282801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83100044e0 con 0x5630d500a5d0 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 -- 172.21.15.94:0/710282801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 msgr2=0x5630d500aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8324d0f640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83080a33b0 0x7f83080a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83180092f0 tx=0x7f8318009280 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 --2- 172.21.15.94:0/710282801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d500aa10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f83100ed750 tx=0x7f83100f8540 comp rx=0 tx=0).stop 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 -- 172.21.15.94:0/710282801 shutdown_connections 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 --2- 172.21.15.94:0/710282801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d501bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 --2- 172.21.15.94:0/710282801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d500aa10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 --2- 172.21.15.94:0/710282801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 0x5630d500a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 -- 172.21.15.94:0/710282801 >> 172.21.15.94:0/710282801 conn(0x5630d4fd3310 msgr2=0x5630d5006c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 -- 172.21.15.94:0/710282801 shutdown_connections 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 -- 172.21.15.94:0/710282801 wait complete. 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.993+0000 7f8328b4b740 1 -- start start 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f8328b4b740 1 Processor -- start 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500a5d0 0x5630d50191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5016480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d5016c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f8328b4b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5630d501c240 con 0x5630d500bb40 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f8328b4b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630d4f2b060 con 0x5630d5019780 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f8328b4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630d501c3c0 con 0x5630d500a5d0 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f8328b4b740 10 client.0 _ll_register_callbacks cb 0x5630d4c7e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5016480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5016480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51884/0 (socket says 172.21.15.94:51884) 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831f7fe640 1 -- 172.21.15.94:0/4077529800 learned_addr learned my addr 172.21.15.94:0/4077529800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831ffff640 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500a5d0 0x5630d50191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f8324d0f640 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d5016c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831f7fe640 1 -- 172.21.15.94:0/4077529800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500a5d0 msgr2=0x5630d50191d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831f7fe640 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500a5d0 0x5630d50191d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831f7fe640 1 -- 172.21.15.94:0/4077529800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 msgr2=0x5630d5016c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831f7fe640 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d5016c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831f7fe640 1 -- 172.21.15.94:0/4077529800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d4ce13c0 con 0x5630d500bb40 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f831f7fe640 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5016480 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8314005f80 tx=0x7f831400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807559832ebc382d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.994+0000 7f82ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f82ff7fe640 1 -- 172.21.15.94:0/4077529800 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831401b440 con 0x5630d500bb40 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f82ff7fe640 1 -- 172.21.15.94:0/4077529800 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f831401b5e0 con 0x5630d500bb40 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f8328b4b740 1 -- 172.21.15.94:0/4077529800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83100bca70 con 0x5630d500bb40 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f82ff7fe640 1 -- 172.21.15.94:0/4077529800 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f831401b7e0 con 0x5630d500bb40 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f8328b4b740 10 client.17718 fetch_fsmap learned FSMap version 58 2024-09-19T04:32:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f8328b4b740 1 -- 172.21.15.94:0/4077529800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5630d5027cf0 con 0x5630d500bb40 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f82ff7fe640 1 -- 172.21.15.94:0/4077529800 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83100bca70 con 0x5630d500bb40 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f82ff7fe640 1 -- 172.21.15.94:0/4077529800 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 58) ==== 8370+0+0 (secure 0 0 0) 0x7f8314019020 con 0x5630d500bb40 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f8328b4b740 10 client.17718 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f8328b4b740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f8328b4b740 20 client.17718 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f8328b4b740 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d50292a0 0x5630d502b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f8328b4b740 4 client.17718 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.995+0000 7f8328b4b740 1 -- 172.21.15.94:0/4077529800 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630d4f7d860 con 0x5630d50292a0 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.996+0000 7f831ffff640 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d50292a0 0x5630d502b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.996+0000 7f831ffff640 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d50292a0 0x5630d502b700 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d906eeca48361524 server_cookie=cf63f5ac199ee9b in_seq=0 out_seq=0 2024-09-19T04:32:59.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.996+0000 7f82ff7fe640 10 client.17718 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f82ff7fe640 1 -- 172.21.15.94:0/4077529800 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630d4f7d860 con 0x5630d50292a0 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f82ff7fe640 10 client.17718 handle_command_reply: tid=0 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 client.17718 shutdown 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 -- 172.21.15.94:0/4077529800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 msgr2=0x5630d5016480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5016480 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8314005f80 tx=0x7f831400be80 comp rx=0 tx=0).stop 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 -- 172.21.15.94:0/4077529800 shutdown_connections 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d50292a0 0x5630d502b700 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d5016c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5016480 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 --2- 172.21.15.94:0/4077529800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500a5d0 0x5630d50191d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 -- 172.21.15.94:0/4077529800 >> 172.21.15.94:0/4077529800 conn(0x5630d4fd3310 msgr2=0x5630d500c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:59.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f82ff7fe640 0 client.17718 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:59.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 -- 172.21.15.94:0/4077529800 shutdown_connections 2024-09-19T04:32:59.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 1 -- 172.21.15.94:0/4077529800 wait complete. 2024-09-19T04:32:59.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:58.997+0000 7f8328b4b740 20 client.17718 trim_cache size 0 max 16384 2024-09-19T04:32:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f8328b4b740 1 Processor -- start 2024-09-19T04:32:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f8328b4b740 1 -- start start 2024-09-19T04:32:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d500a5d0 0x5630d5030fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5031510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d5019780 0x5630d5013760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f8328b4b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83201ae140 con 0x5630d500bb40 2024-09-19T04:32:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f8328b4b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83201ab950 con 0x5630d500a5d0 2024-09-19T04:32:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f8328b4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83201a97c0 con 0x5630d5019780 2024-09-19T04:32:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5031510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5031510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51894/0 (socket says 172.21.15.94:51894) 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831f7fe640 1 -- 172.21.15.94:0/2575185611 learned_addr learned my addr 172.21.15.94:0/2575185611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831ffff640 1 --2- 172.21.15.94:0/2575185611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d500a5d0 0x5630d5030fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831f7fe640 1 -- 172.21.15.94:0/2575185611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d5019780 msgr2=0x5630d5013760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831f7fe640 1 --2- 172.21.15.94:0/2575185611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d5019780 0x5630d5013760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831f7fe640 1 -- 172.21.15.94:0/2575185611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d500a5d0 msgr2=0x5630d5030fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831f7fe640 1 --2- 172.21.15.94:0/2575185611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d500a5d0 0x5630d5030fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831f7fe640 1 -- 172.21.15.94:0/2575185611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d5016f80 con 0x5630d500bb40 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.010+0000 7f831ffff640 1 --2- 172.21.15.94:0/2575185611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d500a5d0 0x5630d5030fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f831f7fe640 1 --2- 172.21.15.94:0/2575185611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5031510 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8314011cb0 tx=0x7f8314003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f019c20c15c203e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8305ffb640 1 -- 172.21.15.94:0/2575185611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831401b640 con 0x5630d500bb40 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8305ffb640 1 -- 172.21.15.94:0/2575185611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83140118f0 con 0x5630d500bb40 2024-09-19T04:32:59.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8305ffb640 1 -- 172.21.15.94:0/2575185611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140053c0 con 0x5630d500bb40 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8328b4b740 1 -- 172.21.15.94:0/2575185611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 msgr2=0x5630d5031510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8328b4b740 1 --2- 172.21.15.94:0/2575185611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5031510 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8314011cb0 tx=0x7f8314003f70 comp rx=0 tx=0).stop 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8328b4b740 1 -- 172.21.15.94:0/2575185611 shutdown_connections 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8328b4b740 1 --2- 172.21.15.94:0/2575185611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d5019780 0x5630d5013760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8328b4b740 1 --2- 172.21.15.94:0/2575185611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500bb40 0x5630d5031510 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8328b4b740 1 --2- 172.21.15.94:0/2575185611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d500a5d0 0x5630d5030fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8328b4b740 1 -- 172.21.15.94:0/2575185611 >> 172.21.15.94:0/2575185611 conn(0x5630d4fd3310 msgr2=0x5630d5011720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.011+0000 7f8328b4b740 1 -- 172.21.15.94:0/2575185611 shutdown_connections 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 1 -- 172.21.15.94:0/2575185611 wait complete. 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 1 -- start start 2024-09-19T04:32:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 1 Processor -- start 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d5021be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 0x5630d50130a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d50135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83201a9e10 con 0x5630d500a5d0 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83201ab280 con 0x5630d5019780 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83201aa4e0 con 0x5630d500bb40 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f8328b4b740 10 client.0 _ll_register_callbacks cb 0x7f8314005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f831ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d5021be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f831ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d5021be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51910/0 (socket says 172.21.15.94:51910) 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.012+0000 7f831ffff640 1 -- 172.21.15.94:0/3722005765 learned_addr learned my addr 172.21.15.94:0/3722005765 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f831f7fe640 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 0x5630d50130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8324d0f640 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d50135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f831ffff640 1 -- 172.21.15.94:0/3722005765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 msgr2=0x5630d50130a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f831ffff640 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 0x5630d50130a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f831ffff640 1 -- 172.21.15.94:0/3722005765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 msgr2=0x5630d50135e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f831ffff640 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d50135e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f831ffff640 1 -- 172.21.15.94:0/3722005765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d5019540 con 0x5630d500a5d0 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8324d0f640 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d50135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f831ffff640 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d5021be0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8310005940 tx=0x7f83100f8ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef29689852820d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f82ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f82ff7fe640 1 -- 172.21.15.94:0/3722005765 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310048070 con 0x5630d500a5d0 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f82ff7fe640 1 -- 172.21.15.94:0/3722005765 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f83100eda60 con 0x5630d500a5d0 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8328b4b740 1 -- 172.21.15.94:0/3722005765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83140118f0 con 0x5630d500a5d0 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f82ff7fe640 1 -- 172.21.15.94:0/3722005765 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f83100bbcd0 con 0x5630d500a5d0 2024-09-19T04:32:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8328b4b740 10 client.17724 fetch_fsmap learned FSMap version 58 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8328b4b740 1 -- 172.21.15.94:0/3722005765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5630d5093bd0 con 0x5630d500a5d0 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f82ff7fe640 1 -- 172.21.15.94:0/3722005765 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83140118f0 con 0x5630d500a5d0 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f82ff7fe640 1 -- 172.21.15.94:0/3722005765 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 58) ==== 8370+0+0 (secure 0 0 0) 0x7f831004b060 con 0x5630d500a5d0 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8328b4b740 10 client.17724 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8328b4b740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8328b4b740 20 client.17724 populate_metadata read hostname 'smithi094' 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8328b4b740 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d5094340 0x5630d5096730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8328b4b740 4 client.17724 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.013+0000 7f8328b4b740 1 -- 172.21.15.94:0/3722005765 --> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630d4f77d70 con 0x5630d5094340 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.014+0000 7f831f7fe640 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d5094340 0x5630d5096730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.014+0000 7f831f7fe640 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d5094340 0x5630d5096730 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54279d6e99046a5c server_cookie=a3252131a8baa158 in_seq=0 out_seq=0 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.014+0000 7f82ff7fe640 10 client.17724 ms_handle_connect on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.014+0000 7f82ff7fe640 1 -- 172.21.15.94:0/3722005765 <== mds.0 v2:172.21.15.104:6838/2749200235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630d4f77d70 con 0x5630d5094340 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.014+0000 7f82ff7fe640 10 client.17724 handle_command_reply: tid=0 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.014+0000 7f8328b4b740 1 client.17724 shutdown 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 -- 172.21.15.94:0/3722005765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 msgr2=0x5630d5021be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d5021be0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8310005940 tx=0x7f83100f8ac0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 -- 172.21.15.94:0/3722005765 shutdown_connections 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.104:6838/2749200235,v1:172.21.15.104:6839/2749200235] conn(0x5630d5094340 0x5630d5096730 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f82ff7fe640 0 client.17724 ms_handle_reset on v2:172.21.15.104:6838/2749200235 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630d5019780 0x5630d50135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630d500bb40 0x5630d50130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 --2- 172.21.15.94:0/3722005765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5630d500a5d0 0x5630d5021be0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 -- 172.21.15.94:0/3722005765 >> 172.21.15.94:0/3722005765 conn(0x5630d4fd3310 msgr2=0x5630d5014db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:59.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 -- 172.21.15.94:0/3722005765 shutdown_connections 2024-09-19T04:32:59.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 1 -- 172.21.15.94:0/3722005765 wait complete. 2024-09-19T04:32:59.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8328b4b740 20 client.17724 trim_cache size 0 max 16384 2024-09-19T04:32:59.024 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-19T04:32:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 -- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83080a33b0 msgr2=0x7f83080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83080a33b0 0x7f83080a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83180092f0 tx=0x7f8318009280 comp rx=0 tx=0).stop 2024-09-19T04:32:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 -- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320100a90 msgr2=0x7f8320198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320100a90 0x7f8320198430 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8310004770 tx=0x7f83100042f0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 -- 172.21.15.94:0/513392886 shutdown_connections 2024-09-19T04:32:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f83080a33b0 0x7f83080a5870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8320106a90 0x7f832019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8320101430 0x7f8320198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 --2- 172.21.15.94:0/513392886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8320100a90 0x7f8320198430 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 -- 172.21.15.94:0/513392886 >> 172.21.15.94:0/513392886 conn(0x7f83200fc7d0 msgr2=0x7f832006aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.015+0000 7f8326772640 1 -- 172.21.15.94:0/513392886 shutdown_connections 2024-09-19T04:32:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.016+0000 7f8326772640 1 -- 172.21.15.94:0/513392886 wait complete. 2024-09-19T04:32:59.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:32:59 smithi094 ceph-mon[30972]: pgmap v563: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 15 op/s 2024-09-19T04:32:59.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:32:59 smithi104 ceph-mon[34145]: pgmap v563: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 15 op/s 2024-09-19T04:32:59.690 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T04:32:59.691 INFO:tasks.fwd_scrub:done joining 2024-09-19T04:32:59.691 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T04:32:59.703 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:32:59.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:32:59 smithi112 ceph-mon[35051]: pgmap v563: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 15 op/s 2024-09-19T04:32:59.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.817+0000 7f6562ed0640 1 -- 172.21.15.94:0/4161764531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c1069a0 msgr2=0x7f655c106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.817+0000 7f6562ed0640 1 --2- 172.21.15.94:0/4161764531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c1069a0 0x7f655c106d80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6550009930 tx=0x7f655002a420 comp rx=0 tx=0).stop 2024-09-19T04:32:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.817+0000 7f6562ed0640 1 -- 172.21.15.94:0/4161764531 shutdown_connections 2024-09-19T04:32:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.817+0000 7f6562ed0640 1 --2- 172.21.15.94:0/4161764531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f655c101330 0x7f655c10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.817+0000 7f6562ed0640 1 --2- 172.21.15.94:0/4161764531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f655c100990 0x7f655c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.817+0000 7f6562ed0640 1 --2- 172.21.15.94:0/4161764531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c1069a0 0x7f655c106d80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.817+0000 7f6562ed0640 1 -- 172.21.15.94:0/4161764531 >> 172.21.15.94:0/4161764531 conn(0x7f655c0fc6d0 msgr2=0x7f655c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.817+0000 7f6562ed0640 1 -- 172.21.15.94:0/4161764531 shutdown_connections 2024-09-19T04:32:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.817+0000 7f6562ed0640 1 -- 172.21.15.94:0/4161764531 wait complete. 2024-09-19T04:32:59.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6562ed0640 1 Processor -- start 2024-09-19T04:32:59.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6562ed0640 1 -- start start 2024-09-19T04:32:59.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6562ed0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c100990 0x7f655c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6562ed0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f655c101330 0x7f655c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6562ed0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f655c1069a0 0x7f655c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6562ed0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f655c110c60 con 0x7f655c100990 2024-09-19T04:32:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6562ed0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f655c110ae0 con 0x7f655c101330 2024-09-19T04:32:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6562ed0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f655c110de0 con 0x7f655c1069a0 2024-09-19T04:32:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6560c6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c100990 0x7f655c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6560c6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c100990 0x7f655c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51922/0 (socket says 172.21.15.94:51922) 2024-09-19T04:32:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6560c6c640 1 -- 172.21.15.94:0/598831149 learned_addr learned my addr 172.21.15.94:0/598831149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:32:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f655bfff640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f655c101330 0x7f655c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f656146d640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f655c1069a0 0x7f655c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6560c6c640 1 -- 172.21.15.94:0/598831149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f655c1069a0 msgr2=0x7f655c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6560c6c640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f655c1069a0 0x7f655c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6560c6c640 1 -- 172.21.15.94:0/598831149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f655c101330 msgr2=0x7f655c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6560c6c640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f655c101330 0x7f655c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f6560c6c640 1 -- 172.21.15.94:0/598831149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c19d2f0 con 0x7f655c100990 2024-09-19T04:32:59.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.818+0000 7f656146d640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f655c1069a0 0x7f655c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:59.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.819+0000 7f6560c6c640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c100990 0x7f655c198300 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6550004770 tx=0x7f65500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:59.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.819+0000 7f6559ffb640 1 -- 172.21.15.94:0/598831149 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655003f070 con 0x7f655c100990 2024-09-19T04:32:59.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.819+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c19d580 con 0x7f655c100990 2024-09-19T04:32:59.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.819+0000 7f6559ffb640 1 -- 172.21.15.94:0/598831149 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f655003b460 con 0x7f655c100990 2024-09-19T04:32:59.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.819+0000 7f6559ffb640 1 -- 172.21.15.94:0/598831149 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550035070 con 0x7f655c100990 2024-09-19T04:32:59.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.819+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c1a4e70 con 0x7f655c100990 2024-09-19T04:32:59.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.820+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655c100df0 con 0x7f655c100990 2024-09-19T04:32:59.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.821+0000 7f6559ffb640 1 -- 172.21.15.94:0/598831149 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f655003b610 con 0x7f655c100990 2024-09-19T04:32:59.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.822+0000 7f6559ffb640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f654c0a33c0 0x7f654c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:59.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.822+0000 7f655bfff640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f654c0a33c0 0x7f654c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:59.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.822+0000 7f6559ffb640 1 -- 172.21.15.94:0/598831149 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f65500f00e0 con 0x7f655c100990 2024-09-19T04:32:59.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.823+0000 7f655bfff640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f654c0a33c0 0x7f654c0a5880 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f655c06b450 tx=0x7f654800a2d0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:59.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.826+0000 7f6559ffb640 1 -- 172.21.15.94:0/598831149 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f65500bca80 con 0x7f655c100990 2024-09-19T04:32:59.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.994+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f655c1a5040 con 0x7f655c100990 2024-09-19T04:32:59.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.995+0000 7f6559ffb640 1 -- 172.21.15.94:0/598831149 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+23375 (secure 0 0 0) 0x7f65500bc380 con 0x7f655c100990 2024-09-19T04:32:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:32:59.997 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":104,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:32:05.721325+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3284064221":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/860358556":"2024-09-20T04:15:01.339086+0000","172.21.15.94:6801/2308569937":"2024-09-20T04:15:01.339086+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/3663117113":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3123077951":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/1265443753":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2308569937":"2024-09-20T04:15:01.339086+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:32:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.997+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f654c0a33c0 msgr2=0x7f654c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.997+0000 7f6562ed0640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f654c0a33c0 0x7f654c0a5880 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f655c06b450 tx=0x7f654800a2d0 comp rx=0 tx=0).stop 2024-09-19T04:32:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c100990 msgr2=0x7f655c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c100990 0x7f655c198300 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6550004770 tx=0x7f65500042f0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 shutdown_connections 2024-09-19T04:33:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f654c0a33c0 0x7f654c0a5880 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f655c1069a0 0x7f655c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f655c101330 0x7f655c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 --2- 172.21.15.94:0/598831149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c100990 0x7f655c198300 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 >> 172.21.15.94:0/598831149 conn(0x7f655c0fc6d0 msgr2=0x7f655c104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 shutdown_connections 2024-09-19T04:33:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:32:59.998+0000 7f6562ed0640 1 -- 172.21.15.94:0/598831149 wait complete. 2024-09-19T04:33:00.011 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:08:46.366796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T04:13:12.965501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T04:13:15.212761+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:33:00.011 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:33:00.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 -- 172.21.15.94:0/3074636335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0106bb0 msgr2=0x7ff7e0106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 --2- 172.21.15.94:0/3074636335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0106bb0 0x7ff7e0106f90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009900 tx=0x7ff7d002a410 comp rx=0 tx=0).stop 2024-09-19T04:33:00.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 -- 172.21.15.94:0/3074636335 shutdown_connections 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 --2- 172.21.15.94:0/3074636335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7e0102100 0x7ff7e010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 --2- 172.21.15.94:0/3074636335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7e0101780 0x7ff7e0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 --2- 172.21.15.94:0/3074636335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0106bb0 0x7ff7e0106f90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 -- 172.21.15.94:0/3074636335 >> 172.21.15.94:0/3074636335 conn(0x7ff7e0075e00 msgr2=0x7ff7e0076210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 -- 172.21.15.94:0/3074636335 shutdown_connections 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 -- 172.21.15.94:0/3074636335 wait complete. 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 Processor -- start 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 -- start start 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0101780 0x7ff7e01972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7e0102100 0x7ff7e0197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7e0106bb0 0x7ff7e019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e0110e10 con 0x7ff7e0101780 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e0110c90 con 0x7ff7e0102100 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.112+0000 7ff7e720e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e0110f90 con 0x7ff7e0106bb0 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e4faa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0101780 0x7ff7e01972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e4faa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0101780 0x7ff7e01972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51940/0 (socket says 172.21.15.94:51940) 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e4faa640 1 -- 172.21.15.94:0/2209804640 learned_addr learned my addr 172.21.15.94:0/2209804640 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7dffff640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7e0102100 0x7ff7e0197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e57ab640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7e0106bb0 0x7ff7e019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e4faa640 1 -- 172.21.15.94:0/2209804640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7e0106bb0 msgr2=0x7ff7e019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e4faa640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7e0106bb0 0x7ff7e019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e4faa640 1 -- 172.21.15.94:0/2209804640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7e0102100 msgr2=0x7ff7e0197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e4faa640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7e0102100 0x7ff7e0197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e4faa640 1 -- 172.21.15.94:0/2209804640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e019c2d0 con 0x7ff7e0101780 2024-09-19T04:33:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e57ab640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7e0106bb0 0x7ff7e019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:33:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e4faa640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0101780 0x7ff7e01972e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0005ab0 tx=0x7ff7d0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7ddffb640 1 -- 172.21.15.94:0/2209804640 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0035070 con 0x7ff7e0101780 2024-09-19T04:33:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7ddffb640 1 -- 172.21.15.94:0/2209804640 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7d0003e40 con 0x7ff7e0101780 2024-09-19T04:33:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e019c560 con 0x7ff7e0101780 2024-09-19T04:33:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7ddffb640 1 -- 172.21.15.94:0/2209804640 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0038bf0 con 0x7ff7e0101780 2024-09-19T04:33:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.113+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e0100670 con 0x7ff7e0101780 2024-09-19T04:33:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.114+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e0069da0 con 0x7ff7e0101780 2024-09-19T04:33:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.115+0000 7ff7ddffb640 1 -- 172.21.15.94:0/2209804640 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff7d0005020 con 0x7ff7e0101780 2024-09-19T04:33:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.116+0000 7ff7ddffb640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7c80a33c0 0x7ff7c80a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.117+0000 7ff7dffff640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7c80a33c0 0x7ff7c80a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.117+0000 7ff7ddffb640 1 -- 172.21.15.94:0/2209804640 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7ff7d00f1f40 con 0x7ff7e0101780 2024-09-19T04:33:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.117+0000 7ff7dffff640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7c80a33c0 0x7ff7c80a5880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00fea50 tx=0x7ff7d400b410 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.120+0000 7ff7ddffb640 1 -- 172.21.15.94:0/2209804640 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff7d00be880 con 0x7ff7e0101780 2024-09-19T04:33:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.303+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7e0198630 con 0x7ff7e0101780 2024-09-19T04:33:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.303+0000 7ff7ddffb640 1 -- 172.21.15.94:0/2209804640 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7ff7d00be180 con 0x7ff7e0101780 2024-09-19T04:33:00.305 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-19T04:33:00.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7c80a33c0 msgr2=0x7ff7c80a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7c80a33c0 0x7ff7c80a5880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00fea50 tx=0x7ff7d400b410 comp rx=0 tx=0).stop 2024-09-19T04:33:00.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0101780 msgr2=0x7ff7e01972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0101780 0x7ff7e01972e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0005ab0 tx=0x7ff7d0003cd0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 shutdown_connections 2024-09-19T04:33:00.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff7c80a33c0 0x7ff7c80a5880 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7e0106bb0 0x7ff7e019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7e0102100 0x7ff7e0197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 --2- 172.21.15.94:0/2209804640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e0101780 0x7ff7e01972e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.305+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 >> 172.21.15.94:0/2209804640 conn(0x7ff7e0075e00 msgr2=0x7ff7e00ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:00.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.306+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 shutdown_connections 2024-09-19T04:33:00.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.306+0000 7ff7e720e640 1 -- 172.21.15.94:0/2209804640 wait complete. 2024-09-19T04:33:00.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T04:33:00.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 -- 172.21.15.94:0/1624553233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c107a90 msgr2=0x7f5b7c107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 --2- 172.21.15.94:0/1624553233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c107a90 0x7f5b7c107e70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009930 tx=0x7f5b6c02a420 comp rx=0 tx=0).stop 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 -- 172.21.15.94:0/1624553233 shutdown_connections 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 --2- 172.21.15.94:0/1624553233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c102420 0x7f5b7c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 --2- 172.21.15.94:0/1624553233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7c101a80 0x7f5b7c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 --2- 172.21.15.94:0/1624553233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c107a90 0x7f5b7c107e70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 -- 172.21.15.94:0/1624553233 >> 172.21.15.94:0/1624553233 conn(0x7f5b7c0fd7c0 msgr2=0x7f5b7c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 -- 172.21.15.94:0/1624553233 shutdown_connections 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 -- 172.21.15.94:0/1624553233 wait complete. 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.421+0000 7f5b81331640 1 Processor -- start 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b81331640 1 -- start start 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b81331640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c101a80 0x7f5b7c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b81331640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c102420 0x7f5b7c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b81331640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7c107a90 0x7f5b7c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b81331640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c111d80 con 0x7f5b7c101a80 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b81331640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c111c00 con 0x7f5b7c102420 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b81331640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c111f00 con 0x7f5b7c107a90 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c101a80 0x7f5b7c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c101a80 0x7f5b7c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51954/0 (socket says 172.21.15.94:51954) 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7affd640 1 -- 172.21.15.94:0/3536359441 learned_addr learned my addr 172.21.15.94:0/3536359441 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7b7fe640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7c107a90 0x7f5b7c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7affd640 1 -- 172.21.15.94:0/3536359441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7c107a90 msgr2=0x7f5b7c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7a7fc640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c102420 0x7f5b7c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7affd640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7c107a90 0x7f5b7c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7affd640 1 -- 172.21.15.94:0/3536359441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c102420 msgr2=0x7f5b7c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7affd640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c102420 0x7f5b7c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7affd640 1 -- 172.21.15.94:0/3536359441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c1a68a0 con 0x7f5b7c101a80 2024-09-19T04:33:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7a7fc640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c102420 0x7f5b7c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:33:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b7affd640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c101a80 0x7f5b7c0744f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c004770 tx=0x7f5b6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.422+0000 7f5b63fff640 1 -- 172.21.15.94:0/3536359441 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c03f070 con 0x7f5b7c101a80 2024-09-19T04:33:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.423+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c1a6b30 con 0x7f5b7c101a80 2024-09-19T04:33:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.423+0000 7f5b63fff640 1 -- 172.21.15.94:0/3536359441 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b6c03b460 con 0x7f5b7c101a80 2024-09-19T04:33:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.423+0000 7f5b63fff640 1 -- 172.21.15.94:0/3536359441 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c035070 con 0x7f5b7c101a80 2024-09-19T04:33:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.423+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c1a7070 con 0x7f5b7c101a80 2024-09-19T04:33:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.424+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7c101ee0 con 0x7f5b7c101a80 2024-09-19T04:33:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.425+0000 7f5b63fff640 1 -- 172.21.15.94:0/3536359441 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5b6c03b610 con 0x7f5b7c101a80 2024-09-19T04:33:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.425+0000 7f5b63fff640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5b640a33c0 0x7f5b640a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.425+0000 7f5b7a7fc640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5b640a33c0 0x7f5b640a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.425+0000 7f5b63fff640 1 -- 172.21.15.94:0/3536359441 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f5b6c0f0460 con 0x7f5b7c101a80 2024-09-19T04:33:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.426+0000 7f5b7a7fc640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5b640a33c0 0x7f5b640a5880 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c075f80 tx=0x7f5b70005e50 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.429+0000 7f5b63fff640 1 -- 172.21.15.94:0/3536359441 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5b6c0bce00 con 0x7f5b7c101a80 2024-09-19T04:33:00.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:00 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/598831149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:33:00.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:00 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2209804640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:33:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.601+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b7c109ff0 con 0x7f5b7c101a80 2024-09-19T04:33:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.601+0000 7f5b63fff640 1 -- 172.21.15.94:0/3536359441 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) ==== 104+0+11 (secure 0 0 0) 0x7f5b6c0bc700 con 0x7f5b7c101a80 2024-09-19T04:33:00.602 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-09-19T04:33:00.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:00 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/598831149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:33:00.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:00 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2209804640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:33:00.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5b640a33c0 msgr2=0x7f5b640a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5b640a33c0 0x7f5b640a5880 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c075f80 tx=0x7f5b70005e50 comp rx=0 tx=0).stop 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c101a80 msgr2=0x7f5b7c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c101a80 0x7f5b7c0744f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c004770 tx=0x7f5b6c0042f0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 shutdown_connections 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f5b640a33c0 0x7f5b640a5880 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7c107a90 0x7f5b7c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c102420 0x7f5b7c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 --2- 172.21.15.94:0/3536359441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b7c101a80 0x7f5b7c0744f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 >> 172.21.15.94:0/3536359441 conn(0x7f5b7c0fd7c0 msgr2=0x7f5b7c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 shutdown_connections 2024-09-19T04:33:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.604+0000 7f5b81331640 1 -- 172.21.15.94:0/3536359441 wait complete. 2024-09-19T04:33:00.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 -- 172.21.15.94:0/2708966310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960107a90 msgr2=0x7f5960107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 --2- 172.21.15.94:0/2708966310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960107a90 0x7f5960107e70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5950009930 tx=0x7f595002a420 comp rx=0 tx=0).stop 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 -- 172.21.15.94:0/2708966310 shutdown_connections 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 --2- 172.21.15.94:0/2708966310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5960102420 0x7f596010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 --2- 172.21.15.94:0/2708966310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5960101a80 0x7f5960101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 --2- 172.21.15.94:0/2708966310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960107a90 0x7f5960107e70 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 -- 172.21.15.94:0/2708966310 >> 172.21.15.94:0/2708966310 conn(0x7f59600fd7c0 msgr2=0x7f59600ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 -- 172.21.15.94:0/2708966310 shutdown_connections 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 -- 172.21.15.94:0/2708966310 wait complete. 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 Processor -- start 2024-09-19T04:33:00.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.718+0000 7f596663c640 1 -- start start 2024-09-19T04:33:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f596663c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960101a80 0x7f59601982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f596663c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5960102420 0x7f5960198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f596663c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5960107a90 0x7f596019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f596663c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5960111d80 con 0x7f5960101a80 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f596663c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5960111c00 con 0x7f5960102420 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f596663c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5960111f00 con 0x7f5960107a90 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960101a80 0x7f59601982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960101a80 0x7f59601982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51974/0 (socket says 172.21.15.94:51974) 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595ffff640 1 -- 172.21.15.94:0/2922507055 learned_addr learned my addr 172.21.15.94:0/2922507055 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f5964bd9640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5960107a90 0x7f596019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595f7fe640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5960102420 0x7f5960198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595ffff640 1 -- 172.21.15.94:0/2922507055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5960107a90 msgr2=0x7f596019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595ffff640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5960107a90 0x7f596019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595ffff640 1 -- 172.21.15.94:0/2922507055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5960102420 msgr2=0x7f5960198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595ffff640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5960102420 0x7f5960198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595ffff640 1 -- 172.21.15.94:0/2922507055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596019d2c0 con 0x7f5960101a80 2024-09-19T04:33:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595ffff640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960101a80 0x7f59601982d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5950005940 tx=0x7f59500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.719+0000 7f595d7fa640 1 -- 172.21.15.94:0/2922507055 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595003f030 con 0x7f5960101a80 2024-09-19T04:33:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.720+0000 7f595d7fa640 1 -- 172.21.15.94:0/2922507055 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5950003cb0 con 0x7f5960101a80 2024-09-19T04:33:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.720+0000 7f595d7fa640 1 -- 172.21.15.94:0/2922507055 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950035070 con 0x7f5960101a80 2024-09-19T04:33:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.720+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596019d550 con 0x7f5960101a80 2024-09-19T04:33:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.720+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59601a4e40 con 0x7f5960101a80 2024-09-19T04:33:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.721+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5960101ee0 con 0x7f5960101a80 2024-09-19T04:33:00.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.722+0000 7f595d7fa640 1 -- 172.21.15.94:0/2922507055 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5950042030 con 0x7f5960101a80 2024-09-19T04:33:00.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.723+0000 7f595d7fa640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f594c0a3440 0x7f594c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:00.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.723+0000 7f595d7fa640 1 -- 172.21.15.94:0/2922507055 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f59500f0210 con 0x7f5960101a80 2024-09-19T04:33:00.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.723+0000 7f595f7fe640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f594c0a3440 0x7f594c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:00.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.723+0000 7f595f7fe640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f594c0a3440 0x7f594c0a5900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f59600fe820 tx=0x7f594800a4c0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:00.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.727+0000 7f595d7fa640 1 -- 172.21.15.94:0/2922507055 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59500bcbb0 con 0x7f5960101a80 2024-09-19T04:33:00.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:00 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/598831149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:33:00.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:00 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2209804640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:33:00.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.896+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5960109ff0 con 0x7f5960101a80 2024-09-19T04:33:00.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.896+0000 7f595d7fa640 1 -- 172.21.15.94:0/2922507055 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) ==== 100+0+11 (secure 0 0 0) 0x7f59500bc4b0 con 0x7f5960101a80 2024-09-19T04:33:00.898 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-09-19T04:33:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f594c0a3440 msgr2=0x7f594c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f594c0a3440 0x7f594c0a5900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f59600fe820 tx=0x7f594800a4c0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960101a80 msgr2=0x7f59601982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960101a80 0x7f59601982d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5950005940 tx=0x7f59500042f0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 shutdown_connections 2024-09-19T04:33:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f594c0a3440 0x7f594c0a5900 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5960107a90 0x7f596019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5960102420 0x7f5960198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 --2- 172.21.15.94:0/2922507055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960101a80 0x7f59601982d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 >> 172.21.15.94:0/2922507055 conn(0x7f59600fd7c0 msgr2=0x7f59600ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 shutdown_connections 2024-09-19T04:33:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:00.898+0000 7f596663c640 1 -- 172.21.15.94:0/2922507055 wait complete. 2024-09-19T04:33:00.911 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T04:33:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.007+0000 7f9d78311640 1 -- 172.21.15.94:0/3972776204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d701069d0 msgr2=0x7f9d70106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.007+0000 7f9d78311640 1 --2- 172.21.15.94:0/3972776204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d701069d0 0x7f9d70106db0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009930 tx=0x7f9d6c02a420 comp rx=0 tx=0).stop 2024-09-19T04:33:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.007+0000 7f9d78311640 1 -- 172.21.15.94:0/3972776204 shutdown_connections 2024-09-19T04:33:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.007+0000 7f9d78311640 1 --2- 172.21.15.94:0/3972776204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d70101360 0x7f9d7010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.007+0000 7f9d78311640 1 --2- 172.21.15.94:0/3972776204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d701009c0 0x7f9d70100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.007+0000 7f9d78311640 1 --2- 172.21.15.94:0/3972776204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d701069d0 0x7f9d70106db0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.007+0000 7f9d78311640 1 -- 172.21.15.94:0/3972776204 >> 172.21.15.94:0/3972776204 conn(0x7f9d700fc6a0 msgr2=0x7f9d700feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.007+0000 7f9d78311640 1 -- 172.21.15.94:0/3972776204 shutdown_connections 2024-09-19T04:33:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.007+0000 7f9d78311640 1 -- 172.21.15.94:0/3972776204 wait complete. 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d78311640 1 Processor -- start 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d78311640 1 -- start start 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d78311640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d701009c0 0x7f9d7019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d78311640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70101360 0x7f9d7019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d78311640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d701069d0 0x7f9d7019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d78311640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70110c80 con 0x7f9d70101360 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d78311640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70110b00 con 0x7f9d701009c0 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d78311640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70110e00 con 0x7f9d701069d0 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d758ac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70101360 0x7f9d7019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d758ac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70101360 0x7f9d7019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51988/0 (socket says 172.21.15.94:51988) 2024-09-19T04:33:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d758ac640 1 -- 172.21.15.94:0/1165001072 learned_addr learned my addr 172.21.15.94:0/1165001072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d760ad640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d701009c0 0x7f9d7019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d768ae640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d701069d0 0x7f9d7019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d758ac640 1 -- 172.21.15.94:0/1165001072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d701069d0 msgr2=0x7f9d7019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d758ac640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d701069d0 0x7f9d7019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d758ac640 1 -- 172.21.15.94:0/1165001072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d701009c0 msgr2=0x7f9d7019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d758ac640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d701009c0 0x7f9d7019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.008+0000 7f9d758ac640 1 -- 172.21.15.94:0/1165001072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7019f550 con 0x7f9d70101360 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.009+0000 7f9d758ac640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70101360 0x7f9d7019aaa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000d950 tx=0x7f9d6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.009+0000 7f9d677fe640 1 -- 172.21.15.94:0/1165001072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60017030 con 0x7f9d70101360 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.009+0000 7f9d677fe640 1 -- 172.21.15.94:0/1165001072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d60004020 con 0x7f9d70101360 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.009+0000 7f9d677fe640 1 -- 172.21.15.94:0/1165001072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d600047a0 con 0x7f9d70101360 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.009+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7019f840 con 0x7f9d70101360 2024-09-19T04:33:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.009+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701a70d0 con 0x7f9d70101360 2024-09-19T04:33:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.010+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d70100e20 con 0x7f9d70101360 2024-09-19T04:33:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.011+0000 7f9d677fe640 1 -- 172.21.15.94:0/1165001072 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9d600041d0 con 0x7f9d70101360 2024-09-19T04:33:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.012+0000 7f9d677fe640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9d5c0a33c0 0x7f9d5c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:01.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.012+0000 7f9d760ad640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9d5c0a33c0 0x7f9d5c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:01.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.012+0000 7f9d677fe640 1 -- 172.21.15.94:0/1165001072 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8585+0+0 (secure 0 0 0) 0x7f9d600cfe10 con 0x7f9d70101360 2024-09-19T04:33:01.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.012+0000 7f9d760ad640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9d5c0a33c0 0x7f9d5c0a5880 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009900 tx=0x7f9d6c000bc0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:01.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.015+0000 7f9d677fe640 1 -- 172.21.15.94:0/1165001072 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d6009c7b0 con 0x7f9d70101360 2024-09-19T04:33:01.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.227+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9d7019b8b0 con 0x7f9d70101360 2024-09-19T04:33:01.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.234+0000 7f9d677fe640 1 -- 172.21.15.94:0/1165001072 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+742871 (secure 0 0 0) 0x7f9d7019b8b0 con 0x7f9d70101360 2024-09-19T04:33:01.246 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:33:01.246 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:33:01.246 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 25, 2024-09-19T04:33:01.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.246 INFO:teuthology.orchestra.run.smithi094.stdout: "active_gid": 15159, 2024-09-19T04:33:01.246 INFO:teuthology.orchestra.run.smithi094.stdout: "active_name": "x", 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addrs": { 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3395191155 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3395191155 2024-09-19T04:33:01.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addr": "172.21.15.94:6801/3395191155", 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: "active_change": "2024-09-19T04:15:01.339292+0000", 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: "standbys": [ 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 24778, 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "y", 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-09-19T04:33:01.248 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.250 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.253 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.254 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-09-19T04:33:01.258 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-09-19T04:33:01.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-09-19T04:33:01.260 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.261 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.265 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.266 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.267 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-09-19T04:33:01.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.269 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.270 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:33:01.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.272 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:33:01.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:33:01.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.273 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-09-19T04:33:01.273 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-09-19T04:33:01.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:01.273 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.275 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.276 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.277 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.277 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.277 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.277 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.278 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.279 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-09-19T04:33:01.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-19T04:33:01.281 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-09-19T04:33:01.282 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.283 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:33:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:33:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:33:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:33:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-09-19T04:33:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-09-19T04:33:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-09-19T04:33:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-09-19T04:33:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-09-19T04:33:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-09-19T04:33:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:33:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-09-19T04:33:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:33:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:33:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:33:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:33:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.374 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.376 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.377 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.377 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.377 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.377 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.377 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:33:01.377 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:33:01.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:33:01.378 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.380 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.381 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.382 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.383 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.385 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.386 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.387 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:33:01.388 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.389 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.390 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.392 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.393 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.394 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.395 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.396 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.397 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.400 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:33:01.401 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:33:01.405 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.405 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.405 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.405 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-09-19T04:33:01.405 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.405 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.405 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.406 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.407 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.408 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:33:01.409 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.410 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.412 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.420 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.422 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.423 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.425 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.427 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.428 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.431 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-19T04:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-09-19T04:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-09-19T04:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-09-19T04:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-09-19T04:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-09-19T04:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.532 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.534 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.535 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.537 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.538 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.539 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.541 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.542 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.543 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.544 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.547 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.548 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.549 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.552 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.553 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.554 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.555 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.556 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.557 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.558 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.561 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:01.562 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-09-19T04:33:01.563 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.564 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.566 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.567 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.568 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.569 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.570 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:33:01.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-09-19T04:33:01.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-19T04:33:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-19T04:33:01.575 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-09-19T04:33:01.576 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.578 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.580 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.581 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.582 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.583 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.585 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.586 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.587 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.588 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.589 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.591 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.592 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.593 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.595 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.597 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.599 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.601 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.602 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.603 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.604 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.605 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.606 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.607 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.608 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.609 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.609 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.609 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.609 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.609 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.609 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.612 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.612 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.612 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.612 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.612 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.612 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.612 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.612 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.613 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.614 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-09-19T04:33:01.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.617 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.618 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.620 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.621 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.622 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-09-19T04:33:01.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-09-19T04:33:01.624 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.626 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.628 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.629 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.630 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.631 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.633 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.634 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-09-19T04:33:01.635 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-09-19T04:33:01.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.637 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.638 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.639 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.640 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.641 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.642 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.643 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.644 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.645 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.649 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.651 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.652 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.654 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-09-19T04:33:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.656 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.658 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.660 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.661 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-09-19T04:33:01.663 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.665 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.666 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.667 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.668 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.669 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.670 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.671 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.672 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.673 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.674 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.675 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-09-19T04:33:01.676 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.679 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.680 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.681 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.682 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.683 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.684 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.685 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.686 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.687 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.689 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.690 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.693 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.694 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.695 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.697 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:33:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.699 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.700 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.701 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.702 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.704 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.706 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.707 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.708 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.709 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.710 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.711 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-09-19T04:33:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:01.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 24809, 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "z", 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.716 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.717 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.718 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.719 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.720 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.721 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.723 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.724 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.725 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.727 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-09-19T04:33:01.728 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.731 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.732 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:01.733 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.734 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.736 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-09-19T04:33:01.737 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-09-19T04:33:01.738 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.739 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.740 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.741 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.742 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.744 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-09-19T04:33:01.746 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.747 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:01.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-09-19T04:33:01.749 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.750 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.751 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.752 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.754 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-09-19T04:33:01.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-09-19T04:33:01.756 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.757 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-09-19T04:33:01.759 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.760 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.761 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.763 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-09-19T04:33:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.765 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.766 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.768 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:33:01.769 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.770 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.771 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.772 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:33:01.773 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:33:01.774 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.775 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:33:01.776 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.778 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:33:01.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-09-19T04:33:01.780 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.782 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.783 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.784 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-09-19T04:33:01.785 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:01.787 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.788 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.789 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.790 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.791 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.792 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.793 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:33:01.794 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.795 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.796 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.797 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.798 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.799 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.802 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-09-19T04:33:01.804 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.806 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-09-19T04:33:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.808 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.810 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.811 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.812 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:33:01.813 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.814 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.816 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.816 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-09-19T04:33:01.816 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-09-19T04:33:01.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.816 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.816 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.816 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.816 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.818 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.819 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.821 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.823 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.824 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.827 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.828 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-09-19T04:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.832 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.833 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.834 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.835 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:33:01.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:33:01.838 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.839 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.840 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.841 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.843 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.844 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.845 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.846 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.847 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:33:01.849 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:33:01.850 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.851 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.854 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.855 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.856 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.857 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.858 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.859 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.860 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.861 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.862 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.864 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.865 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:33:01.866 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.867 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.868 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.869 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.870 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.871 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:33:01.872 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.873 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.874 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.875 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.876 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.883 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.884 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.885 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.886 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.887 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.888 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-19T04:33:01.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-19T04:33:01.890 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.891 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.893 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.894 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-19T04:33:01.895 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.896 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-09-19T04:33:01.897 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.898 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.899 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.900 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.901 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.902 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-09-19T04:33:01.904 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.905 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.906 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.907 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.909 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:33:01.910 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.911 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.912 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.913 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.914 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.915 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.916 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.919 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.920 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-09-19T04:33:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-09-19T04:33:01.923 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.924 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.926 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.928 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.929 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.930 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.931 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.932 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.933 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:01.934 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.935 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.936 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.937 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.941 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.942 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.944 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.946 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.947 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.948 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.949 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.950 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.952 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.953 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.955 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.956 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.957 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.959 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.961 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.963 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.964 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-09-19T04:33:01.965 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.968 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:33:01.969 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.970 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.971 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.972 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.973 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.976 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.979 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-09-19T04:33:01.981 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.982 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-09-19T04:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-09-19T04:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-09-19T04:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-09-19T04:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-09-19T04:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-09-19T04:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-09-19T04:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.077 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.078 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.080 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.081 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.082 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.084 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-09-19T04:33:02.085 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.086 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.089 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.090 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.091 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.092 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.093 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.094 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.095 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-09-19T04:33:02.096 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-09-19T04:33:02.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.099 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.100 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.101 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.101 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.101 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.101 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.101 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.101 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.101 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.101 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.102 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.103 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.104 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.105 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.106 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.107 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.108 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.109 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.110 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.111 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.112 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.113 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.115 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.116 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:02.117 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.118 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.119 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.119 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.119 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.119 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.119 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.119 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.119 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.119 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.119 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.120 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.122 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.123 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.125 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.126 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.127 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-09-19T04:33:02.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-09-19T04:33:02.131 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.133 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.134 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.136 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.137 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.138 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.138 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.138 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.138 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.138 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-09-19T04:33:02.138 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-09-19T04:33:02.138 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.138 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.140 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.141 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.142 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.143 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.144 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.145 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:33:02.147 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-09-19T04:33:02.148 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.149 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.150 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.152 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.153 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.154 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.155 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.156 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.157 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.158 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.159 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.160 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:33:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard", 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "restful", 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_schedule" 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:33:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-09-19T04:33:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-09-19T04:33:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:33:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-09-19T04:33:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-09-19T04:33:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:33:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:33:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-09-19T04:33:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-09-19T04:33:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-09-19T04:33:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-09-19T04:33:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-09-19T04:33:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-09-19T04:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-09-19T04:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-09-19T04:33:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-09-19T04:33:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-09-19T04:33:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:33:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:33:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-09-19T04:33:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:33:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-09-19T04:33:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-09-19T04:33:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-09-19T04:33:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:33:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:33:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:33:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-09-19T04:33:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-09-19T04:33:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-09-19T04:33:02.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-09-19T04:33:02.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-09-19T04:33:02.279 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.280 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.282 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:33:02.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.286 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.288 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.290 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.291 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.294 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.295 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.296 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.303 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.304 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.306 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.307 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:33:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:33:02.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.313 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.316 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.319 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.323 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:33:02.328 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.329 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.331 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.332 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.334 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.335 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.336 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.337 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.340 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:33:02.341 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.342 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.344 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:33:02.345 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.347 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.348 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.351 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.352 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.353 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.354 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.355 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-19T04:33:02.356 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.357 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.358 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-09-19T04:33:02.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-19T04:33:02.360 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.362 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.363 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.364 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.365 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.368 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.369 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.370 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-09-19T04:33:02.371 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.372 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-09-19T04:33:02.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:02.376 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.377 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.379 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.380 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.381 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.383 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.384 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-19T04:33:02.385 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.386 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.387 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.390 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.392 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.393 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.394 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.395 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.396 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.397 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:33:02.401 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.403 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.405 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.406 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.409 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.410 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.411 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.412 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.413 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.415 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.416 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.420 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.421 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.422 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.423 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.424 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.425 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.427 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.429 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.430 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.431 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.432 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-09-19T04:33:02.433 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.434 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.434 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.434 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:33:02.434 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.434 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.436 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.437 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.438 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.439 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.441 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.443 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.445 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.446 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.447 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.448 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.449 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.451 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.452 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.454 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.456 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.457 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.458 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.459 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.462 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-09-19T04:33:02.464 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-09-19T04:33:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:33:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-09-19T04:33:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-09-19T04:33:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-19T04:33:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-19T04:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-09-19T04:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-09-19T04:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-09-19T04:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-09-19T04:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-09-19T04:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-19T04:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-19T04:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.637 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.639 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-19T04:33:02.641 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.643 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-09-19T04:33:02.644 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "services": { 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard": "https://172.21.15.94:8443/" 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "always_on_modules": { 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "octopus": [ 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-19T04:33:02.646 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "pacific": [ 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "quincy": [ 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-19T04:33:02.648 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "reef": [ 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-19T04:33:02.649 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "squid": [ 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-19T04:33:02.650 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: "last_failure_osd_epoch": 82, 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: "active_clients": [ 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2300900557 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:02.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "libcephsqlite", 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2397239557 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1650992217 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-09-19T04:33:02.653 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 865224094 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1765735634 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 847697648 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-19T04:33:02.655 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:33:02.657 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:01 smithi112 ceph-mon[35051]: pgmap v564: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 14 op/s 2024-09-19T04:33:02.657 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:01 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3536359441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:33:02.657 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:01 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2922507055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:33:02.657 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:01 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1165001072' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:33:02.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:01 smithi104 ceph-mon[34145]: pgmap v564: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 14 op/s 2024-09-19T04:33:02.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:01 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3536359441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:33:02.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:01 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2922507055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:33:02.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:01 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1165001072' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:33:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9d5c0a33c0 msgr2=0x7f9d5c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9d5c0a33c0 0x7f9d5c0a5880 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009900 tx=0x7f9d6c000bc0 comp rx=0 tx=0).stop 2024-09-19T04:33:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70101360 msgr2=0x7f9d7019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70101360 0x7f9d7019aaa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000d950 tx=0x7f9d6000de10 comp rx=0 tx=0).stop 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 shutdown_connections 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f9d5c0a33c0 0x7f9d5c0a5880 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d701069d0 0x7f9d7019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70101360 0x7f9d7019aaa0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 --2- 172.21.15.94:0/1165001072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d701009c0 0x7f9d7019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 >> 172.21.15.94:0/1165001072 conn(0x7f9d700fc6a0 msgr2=0x7f9d7006a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 shutdown_connections 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:01.239+0000 7f9d78311640 1 -- 172.21.15.94:0/1165001072 wait complete. 2024-09-19T04:33:02.720 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T04:33:02.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T04:33:02.721 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:01 smithi094 ceph-mon[30972]: pgmap v564: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 14 op/s 2024-09-19T04:33:02.721 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:01 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3536359441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:33:02.721 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:01 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2922507055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:33:02.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:01 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1165001072' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:33:02.984 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:03.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:03 smithi094 ceph-mon[30972]: pgmap v565: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 8 op/s 2024-09-19T04:33:03.512 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.001220608, 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T04:33:03.513 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000183128, 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 333, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 321, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1481968, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 197488, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.055992839, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.021005266, 2024-09-19T04:33:03.515 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.047444775, 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 321, 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.018428593, 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000057409 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.516 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1481968, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 197488 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 3, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 246, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 480, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 279264, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.017408950, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.012496395, 2024-09-19T04:33:03.517 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.001692078, 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 243, 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.020126407, 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000082824 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T04:33:03.518 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 279264 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:03.519 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:03.520 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148704, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 29, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-09-19T04:33:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 332, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1432522, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 332, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 332, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1432522, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:03.524 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:03.525 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:03.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:03.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:03.525 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:33:03.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:03 smithi104 ceph-mon[34145]: pgmap v565: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 8 op/s 2024-09-19T04:33:03.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:03 smithi112 ceph-mon[35051]: pgmap v565: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 8 op/s 2024-09-19T04:33:04.207 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-19T04:33:04.207 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-19T04:33:04.207 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-19T04:33:04.207 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-19T04:33:04.207 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-19T04:33:04.207 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-19T04:33:04.208 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-19T04:33:04.208 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T04:33:04.208 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T04:33:04.452 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 4, 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 4, 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 2992, 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 448, 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.003112716, 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000041188, 2024-09-19T04:33:04.933 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.002220904, 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.003907857, 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001302619 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:33:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 3, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 250, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 480, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 279968, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.018303142, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.013361217, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.002094070, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 247, 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.019197041, 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000077720 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 279968 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.936 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 334, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 324, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1480336, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 200240, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.048286539, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.020057993, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.040469964, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 323, 2024-09-19T04:33:04.937 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.015813890, 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000048959 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1480336, 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 200240 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-09-19T04:33:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:04.939 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:04.940 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:04.941 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148731, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 31, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:04.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-09-19T04:33:04.943 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 337, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1433172, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 337, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 337, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1433172, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:04.944 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:04.945 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:33:05.493 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:05 smithi094 ceph-mon[30972]: pgmap v566: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 118 KiB/s wr, 28 op/s 2024-09-19T04:33:05.522 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T04:33:05.522 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T04:33:05.522 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-19T04:33:05.522 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-19T04:33:05.522 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T04:33:05.522 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T04:33:05.522 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T04:33:05.522 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T04:33:05.523 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T04:33:05.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:05 smithi104 ceph-mon[34145]: pgmap v566: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 118 KiB/s wr, 28 op/s 2024-09-19T04:33:05.768 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:05.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:05 smithi112 ceph-mon[35051]: pgmap v566: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 118 KiB/s wr, 28 op/s 2024-09-19T04:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 335, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 323, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1482512, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 198768, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.059342320, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.020492252, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.051302156, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 323, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.018879489, 2024-09-19T04:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000058450 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1482512, 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 198768 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 3, 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 247, 2024-09-19T04:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 480, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 279440, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.018663008, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.013615868, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.002221740, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 244, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.019061486, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000078120 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 279440 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-09-19T04:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.000310379, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000226754, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148714, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-09-19T04:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 334, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1432782, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 334, 2024-09-19T04:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 334, 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1432782, 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:33:06.945 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-19T04:33:06.945 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-19T04:33:06.945 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-19T04:33:06.945 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-19T04:33:06.945 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-19T04:33:06.946 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-19T04:33:06.946 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-19T04:33:06.946 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T04:33:06.946 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T04:33:07.190 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:07.589 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:07 smithi094 ceph-mon[30972]: pgmap v567: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 101 KiB/s wr, 24 op/s 2024-09-19T04:33:07.713 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 336, 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 326, 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1480880, 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 201520, 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.058096481, 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.021541600, 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.048160309, 2024-09-19T04:33:07.714 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 325, 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.018500218, 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000056923 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1480880, 2024-09-19T04:33:07.715 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 201520 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 7, 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 253, 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 3472, 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 280592, 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.020350726, 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.013323217, 2024-09-19T04:33:07.716 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.003854996, 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 251, 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.024274441, 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000096710 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-19T04:33:07.717 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 280592 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.000409759, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000231348, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-09-19T04:33:07.719 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:07.720 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:07.721 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:07.721 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:07.721 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:07.721 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:07.721 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:07.721 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:07.722 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148721, 2024-09-19T04:33:07.723 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:07.724 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-09-19T04:33:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 339, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1433432, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 339, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 339, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1433432, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:07.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-19T04:33:07.727 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-19T04:33:07.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:07 smithi112 ceph-mon[35051]: pgmap v567: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 101 KiB/s wr, 24 op/s 2024-09-19T04:33:07.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:07 smithi104 ceph-mon[34145]: pgmap v567: 97 pgs: 97 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 101 KiB/s wr, 24 op/s 2024-09-19T04:33:08.338 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T04:33:08.338 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T04:33:08.338 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-19T04:33:08.338 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-19T04:33:08.338 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T04:33:08.338 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T04:33:08.338 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T04:33:08.338 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T04:33:08.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T04:33:08.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:08 smithi094 ceph-mon[30972]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T04:33:08.591 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:33:08.712 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:08 smithi104 ceph-mon[34145]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T04:33:08.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:08 smithi112 ceph-mon[35051]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 8433, 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 8277, 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 2214550, 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 50924722, 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 184, 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.562080672, 2024-09-19T04:33:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.188051486, 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.159244672, 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.168341182, 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 8271, 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.640023260, 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000077381 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 460, 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000069103, 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000150 2024-09-19T04:33:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 98688, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 30768 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 17319, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 20309, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 6524935, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 52571577, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 8, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 1.007497920, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.398453365, 2024-09-19T04:33:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.503432246, 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.148155827, 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 20312, 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.108857754, 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000054591 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 20700, 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.003266248, 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000157 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1694944, 2024-09-19T04:33:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 726768 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 34230, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 38686, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 12474352, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 23901875, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 27, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 1.933775797, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.702119011, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 1.275762152, 2024-09-19T04:33:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.178634379, 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 38683, 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.834650892, 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000021576 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 63278, 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.009357917, 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000147 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 592656, 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 304768 2024-09-19T04:33:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 18580, 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 6.048794386, 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000325554 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-19T04:33:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 383, 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.011795627, 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000030797 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 8702, 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 7442, 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7442, 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 184.556001934, 2024-09-19T04:33:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.024799247 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 1260, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 16, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 47, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 4, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 719, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 74, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 608, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 37, 2024-09-19T04:33:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 612, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 2668, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 412, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 412, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 39, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 13807, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 11823, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 1221, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 34, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 2, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 23, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 4737, 2024-09-19T04:33:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 67, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 2530, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 41, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 1870, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 14124, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 1022, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 1619473741, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 14124, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-19T04:33:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 771, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 23, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 748, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 13353, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 36, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 2883, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 2652, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 1, 2024-09-19T04:33:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 10459, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 5015, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 40, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 1531 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 153, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 16, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 9, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 46, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 344, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-09-19T04:33:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 129, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 85, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 48, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 15, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:33:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 23, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 69, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 4, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 11332, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-09-19T04:33:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 11332, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 28, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 28, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-09-19T04:33:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 101926964, 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3960, 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 26.801529722, 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006768063 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 652, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 3180, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 2528, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 105, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 151, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 46, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 719, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 3485, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 2766, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 412, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 2332, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 1920, 2024-09-19T04:33:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 71608, 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 231712 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 11416, 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 1433, 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 8702, 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 128, 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 2766, 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1039, 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 7.188924382, 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006919080 2024-09-19T04:33:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 603, 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.754208770, 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002909135 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-19T04:33:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 48, 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.268717304, 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005598277 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3730, 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 88.850369133, 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.023820474 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.097 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-19T04:33:09.097 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.097 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.097 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 18, 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.002544243, 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000141346 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 37, 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.042216392, 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001140983 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-19T04:33:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 18, 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.136839991, 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.007602221 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1584, 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 5.743569708, 2024-09-19T04:33:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003625990 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 137, 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.212926666, 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001554209 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 135, 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 80.082031857, 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.593200235 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.010879391, 2024-09-19T04:33:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005439695 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 12, 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.072905744, 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006075478 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-19T04:33:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.003015936, 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001507968 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7, 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.019583943, 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002797706 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 70, 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.167268474, 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002389549 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 5, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 3, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 922, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 461, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 1056, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-19T04:33:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 412687, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 139, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 88, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 1, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8288, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 182, 2024-09-19T04:33:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 2512317, 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 15781, 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-19T04:33:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 19337, 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 98392762, 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 33, 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 19337, 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 19337, 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 41.727524308, 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002157910 2024-09-19T04:33:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 19337, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 19700 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 19337, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 14248, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 5089, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 11, 2024-09-19T04:33:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 68, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 4005, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 113, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 3, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 651, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 14147, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 22, 2024-09-19T04:33:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 2, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-19T04:33:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 105, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-19T04:33:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 26, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 500, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 49, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 129 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:33:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 1 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 21, 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 7, 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 115, 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 85, 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-19T04:33:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 59978, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 15715709, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 59978, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 59978, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 15715709, 2024-09-19T04:33:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 19337, 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 98401172, 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 19337, 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 98401172, 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 19337, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 19337, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 19337, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 19337, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 85, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 9273, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 85, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 45, 2024-09-19T04:33:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 9273, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55b264c6a960": { 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 112, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 11332, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 97732660, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 11332, 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 3960, 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 97732548, 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:09.125 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-19T04:33:09.759 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:09 smithi104 ceph-mon[34145]: pgmap v569: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 28 op/s 2024-09-19T04:33:09.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:09 smithi112 ceph-mon[35051]: pgmap v569: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 28 op/s 2024-09-19T04:33:09.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:09 smithi094 ceph-mon[30972]: pgmap v569: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 28 op/s 2024-09-19T04:33:09.998 INFO:tasks.check_counter:Daemon mds.b mds.exported=67 2024-09-19T04:33:09.999 INFO:tasks.check_counter:Daemon mds.b mds.imported=41 2024-09-19T04:33:09.999 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=153 2024-09-19T04:33:09.999 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=16 2024-09-19T04:33:09.999 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T04:33:09.999 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=18 2024-09-19T04:33:09.999 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=12 2024-09-19T04:33:09.999 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T04:33:09.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T04:33:10.247 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:33:10.744 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-19T04:33:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 402, 2024-09-19T04:33:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 357, 2024-09-19T04:33:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1664242, 2024-09-19T04:33:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 206612, 2024-09-19T04:33:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-09-19T04:33:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 1, 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.051720264, 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.019100994, 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.047564327, 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000020466, 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 356, 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.014155224, 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000039761 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1663408, 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 206288 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1159, 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1404, 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 2208403, 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 968003, 2024-09-19T04:33:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.093158657, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.031488782, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.023904463, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.028182096, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1402, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.052636447, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000037543 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 601344 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 2589, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 2591, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 4952867, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 823360, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 5, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-09-19T04:33:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.194754873, 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.065868267, 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.052005335, 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.060515281, 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2589, 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.068064799, 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000026289 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 6347, 2024-09-19T04:33:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.136479611, 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000021503 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2024-09-19T04:33:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 748, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 16, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 732, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 725, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 13, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2024-09-19T04:33:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-19T04:33:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:33:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-09-19T04:33:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:33:10.757 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-09-19T04:33:10.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 4060, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 7, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-09-19T04:33:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 10306343, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 10299525, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 4060 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 524, 2024-09-19T04:33:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 1130, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 606, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 24, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 32, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 8, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 748, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 1205, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 457, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 53800, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 215328 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.763 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-19T04:33:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-19T04:33:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-19T04:33:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-19T04:33:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-19T04:33:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-19T04:33:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-09-19T04:33:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 4, 2024-09-19T04:33:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 2, 2024-09-19T04:33:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 3, 2024-09-19T04:33:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-19T04:33:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-09-19T04:33:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-09-19T04:33:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 2632, 2024-09-19T04:33:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-19T04:33:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-19T04:33:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 198264, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 57, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8408, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 185, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 1721621, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 17381, 2024-09-19T04:33:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 3743, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 21, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 3, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 3743, 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-19T04:33:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3743, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 3.546959422, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000947624 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3743, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 3744 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 3743, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 3743, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-19T04:33:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 2267, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 1474, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-19T04:33:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 1, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-19T04:33:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-19T04:33:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 105, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 26, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 6, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 7, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 1, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-19T04:33:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-09-19T04:33:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 4146, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 8428970, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 4146, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4146, 2024-09-19T04:33:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 8428970, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 3743, 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 6112044, 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 3743, 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 6112044, 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 3743, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 3743, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 3743, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 3743, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x5567a915ab40": { 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:10.795 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:10.795 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:10.795 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-19T04:33:11.566 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T04:33:11.566 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T04:33:11.566 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-19T04:33:11.566 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-19T04:33:11.566 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-19T04:33:11.567 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:33:11.567 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:33:11.567 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T04:33:11.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T04:33:11.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:11 smithi112 ceph-mon[35051]: pgmap v570: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-09-19T04:33:11.810 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:33:11.841 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:11 smithi094 ceph-mon[30972]: pgmap v570: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-09-19T04:33:11.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:11 smithi104 ceph-mon[34145]: pgmap v570: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-09-19T04:33:12.318 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-19T04:33:12.318 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:12.318 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 3224, 2024-09-19T04:33:12.318 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 3224, 2024-09-19T04:33:12.318 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 15865563, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 1024415, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 2, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.219664628, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.066120429, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.069051504, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.067947322, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3223, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.076129338, 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000023620 2024-09-19T04:33:12.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 948, 2024-09-19T04:33:12.320 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 903, 2024-09-19T04:33:12.321 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 5994792, 2024-09-19T04:33:12.322 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 379974, 2024-09-19T04:33:12.322 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-09-19T04:33:12.322 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-09-19T04:33:12.322 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.088203014, 2024-09-19T04:33:12.322 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.028725221, 2024-09-19T04:33:12.322 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.059916203, 2024-09-19T04:33:12.322 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.012199555, 2024-09-19T04:33:12.322 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:12.323 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 902, 2024-09-19T04:33:12.323 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.029173573, 2024-09-19T04:33:12.323 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000032343 2024-09-19T04:33:12.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.323 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:12.323 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.324 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.324 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.324 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1663136, 2024-09-19T04:33:12.324 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 205648 2024-09-19T04:33:12.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.324 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:12.325 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 246, 2024-09-19T04:33:12.325 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 492, 2024-09-19T04:33:12.325 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 60366, 2024-09-19T04:33:12.325 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 678312, 2024-09-19T04:33:12.325 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 8, 2024-09-19T04:33:12.326 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-09-19T04:33:12.326 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.030943966, 2024-09-19T04:33:12.326 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.014132293, 2024-09-19T04:33:12.326 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.006402091, 2024-09-19T04:33:12.326 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.006647074, 2024-09-19T04:33:12.326 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:12.327 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 489, 2024-09-19T04:33:12.327 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.026290970, 2024-09-19T04:33:12.327 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000053764 2024-09-19T04:33:12.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.327 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:12.327 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.327 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.328 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.328 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.328 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:33:12.328 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 600720 2024-09-19T04:33:12.328 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.328 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-19T04:33:12.328 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-19T04:33:12.329 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-19T04:33:12.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.329 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-19T04:33:12.329 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-19T04:33:12.329 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-19T04:33:12.329 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 6892, 2024-09-19T04:33:12.329 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.128112825, 2024-09-19T04:33:12.329 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000018588 2024-09-19T04:33:12.329 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-19T04:33:12.330 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2024-09-19T04:33:12.331 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-09-19T04:33:12.332 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-09-19T04:33:12.332 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 782, 2024-09-19T04:33:12.332 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 43, 2024-09-19T04:33:12.332 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 739, 2024-09-19T04:33:12.332 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-09-19T04:33:12.332 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 720, 2024-09-19T04:33:12.332 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-09-19T04:33:12.332 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-09-19T04:33:12.332 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 26, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-09-19T04:33:12.333 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:33:12.334 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:33:12.335 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:33:12.335 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:33:12.335 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:33:12.335 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-09-19T04:33:12.335 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:33:12.335 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:33:12.335 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:33:12.335 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-09-19T04:33:12.336 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-19T04:33:12.337 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-19T04:33:12.338 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:33:12.338 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:33:12.338 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:33:12.338 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-09-19T04:33:12.338 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-19T04:33:12.338 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:33:12.339 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-19T04:33:12.339 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-09-19T04:33:12.339 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.339 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-19T04:33:12.339 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 13261, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 16, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 3, 2024-09-19T04:33:12.340 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 23334416, 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 23334355, 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 13261 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.341 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 540, 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 2346, 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 1806, 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 47, 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 141, 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 94, 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 782, 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 2712, 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 1930, 2024-09-19T04:33:12.342 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 53588, 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 215328 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2024-09-19T04:33:12.343 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.344 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.345 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.345 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.345 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-19T04:33:12.346 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.346 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.346 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.346 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.346 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-19T04:33:12.346 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.346 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.347 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.348 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.349 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-19T04:33:12.350 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.351 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-19T04:33:12.352 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.353 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.354 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.355 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.356 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.357 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 4, 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 2, 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 3, 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 4045, 2024-09-19T04:33:12.358 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:12.359 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:12.360 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:12.361 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 199048, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 59, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8408, 2024-09-19T04:33:12.362 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 185, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 1848077, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 23129, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-19T04:33:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 4012, 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 21, 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 3, 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 4012, 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4012, 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 3.134964016, 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000781396 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4012, 2024-09-19T04:33:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 4013 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 4012, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 4012, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 2272, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 1738, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-19T04:33:12.365 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 1, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-19T04:33:12.366 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-19T04:33:12.367 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 105, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 26, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 6, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 7, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 1, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-19T04:33:12.368 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.369 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 4414, 2024-09-19T04:33:12.370 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 21500111, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 4414, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4414, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 21500111, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.371 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 4012, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 19140723, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4012, 2024-09-19T04:33:12.372 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 19140723, 2024-09-19T04:33:12.373 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:12.373 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.373 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.373 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.373 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:12.373 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.373 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-19T04:33:12.374 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:12.374 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-19T04:33:12.374 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:12.374 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:12.374 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:12.374 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:12.374 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:12.375 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 4012, 2024-09-19T04:33:12.375 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 4012, 2024-09-19T04:33:12.375 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4012, 2024-09-19T04:33:12.375 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 4012, 2024-09-19T04:33:12.375 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:12.375 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.376 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.376 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.376 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:12.376 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.376 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:33:12.376 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:12.377 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-19T04:33:12.377 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:12.377 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:12.377 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:12.377 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:12.377 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:12.378 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:12.378 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:12.378 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-19T04:33:12.378 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-19T04:33:12.378 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:12.378 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.379 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.379 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.379 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:12.379 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:12.379 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x557715026b40": { 2024-09-19T04:33:12.379 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:12.380 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-19T04:33:12.380 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:12.380 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-19T04:33:12.380 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-19T04:33:12.380 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:12.380 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:12.381 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:12.381 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:12.381 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-19T04:33:12.381 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-19T04:33:12.381 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:12.381 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:12.382 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:12.382 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:12.382 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:12.382 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:12.382 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-19T04:33:13.020 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T04:33:13.020 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-19T04:33:13.020 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-19T04:33:13.020 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-09-19T04:33:13.020 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-19T04:33:13.020 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:33:13.020 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:33:13.020 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T04:33:13.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T04:33:13.265 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:33:13.646 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:13 smithi104 ceph-mon[34145]: pgmap v571: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-09-19T04:33:13.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:13 smithi112 ceph-mon[35051]: pgmap v571: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-09-19T04:33:13.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:13 smithi094 ceph-mon[30972]: pgmap v571: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-09-19T04:33:13.902 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-19T04:33:13.902 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:13.902 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 0, 2024-09-19T04:33:13.902 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 2, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 0, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 0, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 0, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.001008387, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.000170815, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:13.903 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 7, 2024-09-19T04:33:13.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 255, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 3472, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 280944, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.015699942, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.009306691, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.003857384, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 253, 2024-09-19T04:33:13.905 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.023290041, 2024-09-19T04:33:13.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000092055 2024-09-19T04:33:13.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.906 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:13.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:13.909 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:13.909 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:13.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.909 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 280944 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 338, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 328, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1481424, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 202800, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 2, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 1, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.048859591, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.017657680, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.045613829, 2024-09-19T04:33:13.910 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 327, 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.012380066, 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000037859 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1481424, 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 202800 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.911 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 0, 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:13.912 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:13.913 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:13.914 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 148704, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 29, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:13.915 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 928, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 2, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-19T04:33:13.916 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 341, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1433692, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 341, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-19T04:33:13.917 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-19T04:33:13.918 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 341, 2024-09-19T04:33:13.918 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1433692, 2024-09-19T04:33:13.918 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-19T04:33:13.918 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-19T04:33:13.918 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-19T04:33:13.918 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-19T04:33:13.918 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:13.918 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-19T04:33:13.918 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-19T04:33:14.584 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-19T04:33:14.584 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-19T04:33:14.584 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-19T04:33:14.584 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-19T04:33:14.584 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-19T04:33:14.585 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-19T04:33:14.585 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-19T04:33:14.585 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T04:33:14.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T04:33:14.901 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:33:15.408 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-19T04:33:15.408 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:15.408 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 3349, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 3355, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 838137, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 5738142, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 59, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.217870464, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.061202037, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.061172759, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.073073488, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3351, 2024-09-19T04:33:15.409 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.131253233, 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000039168 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 71, 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000008039, 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000113 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3280, 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 4144 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:15.410 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 902412, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 726117, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 258492194, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 204547842, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 15, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 36.035426821, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 16.210889926, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 25.010935229, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.048013198, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 726116, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 9.364095284, 2024-09-19T04:33:15.411 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000012896 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1615434, 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.217882932, 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000134 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 2688, 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 614352 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:15.412 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 14910, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 12064, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 7170242, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 3941658, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 7, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.729785828, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.274939395, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.489143175, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.057038752, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 12064, 2024-09-19T04:33:15.413 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.313562881, 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000025991 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 20838, 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003179113, 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000152 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1683072, 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 238704 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-09-19T04:33:15.414 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7355, 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.638823769, 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000222817 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-19T04:33:15.415 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 533, 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.018850309, 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000035366 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 3698, 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 3186, 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3186, 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 38.151528451, 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.011974742 2024-09-19T04:33:15.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 512, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 2, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 11, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 652, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 30, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 522, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 100, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 606, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 525, 2024-09-19T04:33:15.417 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 367, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 367, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 31, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 8356, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 6241, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 507, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 68, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 20, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 404, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-09-19T04:33:15.418 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 8, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 462, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 19, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 711, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 5832, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 1025, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 1653028209, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 6, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 5832, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-09-19T04:33:15.419 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 5832, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 45, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 179342, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 179381, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 182615, 2024-09-19T04:33:15.420 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 1419, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 66, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 768 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 20, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 321, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 175, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 292, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 56, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:33:15.421 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 162, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 144, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 23, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 14, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 12, 2024-09-19T04:33:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 4060, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 4060, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-09-19T04:33:15.423 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 7, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 7, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 10306343, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1442, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 3.616872674, 2024-09-19T04:33:15.424 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002508233 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 564, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 1164, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 600, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 87, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 203, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 116, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 652, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 1332, 2024-09-19T04:33:15.425 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 680, 2024-09-19T04:33:15.426 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 367, 2024-09-19T04:33:15.426 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 815, 2024-09-19T04:33:15.426 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 448, 2024-09-19T04:33:15.426 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 63384, 2024-09-19T04:33:15.426 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 223520 2024-09-19T04:33:15.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.426 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-09-19T04:33:15.426 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 5850, 2024-09-19T04:33:15.426 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 576, 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 3698, 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 53, 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 1448, 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 146, 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.325919784, 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002232327 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 70, 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.056552410, 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000807891 2024-09-19T04:33:15.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 23, 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.209659178, 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.009115616 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2157, 2024-09-19T04:33:15.428 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 5.713456783, 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002648797 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.429 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-09-19T04:33:15.430 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.430 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.430 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.430 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-09-19T04:33:15.430 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.430 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.430 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.431 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-09-19T04:33:15.431 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.431 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.431 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.431 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-09-19T04:33:15.431 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.432 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 573, 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.235159029, 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000410399 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 13, 2024-09-19T04:33:15.433 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.009621592, 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000740122 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 179, 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 31.517366590, 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.176074673 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.434 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.435 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.436 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.437 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.438 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-09-19T04:33:15.438 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.438 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.438 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.438 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-09-19T04:33:15.438 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 25, 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.083793085, 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003351723 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-09-19T04:33:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 4, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 2, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 609, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 304, 2024-09-19T04:33:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1061, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 662538, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 208, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 8408, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 185, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 2047138, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 26640, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-19T04:33:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 7885, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 6217048, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 28, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 7885, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7885, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 8.831770534, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001120072 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7885, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 8095 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 7885, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 5903, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 1982, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 0, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 10, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 68, 2024-09-19T04:33:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 1510, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 119, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 157, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 5832, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 20, 2024-09-19T04:33:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-09-19T04:33:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-09-19T04:33:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 105, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 26, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 12, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 12, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 294, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 5, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 80 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-09-19T04:33:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 10, 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-09-19T04:33:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 10, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 144, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 144, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 920667, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 183612880, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 920667, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 920667, 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 183612880, 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 7885, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 6231995, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7885, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 6231995, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 7885, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 7885, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7885, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 7885, 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 144, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 15024, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 144, 2024-09-19T04:33:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 68, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 15024, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x5617c1c28960": { 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 4060, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 6112039, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 4060, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1442, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 6112039, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:15.459 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-19T04:33:15.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:15 smithi112 ceph-mon[35051]: pgmap v572: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-19T04:33:15.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:15 smithi094 ceph-mon[30972]: pgmap v572: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-19T04:33:15.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:15 smithi104 ceph-mon[34145]: pgmap v572: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-19T04:33:16.536 INFO:tasks.check_counter:Daemon mds.c mds.exported=8 2024-09-19T04:33:16.536 INFO:tasks.check_counter:Daemon mds.c mds.imported=19 2024-09-19T04:33:16.536 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=20 2024-09-19T04:33:16.536 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=321 2024-09-19T04:33:16.536 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-19T04:33:16.536 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:33:16.536 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:33:16.536 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T04:33:16.536 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T04:33:16.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:16 smithi112 ceph-mon[35051]: pgmap v573: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-19T04:33:16.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:16 smithi112 ceph-mon[35051]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T04:33:16.785 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:33:17.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:16 smithi094 ceph-mon[30972]: pgmap v573: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-19T04:33:17.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:16 smithi094 ceph-mon[30972]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T04:33:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:16 smithi104 ceph-mon[34145]: pgmap v573: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-19T04:33:17.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:16 smithi104 ceph-mon[34145]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2392, 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2345, 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 32926354, 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 838628, 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 7, 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.225102339, 2024-09-19T04:33:17.302 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.062824843, 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.110018837, 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.053141948, 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2345, 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.080678728, 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000034404 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1667296, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 207712 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1294, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1542, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 38092375, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1101580, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 9, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.112280396, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.033958422, 2024-09-19T04:33:17.304 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.044103980, 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.026092535, 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1539, 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.049875979, 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000032408 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T04:33:17.305 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 690976 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 807, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 809, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 29569952, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 256633, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.074753571, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.018211418, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.032640461, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.019140914, 2024-09-19T04:33:17.306 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 807, 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.025327291, 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000031384 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:33:17.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 6872, 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.146398203, 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000021303 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-09-19T04:33:17.309 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 864, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 75, 2024-09-19T04:33:17.310 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 789, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 757, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 37, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-09-19T04:33:17.311 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 1022, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 1619473741, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 6, 2024-09-19T04:33:17.312 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:33:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-09-19T04:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:33:17.315 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-09-19T04:33:17.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 11333, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 28, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 101959297, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 101926964, 2024-09-19T04:33:17.317 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 11333 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 727, 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 3157, 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 2430, 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 95, 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 141, 2024-09-19T04:33:17.318 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 46, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 864, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 3416, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 2552, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 56876, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 215328 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-09-19T04:33:17.319 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-09-19T04:33:17.320 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.322 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.323 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.324 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.325 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.326 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-09-19T04:33:17.327 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.328 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.329 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.331 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 5, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 3, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 4, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 4936, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.332 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:17.333 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:17.334 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 232976, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 80, 2024-09-19T04:33:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 8408, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 185, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:17.336 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 2501036, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 6742, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 4082, 2024-09-19T04:33:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 26, 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 4, 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 4082, 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4082, 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 2.622931959, 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000642560 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4082, 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 4083 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.338 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 4082, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 4082, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 2365, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 1714, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-09-19T04:33:17.339 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-09-19T04:33:17.340 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-09-19T04:33:17.340 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-09-19T04:33:17.340 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-09-19T04:33:17.340 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 2, 2024-09-19T04:33:17.340 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-09-19T04:33:17.340 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:33:17.340 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-09-19T04:33:17.340 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:33:17.340 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-09-19T04:33:17.341 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 106, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 27, 2024-09-19T04:33:17.342 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 10, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 10, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-09-19T04:33:17.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.344 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 4489, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 100161086, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 4489, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 4489, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 100161086, 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:17.345 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 4082, 2024-09-19T04:33:17.346 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 97786453, 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 4082, 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 97786453, 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-09-19T04:33:17.347 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 4082, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 4082, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 4082, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 4082, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:17.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-09-19T04:33:17.349 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x55a0171aab40": { 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:17.350 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:17.351 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-19T04:33:18.449 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T04:33:18.449 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T04:33:18.450 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-19T04:33:18.450 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-19T04:33:18.450 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-19T04:33:18.450 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:33:18.450 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:33:18.450 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T04:33:18.450 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T04:33:18.709 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:33:18.960 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:18 smithi112 ceph-mon[35051]: pgmap v575: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-19T04:33:19.154 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:18 smithi104 ceph-mon[34145]: pgmap v575: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-19T04:33:19.154 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:18 smithi094 ceph-mon[30972]: pgmap v575: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-19T04:33:19.221 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 373917, 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 727102, 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 78541700, 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 213065580, 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 10, 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 28.783320839, 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 14.258122117, 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 16.319346606, 2024-09-19T04:33:19.222 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.136139873, 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 727100, 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 11.092802050, 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000015256 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1088424, 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.214755496, 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000197 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:33:19.223 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1307, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1615, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 357345, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1805878, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 59, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.114450693, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.043422244, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.035047478, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.025634554, 2024-09-19T04:33:19.224 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1612, 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.097801739, 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000060671 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 154, 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000021665, 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000140 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 5952, 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 627824 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.225 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 405858, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 591756, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 125898437, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 270952559, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 11, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 21.337898071, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 8.980973539, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 13.787989085, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.163209235, 2024-09-19T04:33:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 591756, 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 12.290007360, 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000020768 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 972310, 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.164511386, 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000169 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1687392, 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 232912 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 15434, 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 5.170605700, 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000335013 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 110, 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003036552, 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000027605 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 205630, 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 204470, 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-09-19T04:33:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 204470, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 195.254062739, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000954927 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 1160, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 10, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 45, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 4, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 760, 2024-09-19T04:33:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 37, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 408, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 315, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 707, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 1996, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 320, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 321, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 33, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 411916, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 408735, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 1146, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 61, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-09-19T04:33:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 14, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 399, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 42, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 1846, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 57, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 2257, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 3178, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 1025, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 1653028209, 2024-09-19T04:33:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 6, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 3178, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 27, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 27, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 3151, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 40, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 187675, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 179433, 2024-09-19T04:33:19.233 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 5, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 194906, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 10612, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 32, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 475 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 72, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 208, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 116, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 174, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 112, 2024-09-19T04:33:19.234 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 1, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 138, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 24, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 105, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 17, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:33:19.235 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 47, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 4, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 13261, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-09-19T04:33:19.236 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 13261, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 16, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 16, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 3, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 23334416, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-09-19T04:33:19.237 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 6455, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 18.430596792, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002855243 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 544, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 2358, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 1814, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 88, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 191, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 103, 2024-09-19T04:33:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 760, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 2779, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 2019, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 321, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 1606, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 1285, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 65560, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 223520 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 386226, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 539, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 205630, 2024-09-19T04:33:19.239 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 90, 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 1264, 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 88, 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.183027130, 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002079853 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 201690, 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 165.549039732, 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000820809 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-09-19T04:33:19.240 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.241 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.241 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.241 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-09-19T04:33:19.241 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 105, 2024-09-19T04:33:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.329748098, 2024-09-19T04:33:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003140458 2024-09-19T04:33:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-09-19T04:33:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1794, 2024-09-19T04:33:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 17.640924199, 2024-09-19T04:33:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.009833291 2024-09-19T04:33:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.245 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.246 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-09-19T04:33:19.246 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.246 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.246 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.246 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-09-19T04:33:19.246 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.246 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.247 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.247 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-09-19T04:33:19.247 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.247 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.247 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.247 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-09-19T04:33:19.247 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 516, 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.271850089, 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000526841 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 18, 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.014457142, 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000803174 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 139, 2024-09-19T04:33:19.249 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 10.959847021, 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.078847820 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.250 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.251 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5, 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.008141784, 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001628356 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.253 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 115, 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.297027544, 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002582848 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.254 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 4, 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 2, 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-09-19T04:33:19.255 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 24020, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 12010, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1067, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:19.256 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:19.257 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-19T04:33:19.258 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 662270, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 205, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 8288, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 182, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 2075812, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 28711, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:19.260 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 10112, 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 19592929, 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 28, 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 10112, 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 10112, 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 21.271450934, 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002103584 2024-09-19T04:33:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 10112, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 10466 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 10112, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 3210, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 6902, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 0, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 10, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 13, 2024-09-19T04:33:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 6468, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 82, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 1, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 2, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 44, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 3178, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 20, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:33:19.263 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 6, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-09-19T04:33:19.264 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 106, 2024-09-19T04:33:19.265 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 27, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 12, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 12, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 494, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 29, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 119 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-09-19T04:33:19.266 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 6, 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 27, 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 24, 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.267 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 781078, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 134472670, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 781078, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 781078, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 134472670, 2024-09-19T04:33:19.268 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:19.269 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 10112, 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 19594844, 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 10112, 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 19594844, 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-09-19T04:33:19.270 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 10112, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 10112, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 10112, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 10112, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:19.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.272 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:33:19.272 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:19.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-19T04:33:19.272 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:19.272 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 24, 2024-09-19T04:33:19.272 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 2480, 2024-09-19T04:33:19.272 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:19.272 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 24, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 13, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 2480, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x564e7b842960": { 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-19T04:33:19.273 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 13261, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 19140112, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 13261, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 6455, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 19140112, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:19.275 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:19.275 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:19.275 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-19T04:33:20.048 INFO:tasks.check_counter:Daemon mds.i mds.exported=42 2024-09-19T04:33:20.048 INFO:tasks.check_counter:Daemon mds.i mds.imported=57 2024-09-19T04:33:20.049 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=72 2024-09-19T04:33:20.049 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=208 2024-09-19T04:33:20.049 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T04:33:20.049 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:33:20.049 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:33:20.049 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T04:33:20.049 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T04:33:20.295 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:33:20.791 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-19T04:33:20.791 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:33:20.791 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 341, 2024-09-19T04:33:20.791 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 331, 2024-09-19T04:33:20.791 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1482240, 2024-09-19T04:33:20.791 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 204720, 2024-09-19T04:33:20.791 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-09-19T04:33:20.791 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-09-19T04:33:20.791 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.046783142, 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.017851701, 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.042116123, 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 330, 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.013065085, 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000039591 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:20.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1482240, 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 204720 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 3, 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 255, 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 480, 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 280848, 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-09-19T04:33:20.793 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.013392660, 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.009029888, 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001732044, 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 252, 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.022211920, 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000088142 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:20.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 280848 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 2992, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 448, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.002344623, 2024-09-19T04:33:20.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000035327, 2024-09-19T04:33:20.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002293782, 2024-09-19T04:33:20.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:33:20.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:33:20.796 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-09-19T04:33:20.796 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004103824, 2024-09-19T04:33:20.796 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001367941 2024-09-19T04:33:20.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:33:20.796 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:33:20.797 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:33:20.798 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:33:20.799 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148761, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 31, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-19T04:33:20.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-09-19T04:33:20.801 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 344, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1434082, 2024-09-19T04:33:20.802 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 344, 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 344, 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1434082, 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-19T04:33:20.803 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-19T04:33:21.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:20 smithi112 ceph-mon[35051]: pgmap v576: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-19T04:33:21.093 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:20 smithi094 ceph-mon[30972]: pgmap v576: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-19T04:33:21.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:20 smithi104 ceph-mon[34145]: pgmap v576: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-19T04:33:21.475 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-19T04:33:21.476 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-19T04:33:21.476 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-19T04:33:21.476 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-19T04:33:21.476 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-19T04:33:21.476 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-19T04:33:21.476 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-19T04:33:21.476 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-19T04:33:21.487 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-19T04:33:21.487 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:33:21.487 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:33:21.521 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:33:21.521 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:33:21.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T04:33:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:22.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.375+0000 7f2936b87640 1 -- 172.21.15.94:0/3024023347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930106a80 msgr2=0x7f2930106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.375+0000 7f2936b87640 1 --2- 172.21.15.94:0/3024023347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930106a80 0x7f2930106e60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2924009930 tx=0x7f292402a420 comp rx=0 tx=0).stop 2024-09-19T04:33:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.375+0000 7f2936b87640 1 -- 172.21.15.94:0/3024023347 shutdown_connections 2024-09-19T04:33:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.375+0000 7f2936b87640 1 --2- 172.21.15.94:0/3024023347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930101420 0x7f293010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.375+0000 7f2936b87640 1 --2- 172.21.15.94:0/3024023347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2930100a80 0x7f2930100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.375+0000 7f2936b87640 1 --2- 172.21.15.94:0/3024023347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930106a80 0x7f2930106e60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.375+0000 7f2936b87640 1 -- 172.21.15.94:0/3024023347 >> 172.21.15.94:0/3024023347 conn(0x7f29300fc7a0 msgr2=0x7f29300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.375+0000 7f2936b87640 1 -- 172.21.15.94:0/3024023347 shutdown_connections 2024-09-19T04:33:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.375+0000 7f2936b87640 1 -- 172.21.15.94:0/3024023347 wait complete. 2024-09-19T04:33:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2936b87640 1 Processor -- start 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2936b87640 1 -- start start 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2936b87640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930100a80 0x7f2930102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2936b87640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2930101420 0x7f2930102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2936b87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930106a80 0x7f29301030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2936b87640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2930110d30 con 0x7f2930100a80 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2936b87640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2930110bb0 con 0x7f2930101420 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2936b87640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2930110eb0 con 0x7f2930106a80 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2934923640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930100a80 0x7f2930102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2934923640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930100a80 0x7f2930102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44050/0 (socket says 172.21.15.94:44050) 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2934923640 1 -- 172.21.15.94:0/2509166710 learned_addr learned my addr 172.21.15.94:0/2509166710 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2935124640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930106a80 0x7f29301030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f292ffff640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2930101420 0x7f2930102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2934923640 1 -- 172.21.15.94:0/2509166710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930106a80 msgr2=0x7f29301030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2934923640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930106a80 0x7f29301030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2934923640 1 -- 172.21.15.94:0/2509166710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2930101420 msgr2=0x7f2930102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2934923640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2930101420 0x7f2930102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f2934923640 1 -- 172.21.15.94:0/2509166710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29301a6830 con 0x7f2930100a80 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.376+0000 7f292ffff640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2930101420 0x7f2930102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.377+0000 7f2934923640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930100a80 0x7f2930102650 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2924004770 tx=0x7f29240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.377+0000 7f292dffb640 1 -- 172.21.15.94:0/2509166710 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292403f070 con 0x7f2930100a80 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.377+0000 7f292dffb640 1 -- 172.21.15.94:0/2509166710 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f292403b460 con 0x7f2930100a80 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.377+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29301a6ac0 con 0x7f2930100a80 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.377+0000 7f292dffb640 1 -- 172.21.15.94:0/2509166710 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924035070 con 0x7f2930100a80 2024-09-19T04:33:22.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.377+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29301a6fa0 con 0x7f2930100a80 2024-09-19T04:33:22.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.378+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2930100ee0 con 0x7f2930100a80 2024-09-19T04:33:22.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.379+0000 7f292dffb640 1 -- 172.21.15.94:0/2509166710 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f292403b610 con 0x7f2930100a80 2024-09-19T04:33:22.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.380+0000 7f292dffb640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f29200a33b0 0x7f29200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.380+0000 7f292ffff640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f29200a33b0 0x7f29200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.380+0000 7f292dffb640 1 -- 172.21.15.94:0/2509166710 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8585+0+0 (secure 0 0 0) 0x7f29240f0100 con 0x7f2930100a80 2024-09-19T04:33:22.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.380+0000 7f292ffff640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f29200a33b0 0x7f29200a5870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29301040e0 tx=0x7f291c005e50 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:22.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.384+0000 7f292dffb640 1 -- 172.21.15.94:0/2509166710 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29240bcb40 con 0x7f2930100a80 2024-09-19T04:33:22.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.582+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f29301a71d0 con 0x7f2930100a80 2024-09-19T04:33:22.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.583+0000 7f292dffb640 1 -- 172.21.15.94:0/2509166710 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v106) ==== 81+0+1002 (secure 0 0 0) 0x7f29240bc440 con 0x7f2930100a80 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3284064221 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/860358556 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2308569937 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2229490078 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3663117113 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4021627817 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4018061819 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/3136401482 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/3136401482 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3831757952 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2204846817 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:22.585 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3871891152 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:22.586 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3123077951 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:22.586 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1265443753 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:22.586 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/516140053 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:22.586 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2229490078 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:22.586 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2308569937 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:22.586 INFO:teuthology.orchestra.run.smithi094.stderr:listed 17 entries 2024-09-19T04:33:22.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f29200a33b0 msgr2=0x7f29200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f29200a33b0 0x7f29200a5870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29301040e0 tx=0x7f291c005e50 comp rx=0 tx=0).stop 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930100a80 msgr2=0x7f2930102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930100a80 0x7f2930102650 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2924004770 tx=0x7f29240042f0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 shutdown_connections 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f29200a33b0 0x7f29200a5870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930106a80 0x7f29301030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2930101420 0x7f2930102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 --2- 172.21.15.94:0/2509166710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930100a80 0x7f2930102650 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 >> 172.21.15.94:0/2509166710 conn(0x7f29300fc7a0 msgr2=0x7f293006a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 shutdown_connections 2024-09-19T04:33:22.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:22.586+0000 7f2936b87640 1 -- 172.21.15.94:0/2509166710 wait complete. 2024-09-19T04:33:22.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:22 smithi094 ceph-mon[30972]: pgmap v577: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-19T04:33:22.840 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:22 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2509166710' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:33:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:22 smithi112 ceph-mon[35051]: pgmap v577: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-19T04:33:23.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:22 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2509166710' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:33:23.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:22 smithi104 ceph-mon[34145]: pgmap v577: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-19T04:33:23.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:22 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2509166710' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:33:23.353 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-19T04:33:23.353 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:33:23.370 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T04:33:23.660 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:24.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.150+0000 7f64f5e5f640 1 -- 172.21.15.94:0/1287906185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f01027a0 msgr2=0x7f64f01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.150+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/1287906185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f01027a0 0x7f64f01067f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009900 tx=0x7f64d802a410 comp rx=0 tx=0).stop 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.150+0000 7f64f5e5f640 1 -- 172.21.15.94:0/1287906185 shutdown_connections 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.150+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/1287906185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64f0106f20 0x7f64f010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.150+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/1287906185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f01027a0 0x7f64f01067f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.150+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/1287906185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f0101df0 0x7f64f01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.150+0000 7f64f5e5f640 1 -- 172.21.15.94:0/1287906185 >> 172.21.15.94:0/1287906185 conn(0x7f64f00fd8b0 msgr2=0x7f64f00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.150+0000 7f64f5e5f640 1 -- 172.21.15.94:0/1287906185 shutdown_connections 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.150+0000 7f64f5e5f640 1 -- 172.21.15.94:0/1287906185 wait complete. 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64f5e5f640 1 Processor -- start 2024-09-19T04:33:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64f5e5f640 1 -- start start 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64f5e5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f0101df0 0x7f64f0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64f5e5f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64f01027a0 0x7f64f0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64f5e5f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f0106f20 0x7f64f019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64f5e5f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0111e50 con 0x7f64f0101df0 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64f5e5f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0111cd0 con 0x7f64f0106f20 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64f5e5f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0111fd0 con 0x7f64f01027a0 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64ef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f0101df0 0x7f64f0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64ef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f0101df0 0x7f64f0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44070/0 (socket says 172.21.15.94:44070) 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.151+0000 7f64ef7fe640 1 -- 172.21.15.94:0/68489072 learned_addr learned my addr 172.21.15.94:0/68489072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64effff640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f0106f20 0x7f64f019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64ef7fe640 1 -- 172.21.15.94:0/68489072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64f01027a0 msgr2=0x7f64f0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64ef7fe640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64f01027a0 0x7f64f0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64ef7fe640 1 -- 172.21.15.94:0/68489072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f0106f20 msgr2=0x7f64f019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64ef7fe640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f0106f20 0x7f64f019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64ef7fe640 1 -- 172.21.15.94:0/68489072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f019d3c0 con 0x7f64f0101df0 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64effff640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f0106f20 0x7f64f019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64ef7fe640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f0101df0 0x7f64f0198410 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c920 tx=0x7f64e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64ecff9640 1 -- 172.21.15.94:0/68489072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0007c70 con 0x7f64f0101df0 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f019d6b0 con 0x7f64f0101df0 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64ecff9640 1 -- 172.21.15.94:0/68489072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64e0007e10 con 0x7f64f0101df0 2024-09-19T04:33:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64ecff9640 1 -- 172.21.15.94:0/68489072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0018820 con 0x7f64f0101df0 2024-09-19T04:33:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.152+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f01a4f90 con 0x7f64f0101df0 2024-09-19T04:33:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.153+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f01021d0 con 0x7f64f0101df0 2024-09-19T04:33:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.154+0000 7f64ecff9640 1 -- 172.21.15.94:0/68489072 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f64e00189c0 con 0x7f64f0101df0 2024-09-19T04:33:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.155+0000 7f64ecff9640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64dc0a33b0 0x7f64dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.155+0000 7f64eeffd640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64dc0a33b0 0x7f64dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.155+0000 7f64ecff9640 1 -- 172.21.15.94:0/68489072 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8585+0+0 (secure 0 0 0) 0x7f64e00cff80 con 0x7f64f0101df0 2024-09-19T04:33:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.155+0000 7f64eeffd640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64dc0a33b0 0x7f64dc0a5870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f64d80096c0 tx=0x7f64d8031400 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.159+0000 7f64ecff9640 1 -- 172.21.15.94:0/68489072 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f64e0014030 con 0x7f64f0101df0 2024-09-19T04:33:24.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.336+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f64f010a100 con 0x7f64f0101df0 2024-09-19T04:33:24.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.337+0000 7f64ecff9640 1 -- 172.21.15.94:0/68489072 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v106) ==== 81+0+1002 (secure 0 0 0) 0x7f64e009c920 con 0x7f64f0101df0 2024-09-19T04:33:24.339 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3284064221 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:24.339 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/860358556 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:24.339 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2308569937 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:24.339 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2229490078 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:24.339 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3663117113 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:24.339 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4021627817 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:24.339 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4018061819 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:24.339 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/3136401482 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:24.340 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/3136401482 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:24.340 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3831757952 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:24.340 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2204846817 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:24.340 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3871891152 2024-09-20T04:06:30.053518+0000 2024-09-19T04:33:24.340 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3123077951 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:24.340 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1265443753 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:24.340 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/516140053 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:24.340 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2229490078 2024-09-20T04:06:06.814655+0000 2024-09-19T04:33:24.340 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2308569937 2024-09-20T04:15:01.339086+0000 2024-09-19T04:33:24.341 INFO:teuthology.orchestra.run.smithi094.stderr:listed 17 entries 2024-09-19T04:33:24.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64dc0a33b0 msgr2=0x7f64dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64dc0a33b0 0x7f64dc0a5870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f64d80096c0 tx=0x7f64d8031400 comp rx=0 tx=0).stop 2024-09-19T04:33:24.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f0101df0 msgr2=0x7f64f0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f0101df0 0x7f64f0198410 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c920 tx=0x7f64e000cde0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 shutdown_connections 2024-09-19T04:33:24.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f64dc0a33b0 0x7f64dc0a5870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f0106f20 0x7f64f019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64f01027a0 0x7f64f0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 --2- 172.21.15.94:0/68489072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f0101df0 0x7f64f0198410 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 >> 172.21.15.94:0/68489072 conn(0x7f64f00fd8b0 msgr2=0x7f64f00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:24.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 shutdown_connections 2024-09-19T04:33:24.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:24.339+0000 7f64f5e5f640 1 -- 172.21.15.94:0/68489072 wait complete. 2024-09-19T04:33:24.942 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:24 smithi094 ceph-mon[30972]: pgmap v578: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T04:33:24.942 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:24 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/68489072' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:33:24.970 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-19T04:33:24.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:33:24.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /tmp/cephfs 2024-09-19T04:33:25.002 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[82549]: fuse finished with error 0 and tester_r 0 2024-09-19T04:33:25.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:24 smithi112 ceph-mon[35051]: pgmap v578: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T04:33:25.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:24 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/68489072' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:33:25.081 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:33:25.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:24 smithi104 ceph-mon[34145]: pgmap v578: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T04:33:25.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:24 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/68489072' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:33:27.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:26 smithi112 ceph-mon[35051]: pgmap v579: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T04:33:27.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:26 smithi094 ceph-mon[30972]: pgmap v579: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T04:33:27.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:26 smithi104 ceph-mon[34145]: pgmap v579: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T04:33:29.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:28 smithi112 ceph-mon[35051]: pgmap v580: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-09-19T04:33:29.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:28 smithi094 ceph-mon[30972]: pgmap v580: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-09-19T04:33:29.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:28 smithi104 ceph-mon[34145]: pgmap v580: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-09-19T04:33:31.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:30 smithi112 ceph-mon[35051]: pgmap v581: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-19T04:33:31.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:30 smithi094 ceph-mon[30972]: pgmap v581: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-19T04:33:31.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:30 smithi104 ceph-mon[34145]: pgmap v581: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-19T04:33:31.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T04:33:31.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:33:31.183 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-19T04:33:31.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:33:31.220 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-19T04:33:31.297 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-19T04:33:31.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:33:31.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-19T04:33:31.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:33:31.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-19T04:33:31.465 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T04:33:31.476 INFO:tasks.cephadm:Teardown begin 2024-09-19T04:33:31.477 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:33:31.477 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:33:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.234+0000 7f015cd6f640 1 -- 172.21.15.94:0/2700554001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 msgr2=0x7f015810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.234+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2700554001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 0x7f015810da40 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0150009ca0 tx=0x7f015002a950 comp rx=0 tx=0).stop 2024-09-19T04:33:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.234+0000 7f015cd6f640 1 -- 172.21.15.94:0/2700554001 shutdown_connections 2024-09-19T04:33:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.234+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2700554001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 0x7f015810da40 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.234+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2700554001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158102570 0x7f01581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.234+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2700554001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0158100e20 0x7f0158101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.234+0000 7f015cd6f640 1 -- 172.21.15.94:0/2700554001 >> 172.21.15.94:0/2700554001 conn(0x7f01580fc7d0 msgr2=0x7f01580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.234+0000 7f015cd6f640 1 -- 172.21.15.94:0/2700554001 shutdown_connections 2024-09-19T04:33:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.234+0000 7f015cd6f640 1 -- 172.21.15.94:0/2700554001 wait complete. 2024-09-19T04:33:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f015cd6f640 1 Processor -- start 2024-09-19T04:33:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f015cd6f640 1 -- start start 2024-09-19T04:33:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f015cd6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0158100e20 0x7f0158197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f015cd6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158102570 0x7f0158197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f015cd6f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 0x7f0158197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f015cd6f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0158110e30 con 0x7f0158102ef0 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f015cd6f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0158110cb0 con 0x7f0158102570 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f015cd6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0158110fb0 con 0x7f0158100e20 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f0156d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 0x7f0158197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f0156d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 0x7f0158197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43036/0 (socket says 172.21.15.94:43036) 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f0156d9d640 1 -- 172.21.15.94:0/2607534068 learned_addr learned my addr 172.21.15.94:0/2607534068 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f0155d9b640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158102570 0x7f0158197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f015659c640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0158100e20 0x7f0158197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f0156d9d640 1 -- 172.21.15.94:0/2607534068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0158100e20 msgr2=0x7f0158197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f0156d9d640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0158100e20 0x7f0158197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f0156d9d640 1 -- 172.21.15.94:0/2607534068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158102570 msgr2=0x7f0158197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f0156d9d640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158102570 0x7f0158197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.235+0000 7f0156d9d640 1 -- 172.21.15.94:0/2607534068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01581a68c0 con 0x7f0158102ef0 2024-09-19T04:33:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.236+0000 7f0156d9d640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 0x7f0158197ad0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f015002ae70 tx=0x7f015002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.236+0000 7f014f7fe640 1 -- 172.21.15.94:0/2607534068 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150034040 con 0x7f0158102ef0 2024-09-19T04:33:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.236+0000 7f014f7fe640 1 -- 172.21.15.94:0/2607534068 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01500308f0 con 0x7f0158102ef0 2024-09-19T04:33:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.236+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01581a6b50 con 0x7f0158102ef0 2024-09-19T04:33:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.236+0000 7f014f7fe640 1 -- 172.21.15.94:0/2607534068 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150004780 con 0x7f0158102ef0 2024-09-19T04:33:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.236+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01581a7030 con 0x7f0158102ef0 2024-09-19T04:33:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.237+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0158101200 con 0x7f0158102ef0 2024-09-19T04:33:32.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.238+0000 7f014f7fe640 1 -- 172.21.15.94:0/2607534068 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0150037020 con 0x7f0158102ef0 2024-09-19T04:33:32.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.239+0000 7f014f7fe640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f01440a33b0 0x7f01440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.239+0000 7f015659c640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f01440a33b0 0x7f01440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.239+0000 7f014f7fe640 1 -- 172.21.15.94:0/2607534068 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8585+0+0 (secure 0 0 0) 0x7f01500f0cf0 con 0x7f0158102ef0 2024-09-19T04:33:32.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.239+0000 7f015659c640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f01440a33b0 0x7f01440a5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f014000bbe0 tx=0x7f014000b390 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:32.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.242+0000 7f014f7fe640 1 -- 172.21.15.94:0/2607534068 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f01500bd7a0 con 0x7f0158102ef0 2024-09-19T04:33:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.444+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f015806a560 con 0x7f0158102ef0 2024-09-19T04:33:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.445+0000 7f014f7fe640 1 -- 172.21.15.94:0/2607534068 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11924 (secure 0 0 0) 0x7f015806a560 con 0x7f0158102ef0 2024-09-19T04:33:32.447 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:33:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":58,"btime":"2024-09-19T04:32:53:314324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:32:52.773119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:33:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 58 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f01440a33b0 msgr2=0x7f01440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f01440a33b0 0x7f01440a5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f014000bbe0 tx=0x7f014000b390 comp rx=0 tx=0).stop 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 msgr2=0x7f0158197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 0x7f0158197ad0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f015002ae70 tx=0x7f015002f5d0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 shutdown_connections 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f01440a33b0 0x7f01440a5870 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0158102ef0 0x7f0158197ad0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158102570 0x7f0158197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 --2- 172.21.15.94:0/2607534068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0158100e20 0x7f0158197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 >> 172.21.15.94:0/2607534068 conn(0x7f01580fc7d0 msgr2=0x7f01580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 shutdown_connections 2024-09-19T04:33:32.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:32.447+0000 7f015cd6f640 1 -- 172.21.15.94:0/2607534068 wait complete. 2024-09-19T04:33:33.008 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:32 smithi094 ceph-mon[30972]: pgmap v582: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-19T04:33:33.008 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:32 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2607534068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:33:33.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:32 smithi112 ceph-mon[35051]: pgmap v582: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-19T04:33:33.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:32 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2607534068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:33:33.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:32 smithi104 ceph-mon[34145]: pgmap v582: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-19T04:33:33.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:32 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2607534068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:33:33.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:33:33.711 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:34.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:33 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:33:34.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:33 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:33:34.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:33 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:33:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.209+0000 7f225cfce640 1 -- 172.21.15.94:0/1651668973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258100a80 msgr2=0x7f2258100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.209+0000 7f225cfce640 1 --2- 172.21.15.94:0/1651668973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258100a80 0x7f2258100ee0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2240009930 tx=0x7f224002a420 comp rx=0 tx=0).stop 2024-09-19T04:33:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.209+0000 7f225cfce640 1 -- 172.21.15.94:0/1651668973 shutdown_connections 2024-09-19T04:33:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.209+0000 7f225cfce640 1 --2- 172.21.15.94:0/1651668973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2258101420 0x7f225810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.209+0000 7f225cfce640 1 --2- 172.21.15.94:0/1651668973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258100a80 0x7f2258100ee0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.209+0000 7f225cfce640 1 --2- 172.21.15.94:0/1651668973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2258106a80 0x7f2258106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.209+0000 7f225cfce640 1 -- 172.21.15.94:0/1651668973 >> 172.21.15.94:0/1651668973 conn(0x7f22580fc7a0 msgr2=0x7f22580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.209+0000 7f225cfce640 1 -- 172.21.15.94:0/1651668973 shutdown_connections 2024-09-19T04:33:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.209+0000 7f225cfce640 1 -- 172.21.15.94:0/1651668973 wait complete. 2024-09-19T04:33:34.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f225cfce640 1 Processor -- start 2024-09-19T04:33:34.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f225cfce640 1 -- start start 2024-09-19T04:33:34.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f225cfce640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2258100a80 0x7f22581983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f225cfce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2258101420 0x7f2258198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f225cfce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258106a80 0x7f225819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f225cfce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2258110cf0 con 0x7f2258106a80 2024-09-19T04:33:34.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f225cfce640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2258110b70 con 0x7f2258100a80 2024-09-19T04:33:34.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f225cfce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2258110e70 con 0x7f2258101420 2024-09-19T04:33:34.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f2256d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258106a80 0x7f225819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f2256d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258106a80 0x7f225819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43044/0 (socket says 172.21.15.94:43044) 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f2256d9d640 1 -- 172.21.15.94:0/411342113 learned_addr learned my addr 172.21.15.94:0/411342113 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f224ffff640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2258101420 0x7f2258198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.210+0000 7f225659c640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2258100a80 0x7f22581983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f2256d9d640 1 -- 172.21.15.94:0/411342113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2258101420 msgr2=0x7f2258198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f2256d9d640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2258101420 0x7f2258198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f2256d9d640 1 -- 172.21.15.94:0/411342113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2258100a80 msgr2=0x7f22581983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f2256d9d640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2258100a80 0x7f22581983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f2256d9d640 1 -- 172.21.15.94:0/411342113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225819d370 con 0x7f2258106a80 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f2256d9d640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258106a80 0x7f225819cc90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f225000eaa0 tx=0x7f225000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f224f7fe640 1 -- 172.21.15.94:0/411342113 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225000cdb0 con 0x7f2258106a80 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225819d660 con 0x7f2258106a80 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f224f7fe640 1 -- 172.21.15.94:0/411342113 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2250005020 con 0x7f2258106a80 2024-09-19T04:33:34.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f224f7fe640 1 -- 172.21.15.94:0/411342113 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500116d0 con 0x7f2258106a80 2024-09-19T04:33:34.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.211+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22581a4f40 con 0x7f2258106a80 2024-09-19T04:33:34.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.212+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2258100ee0 con 0x7f2258106a80 2024-09-19T04:33:34.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.213+0000 7f224f7fe640 1 -- 172.21.15.94:0/411342113 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f22500051d0 con 0x7f2258106a80 2024-09-19T04:33:34.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.214+0000 7f224f7fe640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22440a33b0 0x7f22440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.214+0000 7f225659c640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22440a33b0 0x7f22440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.214+0000 7f224f7fe640 1 -- 172.21.15.94:0/411342113 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8585+0+0 (secure 0 0 0) 0x7f22500cfeb0 con 0x7f2258106a80 2024-09-19T04:33:34.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.214+0000 7f225659c640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22440a33b0 0x7f22440a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22480058c0 tx=0x7f224800a2d0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:34.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.218+0000 7f224f7fe640 1 -- 172.21.15.94:0/411342113 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2250016030 con 0x7f2258106a80 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.417+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2258109a90 con 0x7f2258106a80 2024-09-19T04:33:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.417+0000 7f224f7fe640 1 -- 172.21.15.94:0/411342113 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11924 (secure 0 0 0) 0x7f225009c200 con 0x7f2258106a80 2024-09-19T04:33:34.419 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:33:34.420 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 58 2024-09-19T04:33:34.420 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":58,"btime":"2024-09-19T04:32:53:314324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/422186833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":422186833},{"type":"v1","addr":"172.21.15.94:6835","nonce":422186833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3764912957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3764912957},{"type":"v1","addr":"172.21.15.94:6837","nonce":3764912957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/4157648434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":4157648434},{"type":"v1","addr":"172.21.15.94:6839","nonce":4157648434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/44219912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":44219912},{"type":"v1","addr":"172.21.15.94:6841","nonce":44219912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1061074349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1061074349},{"type":"v1","addr":"172.21.15.104:6833","nonce":1061074349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1795597710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1795597710},{"type":"v1","addr":"172.21.15.112:6833","nonce":1795597710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:13:17.025380+0000","modified":"2024-09-19T04:32:52.773119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24466,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/2064839900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2064839900},{"type":"v1","addr":"172.21.15.104:6835","nonce":2064839900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2682504501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2682504501},{"type":"v1","addr":"172.21.15.104:6837","nonce":2682504501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/1098195473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1098195473},{"type":"v1","addr":"172.21.15.112:6835","nonce":1098195473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/134813546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":134813546},{"type":"v1","addr":"172.21.15.112:6837","nonce":134813546}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6839/3588924475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3588924475},{"type":"v1","addr":"172.21.15.112:6839","nonce":3588924475}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2749200235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2749200235},{"type":"v1","addr":"172.21.15.104:6839","nonce":2749200235}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24454,24467,24466]},"id":1}]} 2024-09-19T04:33:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22440a33b0 msgr2=0x7f22440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22440a33b0 0x7f22440a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22480058c0 tx=0x7f224800a2d0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258106a80 msgr2=0x7f225819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258106a80 0x7f225819cc90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f225000eaa0 tx=0x7f225000ef60 comp rx=0 tx=0).stop 2024-09-19T04:33:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 shutdown_connections 2024-09-19T04:33:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f22440a33b0 0x7f22440a5870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2258106a80 0x7f225819cc90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2258101420 0x7f2258198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 --2- 172.21.15.94:0/411342113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2258100a80 0x7f22581983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 >> 172.21.15.94:0/411342113 conn(0x7f22580fc7a0 msgr2=0x7f225806aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 shutdown_connections 2024-09-19T04:33:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:34.420+0000 7f225cfce640 1 -- 172.21.15.94:0/411342113 wait complete. 2024-09-19T04:33:35.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:33:35.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:34 smithi112 ceph-mon[35051]: pgmap v583: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-19T04:33:35.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:34 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:33:35.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:34 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:33:35.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:34 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:33:35.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:34 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/411342113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:33:35.046 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:34 smithi094 ceph-mon[30972]: pgmap v583: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-19T04:33:35.046 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:34 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:33:35.046 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:34 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:33:35.046 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:34 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:33:35.046 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:34 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/411342113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:33:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:34 smithi104 ceph-mon[34145]: pgmap v583: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-19T04:33:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:34 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:33:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:34 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:33:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:34 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' 2024-09-19T04:33:35.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:34 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/411342113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:33:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:35.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.722+0000 7febeac62640 1 -- 172.21.15.94:0/61334157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 msgr2=0x7febe410c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:35.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.722+0000 7febeac62640 1 --2- 172.21.15.94:0/61334157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 0x7febe410c9f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7febd0009900 tx=0x7febd002a410 comp rx=0 tx=0).stop 2024-09-19T04:33:35.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.722+0000 7febeac62640 1 -- 172.21.15.94:0/61334157 shutdown_connections 2024-09-19T04:33:35.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.722+0000 7febeac62640 1 --2- 172.21.15.94:0/61334157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febe410cf30 0x7febe4119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:35.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.722+0000 7febeac62640 1 --2- 172.21.15.94:0/61334157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 0x7febe410c9f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:35.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.722+0000 7febeac62640 1 --2- 172.21.15.94:0/61334157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe4112640 0x7febe4112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:35.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.722+0000 7febeac62640 1 -- 172.21.15.94:0/61334157 >> 172.21.15.94:0/61334157 conn(0x7febe406cdc0 msgr2=0x7febe406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:35.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 -- 172.21.15.94:0/61334157 shutdown_connections 2024-09-19T04:33:35.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 -- 172.21.15.94:0/61334157 wait complete. 2024-09-19T04:33:35.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 Processor -- start 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 -- start start 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 0x7febe41a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febe410cf30 0x7febe41a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe4112640 0x7febe41a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febe411c850 con 0x7febe410c590 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7febe411c6d0 con 0x7febe4112640 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febeac62640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febe411c9d0 con 0x7febe410cf30 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febe89fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 0x7febe41a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febe89fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 0x7febe41a8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43062/0 (socket says 172.21.15.94:43062) 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.723+0000 7febe89fe640 1 -- 172.21.15.94:0/3663479193 learned_addr learned my addr 172.21.15.94:0/3663479193 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe91ff640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe4112640 0x7febe41a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe3fff640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febe410cf30 0x7febe41a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe89fe640 1 -- 172.21.15.94:0/3663479193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febe410cf30 msgr2=0x7febe41a8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe89fe640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febe410cf30 0x7febe41a8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe89fe640 1 -- 172.21.15.94:0/3663479193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe4112640 msgr2=0x7febe41a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe89fe640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe4112640 0x7febe41a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe89fe640 1 -- 172.21.15.94:0/3663479193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe41a3390 con 0x7febe410c590 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe89fe640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 0x7febe41a8a50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7febd800d950 tx=0x7febd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe1ffb640 1 -- 172.21.15.94:0/3663479193 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8017070 con 0x7febe410c590 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe1ffb640 1 -- 172.21.15.94:0/3663479193 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febd8009c30 con 0x7febe410c590 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe41a3680 con 0x7febe410c590 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febe1ffb640 1 -- 172.21.15.94:0/3663479193 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd800f620 con 0x7febe410c590 2024-09-19T04:33:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.724+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe41af9d0 con 0x7febe410c590 2024-09-19T04:33:35.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.725+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe41a39f0 con 0x7febe410c590 2024-09-19T04:33:35.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.726+0000 7febe1ffb640 1 -- 172.21.15.94:0/3663479193 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7febd80109a0 con 0x7febe410c590 2024-09-19T04:33:35.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.727+0000 7febe1ffb640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7febd40a33b0 0x7febd40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:35.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.727+0000 7febe1ffb640 1 -- 172.21.15.94:0/3663479193 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8585+0+0 (secure 0 0 0) 0x7febd80cfa60 con 0x7febe410c590 2024-09-19T04:33:35.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.727+0000 7febe3fff640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7febd40a33b0 0x7febd40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:35.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.728+0000 7febe3fff640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7febd40a33b0 0x7febd40a5870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febe41a4300 tx=0x7febd0005a20 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.731+0000 7febe1ffb640 1 -- 172.21.15.94:0/3663479193 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7febd8009de0 con 0x7febe410c590 2024-09-19T04:33:35.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.899+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febe4114a60 con 0x7febe410c590 2024-09-19T04:33:35.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.901+0000 7febe1ffb640 1 -- 172.21.15.94:0/3663479193 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+23375 (secure 0 0 0) 0x7febd809c400 con 0x7febe410c590 2024-09-19T04:33:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-19T04:33:35.903 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":106,"fsid":"503c4526-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:05:34.057583+0000","modified":"2024-09-19T04:33:15.743563+0000","last_up_change":"2024-09-19T04:12:26.996096+0000","last_in_change":"2024-09-19T04:12:03.838508+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:08:46.366796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:12.965501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:15.212761+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4c09ef65-3d0d-4a11-99b4-b3d3b4c63fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6803","nonce":2563797567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6805","nonce":2563797567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6809","nonce":2563797567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2563797567},{"type":"v1","addr":"172.21.15.94:6807","nonce":2563797567}]},"public_addr":"172.21.15.94:6803/2563797567","cluster_addr":"172.21.15.94:6805/2563797567","heartbeat_back_addr":"172.21.15.94:6809/2563797567","heartbeat_front_addr":"172.21.15.94:6807/2563797567","state":["exists","up"]},{"osd":1,"uuid":"cb14f7e1-6d75-4f55-87f6-6cfe66ab6f18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6801","nonce":1485019338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6803","nonce":1485019338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6807","nonce":1485019338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1485019338},{"type":"v1","addr":"172.21.15.104:6805","nonce":1485019338}]},"public_addr":"172.21.15.104:6801/1485019338","cluster_addr":"172.21.15.104:6803/1485019338","heartbeat_back_addr":"172.21.15.104:6807/1485019338","heartbeat_front_addr":"172.21.15.104:6805/1485019338","state":["exists","up"]},{"osd":2,"uuid":"92ebda5b-3f73-46b3-b026-a9d41639aa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6801","nonce":1893936170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6803","nonce":1893936170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6807","nonce":1893936170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1893936170},{"type":"v1","addr":"172.21.15.112:6805","nonce":1893936170}]},"public_addr":"172.21.15.112:6801/1893936170","cluster_addr":"172.21.15.112:6803/1893936170","heartbeat_back_addr":"172.21.15.112:6807/1893936170","heartbeat_front_addr":"172.21.15.112:6805/1893936170","state":["exists","up"]},{"osd":3,"uuid":"dcb93a96-92d9-4cf2-a2f0-f547d49835cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6811","nonce":1707310526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6813","nonce":1707310526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6817","nonce":1707310526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1707310526},{"type":"v1","addr":"172.21.15.94:6815","nonce":1707310526}]},"public_addr":"172.21.15.94:6811/1707310526","cluster_addr":"172.21.15.94:6813/1707310526","heartbeat_back_addr":"172.21.15.94:6817/1707310526","heartbeat_front_addr":"172.21.15.94:6815/1707310526","state":["exists","up"]},{"osd":4,"uuid":"0edd43c9-dec2-4df0-b10e-cf68a6e809b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6809","nonce":571438521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6811","nonce":571438521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6815","nonce":571438521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":571438521},{"type":"v1","addr":"172.21.15.104:6813","nonce":571438521}]},"public_addr":"172.21.15.104:6809/571438521","cluster_addr":"172.21.15.104:6811/571438521","heartbeat_back_addr":"172.21.15.104:6815/571438521","heartbeat_front_addr":"172.21.15.104:6813/571438521","state":["exists","up"]},{"osd":5,"uuid":"abe550dd-07b4-46e6-a55e-b65a9b15c373","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6809","nonce":3131983830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6811","nonce":3131983830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6815","nonce":3131983830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3131983830},{"type":"v1","addr":"172.21.15.112:6813","nonce":3131983830}]},"public_addr":"172.21.15.112:6809/3131983830","cluster_addr":"172.21.15.112:6811/3131983830","heartbeat_back_addr":"172.21.15.112:6815/3131983830","heartbeat_front_addr":"172.21.15.112:6813/3131983830","state":["exists","up"]},{"osd":6,"uuid":"26a90049-fa63-414c-a4e2-6a779aa15473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6819","nonce":1178759966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6821","nonce":1178759966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6825","nonce":1178759966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1178759966},{"type":"v1","addr":"172.21.15.94:6823","nonce":1178759966}]},"public_addr":"172.21.15.94:6819/1178759966","cluster_addr":"172.21.15.94:6821/1178759966","heartbeat_back_addr":"172.21.15.94:6825/1178759966","heartbeat_front_addr":"172.21.15.94:6823/1178759966","state":["exists","up"]},{"osd":7,"uuid":"699d883d-3aa9-4430-80ef-5a3de8cd773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6817","nonce":2625303953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6819","nonce":2625303953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6823","nonce":2625303953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2625303953},{"type":"v1","addr":"172.21.15.104:6821","nonce":2625303953}]},"public_addr":"172.21.15.104:6817/2625303953","cluster_addr":"172.21.15.104:6819/2625303953","heartbeat_back_addr":"172.21.15.104:6823/2625303953","heartbeat_front_addr":"172.21.15.104:6821/2625303953","state":["exists","up"]},{"osd":8,"uuid":"a4ed7d18-c753-4ddc-b1cc-518b5c67923d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6817","nonce":3390158844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6819","nonce":3390158844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6823","nonce":3390158844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3390158844},{"type":"v1","addr":"172.21.15.112:6821","nonce":3390158844}]},"public_addr":"172.21.15.112:6817/3390158844","cluster_addr":"172.21.15.112:6819/3390158844","heartbeat_back_addr":"172.21.15.112:6823/3390158844","heartbeat_front_addr":"172.21.15.112:6821/3390158844","state":["exists","up"]},{"osd":9,"uuid":"d4dd28b1-8e3e-4871-9391-4d74b4c49b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6827","nonce":704568533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6829","nonce":704568533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6833","nonce":704568533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":704568533},{"type":"v1","addr":"172.21.15.94:6831","nonce":704568533}]},"public_addr":"172.21.15.94:6827/704568533","cluster_addr":"172.21.15.94:6829/704568533","heartbeat_back_addr":"172.21.15.94:6833/704568533","heartbeat_front_addr":"172.21.15.94:6831/704568533","state":["exists","up"]},{"osd":10,"uuid":"ab8a22e7-d51e-499d-8154-301d97f3790f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6825","nonce":1626292599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6827","nonce":1626292599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6831","nonce":1626292599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1626292599},{"type":"v1","addr":"172.21.15.104:6829","nonce":1626292599}]},"public_addr":"172.21.15.104:6825/1626292599","cluster_addr":"172.21.15.104:6827/1626292599","heartbeat_back_addr":"172.21.15.104:6831/1626292599","heartbeat_front_addr":"172.21.15.104:6829/1626292599","state":["exists","up"]},{"osd":11,"uuid":"47a94e97-ed76-4aa0-918c-a813e88ca539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6825","nonce":3391428889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6827","nonce":3391428889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6831","nonce":3391428889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3391428889},{"type":"v1","addr":"172.21.15.112:6829","nonce":3391428889}]},"public_addr":"172.21.15.112:6825/3391428889","cluster_addr":"172.21.15.112:6827/3391428889","heartbeat_back_addr":"172.21.15.112:6831/3391428889","heartbeat_front_addr":"172.21.15.112:6829/3391428889","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:07:51.419138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:17.300529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:08:42.429599+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:07.452464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:32.806181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:09:57.169871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:21.377595+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:10:46.587548+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:10.894082+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:11:36.092713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:00.904130+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.110725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3284064221":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/860358556":"2024-09-20T04:15:01.339086+0000","172.21.15.94:6801/2308569937":"2024-09-20T04:15:01.339086+0000","172.21.15.94:6801/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/3663117113":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/4021627817":"2024-09-20T04:06:06.814655+0000","172.21.15.94:0/4018061819":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:6801/3136401482":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3831757952":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/2204846817":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3871891152":"2024-09-20T04:06:30.053518+0000","172.21.15.94:0/3123077951":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/1265443753":"2024-09-20T04:15:01.339086+0000","172.21.15.94:0/516140053":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2229490078":"2024-09-20T04:06:06.814655+0000","172.21.15.94:6800/2308569937":"2024-09-20T04:15:01.339086+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:33:35.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7febd40a33b0 msgr2=0x7febd40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7febd40a33b0 0x7febd40a5870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febe41a4300 tx=0x7febd0005a20 comp rx=0 tx=0).stop 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 msgr2=0x7febe41a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 0x7febe41a8a50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7febd800d950 tx=0x7febd800de10 comp rx=0 tx=0).stop 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 shutdown_connections 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7febd40a33b0 0x7febd40a5870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe4112640 0x7febe41a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febe410cf30 0x7febe41a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 --2- 172.21.15.94:0/3663479193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe410c590 0x7febe41a8a50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 >> 172.21.15.94:0/3663479193 conn(0x7febe406cdc0 msgr2=0x7febe410afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 shutdown_connections 2024-09-19T04:33:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:35.903+0000 7febeac62640 1 -- 172.21.15.94:0/3663479193 wait complete. 2024-09-19T04:33:36.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T04:33:36.772 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:36.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:36 smithi104 ceph-mon[34145]: pgmap v584: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-19T04:33:36.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:36 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3663479193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:33:37.031 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:36 smithi094 ceph-mon[30972]: pgmap v584: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-19T04:33:37.032 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:36 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3663479193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:33:37.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:36 smithi112 ceph-mon[35051]: pgmap v584: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-19T04:33:37.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:36 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3663479193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:33:37.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.232+0000 7fed7d8f8640 1 -- 172.21.15.94:0/2270582273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78100a70 msgr2=0x7fed78100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.232+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/2270582273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78100a70 0x7fed78100ed0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fed60009930 tx=0x7fed6002a420 comp rx=0 tx=0).stop 2024-09-19T04:33:37.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.232+0000 7fed7d8f8640 1 -- 172.21.15.94:0/2270582273 shutdown_connections 2024-09-19T04:33:37.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.232+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/2270582273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed78101410 0x7fed7810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.232+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/2270582273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78100a70 0x7fed78100ed0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.232+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/2270582273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed78106a70 0x7fed78106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 -- 172.21.15.94:0/2270582273 >> 172.21.15.94:0/2270582273 conn(0x7fed780fc790 msgr2=0x7fed780febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 -- 172.21.15.94:0/2270582273 shutdown_connections 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 -- 172.21.15.94:0/2270582273 wait complete. 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 Processor -- start 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 -- start start 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed78100a70 0x7fed781972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed78101410 0x7fed78197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78106a70 0x7fed7819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed78110d20 con 0x7fed78106a70 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed78110ba0 con 0x7fed78100a70 2024-09-19T04:33:37.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.233+0000 7fed7d8f8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed78110ea0 con 0x7fed78101410 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78106a70 0x7fed7819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78106a70 0x7fed7819bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43082/0 (socket says 172.21.15.94:43082) 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed777fe640 1 -- 172.21.15.94:0/1122299302 learned_addr learned my addr 172.21.15.94:0/1122299302 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed6e7fc640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed78101410 0x7fed78197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed76ffd640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed78100a70 0x7fed781972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed777fe640 1 -- 172.21.15.94:0/1122299302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed78101410 msgr2=0x7fed78197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed777fe640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed78101410 0x7fed78197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed777fe640 1 -- 172.21.15.94:0/1122299302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed78100a70 msgr2=0x7fed781972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed777fe640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed78100a70 0x7fed781972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed777fe640 1 -- 172.21.15.94:0/1122299302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7819c290 con 0x7fed78106a70 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed777fe640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78106a70 0x7fed7819bbb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fed7000eaa0 tx=0x7fed7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed7c8f6640 1 -- 172.21.15.94:0/1122299302 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7000cdb0 con 0x7fed78106a70 2024-09-19T04:33:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7819c580 con 0x7fed78106a70 2024-09-19T04:33:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed7c8f6640 1 -- 172.21.15.94:0/1122299302 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed70005020 con 0x7fed78106a70 2024-09-19T04:33:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed7c8f6640 1 -- 172.21.15.94:0/1122299302 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700116d0 con 0x7fed78106a70 2024-09-19T04:33:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.234+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed781a3e60 con 0x7fed78106a70 2024-09-19T04:33:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.236+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed78100ed0 con 0x7fed78106a70 2024-09-19T04:33:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.237+0000 7fed7c8f6640 1 -- 172.21.15.94:0/1122299302 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fed700051d0 con 0x7fed78106a70 2024-09-19T04:33:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.237+0000 7fed7c8f6640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed640a33b0 0x7fed640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.238+0000 7fed76ffd640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed640a33b0 0x7fed640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.238+0000 7fed7c8f6640 1 -- 172.21.15.94:0/1122299302 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8585+0+0 (secure 0 0 0) 0x7fed700cfec0 con 0x7fed78106a70 2024-09-19T04:33:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.238+0000 7fed76ffd640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed640a33b0 0x7fed640a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fed68005d00 tx=0x7fed6800b410 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.242+0000 7fed7c8f6640 1 -- 172.21.15.94:0/1122299302 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fed70016030 con 0x7fed78106a70 2024-09-19T04:33:37.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.444+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fed78108a70 con 0x7fed78106a70 2024-09-19T04:33:37.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.446+0000 7fed7c8f6640 1 -- 172.21.15.94:0/1122299302 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7fed7009c1b0 con 0x7fed78106a70 2024-09-19T04:33:37.447 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed640a33b0 msgr2=0x7fed640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed640a33b0 0x7fed640a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fed68005d00 tx=0x7fed6800b410 comp rx=0 tx=0).stop 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78106a70 msgr2=0x7fed7819bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78106a70 0x7fed7819bbb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fed7000eaa0 tx=0x7fed7000ef60 comp rx=0 tx=0).stop 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 shutdown_connections 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed640a33b0 0x7fed640a5870 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed78106a70 0x7fed7819bbb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed78101410 0x7fed78197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 --2- 172.21.15.94:0/1122299302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed78100a70 0x7fed781972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 >> 172.21.15.94:0/1122299302 conn(0x7fed780fc790 msgr2=0x7fed780fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 shutdown_connections 2024-09-19T04:33:37.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:37.449+0000 7fed7d8f8640 1 -- 172.21.15.94:0/1122299302 wait complete. 2024-09-19T04:33:37.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:37 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a[30967]: 2024-09-19T04:33:37.445+0000 7f427bd7b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:33:37.783 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e[63352]: ignoring --setuser ceph since I am not root 2024-09-19T04:33:37.783 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e[63352]: ignoring --setgroup ceph since I am not root 2024-09-19T04:33:37.783 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-mds[63356]: -- 172.21.15.104:0/2438691254 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2301b5860 con 0x55b230193000 2024-09-19T04:33:37.783 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e[63352]: starting mds.e at 2024-09-19T04:33:37.784 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b[63958]: ignoring --setuser ceph since I am not root 2024-09-19T04:33:37.784 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b[63958]: ignoring --setgroup ceph since I am not root 2024-09-19T04:33:37.784 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-mds[63962]: -- 172.21.15.104:0/4000632141 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0d3623860 con 0x55d0d3600800 2024-09-19T04:33:37.784 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b[63958]: starting mds.b at 2024-09-19T04:33:37.784 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h[63654]: ignoring --setuser ceph since I am not root 2024-09-19T04:33:37.784 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h[63654]: ignoring --setgroup ceph since I am not root 2024-09-19T04:33:37.784 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-mds[63658]: -- 172.21.15.104:0/4091960493 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556975185860 con 0x556975162800 2024-09-19T04:33:37.784 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h[63654]: starting mds.h at 2024-09-19T04:33:37.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f[64832]: ignoring --setuser ceph since I am not root 2024-09-19T04:33:37.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f[64832]: ignoring --setgroup ceph since I am not root 2024-09-19T04:33:37.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-mds[64836]: -- 172.21.15.112:0/284770867 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55747c6c9860 con 0x55747c6a7400 2024-09-19T04:33:37.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f[64832]: starting mds.f at 2024-09-19T04:33:37.785 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c[65135]: ignoring --setuser ceph since I am not root 2024-09-19T04:33:37.785 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c[65135]: ignoring --setgroup ceph since I am not root 2024-09-19T04:33:37.786 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-mds[65139]: -- 172.21.15.112:0/1648790153 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b98cbf860 con 0x563b98c9c800 2024-09-19T04:33:37.786 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c[65135]: starting mds.c at 2024-09-19T04:33:37.786 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i[65435]: ignoring --setuser ceph since I am not root 2024-09-19T04:33:37.786 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i[65435]: ignoring --setgroup ceph since I am not root 2024-09-19T04:33:37.786 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-mds[65439]: -- 172.21.15.112:0/3443758860 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654a6405860 con 0x5654a63e2800 2024-09-19T04:33:37.786 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i[65435]: starting mds.i at 2024-09-19T04:33:38.034 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1122299302' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:33:38.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-mon[35051]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:33:38.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-mon[35051]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:33:38.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1122299302' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:33:38.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-mon[35051]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T04:33:38.035 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:37 smithi112 ceph-mon[35051]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T04:33:38.049 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:37 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1122299302' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:33:38.049 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:37 smithi094 ceph-mon[30972]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:33:38.049 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:37 smithi094 ceph-mon[30972]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:33:38.049 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:37 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1122299302' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:33:38.049 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:37 smithi094 ceph-mon[30972]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T04:33:38.049 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:37 smithi094 ceph-mon[30972]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T04:33:38.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:33:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1122299302' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:33:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-mon[34145]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:33:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-mon[34145]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:33:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1122299302' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:33:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-mon[34145]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T04:33:38.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:37 smithi104 ceph-mon[34145]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T04:33:38.321 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.774+0000 7f34d098d640 1 -- 172.21.15.94:0/3705184315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102570 msgr2=0x7f34cc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.774+0000 7f34d098d640 1 --2- 172.21.15.94:0/3705184315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102570 0x7f34cc1029b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009900 tx=0x7f34b402a480 comp rx=0 tx=0).stop 2024-09-19T04:33:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.774+0000 7f34d098d640 1 -- 172.21.15.94:0/3705184315 shutdown_connections 2024-09-19T04:33:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.774+0000 7f34d098d640 1 --2- 172.21.15.94:0/3705184315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc102ef0 0x7f34cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.774+0000 7f34d098d640 1 --2- 172.21.15.94:0/3705184315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102570 0x7f34cc1029b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.774+0000 7f34d098d640 1 --2- 172.21.15.94:0/3705184315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34cc100e20 0x7f34cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.774+0000 7f34d098d640 1 -- 172.21.15.94:0/3705184315 >> 172.21.15.94:0/3705184315 conn(0x7f34cc0fc7d0 msgr2=0x7f34cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.774+0000 7f34d098d640 1 -- 172.21.15.94:0/3705184315 shutdown_connections 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.774+0000 7f34d098d640 1 -- 172.21.15.94:0/3705184315 wait complete. 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34d098d640 1 Processor -- start 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34d098d640 1 -- start start 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34d098d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc100e20 0x7f34cc198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34d098d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34cc102570 0x7f34cc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34d098d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102ef0 0x7f34cc19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34d098d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc110f20 con 0x7f34cc102ef0 2024-09-19T04:33:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34d098d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc110da0 con 0x7f34cc100e20 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34d098d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1110a0 con 0x7f34cc102570 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102ef0 0x7f34cc19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102ef0 0x7f34cc19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43106/0 (socket says 172.21.15.94:43106) 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34cad9d640 1 -- 172.21.15.94:0/3899615437 learned_addr learned my addr 172.21.15.94:0/3899615437 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34c3fff640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34cc102570 0x7f34cc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34ca59c640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc100e20 0x7f34cc198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34cad9d640 1 -- 172.21.15.94:0/3899615437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34cc102570 msgr2=0x7f34cc198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34cad9d640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34cc102570 0x7f34cc198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34cad9d640 1 -- 172.21.15.94:0/3899615437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc100e20 msgr2=0x7f34cc198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.775+0000 7f34cad9d640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc100e20 0x7f34cc198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.776+0000 7f34cad9d640 1 -- 172.21.15.94:0/3899615437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc19d500 con 0x7f34cc102ef0 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.776+0000 7f34ca59c640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc100e20 0x7f34cc198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:33:38.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.776+0000 7f34cad9d640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102ef0 0x7f34cc19ce20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f34c400b970 tx=0x7f34c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.776+0000 7f34c37fe640 1 -- 172.21.15.94:0/3899615437 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4019070 con 0x7f34cc102ef0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.776+0000 7f34c37fe640 1 -- 172.21.15.94:0/3899615437 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34c4014440 con 0x7f34cc102ef0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.776+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc19d7f0 con 0x7f34cc102ef0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.776+0000 7f34c37fe640 1 -- 172.21.15.94:0/3899615437 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c40075d0 con 0x7f34cc102ef0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.776+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc1a50d0 con 0x7f34cc102ef0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.777+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc101200 con 0x7f34cc102ef0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.778+0000 7f34c37fe640 1 -- 172.21.15.94:0/3899615437 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f34c401c070 con 0x7f34cc102ef0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.778+0000 7f34c37fe640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f34b80a33b0 0x7f34b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.779+0000 7f34ca59c640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f34b80a33b0 0x7f34b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.779+0000 7f34c37fe640 1 -- 172.21.15.94:0/3899615437 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8799+0+0 (secure 0 0 0) 0x7f34c40cf2f0 con 0x7f34cc102ef0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.779+0000 7f34ca59c640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f34b80a33b0 0x7f34b80a5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34bc006fd0 tx=0x7f34bc008040 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:38.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.783+0000 7f34c37fe640 1 -- 172.21.15.94:0/3899615437 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f34c409b680 con 0x7f34cc102ef0 2024-09-19T04:33:38.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:38.983+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f34cc10a0a0 con 0x7f34cc102ef0 2024-09-19T04:33:39.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.453+0000 7f34c37fe640 1 -- 172.21.15.94:0/3899615437 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7f34c409f980 con 0x7f34cc102ef0 2024-09-19T04:33:39.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.456+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f34b80a33b0 msgr2=0x7f34b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.456+0000 7f34d098d640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f34b80a33b0 0x7f34b80a5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34bc006fd0 tx=0x7f34bc008040 comp rx=0 tx=0).stop 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102ef0 msgr2=0x7f34cc19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102ef0 0x7f34cc19ce20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f34c400b970 tx=0x7f34c400be30 comp rx=0 tx=0).stop 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 shutdown_connections 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7f34b80a33b0 0x7f34b80a5870 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34cc102ef0 0x7f34cc19ce20 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34cc102570 0x7f34cc198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 --2- 172.21.15.94:0/3899615437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc100e20 0x7f34cc198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 >> 172.21.15.94:0/3899615437 conn(0x7f34cc0fc7d0 msgr2=0x7f34cc06a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 shutdown_connections 2024-09-19T04:33:39.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:39.457+0000 7f34d098d640 1 -- 172.21.15.94:0/3899615437 wait complete. 2024-09-19T04:33:39.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: pgmap v586: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-09-19T04:33:39.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.104:6834/930665204,v1:172.21.15.104:6835/930665204] up:boot 2024-09-19T04:33:39.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.112:6834/2039849706,v1:172.21.15.112:6835/2039849706] up:boot 2024-09-19T04:33:39.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.104:6836/909765046,v1:172.21.15.104:6837/909765046] up:boot 2024-09-19T04:33:39.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.112:6836/1122590996,v1:172.21.15.112:6838/1122590996] up:boot 2024-09-19T04:33:39.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.104:6838/1706015659,v1:172.21.15.104:6839/1706015659] up:boot 2024-09-19T04:33:39.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: mds.? [v2:172.21.15.112:6837/333376949,v1:172.21.15.112:6839/333376949] up:boot 2024-09-19T04:33:39.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T04:33:39.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:33:39.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:33:39.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:33:39.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:33:39.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:33:39.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:33:39.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:39 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3899615437' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: pgmap v586: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.104:6834/930665204,v1:172.21.15.104:6835/930665204] up:boot 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.112:6834/2039849706,v1:172.21.15.112:6835/2039849706] up:boot 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.104:6836/909765046,v1:172.21.15.104:6837/909765046] up:boot 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.112:6836/1122590996,v1:172.21.15.112:6838/1122590996] up:boot 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.104:6838/1706015659,v1:172.21.15.104:6839/1706015659] up:boot 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: mds.? [v2:172.21.15.112:6837/333376949,v1:172.21.15.112:6839/333376949] up:boot 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:33:39.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:33:39.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:33:39.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:33:39.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:33:39.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:33:39.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:39 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3899615437' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:33:39.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: pgmap v586: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.104:6834/930665204,v1:172.21.15.104:6835/930665204] up:boot 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.112:6834/2039849706,v1:172.21.15.112:6835/2039849706] up:boot 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.104:6836/909765046,v1:172.21.15.104:6837/909765046] up:boot 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.112:6836/1122590996,v1:172.21.15.112:6838/1122590996] up:boot 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.104:6838/1706015659,v1:172.21.15.104:6839/1706015659] up:boot 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: mds.? [v2:172.21.15.112:6837/333376949,v1:172.21.15.112:6839/333376949] up:boot 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:33:39.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:33:39.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:33:39.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: from='mgr.15159 172.21.15.94:0/1030701865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:33:39.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:39 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3899615437' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:33:40.088 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:33:40.337 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:40.709 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:40 smithi094 ceph-mon[30972]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:33:40.709 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:40 smithi094 ceph-mon[30972]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:33:40.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:40 smithi094 ceph-mon[30972]: Cluster is now healthy 2024-09-19T04:33:40.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:40 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/3899615437' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:33:40.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:40 smithi094 ceph-mon[30972]: fsmap 12 up:standby 2024-09-19T04:33:40.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:40 smithi112 ceph-mon[35051]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:33:40.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:40 smithi112 ceph-mon[35051]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:33:40.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:40 smithi112 ceph-mon[35051]: Cluster is now healthy 2024-09-19T04:33:40.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:40 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/3899615437' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:33:40.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:40 smithi112 ceph-mon[35051]: fsmap 12 up:standby 2024-09-19T04:33:40.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.791+0000 7fed833cf640 1 -- 172.21.15.94:0/1502978736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c101de0 msgr2=0x7fed7c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:40.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.791+0000 7fed833cf640 1 --2- 172.21.15.94:0/1502978736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c101de0 0x7fed7c1021c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fed78009900 tx=0x7fed7802a410 comp rx=0 tx=0).stop 2024-09-19T04:33:40.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 -- 172.21.15.94:0/1502978736 shutdown_connections 2024-09-19T04:33:40.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 --2- 172.21.15.94:0/1502978736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed7c106f10 0x7fed7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:40.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 --2- 172.21.15.94:0/1502978736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c102790 0x7fed7c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 --2- 172.21.15.94:0/1502978736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c101de0 0x7fed7c1021c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 -- 172.21.15.94:0/1502978736 >> 172.21.15.94:0/1502978736 conn(0x7fed7c0fd8c0 msgr2=0x7fed7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 -- 172.21.15.94:0/1502978736 shutdown_connections 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 -- 172.21.15.94:0/1502978736 wait complete. 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 Processor -- start 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 -- start start 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.792+0000 7fed833cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c101de0 0x7fed7c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed833cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c102790 0x7fed7c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed833cf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed7c106f10 0x7fed7c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed833cf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c111de0 con 0x7fed7c102790 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed833cf640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c111c60 con 0x7fed7c106f10 2024-09-19T04:33:40.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed833cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c111f60 con 0x7fed7c101de0 2024-09-19T04:33:40.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8096a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c102790 0x7fed7c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:40.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8096a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c102790 0x7fed7c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43122/0 (socket says 172.21.15.94:43122) 2024-09-19T04:33:40.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8096a640 1 -- 172.21.15.94:0/1098460577 learned_addr learned my addr 172.21.15.94:0/1098460577 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:40.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8196c640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed7c106f10 0x7fed7c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:40.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8116b640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c101de0 0x7fed7c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:40.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8096a640 1 -- 172.21.15.94:0/1098460577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c101de0 msgr2=0x7fed7c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:40.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8096a640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c101de0 0x7fed7c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:40.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8096a640 1 -- 172.21.15.94:0/1098460577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed7c106f10 msgr2=0x7fed7c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:40.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8096a640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed7c106f10 0x7fed7c19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8096a640 1 -- 172.21.15.94:0/1098460577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c19d260 con 0x7fed7c102790 2024-09-19T04:33:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8116b640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c101de0 0x7fed7c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:33:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed8096a640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c102790 0x7fed7c1987f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c920 tx=0x7fed6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed727fc640 1 -- 172.21.15.94:0/1098460577 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c007c70 con 0x7fed7c102790 2024-09-19T04:33:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7c19d550 con 0x7fed7c102790 2024-09-19T04:33:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed727fc640 1 -- 172.21.15.94:0/1098460577 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed6c007e10 con 0x7fed7c102790 2024-09-19T04:33:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed727fc640 1 -- 172.21.15.94:0/1098460577 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c018820 con 0x7fed7c102790 2024-09-19T04:33:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.793+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7c1a4e30 con 0x7fed7c102790 2024-09-19T04:33:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.794+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7c1021c0 con 0x7fed7c102790 2024-09-19T04:33:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.795+0000 7fed727fc640 1 -- 172.21.15.94:0/1098460577 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fed6c0189c0 con 0x7fed7c102790 2024-09-19T04:33:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.796+0000 7fed727fc640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed680a33b0 0x7fed680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.796+0000 7fed8116b640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed680a33b0 0x7fed680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.796+0000 7fed727fc640 1 -- 172.21.15.94:0/1098460577 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8799+0+0 (secure 0 0 0) 0x7fed6c0cff60 con 0x7fed7c102790 2024-09-19T04:33:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.797+0000 7fed8116b640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed680a33b0 0x7fed680a5870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed78031440 tx=0x7fed780095f0 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.801+0000 7fed727fc640 1 -- 172.21.15.94:0/1098460577 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fed6c014030 con 0x7fed7c102790 2024-09-19T04:33:40.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:40 smithi104 ceph-mon[34145]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:33:40.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:40 smithi104 ceph-mon[34145]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:33:40.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:40 smithi104 ceph-mon[34145]: Cluster is now healthy 2024-09-19T04:33:40.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:40 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/3899615437' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:33:40.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:40 smithi104 ceph-mon[34145]: fsmap 12 up:standby 2024-09-19T04:33:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:40.971+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed7c02d7c0 con 0x7fed7c102790 2024-09-19T04:33:41.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.466+0000 7fed727fc640 1 -- 172.21.15.94:0/1098460577 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7fed6c09c830 con 0x7fed7c102790 2024-09-19T04:33:41.468 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:33:41.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed680a33b0 msgr2=0x7fed680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:41.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed680a33b0 0x7fed680a5870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed78031440 tx=0x7fed780095f0 comp rx=0 tx=0).stop 2024-09-19T04:33:41.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c102790 msgr2=0x7fed7c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:41.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c102790 0x7fed7c1987f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c920 tx=0x7fed6c00cde0 comp rx=0 tx=0).stop 2024-09-19T04:33:41.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 shutdown_connections 2024-09-19T04:33:41.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7fed680a33b0 0x7fed680a5870 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:41.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed7c106f10 0x7fed7c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:41.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed7c102790 0x7fed7c1987f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:41.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 --2- 172.21.15.94:0/1098460577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c101de0 0x7fed7c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:41.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.469+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 >> 172.21.15.94:0/1098460577 conn(0x7fed7c0fd8c0 msgr2=0x7fed7c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:41.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.470+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 shutdown_connections 2024-09-19T04:33:41.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:41.470+0000 7fed833cf640 1 -- 172.21.15.94:0/1098460577 wait complete. 2024-09-19T04:33:41.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:41 smithi112 ceph-mon[35051]: pgmap v587: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-19T04:33:41.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:41 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1098460577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:33:41.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:41 smithi094 ceph-mon[30972]: pgmap v587: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-19T04:33:41.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:41 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1098460577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:33:41.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:41 smithi104 ceph-mon[34145]: pgmap v587: 97 pgs: 97 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-19T04:33:41.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:41 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1098460577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:33:42.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 503c4526-763c-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:33:42.344 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:42.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:42 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/1098460577' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:33:42.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:42 smithi112 ceph-mon[35051]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T04:33:42.834 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:42 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/1098460577' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:33:42.834 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:42 smithi094 ceph-mon[30972]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T04:33:42.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:42 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/1098460577' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:33:42.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:42 smithi104 ceph-mon[34145]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T04:33:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.918+0000 7ff49664b640 1 -- 172.21.15.94:0/2752137508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490100a80 msgr2=0x7ff490100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.918+0000 7ff49664b640 1 --2- 172.21.15.94:0/2752137508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490100a80 0x7ff490100ee0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff478009930 tx=0x7ff47802a420 comp rx=0 tx=0).stop 2024-09-19T04:33:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.918+0000 7ff49664b640 1 -- 172.21.15.94:0/2752137508 shutdown_connections 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.918+0000 7ff49664b640 1 --2- 172.21.15.94:0/2752137508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff490101420 0x7ff49010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.918+0000 7ff49664b640 1 --2- 172.21.15.94:0/2752137508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490100a80 0x7ff490100ee0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.918+0000 7ff49664b640 1 --2- 172.21.15.94:0/2752137508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff490106a80 0x7ff490106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.918+0000 7ff49664b640 1 -- 172.21.15.94:0/2752137508 >> 172.21.15.94:0/2752137508 conn(0x7ff4900fc7a0 msgr2=0x7ff4900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.918+0000 7ff49664b640 1 -- 172.21.15.94:0/2752137508 shutdown_connections 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.918+0000 7ff49664b640 1 -- 172.21.15.94:0/2752137508 wait complete. 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff49664b640 1 Processor -- start 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff49664b640 1 -- start start 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff49664b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff490100a80 0x7ff490199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff49664b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490101420 0x7ff4901996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff49664b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff490106a80 0x7ff4901a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff49664b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff490110d30 con 0x7ff490101420 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff49664b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff490110bb0 con 0x7ff490106a80 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff49664b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff490110eb0 con 0x7ff490100a80 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff487fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490101420 0x7ff4901996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff487fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490101420 0x7ff4901996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59734/0 (socket says 172.21.15.94:59734) 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff487fff640 1 -- 172.21.15.94:0/2460409150 learned_addr learned my addr 172.21.15.94:0/2460409150 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff494be8640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff490106a80 0x7ff4901a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff48ffff640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff490100a80 0x7ff490199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff487fff640 1 -- 172.21.15.94:0/2460409150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff490100a80 msgr2=0x7ff490199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff487fff640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff490100a80 0x7ff490199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff487fff640 1 -- 172.21.15.94:0/2460409150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff490106a80 msgr2=0x7ff4901a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff487fff640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff490106a80 0x7ff4901a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff487fff640 1 -- 172.21.15.94:0/2460409150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4901a4720 con 0x7ff490101420 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.919+0000 7ff487fff640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490101420 0x7ff4901996a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff478005940 tx=0x7ff4780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.920+0000 7ff48dffb640 1 -- 172.21.15.94:0/2460409150 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47803f030 con 0x7ff490101420 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.920+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4901a48f0 con 0x7ff490101420 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.920+0000 7ff48dffb640 1 -- 172.21.15.94:0/2460409150 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4780318a0 con 0x7ff490101420 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.920+0000 7ff48dffb640 1 -- 172.21.15.94:0/2460409150 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478035070 con 0x7ff490101420 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.920+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901a4d40 con 0x7ff490101420 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.921+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff490100ee0 con 0x7ff490101420 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.922+0000 7ff48dffb640 1 -- 172.21.15.94:0/2460409150 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff478042030 con 0x7ff490101420 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.922+0000 7ff48dffb640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff47c0a33b0 0x7ff47c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.922+0000 7ff48ffff640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff47c0a33b0 0x7ff47c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.923+0000 7ff48dffb640 1 -- 172.21.15.94:0/2460409150 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8303+0+0 (secure 0 0 0) 0x7ff4780f0490 con 0x7ff490101420 2024-09-19T04:33:42.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.923+0000 7ff48ffff640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff47c0a33b0 0x7ff47c0a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff480009ae0 tx=0x7ff480005f90 comp rx=0 tx=0).ready entity=mgr.15159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:42.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:42.926+0000 7ff48dffb640 1 -- 172.21.15.94:0/2460409150 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff4780bcf50 con 0x7ff490101420 2024-09-19T04:33:43.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.095+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff490108ff0 con 0x7ff490101420 2024-09-19T04:33:43.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.480+0000 7ff48dffb640 1 -- 172.21.15.94:0/2460409150 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7ff4780bc850 con 0x7ff490101420 2024-09-19T04:33:43.482 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-19T04:33:43.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.483+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff47c0a33b0 msgr2=0x7ff47c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:43.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.483+0000 7ff49664b640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff47c0a33b0 0x7ff47c0a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff480009ae0 tx=0x7ff480005f90 comp rx=0 tx=0).stop 2024-09-19T04:33:43.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.483+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490101420 msgr2=0x7ff4901996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:43.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.483+0000 7ff49664b640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490101420 0x7ff4901996a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff478005940 tx=0x7ff4780042f0 comp rx=0 tx=0).stop 2024-09-19T04:33:43.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.484+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 shutdown_connections 2024-09-19T04:33:43.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.484+0000 7ff49664b640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:6800/3395191155,v1:172.21.15.94:6801/3395191155] conn(0x7ff47c0a33b0 0x7ff47c0a5870 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:43.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.484+0000 7ff49664b640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff490106a80 0x7ff4901a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:43.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.484+0000 7ff49664b640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff490101420 0x7ff4901996a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:43.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.484+0000 7ff49664b640 1 --2- 172.21.15.94:0/2460409150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff490100a80 0x7ff490199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:43.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.484+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 >> 172.21.15.94:0/2460409150 conn(0x7ff4900fc7a0 msgr2=0x7ff4900fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:43.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.485+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 shutdown_connections 2024-09-19T04:33:43.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-19T04:33:43.485+0000 7ff49664b640 1 -- 172.21.15.94:0/2460409150 wait complete. 2024-09-19T04:33:43.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:43 smithi112 ceph-mon[35051]: pgmap v589: 65 pgs: 65 active+clean; 2.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:33:43.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:43 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2460409150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:33:43.839 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:43 smithi094 ceph-mon[30972]: pgmap v589: 65 pgs: 65 active+clean; 2.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:33:43.839 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:43 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2460409150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:33:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:43 smithi104 ceph-mon[34145]: pgmap v589: 65 pgs: 65 active+clean; 2.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:33:43.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:43 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2460409150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:33:44.198 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:33:44.283 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:33:44.311 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:33:44.338 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T04:33:44.338 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:33:44.364 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:33:44.379 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:33:44.407 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T04:33:44.407 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T04:33:44.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.a 2024-09-19T04:33:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:44 smithi104 ceph-mon[34145]: from='client.? 172.21.15.94:0/2460409150' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:33:44.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:44 smithi104 ceph-mon[34145]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T04:33:44.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:44 smithi094 systemd[1]: Stopping Ceph mon.a for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:33:44.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:44 smithi094 ceph-mon[30972]: from='client.? 172.21.15.94:0/2460409150' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:33:44.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:44 smithi094 ceph-mon[30972]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T04:33:44.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:44 smithi112 ceph-mon[35051]: from='client.? 172.21.15.94:0/2460409150' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:33:44.784 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:44 smithi112 ceph-mon[35051]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T04:33:45.090 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a[30967]: 2024-09-19T04:33:44.760+0000 7f4281586640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:33:45.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a[30967]: 2024-09-19T04:33:44.760+0000 7f4281586640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T04:33:45.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:44 smithi094 podman[122190]: 2024-09-19 04:33:44.97533953 +0000 UTC m=+0.332318749 container died cf9c5201f28527b683791423b5d1b0c262f45c959cba0fee8faf1de2ddb7ed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T04:33:45.553 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:45 smithi094 podman[122190]: 2024-09-19 04:33:45.343087845 +0000 UTC m=+0.700067056 container cleanup cf9c5201f28527b683791423b5d1b0c262f45c959cba0fee8faf1de2ddb7ed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-19T04:33:45.553 INFO:journalctl@ceph.mon.a.smithi094.stdout:Sep 19 04:33:45 smithi094 bash[122190]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-a 2024-09-19T04:33:45.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T04:33:45.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:33:45.826 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T04:33:45.826 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T04:33:45.826 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.b 2024-09-19T04:33:46.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:45 smithi104 systemd[1]: Stopping Ceph mon.b for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:33:46.524 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:46 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-b[34141]: 2024-09-19T04:33:46.131+0000 7f3a162d7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:33:46.524 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:46 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-b[34141]: 2024-09-19T04:33:46.131+0000 7f3a162d7640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T04:33:46.524 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:46 smithi104 podman[71079]: 2024-09-19 04:33:46.254134753 +0000 UTC m=+0.230936347 container died c97384baf7fc62c69677d163de40a388d2919a20678cb75e560c713850e557b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:33:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:46 smithi104 podman[71079]: 2024-09-19 04:33:46.62287249 +0000 UTC m=+0.599674061 container cleanup c97384baf7fc62c69677d163de40a388d2919a20678cb75e560c713850e557b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T04:33:46.854 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 19 04:33:46 smithi104 bash[71079]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-b 2024-09-19T04:33:47.057 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T04:33:47.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:33:47.104 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T04:33:47.104 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T04:33:47.105 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.c 2024-09-19T04:33:47.495 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:47 smithi112 systemd[1]: Stopping Ceph mon.c for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:33:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-c[35047]: 2024-09-19T04:33:47.493+0000 7fe8e980f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:33:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-c[35047]: 2024-09-19T04:33:47.493+0000 7fe8e980f640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T04:33:47.785 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:47 smithi112 podman[72294]: 2024-09-19 04:33:47.525004581 +0000 UTC m=+0.158332750 container died 5450f6bd98af8715ce3df9d89be17a04b843f698c19716c835cd8e718ab83dac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T04:33:48.171 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:47 smithi112 podman[72294]: 2024-09-19 04:33:47.902875678 +0000 UTC m=+0.536203846 container cleanup 5450f6bd98af8715ce3df9d89be17a04b843f698c19716c835cd8e718ab83dac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T04:33:48.172 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 19 04:33:47 smithi112 bash[72294]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mon-c 2024-09-19T04:33:48.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T04:33:48.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:33:48.353 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T04:33:48.353 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T04:33:48.353 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.x 2024-09-19T04:33:48.665 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:33:48 smithi094 systemd[1]: Stopping Ceph mgr.x for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:33:49.007 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:33:48 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:33:48.663+0000 7fa01377f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:33:49.007 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:33:48 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x[31210]: 2024-09-19T04:33:48.663+0000 7fa01377f640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-19T04:33:49.007 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:33:48 smithi094 podman[122314]: 2024-09-19 04:33:48.695047942 +0000 UTC m=+0.141775925 container died d829d7ba3425b3443c2a93377f0057023937660b4d71b1a8028aa741cd0a5e21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True) 2024-09-19T04:33:49.340 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:33:49 smithi094 podman[122314]: 2024-09-19 04:33:49.106227011 +0000 UTC m=+0.552954979 container cleanup d829d7ba3425b3443c2a93377f0057023937660b4d71b1a8028aa741cd0a5e21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T04:33:49.341 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Sep 19 04:33:49 smithi094 bash[122314]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-x 2024-09-19T04:33:49.555 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T04:33:49.591 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:33:49.591 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T04:33:49.592 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T04:33:49.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.y 2024-09-19T04:33:49.915 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:33:49 smithi104 systemd[1]: Stopping Ceph mgr.y for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:33:50.186 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:33:49 smithi104 podman[71201]: 2024-09-19 04:33:49.914394063 +0000 UTC m=+0.142299612 container died 56905ac5daf2ee0f625d4dfadea952d2bb57137abceb85973425cea40e6aeb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T04:33:50.570 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:33:50 smithi104 podman[71201]: 2024-09-19 04:33:50.268287801 +0000 UTC m=+0.496193338 container cleanup 56905ac5daf2ee0f625d4dfadea952d2bb57137abceb85973425cea40e6aeb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T04:33:50.570 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 19 04:33:50 smithi104 bash[71201]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-y 2024-09-19T04:33:50.698 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T04:33:50.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:33:50.733 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T04:33:50.733 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T04:33:50.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.z 2024-09-19T04:33:51.034 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:33:50 smithi112 systemd[1]: Stopping Ceph mgr.z for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:33:51.334 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:33:51 smithi112 podman[72428]: 2024-09-19 04:33:51.064113131 +0000 UTC m=+0.139484228 container died 6a914845d4b99b9f2221d84df623be72f4ab19e1b2b2544a994208c13da38c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4) 2024-09-19T04:33:51.676 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:33:51 smithi112 podman[72428]: 2024-09-19 04:33:51.424678865 +0000 UTC m=+0.500049949 container cleanup 6a914845d4b99b9f2221d84df623be72f4ab19e1b2b2544a994208c13da38c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T04:33:51.676 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 19 04:33:51 smithi112 bash[72428]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mgr-z 2024-09-19T04:33:51.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T04:33:51.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:33:51.881 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T04:33:51.881 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T04:33:51.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.0 2024-09-19T04:33:52.195 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:51 smithi094 systemd[1]: Stopping Ceph osd.0 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:33:52.452 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0[45247]: 2024-09-19T04:33:52.194+0000 7f86b2839640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:33:52.453 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0[45247]: 2024-09-19T04:33:52.194+0000 7f86b2839640 -1 osd.0 109 *** Got signal Terminated *** 2024-09-19T04:33:52.453 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0[45247]: 2024-09-19T04:33:52.194+0000 7f86b2839640 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:33:58.179 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:57 smithi094 podman[122436]: 2024-09-19 04:33:57.920388873 +0000 UTC m=+5.844833863 container died 0b165cdf9f58bf1f614779e1bfae3e73d8dc16189235464c057180f39fb70895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T04:33:58.459 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:58 smithi094 podman[122436]: 2024-09-19 04:33:58.261061281 +0000 UTC m=+6.185506264 container cleanup 0b165cdf9f58bf1f614779e1bfae3e73d8dc16189235464c057180f39fb70895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T04:33:58.460 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:58 smithi094 bash[122436]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0 2024-09-19T04:33:58.841 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:58 smithi094 podman[122465]: 2024-09-19 04:33:58.459056707 +0000 UTC m=+0.534389783 container remove 0b165cdf9f58bf1f614779e1bfae3e73d8dc16189235464c057180f39fb70895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T04:33:59.183 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:58 smithi094 podman[122550]: 2024-09-19 04:33:58.760474372 +0000 UTC m=+0.015897728 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:33:59.183 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:58 smithi094 podman[122550]: 2024-09-19 04:33:58.871576197 +0000 UTC m=+0.126999538 container create 94d110f4294f11b3acec950993bf07c5342db262a4400474a78a7309deffc5ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T04:33:59.545 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:59 smithi094 podman[122550]: 2024-09-19 04:33:59.182734901 +0000 UTC m=+0.438158249 container init 94d110f4294f11b3acec950993bf07c5342db262a4400474a78a7309deffc5ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T04:33:59.545 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:59 smithi094 podman[122550]: 2024-09-19 04:33:59.187296377 +0000 UTC m=+0.442719719 container start 94d110f4294f11b3acec950993bf07c5342db262a4400474a78a7309deffc5ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T04:33:59.545 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:59 smithi094 podman[122550]: 2024-09-19 04:33:59.257820993 +0000 UTC m=+0.513244341 container attach 94d110f4294f11b3acec950993bf07c5342db262a4400474a78a7309deffc5ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T04:33:59.841 INFO:journalctl@ceph.osd.0.smithi094.stdout:Sep 19 04:33:59 smithi094 podman[122550]: 2024-09-19 04:33:59.544688503 +0000 UTC m=+0.800111862 container died 94d110f4294f11b3acec950993bf07c5342db262a4400474a78a7309deffc5ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T04:34:00.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T04:34:00.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:34:00.240 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T04:34:00.240 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T04:34:00.240 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.1 2024-09-19T04:34:00.565 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:00 smithi104 systemd[1]: Stopping Ceph osd.1 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:34:00.854 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1[38626]: 2024-09-19T04:34:00.563+0000 7fb5b6fd2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:34:00.854 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1[38626]: 2024-09-19T04:34:00.563+0000 7fb5b6fd2640 -1 osd.1 109 *** Got signal Terminated *** 2024-09-19T04:34:00.854 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1[38626]: 2024-09-19T04:34:00.563+0000 7fb5b6fd2640 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:34:06.548 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:06 smithi104 podman[71336]: 2024-09-19 04:34:06.242319666 +0000 UTC m=+5.821794802 container died b2be2720d26a21bfd6f344d058acea6d3f55387115bac1bf8c19e317ec375c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-19T04:34:06.854 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:06 smithi104 podman[71336]: 2024-09-19 04:34:06.651501906 +0000 UTC m=+6.230977087 container cleanup b2be2720d26a21bfd6f344d058acea6d3f55387115bac1bf8c19e317ec375c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True) 2024-09-19T04:34:06.854 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:06 smithi104 bash[71336]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1 2024-09-19T04:34:07.209 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:06 smithi104 podman[71351]: 2024-09-19 04:34:06.889071182 +0000 UTC m=+0.646643724 container remove b2be2720d26a21bfd6f344d058acea6d3f55387115bac1bf8c19e317ec375c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T04:34:07.476 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:07 smithi104 podman[71442]: 2024-09-19 04:34:07.111144268 +0000 UTC m=+0.015591678 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:34:07.476 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:07 smithi104 podman[71442]: 2024-09-19 04:34:07.256587759 +0000 UTC m=+0.161035215 container create 1f37ef26b1f13fd6002ade8c885472f8bf5816d6b77e0bfb94318c33b2f08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:34:07.854 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:07 smithi104 podman[71442]: 2024-09-19 04:34:07.475549043 +0000 UTC m=+0.379996458 container init 1f37ef26b1f13fd6002ade8c885472f8bf5816d6b77e0bfb94318c33b2f08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T04:34:07.854 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:07 smithi104 podman[71442]: 2024-09-19 04:34:07.479297588 +0000 UTC m=+0.383744999 container start 1f37ef26b1f13fd6002ade8c885472f8bf5816d6b77e0bfb94318c33b2f08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T04:34:07.854 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:07 smithi104 podman[71442]: 2024-09-19 04:34:07.550452654 +0000 UTC m=+0.454900078 container attach 1f37ef26b1f13fd6002ade8c885472f8bf5816d6b77e0bfb94318c33b2f08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T04:34:07.855 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 19 04:34:07 smithi104 podman[71465]: 2024-09-19 04:34:07.696450492 +0000 UTC m=+0.016195936 container died 1f37ef26b1f13fd6002ade8c885472f8bf5816d6b77e0bfb94318c33b2f08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:34:08.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T04:34:08.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:34:08.313 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T04:34:08.313 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T04:34:08.313 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.2 2024-09-19T04:34:08.613 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:08 smithi112 systemd[1]: Stopping Ceph osd.2 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:34:09.034 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:08 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2[40044]: 2024-09-19T04:34:08.610+0000 7f020bfd1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:34:09.035 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:08 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2[40044]: 2024-09-19T04:34:08.610+0000 7f020bfd1640 -1 osd.2 109 *** Got signal Terminated *** 2024-09-19T04:34:09.035 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:08 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2[40044]: 2024-09-19T04:34:08.610+0000 7f020bfd1640 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:34:14.454 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:14 smithi112 podman[72557]: 2024-09-19 04:34:14.161170862 +0000 UTC m=+5.651923431 container died 60bb56813460a2e6eb82f50a9b666257efd7826d7cdf4d06c7c7a49af4eb92a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T04:34:14.714 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:14 smithi112 podman[72557]: 2024-09-19 04:34:14.453756346 +0000 UTC m=+5.944508914 container cleanup 60bb56813460a2e6eb82f50a9b666257efd7826d7cdf4d06c7c7a49af4eb92a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T04:34:14.714 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:14 smithi112 bash[72557]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2 2024-09-19T04:34:15.020 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:14 smithi112 podman[72572]: 2024-09-19 04:34:14.713412329 +0000 UTC m=+0.549177310 container remove 60bb56813460a2e6eb82f50a9b666257efd7826d7cdf4d06c7c7a49af4eb92a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:34:15.285 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:15 smithi112 podman[72664]: 2024-09-19 04:34:14.922810634 +0000 UTC m=+0.015720439 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:34:15.285 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:15 smithi112 podman[72664]: 2024-09-19 04:34:15.022568933 +0000 UTC m=+0.115478784 container create 97a35ad57c3621d747ad1f9af3ff2e195103a970283b50142a46d5a0aa8b4036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-19T04:34:15.285 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:15 smithi112 podman[72664]: 2024-09-19 04:34:15.258323423 +0000 UTC m=+0.351233228 container init 97a35ad57c3621d747ad1f9af3ff2e195103a970283b50142a46d5a0aa8b4036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:34:15.285 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:15 smithi112 podman[72664]: 2024-09-19 04:34:15.261753637 +0000 UTC m=+0.354663446 container start 97a35ad57c3621d747ad1f9af3ff2e195103a970283b50142a46d5a0aa8b4036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T04:34:15.778 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:15 smithi112 podman[72664]: 2024-09-19 04:34:15.383540309 +0000 UTC m=+0.476450140 container attach 97a35ad57c3621d747ad1f9af3ff2e195103a970283b50142a46d5a0aa8b4036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:34:15.779 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 19 04:34:15 smithi112 podman[72664]: 2024-09-19 04:34:15.495400029 +0000 UTC m=+0.588309835 container died 97a35ad57c3621d747ad1f9af3ff2e195103a970283b50142a46d5a0aa8b4036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T04:34:16.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T04:34:16.107 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:34:16.108 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T04:34:16.108 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T04:34:16.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.3 2024-09-19T04:34:16.423 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:16 smithi094 systemd[1]: Stopping Ceph osd.3 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:34:16.840 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:16 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3[52047]: 2024-09-19T04:34:16.420+0000 7f1c7c9fa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:34:16.841 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:16 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3[52047]: 2024-09-19T04:34:16.420+0000 7f1c7c9fa640 -1 osd.3 109 *** Got signal Terminated *** 2024-09-19T04:34:16.841 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:16 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3[52047]: 2024-09-19T04:34:16.420+0000 7f1c7c9fa640 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:34:19.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:18 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:18.778+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:19.218 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:18 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:18.842+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:19.591 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:19 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3[52047]: 2024-09-19T04:34:19.217+0000 7f1c78fff640 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.409932+0000 front 2024-09-19T04:33:54.409890+0000 (oldest deadline 2024-09-19T04:34:19.110050+0000) 2024-09-19T04:34:19.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:19 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:19.272+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:20.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:19 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:19.769+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:20.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:19 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:19.849+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:20.200 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:19 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:19.889+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:20.590 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:20 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3[52047]: 2024-09-19T04:34:20.199+0000 7f1c78fff640 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.409932+0000 front 2024-09-19T04:33:54.409890+0000 (oldest deadline 2024-09-19T04:34:19.110050+0000) 2024-09-19T04:34:20.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:20 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:20.249+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:21.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:20.758+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:21.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:20.883+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:21.151 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:20 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:20.846+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:21.590 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:21 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3[52047]: 2024-09-19T04:34:21.149+0000 7f1c78fff640 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.409932+0000 front 2024-09-19T04:33:54.409890+0000 (oldest deadline 2024-09-19T04:34:19.110050+0000) 2024-09-19T04:34:21.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:21 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:21.201+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:22.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:21 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:21.725+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:22.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:21 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:21.892+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:22.124 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:21 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:21.849+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:22.381 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:22 smithi094 podman[122658]: 2024-09-19 04:34:22.124111152 +0000 UTC m=+5.817957553 container died 17bd7afc89b22cdabf12f5374500de77f68545305392bebdf58ba8640db3cdba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:34:22.381 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:22 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:22.217+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:22.816 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:22 smithi094 podman[122658]: 2024-09-19 04:34:22.588955258 +0000 UTC m=+6.282801651 container cleanup 17bd7afc89b22cdabf12f5374500de77f68545305392bebdf58ba8640db3cdba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:34:22.816 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:22 smithi094 bash[122658]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3 2024-09-19T04:34:22.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:22 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:22.529+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:23.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:22 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:22.749+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:23.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:22 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:22.926+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:23.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:22 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:22.861+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:23.091 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:22 smithi094 podman[122674]: 2024-09-19 04:34:22.81538871 +0000 UTC m=+0.693489937 container remove 17bd7afc89b22cdabf12f5374500de77f68545305392bebdf58ba8640db3cdba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:34:23.519 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:23 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:23.215+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:23.520 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:23 smithi094 podman[122775]: 2024-09-19 04:34:23.149821454 +0000 UTC m=+0.016065854 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:34:23.520 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:23 smithi094 podman[122775]: 2024-09-19 04:34:23.266168353 +0000 UTC m=+0.132412749 container create de7fd9c2ff83d23da3785375d7a0a8abcd761131ae2a30cdf5659bc3e8229de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T04:34:23.520 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:23 smithi094 podman[122775]: 2024-09-19 04:34:23.518603356 +0000 UTC m=+0.384847744 container init de7fd9c2ff83d23da3785375d7a0a8abcd761131ae2a30cdf5659bc3e8229de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T04:34:23.841 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:23 smithi094 podman[122775]: 2024-09-19 04:34:23.522193101 +0000 UTC m=+0.388437492 container start de7fd9c2ff83d23da3785375d7a0a8abcd761131ae2a30cdf5659bc3e8229de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T04:34:23.841 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:23 smithi094 podman[122775]: 2024-09-19 04:34:23.626958518 +0000 UTC m=+0.493202912 container attach de7fd9c2ff83d23da3785375d7a0a8abcd761131ae2a30cdf5659bc3e8229de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T04:34:23.841 INFO:journalctl@ceph.osd.3.smithi094.stdout:Sep 19 04:34:23 smithi094 podman[122775]: 2024-09-19 04:34:23.708826792 +0000 UTC m=+0.575071183 container died de7fd9c2ff83d23da3785375d7a0a8abcd761131ae2a30cdf5659bc3e8229de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T04:34:23.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:23 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:23.496+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:24.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:23.948+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:24.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:23.789+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:24.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:23.758+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:24.218 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:23 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:23.890+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:24.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T04:34:24.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:34:24.373 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T04:34:24.373 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T04:34:24.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.4 2024-09-19T04:34:24.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:24 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:24.216+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:24.706 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:24 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:24.519+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:24.706 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:24 smithi104 systemd[1]: Stopping Ceph osd.4 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:34:25.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:24.749+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:25.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:24.790+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:25.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:24.932+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:25.104 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:24 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4[45506]: 2024-09-19T04:34:24.704+0000 7f5c8aabd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:34:25.104 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:24 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4[45506]: 2024-09-19T04:34:24.704+0000 7f5c8aabd640 -1 osd.4 109 *** Got signal Terminated *** 2024-09-19T04:34:25.104 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:24 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4[45506]: 2024-09-19T04:34:24.704+0000 7f5c8aabd640 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:34:25.252 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:24 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:24.933+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:25.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:25.250+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:25.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:25 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:25.551+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:26.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:25 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:25.755+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:26.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:25 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:25.763+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:26.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:25 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:25.951+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:26.289 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:25 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:25.949+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:26.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:26 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:26.288+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:26.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:26 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:26.555+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:27.034 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:26 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:26.712+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:27.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:26 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:26.727+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:27.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:26 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:26.940+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:27.243 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:26 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:26.903+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:27.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:27 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:27.241+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:27.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:27 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:27.554+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:28.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:27.703+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:28.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:27.939+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:28.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:27.729+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:28.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:27 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:27.729+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:28.291 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:27 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:27.917+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:28.552 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:28 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:28.208+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:28.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:28.289+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:28.834 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:28 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:28.550+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:29.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:28.911+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:29.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:28.757+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:29.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:28.757+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:29.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:28.718+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:29.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:28 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:28.718+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:29.277 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:28.872+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:29.277 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:28 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:28.872+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:29.552 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:29 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:29.255+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:29.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:29 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:29.275+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:29.808 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:29 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:29.551+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:29.808 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:29 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:29.551+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:30.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:29 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:29.700+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:30.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:29 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:29.700+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:30.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:29 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:29.881+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:30.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:29 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:29.881+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:30.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:29 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:29.709+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:30.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:29 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:29.709+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:30.291 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:29 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:29.872+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:30.291 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:29 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:29.872+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:30.574 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:30 smithi104 podman[71545]: 2024-09-19 04:34:30.371349593 +0000 UTC m=+5.808397915 container died 8eabfde5842e19f5e9c5dc251237ebce98ed88a5995e90d959614b678a90b212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T04:34:30.575 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:30.285+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:30.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:30.289+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:30.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:30.289+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:30.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:30.572+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:30.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:30 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:30.572+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:30.854 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:30 smithi104 podman[71545]: 2024-09-19 04:34:30.753983418 +0000 UTC m=+6.191031734 container cleanup 8eabfde5842e19f5e9c5dc251237ebce98ed88a5995e90d959614b678a90b212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T04:34:30.854 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:30 smithi104 bash[71545]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4 2024-09-19T04:34:31.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:30 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:30.713+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:31.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:30 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:30.713+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:31.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:30 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:30.714+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:31.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:30 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:30.714+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:31.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:30 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:30.927+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:31.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:30 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:30.927+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:31.262 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:30.851+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:31.262 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:30 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:30.851+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:31.308 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:31 smithi104 podman[71572]: 2024-09-19 04:34:31.00518839 +0000 UTC m=+0.630741567 container remove 8eabfde5842e19f5e9c5dc251237ebce98ed88a5995e90d959614b678a90b212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T04:34:31.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:31.261+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:31.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:31.261+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:31.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:31 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:31.307+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:31.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:31 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:31.532+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:31.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:31 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:31.532+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:31.604 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:31 smithi104 podman[71659]: 2024-09-19 04:34:31.217350022 +0000 UTC m=+0.015522278 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:34:31.605 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:31 smithi104 podman[71659]: 2024-09-19 04:34:31.330873131 +0000 UTC m=+0.129045371 container create 43f5aa05fb71597791790feec7ca081c90df31e47d0078b06a8c0f6bb40033c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T04:34:31.605 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:31 smithi104 podman[71659]: 2024-09-19 04:34:31.550248458 +0000 UTC m=+0.348420712 container init 43f5aa05fb71597791790feec7ca081c90df31e47d0078b06a8c0f6bb40033c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T04:34:31.605 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:31 smithi104 podman[71659]: 2024-09-19 04:34:31.553615492 +0000 UTC m=+0.351787734 container start 43f5aa05fb71597791790feec7ca081c90df31e47d0078b06a8c0f6bb40033c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T04:34:32.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:31 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:31.761+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:32.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:31 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:31.761+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:32.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:31 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:31.900+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:32.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:31 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:31.900+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:32.036 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:31 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:31.745+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:32.036 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:31 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:31.745+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:32.087 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:31 smithi104 podman[71659]: 2024-09-19 04:34:31.644052146 +0000 UTC m=+0.442224396 container attach 43f5aa05fb71597791790feec7ca081c90df31e47d0078b06a8c0f6bb40033c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:34:32.087 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 19 04:34:31 smithi104 podman[71659]: 2024-09-19 04:34:31.741229142 +0000 UTC m=+0.539401391 container died 43f5aa05fb71597791790feec7ca081c90df31e47d0078b06a8c0f6bb40033c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T04:34:32.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:31.820+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:32.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:31 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:31.820+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:32.379 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T04:34:32.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:34:32.419 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T04:34:32.419 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T04:34:32.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.5 2024-09-19T04:34:32.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:32 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:32.244+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:32.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:32 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:32.244+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:32.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:32 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:32.338+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:32.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:32 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:32.571+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:32.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:32 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:32.571+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:32.716 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:32 smithi112 systemd[1]: Stopping Ceph osd.5 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:34:33.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:32 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:32.780+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:33.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:32 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:32.780+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:33.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:32 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:32.908+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:33.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:32 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:32.908+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:33.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:32 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:32.715+0000 7f0a381ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:34:33.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:32 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:32.715+0000 7f0a381ab640 -1 osd.5 109 *** Got signal Terminated *** 2024-09-19T04:34:33.036 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:32 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:32.715+0000 7f0a381ab640 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:34:33.036 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:32 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:32.739+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:33.036 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:32 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:32.739+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:33.237 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:32 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:32.843+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:33.237 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:32 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:32.843+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:33.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:33 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:33.236+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:33.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:33 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:33.236+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:33.854 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:33 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:33.363+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:33.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:33 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:33.545+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:33.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:33 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:33.545+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:34.034 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:33 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:33.767+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:34.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:33 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:33.767+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:34.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:33 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:33.776+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:34.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:33 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:33.776+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:34.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:33 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:33.927+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:34.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:33 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:33.927+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:34.280 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:33 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:33.851+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:34.280 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:33 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:33.851+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:34.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:34 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:34.278+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:34.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:34 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:34.278+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:34.854 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:34 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:34.376+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:34.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:34 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:34.569+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:34.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:34 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:34.569+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:35.034 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:34 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:34.890+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:35.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:34 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:34.890+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:35.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:34 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:34.738+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:35.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:34 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:34.738+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:35.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:34 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:34.738+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:12.492543+0000 front 2024-09-19T04:34:12.492608+0000 (oldest deadline 2024-09-19T04:34:34.192740+0000) 2024-09-19T04:34:35.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:34 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:34.815+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:35.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:34 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:34.815+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:35.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:34 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:34.815+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:35.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:34 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:34.811+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:35.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:34 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:34.811+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:35.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:35 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:35.265+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:35.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:35 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:35.265+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:35.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:35 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:35.336+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:35.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:35 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:35.580+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:35.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:35 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:35.580+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:36.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:35 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:35.871+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:36.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:35 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:35.871+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:36.036 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:35 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:35.739+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:36.036 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:35 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:35.739+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:36.036 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:35 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:35.739+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:12.492543+0000 front 2024-09-19T04:34:12.492608+0000 (oldest deadline 2024-09-19T04:34:34.192740+0000) 2024-09-19T04:34:36.037 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:35 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:35.830+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:36.037 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:35 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:35.830+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:36.037 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:35 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:35.830+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:36.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:35 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:35.790+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:36.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:35 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:35.790+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:36.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:35 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:35.790+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:36.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:36 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:36.252+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:36.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:36 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:36.252+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:36.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:36 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:36.295+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:36.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:36 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:36.295+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:37.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:36 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:36.859+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:37.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:36 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:36.859+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:37.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:36 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:36.724+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.389889+0000 front 2024-09-19T04:33:54.390031+0000 (oldest deadline 2024-09-19T04:34:23.190369+0000) 2024-09-19T04:34:37.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:36 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:36.724+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.691465+0000 front 2024-09-19T04:34:03.691543+0000 (oldest deadline 2024-09-19T04:34:27.191628+0000) 2024-09-19T04:34:37.035 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:36 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5[46808]: 2024-09-19T04:34:36.724+0000 7f0a307a8640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:12.492543+0000 front 2024-09-19T04:34:12.492608+0000 (oldest deadline 2024-09-19T04:34:34.192740+0000) 2024-09-19T04:34:37.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:36 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:36.873+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:37.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:36 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:36.873+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:37.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:36 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:36.873+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:37.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:36 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:36.754+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:37.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:36 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:36.754+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:37.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:36 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:36.754+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:37.104 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:36 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:36.611+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:37.104 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:36 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:36.611+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:37.574 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:37.259+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:37.574 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:37.259+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:37.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:37 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:37.235+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:37.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:37 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:37.235+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:37.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:37.573+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:37.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:37 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:37.573+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:38.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:37 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:37.779+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:38.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:37 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:37.779+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:38.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:37 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:37.779+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:38.265 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:37.844+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:38.265 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:37.844+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:38.265 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:37.844+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:38.265 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:37.909+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:38.265 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:37.909+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:38.266 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:37 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:37.909+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:38.534 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:38 smithi112 podman[72779]: 2024-09-19 04:34:38.263970766 +0000 UTC m=+5.647794606 container died ea4cf25ca7c8bc2db6068de31688b808d5acdc502c9cc7abb0fe5ce8388aa576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T04:34:38.566 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:38 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:38.303+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:38.566 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:38 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:38.303+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:38.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:38 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:38.198+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:38.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:38 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:38.198+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:38.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:38 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:38.198+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.415791+0000 front 2024-09-19T04:34:13.415725+0000 (oldest deadline 2024-09-19T04:34:37.515671+0000) 2024-09-19T04:34:38.846 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:38 smithi112 podman[72779]: 2024-09-19 04:34:38.632182946 +0000 UTC m=+6.016006825 container cleanup ea4cf25ca7c8bc2db6068de31688b808d5acdc502c9cc7abb0fe5ce8388aa576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T04:34:38.846 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:38 smithi112 bash[72779]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5 2024-09-19T04:34:38.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:38 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:38.565+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:38.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:38 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:38.565+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:38.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:38 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:38.565+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:39.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:38 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:38.806+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:39.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:38 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:38.806+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:39.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:38 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:38.806+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:39.237 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:38 smithi112 podman[72796]: 2024-09-19 04:34:38.925204202 +0000 UTC m=+0.653198621 container remove ea4cf25ca7c8bc2db6068de31688b808d5acdc502c9cc7abb0fe5ce8388aa576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T04:34:39.237 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:38 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:38.860+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:39.237 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:38 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:38.860+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:39.238 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:38 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:38.860+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:39.238 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:38 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:38.844+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:39.238 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:38 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:38.844+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:39.238 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:38 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:38.844+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:39.535 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:39 smithi112 podman[72885]: 2024-09-19 04:34:39.139248866 +0000 UTC m=+0.015231866 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:34:39.535 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:39 smithi112 podman[72885]: 2024-09-19 04:34:39.251045463 +0000 UTC m=+0.127028513 container create 9057eaf2646bf06276110aa580e97edd630b0be41c9427c4bde5bf427556ebdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T04:34:39.535 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:39 smithi112 podman[72885]: 2024-09-19 04:34:39.470045431 +0000 UTC m=+0.346028434 container init 9057eaf2646bf06276110aa580e97edd630b0be41c9427c4bde5bf427556ebdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T04:34:39.535 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:39 smithi112 podman[72885]: 2024-09-19 04:34:39.473323193 +0000 UTC m=+0.349306194 container start 9057eaf2646bf06276110aa580e97edd630b0be41c9427c4bde5bf427556ebdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T04:34:39.570 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:39 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:39.303+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:39.570 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:39 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:39.303+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:39.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:39 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:39.235+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:39.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:39 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:39.235+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:39.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:39 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:39.235+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.415791+0000 front 2024-09-19T04:34:13.415725+0000 (oldest deadline 2024-09-19T04:34:37.515671+0000) 2024-09-19T04:34:39.830 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:39 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:39.828+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:39.830 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:39 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:39.828+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:39.830 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:39 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:39.828+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:39.830 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:39 smithi112 podman[72885]: 2024-09-19 04:34:39.553392407 +0000 UTC m=+0.429375423 container attach 9057eaf2646bf06276110aa580e97edd630b0be41c9427c4bde5bf427556ebdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T04:34:39.830 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 19 04:34:39 smithi112 podman[72885]: 2024-09-19 04:34:39.663098069 +0000 UTC m=+0.539081064 container died 9057eaf2646bf06276110aa580e97edd630b0be41c9427c4bde5bf427556ebdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T04:34:39.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:39 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:39.569+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:39.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:39 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:39.569+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:39.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:39 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:39.569+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:40.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:39 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:39.802+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:40.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:39 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:39.802+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:40.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:39 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:39.802+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:40.224 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:39 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:39.900+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:40.224 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:39 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:39.900+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:40.224 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:39 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:39.900+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:40.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T04:34:40.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:34:40.280 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T04:34:40.280 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T04:34:40.280 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.6 2024-09-19T04:34:40.575 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:40 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:40.309+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:40.576 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:40 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:40.309+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:40.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:40 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:40.189+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:40.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:40 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:40.189+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:40.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:40 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:40.189+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.415791+0000 front 2024-09-19T04:34:13.415725+0000 (oldest deadline 2024-09-19T04:34:37.515671+0000) 2024-09-19T04:34:40.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:40 smithi094 systemd[1]: Stopping Ceph osd.6 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:34:40.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:40 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:40.575+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:40.893 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:40 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:40.575+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:40.893 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:40 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:40.575+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:41.090 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:40 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:40.590+0000 7fe8130ff640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:34:41.091 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:40 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:40.590+0000 7fe8130ff640 -1 osd.6 109 *** Got signal Terminated *** 2024-09-19T04:34:41.091 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:40 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:40.590+0000 7fe8130ff640 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:34:41.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:40 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:40.768+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:41.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:40 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:40.768+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:41.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:40 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:40.768+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:41.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:40 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:40.873+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:41.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:40 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:40.873+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:41.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:40 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:40.873+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:41.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:40 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:40.890+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:41.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:40 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:40.890+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:41.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:40 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:40.890+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:41.557 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:41 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:41.284+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:41.557 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:41 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:41.284+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:41.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:41 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:41.172+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:41.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:41 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:41.172+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:41.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:41 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:41.172+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.415791+0000 front 2024-09-19T04:34:13.415725+0000 (oldest deadline 2024-09-19T04:34:37.515671+0000) 2024-09-19T04:34:41.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:41 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:41.556+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:41.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:41 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:41.556+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:41.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:41 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:41.556+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:42.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:41 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:41.802+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:42.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:41 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:41.802+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:42.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:41 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:41.802+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:42.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:41 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:41.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:42.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:41 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:41.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:42.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:41 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:41.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:42.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:41 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:41.882+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:42.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:41 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:41.882+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:42.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:41 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:41.882+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:42.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:42 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:42.122+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:42.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:42 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:42.122+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:42.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:42 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:42.122+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.415791+0000 front 2024-09-19T04:34:13.415725+0000 (oldest deadline 2024-09-19T04:34:37.515671+0000) 2024-09-19T04:34:42.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:42 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:42.315+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:42.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:42 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:42.315+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:42.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:42 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:42.524+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:42.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:42 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:42.524+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:42.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:42 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:42.524+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:42.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:42 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:42.524+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:43.116 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:42 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:42.847+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:43.116 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:42 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:42.847+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:43.116 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:42 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:42.847+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:43.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:42 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:42.865+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:43.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:42 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:42.865+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:43.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:42 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:42.865+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:43.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:42 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:42.819+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:43.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:42 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:42.819+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:43.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:42 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:42.819+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:43.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:43 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:43.114+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:43.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:43 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:43.114+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:43.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:43 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:43.114+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.415791+0000 front 2024-09-19T04:34:13.415725+0000 (oldest deadline 2024-09-19T04:34:37.515671+0000) 2024-09-19T04:34:43.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:43 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:43.309+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:43.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:43 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:43.309+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:43.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:43 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:43.309+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:43.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:43 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:43.487+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:43.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:43 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:43.487+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:43.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:43 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:43.487+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:43.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:43 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:43.487+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:44.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:43 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:43.797+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:44.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:43 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:43.797+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:44.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:43 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:43.797+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:44.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:43 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:43.797+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:44.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:43.892+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:44.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:43.892+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:44.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:43.892+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:44.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:43.892+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:44.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:43.816+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:44.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:43.816+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:44.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:43.816+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:44.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:43 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:43.816+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:44.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:44.089+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:44.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:44.089+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:44.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:44.089+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.415791+0000 front 2024-09-19T04:34:13.415725+0000 (oldest deadline 2024-09-19T04:34:37.515671+0000) 2024-09-19T04:34:44.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:44.089+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:17.516097+0000 front 2024-09-19T04:34:17.516126+0000 (oldest deadline 2024-09-19T04:34:43.416231+0000) 2024-09-19T04:34:44.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:44 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:44.318+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:44.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:44 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:44.318+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:44.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:44 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:44.318+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:44.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:44 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:44.473+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:44.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:44 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:44.473+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:44.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:44 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:44.473+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:44.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:44 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:44.473+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:45.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:44.807+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:45.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:44.807+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:45.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:44.807+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:45.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:44 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:44.807+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:45.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:44 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:44.886+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:45.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:44 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:44.886+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:45.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:44 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:44.886+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:45.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:44 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:44.886+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:45.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:44 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:44.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:45.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:44 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:44.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:45.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:44 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:44.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:45.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:44 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:44.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:45.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:45 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:45.117+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.712273+0000 front 2024-09-19T04:33:56.712030+0000 (oldest deadline 2024-09-19T04:34:18.912647+0000) 2024-09-19T04:34:45.590 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:45 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:45.117+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.314174+0000 front 2024-09-19T04:34:05.314157+0000 (oldest deadline 2024-09-19T04:34:29.414057+0000) 2024-09-19T04:34:45.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:45 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:45.117+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.415791+0000 front 2024-09-19T04:34:13.415725+0000 (oldest deadline 2024-09-19T04:34:37.515671+0000) 2024-09-19T04:34:45.591 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:45 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6[58572]: 2024-09-19T04:34:45.117+0000 7fe80b6fc640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:17.516097+0000 front 2024-09-19T04:34:17.516126+0000 (oldest deadline 2024-09-19T04:34:43.416231+0000) 2024-09-19T04:34:45.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:45 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:45.284+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:45.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:45 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:45.284+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:45.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:45 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:45.284+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:45.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:45 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:45.450+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:45.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:45 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:45.450+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:45.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:45 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:45.450+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:45.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:45 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:45.450+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:46.061 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:45 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:45.771+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:46.061 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:45 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:45.771+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:46.061 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:45 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:45.771+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:46.061 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:45 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:45.771+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:46.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:45 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:45.852+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:46.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:45 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:45.852+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:46.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:45 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:45.852+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:46.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:45 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:45.852+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:46.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:45 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:45.927+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:46.288 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:45 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:45.927+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:46.288 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:45 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:45.927+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:46.288 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:45 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:45.927+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:46.340 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:46 smithi094 podman[122880]: 2024-09-19 04:34:46.060254156 +0000 UTC m=+5.586437888 container died 7b2feba17e08fb7f912d533280705bc3189994cb25b6bea3c0b8395c6c3259ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:34:46.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:46 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:46.327+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:46.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:46 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:46.327+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:46.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:46 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:46.327+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:46.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:46 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:46.416+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:46.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:46 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:46.416+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:46.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:46 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:46.416+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:46.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:46 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:46.416+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:46.811 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:46 smithi094 podman[122880]: 2024-09-19 04:34:46.340491743 +0000 UTC m=+5.866675476 container cleanup 7b2feba17e08fb7f912d533280705bc3189994cb25b6bea3c0b8395c6c3259ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T04:34:46.812 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:46 smithi094 bash[122880]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6 2024-09-19T04:34:46.812 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:46 smithi094 podman[122895]: 2024-09-19 04:34:46.533863435 +0000 UTC m=+0.467909142 container remove 7b2feba17e08fb7f912d533280705bc3189994cb25b6bea3c0b8395c6c3259ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T04:34:47.090 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:46 smithi094 podman[122985]: 2024-09-19 04:34:46.859679237 +0000 UTC m=+0.015521918 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:34:47.091 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:46 smithi094 podman[122985]: 2024-09-19 04:34:46.993223815 +0000 UTC m=+0.149066527 container create 0ce256838393863e94df9194375ca7ddf0f0bd865f7298592f9991e416cf937d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T04:34:47.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:46 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:46.810+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:47.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:46 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:46.810+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:47.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:46 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:46.810+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:47.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:46 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:46.810+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:47.284 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:46 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:46.924+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:47.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:46 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:46.924+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:47.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:46 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:46.924+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:47.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:46 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:46.924+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:47.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:46 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:46.834+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:47.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:46 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:46.834+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:47.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:46 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:46.834+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:47.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:46 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:46.834+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:47.403 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:47 smithi094 podman[122985]: 2024-09-19 04:34:47.203680859 +0000 UTC m=+0.359523534 container init 0ce256838393863e94df9194375ca7ddf0f0bd865f7298592f9991e416cf937d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T04:34:47.403 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:47 smithi094 podman[122985]: 2024-09-19 04:34:47.206929146 +0000 UTC m=+0.362771813 container start 0ce256838393863e94df9194375ca7ddf0f0bd865f7298592f9991e416cf937d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-19T04:34:47.404 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:47 smithi094 podman[122985]: 2024-09-19 04:34:47.303528448 +0000 UTC m=+0.459371117 container attach 0ce256838393863e94df9194375ca7ddf0f0bd865f7298592f9991e416cf937d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:34:47.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:47.310+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:47.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:47.310+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:47.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:47.310+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:47.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:47.447+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:47.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:47.447+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:47.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:47.447+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:47.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:47.447+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:47.757 INFO:journalctl@ceph.osd.6.smithi094.stdout:Sep 19 04:34:47 smithi094 podman[122985]: 2024-09-19 04:34:47.403134123 +0000 UTC m=+0.558976790 container died 0ce256838393863e94df9194375ca7ddf0f0bd865f7298592f9991e416cf937d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T04:34:48.032 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:47 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:47.780+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:48.033 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:47 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:47.780+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:48.033 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:47 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:47.780+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:48.033 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:47 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:47.780+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:48.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T04:34:48.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:34:48.097 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T04:34:48.098 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T04:34:48.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.7 2024-09-19T04:34:48.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:47.886+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:48.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:47.886+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:48.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:47.886+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:48.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:47.886+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:48.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:47.832+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:48.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:47.832+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:48.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:47.832+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:48.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:47 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:47.832+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:48.405 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:48 smithi104 systemd[1]: Stopping Ceph osd.7 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:34:48.405 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:48.288+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:48.405 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:48.288+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:48.405 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:48.288+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:48.405 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:48.403+0000 7fe4d6b00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:34:48.405 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:48.403+0000 7fe4d6b00640 -1 osd.7 109 *** Got signal Terminated *** 2024-09-19T04:34:48.405 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:48.403+0000 7fe4d6b00640 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:34:48.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:48.461+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:48.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:48.461+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:48.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:48.461+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:48.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:48 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:48.461+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:49.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:48 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:48.769+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:49.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:48 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:48.769+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:49.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:48 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:48.769+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:49.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:48 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:48.769+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:49.186 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:48 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:48.791+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:49.186 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:48 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:48.791+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:49.187 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:48 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:48.791+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:49.187 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:48 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:48.791+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:49.187 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:48 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:48.926+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:49.187 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:48 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:48.926+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:49.187 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:48 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:48.926+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:49.187 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:48 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:48.926+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:49.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:49 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:49.285+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:49.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:49 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:49.285+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:49.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:49 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:49.285+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:49.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:49 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:49.421+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:49.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:49 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:49.421+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:49.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:49 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:49.421+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:49.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:49 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:49.421+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:50.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:49 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:49.781+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:50.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:49 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:49.781+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:50.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:49 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:49.781+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:50.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:49 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:49.781+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:50.284 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:49 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:49.947+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:50.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:49 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:49.947+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:50.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:49 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:49.947+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:50.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:49 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:49.947+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:50.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:49 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:49.803+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:50.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:49 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:49.803+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:50.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:49 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:49.803+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:50.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:49 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:49.803+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:50.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:50 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:50.320+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:50.605 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:50 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:50.320+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:50.605 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:50 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:50.320+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:50.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:50 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:50.436+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:50.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:50 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:50.436+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:50.606 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:50 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:50.436+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:50.606 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:50 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:50.436+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:51.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:50 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:50.749+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:51.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:50 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:50.749+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:51.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:50 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:50.749+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:51.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:50 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:50.749+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:51.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:50 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:50.749+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:34:51.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:50 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:50.983+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:51.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:50 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:50.983+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:51.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:50 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:50.983+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:51.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:50 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:50.983+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:51.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:50 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:50.791+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:51.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:50 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:50.791+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:51.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:50 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:50.791+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:51.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:50 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:50.791+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:51.854 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:51 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:51.356+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:51.854 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:51 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:51.356+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:51.855 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:51 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:51.356+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:51.855 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:51 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:51.356+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.590442+0000 front 2024-09-19T04:34:26.590502+0000 (oldest deadline 2024-09-19T04:34:50.591119+0000) 2024-09-19T04:34:51.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:51 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:51.391+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:51.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:51 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:51.391+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:51.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:51 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:51.391+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:51.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:51 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:51.391+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:52.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:51 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:51.764+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:52.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:51 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:51.764+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:52.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:51 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:51.764+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:52.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:51 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:51.764+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:52.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:51 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:51.764+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:34:52.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:52.008+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:52.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:52.008+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:52.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:52.008+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:52.036 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:52.008+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:52.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:51 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:51.757+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:52.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:51 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:51.757+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:52.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:51 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:51.757+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:52.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:51 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:51.757+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:52.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:51 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:51.757+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:34:52.854 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:52 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:52.357+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:52.854 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:52 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:52.357+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:52.854 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:52 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:52.357+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:52.855 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:52 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:52.357+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.590442+0000 front 2024-09-19T04:34:26.590502+0000 (oldest deadline 2024-09-19T04:34:50.591119+0000) 2024-09-19T04:34:52.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:52 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:52.415+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:52.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:52 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:52.415+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:52.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:52 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:52.415+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:52.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:52 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:52.415+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:52.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:52 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:52.415+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:34:53.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:53.017+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:53.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:53.017+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:53.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:53.017+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:53.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:53.017+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:53.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:52.769+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:53.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:52.769+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:53.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:52.769+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:53.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:52.769+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:53.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:52 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:52.769+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:34:53.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:52.789+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:53.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:52.789+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:53.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:52.789+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:53.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:52.789+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:53.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:52 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:52.789+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:34:53.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:53 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:53.345+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.087462+0000 front 2024-09-19T04:34:02.087500+0000 (oldest deadline 2024-09-19T04:34:27.987569+0000) 2024-09-19T04:34:53.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:53 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:53.345+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.288399+0000 front 2024-09-19T04:34:10.288355+0000 (oldest deadline 2024-09-19T04:34:36.188464+0000) 2024-09-19T04:34:53.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:53 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:53.345+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:16.189069+0000 front 2024-09-19T04:34:16.188954+0000 (oldest deadline 2024-09-19T04:34:43.189390+0000) 2024-09-19T04:34:53.604 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:53 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7[52025]: 2024-09-19T04:34:53.345+0000 7fe4d3105640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.590442+0000 front 2024-09-19T04:34:26.590502+0000 (oldest deadline 2024-09-19T04:34:50.591119+0000) 2024-09-19T04:34:53.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:53 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:53.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:53.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:53 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:53.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:53.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:53 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:53.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:53.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:53 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:53.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:53.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:53 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:53.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:34:54.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:53 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:53.798+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:54.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:53 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:53.798+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:54.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:53 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:53.798+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:54.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:53 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:53.798+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:54.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:53 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:53.798+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:34:54.155 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:53 smithi104 podman[71759]: 2024-09-19 04:34:53.86233156 +0000 UTC m=+5.561795261 container died be2e02635f589324ec7ac56b4a7b72fd8b02fcf2295939e65b8dea1572a0f1e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:34:54.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:53.978+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:54.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:53.978+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:54.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:53.978+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:54.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:53.978+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:54.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:53.800+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:54.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:53.800+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:54.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:53.800+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:54.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:53.800+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:54.286 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:53 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:53.800+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:34:54.458 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:54 smithi104 podman[71759]: 2024-09-19 04:34:54.253558869 +0000 UTC m=+5.953022559 container cleanup be2e02635f589324ec7ac56b4a7b72fd8b02fcf2295939e65b8dea1572a0f1e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T04:34:54.458 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:54 smithi104 bash[71759]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7 2024-09-19T04:34:54.820 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:54 smithi104 podman[71773]: 2024-09-19 04:34:54.512903579 +0000 UTC m=+0.651870437 container remove be2e02635f589324ec7ac56b4a7b72fd8b02fcf2295939e65b8dea1572a0f1e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:34:54.820 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:54 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:54.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:54.820 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:54 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:54.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:54.820 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:54 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:54.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:54.820 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:54 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:54.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:54.820 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:54 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:54.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:34:55.104 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:54 smithi104 podman[71867]: 2024-09-19 04:34:54.72191631 +0000 UTC m=+0.015562305 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:34:55.104 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:54 smithi104 podman[71867]: 2024-09-19 04:34:54.830219735 +0000 UTC m=+0.123865721 container create 1b8d6d50d5c2749e900eaab673276cd043e43bbaa19509901cb8b2156a4a316d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T04:34:55.104 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:55 smithi104 podman[71867]: 2024-09-19 04:34:55.057525586 +0000 UTC m=+0.351171569 container init 1b8d6d50d5c2749e900eaab673276cd043e43bbaa19509901cb8b2156a4a316d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T04:34:55.104 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:55 smithi104 podman[71867]: 2024-09-19 04:34:55.060701697 +0000 UTC m=+0.354347683 container start 1b8d6d50d5c2749e900eaab673276cd043e43bbaa19509901cb8b2156a4a316d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T04:34:55.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:54 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:54.989+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:55.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:54 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:54.989+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:55.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:54 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:54.989+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:55.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:54 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:54.989+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:55.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:54 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:54.819+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:55.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:54 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:54.819+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:55.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:54 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:54.819+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:55.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:54 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:54.819+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:55.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:54 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:54.819+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:34:55.340 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:54 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:54.848+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:55.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:54 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:54.848+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:55.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:54 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:54.848+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:55.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:54 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:54.848+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:55.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:54 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:54.848+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:34:55.437 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:55 smithi104 podman[71867]: 2024-09-19 04:34:55.149293737 +0000 UTC m=+0.442939723 container attach 1b8d6d50d5c2749e900eaab673276cd043e43bbaa19509901cb8b2156a4a316d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:34:55.438 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 19 04:34:55 smithi104 podman[71867]: 2024-09-19 04:34:55.251545046 +0000 UTC m=+0.545191039 container died 1b8d6d50d5c2749e900eaab673276cd043e43bbaa19509901cb8b2156a4a316d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T04:34:55.803 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:55 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:55.435+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:55.803 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:55 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:55.435+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:55.803 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:55 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:55.435+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:55.803 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:55 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:55.435+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:55.803 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:55 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:55.435+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:34:55.832 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T04:34:55.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:34:55.871 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T04:34:55.871 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T04:34:55.871 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.8 2024-09-19T04:34:56.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:55.825+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:56.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:55.825+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:56.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:55.825+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:56.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:55.825+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:56.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:55 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:55.825+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:34:56.169 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:55 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:55.795+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:56.170 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:55 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:55.795+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:56.170 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:55 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:55.795+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:56.170 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:55 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:55.795+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:56.170 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:55 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:55.795+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:34:56.170 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:55 smithi112 systemd[1]: Stopping Ceph osd.8 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:34:56.170 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:55 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:55.971+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:56.170 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:55 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:55.971+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:56.170 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:55 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:55.971+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:56.171 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:55 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:55.971+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:56.534 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:56.168+0000 7f3e7ebbb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:34:56.535 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:56.168+0000 7f3e7ebbb640 -1 osd.8 109 *** Got signal Terminated *** 2024-09-19T04:34:56.535 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:56.168+0000 7f3e7ebbb640 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:34:56.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:56.439+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:56.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:56.439+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:56.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:56.439+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:56.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:56.439+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:56.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:56 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:56.439+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:34:57.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:56.979+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:57.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:56.979+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:57.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:56.979+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:57.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:56.979+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:57.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:56.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:57.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:56.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:57.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:56.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:57.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:56.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:57.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:56 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:56.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:34:57.340 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:56.849+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:57.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:56.849+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:57.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:56.849+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:57.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:56.849+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:57.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:56 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:56.849+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:34:57.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:57 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:57.412+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:57.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:57 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:57.412+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:57.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:57 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:57.412+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:57.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:57 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:57.412+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:57.605 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:57 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:57.412+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:34:58.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:57.933+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:58.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:57.933+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:58.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:57.933+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:58.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:57.933+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:58.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:57.744+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:58.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:57.744+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:58.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:57.744+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:58.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:57.744+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:58.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:57 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:57.744+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:34:58.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:57.855+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:58.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:57.855+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:58.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:57.855+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:58.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:57.855+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:58.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:57 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:57.855+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:34:58.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:58 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:58.429+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:58.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:58 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:58.429+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:58.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:58 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:58.429+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:58.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:58 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:58.429+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:58.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:58 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:58.429+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:34:58.980 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:58.712+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:34:58.980 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:58.712+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:34:58.980 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:58.712+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:34:58.980 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:58.712+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:34:58.980 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:58.712+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:34:59.090 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:58 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:58.900+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:34:59.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:58 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:58.900+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:34:59.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:58 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:58.900+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:34:59.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:58 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:58.900+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:34:59.091 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:58 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:58.900+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:34:59.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:58.978+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:34:59.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:58.978+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:34:59.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:58.978+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:34:59.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:58 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:58.978+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:34:59.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:59.476+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:34:59.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:59.476+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:34:59.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:59.476+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:34:59.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:59.476+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:34:59.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:59.476+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:34:59.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:34:59 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:34:59.476+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:00.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:59.950+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:35:00.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:59.950+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:35:00.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:59.950+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:35:00.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:34:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:34:59.950+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:35:00.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:59.745+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:00.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:59.745+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:00.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:59.745+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:00.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:59.745+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:00.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:34:59 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:34:59.745+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:00.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:59.914+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:35:00.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:59.914+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:35:00.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:59.914+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:35:00.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:59.914+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:35:00.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:59.914+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:35:00.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:34:59 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:34:59.914+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:34.731113+0000 front 2024-09-19T04:34:34.731125+0000 (oldest deadline 2024-09-19T04:34:59.431096+0000) 2024-09-19T04:35:00.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:00.497+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:00.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:00.497+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:00.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:00.497+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:00.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:00.497+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:00.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:00.497+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:00.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:00 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:00.497+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:01.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:35:00.983+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:54.415573+0000 front 2024-09-19T04:33:54.415518+0000 (oldest deadline 2024-09-19T04:34:17.915561+0000) 2024-09-19T04:35:01.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:35:00.983+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:02.616407+0000 front 2024-09-19T04:34:02.616543+0000 (oldest deadline 2024-09-19T04:34:28.516453+0000) 2024-09-19T04:35:01.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:35:00.983+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:10.817171+0000 front 2024-09-19T04:34:10.817218+0000 (oldest deadline 2024-09-19T04:34:34.317325+0000) 2024-09-19T04:35:01.035 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8[53738]: 2024-09-19T04:35:00.983+0000 7f3e7b1c0640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.218520+0000 front 2024-09-19T04:34:20.218629+0000 (oldest deadline 2024-09-19T04:34:43.718434+0000) 2024-09-19T04:35:01.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:00.751+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:01.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:00.751+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:01.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:00.751+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:01.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:00.751+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:01.036 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:00 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:00.751+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:01.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:00.943+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:35:01.368 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:00.943+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:35:01.368 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:00.943+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:35:01.368 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:00.943+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:35:01.368 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:00.943+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:35:01.368 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:00 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:00.943+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:34.731113+0000 front 2024-09-19T04:34:34.731125+0000 (oldest deadline 2024-09-19T04:34:59.431096+0000) 2024-09-19T04:35:01.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:01 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:01.539+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:01.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:01 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:01.539+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:01.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:01 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:01.539+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:01.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:01 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:01.539+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:01.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:01 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:01.539+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:01.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:01 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:01.539+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:01.928 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:01 smithi112 podman[72999]: 2024-09-19 04:35:01.651013536 +0000 UTC m=+5.579190243 container died f8fc7241ac757c9d746fe284b35f62f93a8d684ce07c83c5b81d2681b8c4f3aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:35:01.928 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:01 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:01.703+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:01.928 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:01 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:01.703+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:01.928 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:01 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:01.703+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:01.928 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:01 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:01.703+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:01.928 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:01 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:01.703+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:01.928 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:01 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:01.703+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:02.270 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:02 smithi112 podman[72999]: 2024-09-19 04:35:02.026240516 +0000 UTC m=+5.954417221 container cleanup f8fc7241ac757c9d746fe284b35f62f93a8d684ce07c83c5b81d2681b8c4f3aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T04:35:02.270 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:02 smithi112 bash[72999]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8 2024-09-19T04:35:02.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:01.956+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:35:02.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:01.956+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:35:02.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:01.956+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:35:02.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:01.956+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:35:02.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:01.956+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:35:02.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:01 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:01.956+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:34.731113+0000 front 2024-09-19T04:34:34.731125+0000 (oldest deadline 2024-09-19T04:34:59.431096+0000) 2024-09-19T04:35:02.535 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:02 smithi112 podman[73013]: 2024-09-19 04:35:02.26928869 +0000 UTC m=+0.617089488 container remove f8fc7241ac757c9d746fe284b35f62f93a8d684ce07c83c5b81d2681b8c4f3aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T04:35:02.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:02 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:02.506+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:02.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:02 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:02.506+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:02.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:02 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:02.506+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:02.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:02 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:02.506+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:02.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:02 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:02.506+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:02.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:02 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:02.506+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:02.932 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:02 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:02.726+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:02.932 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:02 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:02.726+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:02.932 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:02 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:02.726+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:02.932 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:02 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:02.726+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:02.932 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:02 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:02.726+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:02.932 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:02 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:02.726+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:02.933 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:02 smithi112 podman[73098]: 2024-09-19 04:35:02.527499481 +0000 UTC m=+0.015558627 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:35:02.933 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:02 smithi112 podman[73098]: 2024-09-19 04:35:02.628271059 +0000 UTC m=+0.116330192 container create 6b4f477f36fe101b64f19e66e422292e1100c004796fa68d70748821358f935c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T04:35:02.933 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:02 smithi112 podman[73098]: 2024-09-19 04:35:02.856027132 +0000 UTC m=+0.344086272 container init 6b4f477f36fe101b64f19e66e422292e1100c004796fa68d70748821358f935c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T04:35:02.933 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:02 smithi112 podman[73098]: 2024-09-19 04:35:02.859370077 +0000 UTC m=+0.347429211 container start 6b4f477f36fe101b64f19e66e422292e1100c004796fa68d70748821358f935c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:03.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:02 smithi112 podman[73098]: 2024-09-19 04:35:02.930935824 +0000 UTC m=+0.418994959 container attach 6b4f477f36fe101b64f19e66e422292e1100c004796fa68d70748821358f935c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:03.285 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 19 04:35:03 smithi112 podman[73098]: 2024-09-19 04:35:03.049946993 +0000 UTC m=+0.538006130 container died 6b4f477f36fe101b64f19e66e422292e1100c004796fa68d70748821358f935c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T04:35:03.340 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:02.916+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:35:03.340 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:02.916+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:35:03.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:02.916+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:35:03.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:02.916+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:35:03.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:02.916+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:35:03.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:02 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:02.916+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:34.731113+0000 front 2024-09-19T04:34:34.731125+0000 (oldest deadline 2024-09-19T04:34:59.431096+0000) 2024-09-19T04:35:03.630 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T04:35:03.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:03.669 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T04:35:03.669 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T04:35:03.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.9 2024-09-19T04:35:03.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:03 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:03.502+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:03.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:03 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:03.502+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:03.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:03 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:03.502+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:03.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:03 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:03.502+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:03.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:03 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:03.502+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:03.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:03 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:03.502+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:03.969 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 systemd[1]: Stopping Ceph osd.9 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:03.969 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:03.938+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:35:03.969 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:03.938+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:35:03.969 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:03.938+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:35:03.970 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:03.938+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:35:03.970 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:03.938+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:35:03.970 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:03.938+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:34.731113+0000 front 2024-09-19T04:34:34.731125+0000 (oldest deadline 2024-09-19T04:34:59.431096+0000) 2024-09-19T04:35:03.970 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:03.967+0000 7f17d83b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:03.970 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:03.967+0000 7f17d83b5640 -1 osd.9 109 *** Got signal Terminated *** 2024-09-19T04:35:03.970 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:03 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:03.967+0000 7f17d83b5640 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:35:04.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:03 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:03.717+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:04.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:03 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:03.717+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:04.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:03 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:03.717+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:04.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:03 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:03.717+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:04.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:03 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:03.717+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:04.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:03 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:03.717+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:04.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:04 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:04.499+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:04.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:04 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:04.499+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:04.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:04 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:04.499+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:04.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:04 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:04.499+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:04.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:04 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:04.499+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:04.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:04 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:04.499+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:05.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:04 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:04.683+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:05.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:04 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:04.683+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:05.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:04 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:04.683+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:05.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:04 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:04.683+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:05.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:04 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:04.683+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:05.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:04 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:04.683+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:05.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:04.964+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:35:05.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:04.965+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:35:05.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:04.965+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:35:05.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:04.965+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:35:05.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:04.965+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:35:05.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:04 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:04.965+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:34.731113+0000 front 2024-09-19T04:34:34.731125+0000 (oldest deadline 2024-09-19T04:34:59.431096+0000) 2024-09-19T04:35:05.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:05 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:05.485+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:05.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:05 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:05.485+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:05.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:05 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:05.485+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:05.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:05 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:05.485+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:05.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:05 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:05.485+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:05.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:05 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:05.485+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:06.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:05 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:05.652+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:06.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:05 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:05.652+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:06.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:05 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:05.652+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:06.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:05 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:05.652+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:06.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:05 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:05.652+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:06.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:05 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:05.652+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:06.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:05.967+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:35:06.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:05.967+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:35:06.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:05.967+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:35:06.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:05.967+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:35:06.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:05.967+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:35:06.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:05 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:05.967+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:34.731113+0000 front 2024-09-19T04:34:34.731125+0000 (oldest deadline 2024-09-19T04:34:59.431096+0000) 2024-09-19T04:35:06.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:06 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:06.455+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:06.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:06 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:06.455+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:06.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:06 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:06.455+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:06.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:06 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:06.455+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:06.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:06 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:06.455+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:06.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:06 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:06.455+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:07.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:06 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:06.697+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:07.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:06 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:06.697+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:07.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:06 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:06.697+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:07.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:06 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:06.697+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:07.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:06 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:06.697+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:07.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:06 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:06.697+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:07.340 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:06 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:06.985+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:35:07.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:06 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:06.985+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:35:07.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:06 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:06.985+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:35:07.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:06 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:06.985+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:35:07.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:06 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:06.985+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:35:07.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:06 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:06.985+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:34.731113+0000 front 2024-09-19T04:34:34.731125+0000 (oldest deadline 2024-09-19T04:34:59.431096+0000) 2024-09-19T04:35:07.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:07 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:07.501+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:07.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:07 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:07.501+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:07.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:07 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:07.501+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:07.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:07 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:07.501+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:07.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:07 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:07.501+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:07.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:07 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:07.501+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:07.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:07 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:07.501+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:08.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:07 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:07.681+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:08.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:07 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:07.681+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:08.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:07 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:07.681+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:08.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:07 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:07.681+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:08.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:07 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:07.681+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:08.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:07 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:07.681+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:08.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:08.025+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:52.622687+0000 front 2024-09-19T04:33:52.622674+0000 (oldest deadline 2024-09-19T04:34:18.522489+0000) 2024-09-19T04:35:08.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:08.025+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:05.523964+0000 front 2024-09-19T04:34:05.523829+0000 (oldest deadline 2024-09-19T04:34:28.423965+0000) 2024-09-19T04:35:08.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:08.025+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:08.424598+0000 front 2024-09-19T04:34:08.424565+0000 (oldest deadline 2024-09-19T04:34:34.825140+0000) 2024-09-19T04:35:08.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:08.025+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:21.127192+0000 front 2024-09-19T04:34:21.127220+0000 (oldest deadline 2024-09-19T04:34:43.427306+0000) 2024-09-19T04:35:08.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:08.025+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:28.529348+0000 front 2024-09-19T04:34:28.529420+0000 (oldest deadline 2024-09-19T04:34:50.729610+0000) 2024-09-19T04:35:08.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:08 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9[65586]: 2024-09-19T04:35:08.025+0000 7f17d09b2640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:34.731113+0000 front 2024-09-19T04:34:34.731125+0000 (oldest deadline 2024-09-19T04:34:59.431096+0000) 2024-09-19T04:35:08.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:08 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:08.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:08.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:08 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:08.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:08.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:08 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:08.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:08.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:08 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:08.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:08.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:08 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:08.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:08.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:08 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:08.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:08.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:08 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:08.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:09.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:08 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:08.699+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:09.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:08 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:08.699+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:09.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:08 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:08.699+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:09.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:08 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:08.699+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:09.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:08 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:08.699+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:09.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:08 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:08.699+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:09.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:09 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:09.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:09.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:09 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:09.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:09.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:09 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:09.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:09.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:09 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:09.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:09.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:09 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:09.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:09.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:09 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:09.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:09.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:09 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:09.456+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:09.862 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:09 smithi094 podman[123099]: 2024-09-19 04:35:09.598256308 +0000 UTC m=+5.738596532 container died f6265443b2db169dc10a202e98545f5543f6489baeb64525ba93dbf9b9b735ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T04:35:10.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:09 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:09.725+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:10.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:09 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:09.725+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:10.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:09 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:09.725+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:10.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:09 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:09.725+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:10.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:09 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:09.725+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:10.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:09 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:09.725+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:10.245 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:10 smithi094 podman[123099]: 2024-09-19 04:35:10.042838294 +0000 UTC m=+6.183178472 container cleanup f6265443b2db169dc10a202e98545f5543f6489baeb64525ba93dbf9b9b735ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:10.245 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:10 smithi094 bash[123099]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9 2024-09-19T04:35:10.590 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:10 smithi094 podman[123116]: 2024-09-19 04:35:10.244363806 +0000 UTC m=+0.639337552 container remove f6265443b2db169dc10a202e98545f5543f6489baeb64525ba93dbf9b9b735ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:10.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:10 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:10.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:10.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:10 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:10.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:10.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:10 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:10.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:10.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:10 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:10.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:10.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:10 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:10.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:10.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:10 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:10.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:10.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:10 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:10.468+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:10.914 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:10 smithi094 podman[123202]: 2024-09-19 04:35:10.558672745 +0000 UTC m=+0.015952223 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:35:10.914 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:10 smithi094 podman[123202]: 2024-09-19 04:35:10.703167361 +0000 UTC m=+0.160446827 container create dd8c557915aeda7e501246fa97e4eec403e5c67cf6493e6943dc65a45dcd2788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:10.914 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:10 smithi094 podman[123202]: 2024-09-19 04:35:10.913654189 +0000 UTC m=+0.370933655 container init dd8c557915aeda7e501246fa97e4eec403e5c67cf6493e6943dc65a45dcd2788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T04:35:11.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:10.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:11.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:10.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:11.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:10.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:11.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:10.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:11.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:10.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:11.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:10.772+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:11.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:10 smithi094 podman[123202]: 2024-09-19 04:35:10.917088466 +0000 UTC m=+0.374367932 container start dd8c557915aeda7e501246fa97e4eec403e5c67cf6493e6943dc65a45dcd2788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:11.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:11 smithi094 podman[123202]: 2024-09-19 04:35:11.005382453 +0000 UTC m=+0.462661929 container attach dd8c557915aeda7e501246fa97e4eec403e5c67cf6493e6943dc65a45dcd2788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T04:35:11.341 INFO:journalctl@ceph.osd.9.smithi094.stdout:Sep 19 04:35:11 smithi094 podman[123202]: 2024-09-19 04:35:11.099116186 +0000 UTC m=+0.556395662 container died dd8c557915aeda7e501246fa97e4eec403e5c67cf6493e6943dc65a45dcd2788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T04:35:11.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T04:35:11.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:11.765 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T04:35:11.765 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T04:35:11.765 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.10 2024-09-19T04:35:11.794 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:11 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:11.480+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:11.794 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:11 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:11.480+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:11.794 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:11 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:11.480+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:11.794 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:11 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:11.480+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:11.795 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:11 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:11.480+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:11.795 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:11 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:11.480+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:11.795 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:11 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:11.480+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:12.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:11 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:11.781+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:12.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:11 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:11.781+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:12.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:11 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:11.781+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:12.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:11 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:11.781+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:12.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:11 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:11.781+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:12.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:11 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:11.781+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:12.071 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:11 smithi104 systemd[1]: Stopping Ceph osd.10 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:12.072 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.069+0000 7f57fa8cc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:12.072 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.069+0000 7f57fa8cc640 -1 osd.10 109 *** Got signal Terminated *** 2024-09-19T04:35:12.072 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.069+0000 7f57fa8cc640 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:35:12.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.493+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:12.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.493+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:12.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.493+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:12.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.493+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:12.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.493+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:12.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.493+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:12.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:12 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:12.493+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:13.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:12 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:12.828+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:13.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:12 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:12.828+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:13.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:12 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:12.828+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:13.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:12 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:12.828+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:13.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:12 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:12.828+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:13.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:12 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:12.828+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:13.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:13 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:13.504+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:13.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:13 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:13.504+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:13.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:13 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:13.504+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:13.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:13 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:13.504+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:13.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:13 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:13.504+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:13.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:13 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:13.504+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:13.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:13 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:13.504+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:14.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:13 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:13.865+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:14.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:13 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:13.865+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:14.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:13 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:13.865+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:14.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:13 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:13.865+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:14.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:13 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:13.865+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:14.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:13 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:13.865+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:14.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:14 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:14.535+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:14.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:14 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:14.535+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:14.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:14 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:14.535+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:14.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:14 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:14.535+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:14.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:14 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:14.535+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:14.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:14 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:14.535+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:14.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:14 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:14.535+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:15.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:14 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:14.820+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:15.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:14 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:14.820+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:15.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:14 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:14.820+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:15.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:14 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:14.820+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:15.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:14 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:14.820+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:15.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:14 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:14.820+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:15.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:15 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:15.558+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:15.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:15 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:15.558+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:15.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:15 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:15.558+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:15.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:15 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:15.558+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:15.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:15 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:15.558+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:15.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:15 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:15.558+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:15.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:15 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:15.558+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:16.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:15 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:15.814+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:16.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:15 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:15.814+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:16.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:15 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:15.814+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:16.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:15 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:15.814+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:16.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:15 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:15.814+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:16.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:15 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:15.814+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:16.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:16 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:16.599+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:56.717035+0000 front 2024-09-19T04:33:56.716974+0000 (oldest deadline 2024-09-19T04:34:22.517485+0000) 2024-09-19T04:35:16.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:16 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:16.599+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:03.618383+0000 front 2024-09-19T04:34:03.618399+0000 (oldest deadline 2024-09-19T04:34:28.918581+0000) 2024-09-19T04:35:16.854 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:16 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:16.599+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:13.020052+0000 front 2024-09-19T04:34:13.020242+0000 (oldest deadline 2024-09-19T04:34:38.320061+0000) 2024-09-19T04:35:16.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:16 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:16.599+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:18.821084+0000 front 2024-09-19T04:34:18.821131+0000 (oldest deadline 2024-09-19T04:34:42.320950+0000) 2024-09-19T04:35:16.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:16 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:16.599+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:26.922494+0000 front 2024-09-19T04:34:26.922454+0000 (oldest deadline 2024-09-19T04:34:52.222522+0000) 2024-09-19T04:35:16.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:16 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:16.599+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.923589+0000 front 2024-09-19T04:34:36.923560+0000 (oldest deadline 2024-09-19T04:34:59.223425+0000) 2024-09-19T04:35:16.855 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:16 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10[58757]: 2024-09-19T04:35:16.599+0000 7f57f2ec9640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-09-19T04:34:43.324491+0000 front 2024-09-19T04:34:43.324412+0000 (oldest deadline 2024-09-19T04:35:07.424216+0000) 2024-09-19T04:35:17.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:16 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:16.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:17.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:16 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:16.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:17.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:16 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:16.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:17.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:16 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:16.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:17.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:16 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:16.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:17.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:16 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:16.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:17.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:16 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:16.857+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-19T04:34:51.325051+0000 front 2024-09-19T04:34:51.325089+0000 (oldest deadline 2024-09-19T04:35:15.925290+0000) 2024-09-19T04:35:17.996 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:17 smithi104 podman[71982]: 2024-09-19 04:35:17.714000889 +0000 UTC m=+5.751167294 container died 0a908449f74636215b9b344a527e343c414183aee53fbd4aa8529f1ea2a979e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T04:35:18.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:17 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:17.887+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:18.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:17 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:17.887+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:18.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:17 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:17.887+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:18.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:17 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:17.887+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:18.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:17 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:17.887+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:18.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:17 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:17.887+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:18.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:17 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:17.887+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-19T04:34:51.325051+0000 front 2024-09-19T04:34:51.325089+0000 (oldest deadline 2024-09-19T04:35:15.925290+0000) 2024-09-19T04:35:18.345 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:18 smithi104 podman[71982]: 2024-09-19 04:35:18.077570005 +0000 UTC m=+6.114736399 container cleanup 0a908449f74636215b9b344a527e343c414183aee53fbd4aa8529f1ea2a979e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T04:35:18.345 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:18 smithi104 bash[71982]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10 2024-09-19T04:35:18.604 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:18 smithi104 podman[71998]: 2024-09-19 04:35:18.345128739 +0000 UTC m=+0.626171116 container remove 0a908449f74636215b9b344a527e343c414183aee53fbd4aa8529f1ea2a979e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:18.957 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:18 smithi104 podman[72089]: 2024-09-19 04:35:18.593074801 +0000 UTC m=+0.015622325 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:35:18.957 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:18 smithi104 podman[72089]: 2024-09-19 04:35:18.720987435 +0000 UTC m=+0.143534944 container create 09cdd9850db29aab216b7cf0ec9dcf5fe2a0f6a3dbf27bb07872bd2667ce4f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T04:35:18.957 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:18 smithi104 podman[72089]: 2024-09-19 04:35:18.956677872 +0000 UTC m=+0.379225397 container init 09cdd9850db29aab216b7cf0ec9dcf5fe2a0f6a3dbf27bb07872bd2667ce4f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T04:35:19.186 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:18 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:18.871+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:19.186 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:18 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:18.871+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:19.186 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:18 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:18.871+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:19.186 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:18 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:18.871+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:19.187 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:18 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:18.871+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:19.187 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:18 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:18.871+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:19.187 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:18 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:18.871+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-19T04:34:51.325051+0000 front 2024-09-19T04:34:51.325089+0000 (oldest deadline 2024-09-19T04:35:15.925290+0000) 2024-09-19T04:35:19.354 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:18 smithi104 podman[72089]: 2024-09-19 04:35:18.959937421 +0000 UTC m=+0.382484928 container start 09cdd9850db29aab216b7cf0ec9dcf5fe2a0f6a3dbf27bb07872bd2667ce4f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T04:35:19.354 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:19 smithi104 podman[72089]: 2024-09-19 04:35:19.031800259 +0000 UTC m=+0.454347777 container attach 09cdd9850db29aab216b7cf0ec9dcf5fe2a0f6a3dbf27bb07872bd2667ce4f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:19.354 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 19 04:35:19 smithi104 podman[72089]: 2024-09-19 04:35:19.153688771 +0000 UTC m=+0.576236286 container died 09cdd9850db29aab216b7cf0ec9dcf5fe2a0f6a3dbf27bb07872bd2667ce4f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-10-deactivate, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T04:35:19.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T04:35:19.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:19.817 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T04:35:19.818 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T04:35:19.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.11 2024-09-19T04:35:20.122 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:19 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:19.843+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:20.122 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:19 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:19.843+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:20.123 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:19 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:19.843+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:20.123 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:19 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:19.843+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:20.123 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:19 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:19.843+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:20.123 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:19 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:19.843+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:20.123 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:19 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:19.843+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-19T04:34:51.325051+0000 front 2024-09-19T04:34:51.325089+0000 (oldest deadline 2024-09-19T04:35:15.925290+0000) 2024-09-19T04:35:20.123 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:19 smithi112 systemd[1]: Stopping Ceph osd.11 for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:20.123 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.120+0000 7f816bd4c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:20.123 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.120+0000 7f816bd4c640 -1 osd.11 109 *** Got signal Terminated *** 2024-09-19T04:35:20.123 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.120+0000 7f816bd4c640 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:35:21.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:21.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:21.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:21.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:21.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:21.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:21.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:20 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:20.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-19T04:34:51.325051+0000 front 2024-09-19T04:34:51.325089+0000 (oldest deadline 2024-09-19T04:35:15.925290+0000) 2024-09-19T04:35:22.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:21 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:21.767+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:22.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:21 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:21.767+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:22.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:21 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:21.767+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:22.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:21 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:21.767+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:22.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:21 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:21.767+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:22.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:21 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:21.767+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:22.035 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:21 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:21.767+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-19T04:34:51.325051+0000 front 2024-09-19T04:34:51.325089+0000 (oldest deadline 2024-09-19T04:35:15.925290+0000) 2024-09-19T04:35:23.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:22 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:22.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:23.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:22 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:22.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:23.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:22 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:22.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:23.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:22 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:22.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:23.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:22 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:22.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:23.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:22 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:22.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:23.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:22 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:22.805+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-19T04:34:51.325051+0000 front 2024-09-19T04:34:51.325089+0000 (oldest deadline 2024-09-19T04:35:15.925290+0000) 2024-09-19T04:35:24.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:23.789+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:24.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:23.789+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:24.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:23.789+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:24.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:23.789+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:24.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:23.789+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:24.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:23.789+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:24.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:23 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:23.789+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-19T04:34:51.325051+0000 front 2024-09-19T04:34:51.325089+0000 (oldest deadline 2024-09-19T04:35:15.925290+0000) 2024-09-19T04:35:25.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:24.830+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-09-19T04:33:55.515490+0000 front 2024-09-19T04:33:55.515601+0000 (oldest deadline 2024-09-19T04:34:19.015616+0000) 2024-09-19T04:35:25.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:24.830+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-19T04:34:04.916602+0000 front 2024-09-19T04:34:04.916658+0000 (oldest deadline 2024-09-19T04:34:29.016688+0000) 2024-09-19T04:35:25.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:24.830+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-19T04:34:11.917701+0000 front 2024-09-19T04:34:11.917642+0000 (oldest deadline 2024-09-19T04:34:37.217737+0000) 2024-09-19T04:35:25.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:24.830+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-09-19T04:34:20.118901+0000 front 2024-09-19T04:34:20.118934+0000 (oldest deadline 2024-09-19T04:34:43.618964+0000) 2024-09-19T04:35:25.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:24.830+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-19T04:34:27.020616+0000 front 2024-09-19T04:34:27.020739+0000 (oldest deadline 2024-09-19T04:34:51.720522+0000) 2024-09-19T04:35:25.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:24.830+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-19T04:34:36.321979+0000 front 2024-09-19T04:34:36.321910+0000 (oldest deadline 2024-09-19T04:35:01.621996+0000) 2024-09-19T04:35:25.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:24 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11[60292]: 2024-09-19T04:35:24.830+0000 7f8164349640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-19T04:34:51.325051+0000 front 2024-09-19T04:34:51.325089+0000 (oldest deadline 2024-09-19T04:35:15.925290+0000) 2024-09-19T04:35:25.864 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:25 smithi112 podman[73212]: 2024-09-19 04:35:25.607785768 +0000 UTC m=+5.597339140 container died c581fb000b95d0266eca8f1c9654cd2846538b07f83f991d0bcaa35501369083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T04:35:26.264 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:25 smithi112 podman[73212]: 2024-09-19 04:35:25.953584196 +0000 UTC m=+5.943137559 container cleanup c581fb000b95d0266eca8f1c9654cd2846538b07f83f991d0bcaa35501369083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T04:35:26.264 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:25 smithi112 bash[73212]: ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11 2024-09-19T04:35:26.534 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:26 smithi112 podman[73227]: 2024-09-19 04:35:26.263282704 +0000 UTC m=+0.653965161 container remove c581fb000b95d0266eca8f1c9654cd2846538b07f83f991d0bcaa35501369083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:26.934 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:26 smithi112 podman[73316]: 2024-09-19 04:35:26.51011158 +0000 UTC m=+0.015880051 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:35:26.934 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:26 smithi112 podman[73316]: 2024-09-19 04:35:26.647427227 +0000 UTC m=+0.153195684 container create e51e1163062b83c8e44d9b378e6074e621d97fff0c002caea3e7ff448cd1f450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T04:35:26.934 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:26 smithi112 podman[73316]: 2024-09-19 04:35:26.850172601 +0000 UTC m=+0.355941058 container init e51e1163062b83c8e44d9b378e6074e621d97fff0c002caea3e7ff448cd1f450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T04:35:26.934 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:26 smithi112 podman[73316]: 2024-09-19 04:35:26.853580324 +0000 UTC m=+0.359348780 container start e51e1163062b83c8e44d9b378e6074e621d97fff0c002caea3e7ff448cd1f450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:35:26.934 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:26 smithi112 podman[73316]: 2024-09-19 04:35:26.933299519 +0000 UTC m=+0.439067988 container attach e51e1163062b83c8e44d9b378e6074e621d97fff0c002caea3e7ff448cd1f450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T04:35:27.285 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:27 smithi112 podman[73316]: 2024-09-19 04:35:27.053695286 +0000 UTC m=+0.559463743 container died e51e1163062b83c8e44d9b378e6074e621d97fff0c002caea3e7ff448cd1f450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T04:35:27.579 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 19 04:35:27 smithi112 podman[73339]: 2024-09-19 04:35:27.561739599 +0000 UTC m=+0.498140124 container remove e51e1163062b83c8e44d9b378e6074e621d97fff0c002caea3e7ff448cd1f450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T04:35:27.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T04:35:27.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:27.621 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T04:35:27.621 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T04:35:27.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.a 2024-09-19T04:35:27.912 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:35:27 smithi094 systemd[1]: Stopping Ceph mds.a for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:28.340 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:35:27 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-a[72332]: 2024-09-19T04:35:27.910+0000 7f33a96e6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:28.341 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:35:27 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-a[72332]: 2024-09-19T04:35:27.910+0000 7f33a96e6640 -1 mds.a *** got signal Terminated *** 2024-09-19T04:35:32.287 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:35:32 smithi094 podman[123303]: 2024-09-19 04:35:32.008197966 +0000 UTC m=+4.182568190 container died f2db2a61c56b3d5ecab580394510e12bdfd6f1acc1816d1a9670ffc1817c68f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:32.590 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:35:32 smithi094 podman[123303]: 2024-09-19 04:35:32.385085224 +0000 UTC m=+4.559455449 container cleanup f2db2a61c56b3d5ecab580394510e12bdfd6f1acc1816d1a9670ffc1817c68f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:35:32.591 INFO:journalctl@ceph.mds.a.smithi094.stdout:Sep 19 04:35:32 smithi094 bash[123303]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-a 2024-09-19T04:35:32.814 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T04:35:32.851 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:32.852 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T04:35:32.852 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T04:35:32.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.d 2024-09-19T04:35:33.341 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:35:32 smithi094 systemd[1]: Stopping Ceph mds.d for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:33.341 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:35:33 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-d[71147]: 2024-09-19T04:35:33.183+0000 7fc667b2d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:33.341 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:35:33 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-d[71147]: 2024-09-19T04:35:33.183+0000 7fc667b2d640 -1 mds.d *** got signal Terminated *** 2024-09-19T04:35:37.476 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:35:37 smithi094 podman[123436]: 2024-09-19 04:35:37.193076436 +0000 UTC m=+4.118115177 container died 094be904acb43a0c7d203a1c1846bbf23336abd159cc359d854329bc8116bc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T04:35:37.840 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:35:37 smithi094 podman[123436]: 2024-09-19 04:35:37.583020619 +0000 UTC m=+4.508059343 container cleanup 094be904acb43a0c7d203a1c1846bbf23336abd159cc359d854329bc8116bc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T04:35:37.840 INFO:journalctl@ceph.mds.d.smithi094.stdout:Sep 19 04:35:37 smithi094 bash[123436]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-d 2024-09-19T04:35:37.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T04:35:38.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:38.009 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T04:35:38.009 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T04:35:38.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.g 2024-09-19T04:35:38.338 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:35:38 smithi094 systemd[1]: Stopping Ceph mds.g for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:38.590 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:35:38 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-g[71918]: 2024-09-19T04:35:38.336+0000 7f03c3e2a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:38.591 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:35:38 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-g[71918]: 2024-09-19T04:35:38.336+0000 7f03c3e2a640 -1 mds.g *** got signal Terminated *** 2024-09-19T04:35:41.205 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:35:40 smithi094 podman[123558]: 2024-09-19 04:35:40.8905715 +0000 UTC m=+2.663488407 container died d640449f79613d7332d8e12aeb8bf9c02b5ce4f8d860aba97640d77e62f16ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T04:35:41.590 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:35:41 smithi094 podman[123558]: 2024-09-19 04:35:41.387227142 +0000 UTC m=+3.160144049 container cleanup d640449f79613d7332d8e12aeb8bf9c02b5ce4f8d860aba97640d77e62f16ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-g, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:41.591 INFO:journalctl@ceph.mds.g.smithi094.stdout:Sep 19 04:35:41 smithi094 bash[123558]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-g 2024-09-19T04:35:41.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T04:35:41.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:41.848 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T04:35:41.848 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T04:35:41.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.j 2024-09-19T04:35:42.160 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:35:41 smithi094 systemd[1]: Stopping Ceph mds.j for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:42.160 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:35:42 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-j[70750]: 2024-09-19T04:35:42.158+0000 7f2f3b84e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:42.160 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:35:42 smithi094 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-j[70750]: 2024-09-19T04:35:42.158+0000 7f2f3b84e640 -1 mds.j *** got signal Terminated *** 2024-09-19T04:35:46.090 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:35:45 smithi094 podman[123676]: 2024-09-19 04:35:45.835115439 +0000 UTC m=+3.769394801 container died 8d883c184a160f6d085691989bc4f5c4db033e68a7946f3626bcaace6f5f955e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T04:35:46.447 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:35:46 smithi094 podman[123676]: 2024-09-19 04:35:46.433794233 +0000 UTC m=+4.368073638 container cleanup 8d883c184a160f6d085691989bc4f5c4db033e68a7946f3626bcaace6f5f955e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T04:35:46.447 INFO:journalctl@ceph.mds.j.smithi094.stdout:Sep 19 04:35:46 smithi094 bash[123676]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-j 2024-09-19T04:35:46.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T04:35:46.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:46.817 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T04:35:46.817 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T04:35:46.817 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.b 2024-09-19T04:35:47.104 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:35:46 smithi104 systemd[1]: Stopping Ceph mds.b for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:47.562 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:35:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b[63958]: 2024-09-19T04:35:47.134+0000 7f8e9a3a2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:47.562 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:35:47 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b[63958]: 2024-09-19T04:35:47.134+0000 7f8e9a3a2640 -1 mds.b *** got signal Terminated *** 2024-09-19T04:35:49.817 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:35:49 smithi104 podman[72192]: 2024-09-19 04:35:49.523246181 +0000 UTC m=+2.510854062 container died 707f082b4b8faf27c92a614b089cccd85595304e745bdfe1ed697695e887477a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T04:35:50.104 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:35:49 smithi104 podman[72192]: 2024-09-19 04:35:49.924333171 +0000 UTC m=+2.911941051 container cleanup 707f082b4b8faf27c92a614b089cccd85595304e745bdfe1ed697695e887477a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T04:35:50.104 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 19 04:35:49 smithi104 bash[72192]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-b 2024-09-19T04:35:50.352 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T04:35:50.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:50.387 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T04:35:50.387 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T04:35:50.387 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.e 2024-09-19T04:35:50.721 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:35:50 smithi104 systemd[1]: Stopping Ceph mds.e for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:51.104 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:35:50 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e[63352]: 2024-09-19T04:35:50.719+0000 7fc771ae3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:51.104 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:35:50 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e[63352]: 2024-09-19T04:35:50.719+0000 7fc771ae3640 -1 mds.e *** got signal Terminated *** 2024-09-19T04:35:53.837 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:35:53 smithi104 podman[72325]: 2024-09-19 04:35:53.521674152 +0000 UTC m=+2.907146552 container died 91311a4e8208cd9029a0fc7ae900609f98acdbdcb82d7feddda3d3df797bd415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T04:35:54.104 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:35:53 smithi104 podman[72325]: 2024-09-19 04:35:53.919467793 +0000 UTC m=+3.304940199 container cleanup 91311a4e8208cd9029a0fc7ae900609f98acdbdcb82d7feddda3d3df797bd415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T04:35:54.104 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:35:53 smithi104 bash[72325]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e 2024-09-19T04:35:54.405 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 19 04:35:54 smithi104 podman[72337]: 2024-09-19 04:35:54.153834673 +0000 UTC m=+0.631052691 container remove 91311a4e8208cd9029a0fc7ae900609f98acdbdcb82d7feddda3d3df797bd415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T04:35:54.409 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T04:35:54.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:54.443 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T04:35:54.443 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T04:35:54.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.h 2024-09-19T04:35:54.765 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:35:54 smithi104 systemd[1]: Stopping Ceph mds.h for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:55.104 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:35:54 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h[63654]: 2024-09-19T04:35:54.764+0000 7f3679817640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:55.104 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:35:54 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h[63654]: 2024-09-19T04:35:54.764+0000 7f3679817640 -1 mds.h *** got signal Terminated *** 2024-09-19T04:35:57.790 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:35:57 smithi104 podman[72448]: 2024-09-19 04:35:57.512743668 +0000 UTC m=+2.854567509 container died fd7a16416277a40958965506b5d5588a722d16cf5a3806f26437010301204a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:35:58.104 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:35:57 smithi104 podman[72448]: 2024-09-19 04:35:57.888984903 +0000 UTC m=+3.230808744 container cleanup fd7a16416277a40958965506b5d5588a722d16cf5a3806f26437010301204a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:58.104 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 19 04:35:57 smithi104 bash[72448]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-h 2024-09-19T04:35:58.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T04:35:58.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:58.344 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T04:35:58.344 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T04:35:58.344 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.k 2024-09-19T04:35:58.660 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:35:58 smithi104 systemd[1]: Stopping Ceph mds.k for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:59.104 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:35:58 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-k[63040]: 2024-09-19T04:35:58.658+0000 7f821c78b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:59.104 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:35:58 smithi104 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-k[63040]: 2024-09-19T04:35:58.658+0000 7f821c78b640 -1 mds.k *** got signal Terminated *** 2024-09-19T04:36:02.935 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:36:02 smithi104 podman[72568]: 2024-09-19 04:36:02.673666962 +0000 UTC m=+4.112661803 container died 5bd23669d672323b7483b798c4f803cb5255158332d12dd2fb6b8200de107c13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-k, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T04:36:03.201 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:36:03 smithi104 podman[72568]: 2024-09-19 04:36:03.008447057 +0000 UTC m=+4.447441897 container cleanup 5bd23669d672323b7483b798c4f803cb5255158332d12dd2fb6b8200de107c13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-k, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T04:36:03.201 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 19 04:36:03 smithi104 bash[72568]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-k 2024-09-19T04:36:03.424 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T04:36:03.456 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:03.456 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T04:36:03.456 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T04:36:03.456 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.c 2024-09-19T04:36:03.785 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:36:03 smithi112 systemd[1]: Stopping Ceph mds.c for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:04.052 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:36:03 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c[65135]: 2024-09-19T04:36:03.835+0000 7efdb2de0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:04.052 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:36:03 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c[65135]: 2024-09-19T04:36:03.835+0000 7efdb2de0640 -1 mds.c *** got signal Terminated *** 2024-09-19T04:36:05.780 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:36:05 smithi112 podman[73418]: 2024-09-19 04:36:05.525156659 +0000 UTC m=+1.877804483 container died fc3ebc87679e78d69e922b17d133249eedacb810eac88531c8e9e92a72813af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:36:06.035 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:36:05 smithi112 podman[73418]: 2024-09-19 04:36:05.878158934 +0000 UTC m=+2.230806753 container cleanup fc3ebc87679e78d69e922b17d133249eedacb810eac88531c8e9e92a72813af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-09-19T04:36:06.035 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 19 04:36:05 smithi112 bash[73418]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-c 2024-09-19T04:36:06.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T04:36:06.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:06.342 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T04:36:06.342 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T04:36:06.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.f 2024-09-19T04:36:06.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:36:06 smithi112 systemd[1]: Stopping Ceph mds.f for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:06.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:36:06 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f[64832]: 2024-09-19T04:36:06.648+0000 7fb683a74640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:06.785 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:36:06 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f[64832]: 2024-09-19T04:36:06.648+0000 7fb683a74640 -1 mds.f *** got signal Terminated *** 2024-09-19T04:36:09.784 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:36:09 smithi112 podman[73551]: 2024-09-19 04:36:09.518176526 +0000 UTC m=+2.964686948 container died 8f1c7824f25b3978db8ce5a3ebf553697e4fde293deca684cd7268b9d513d62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:36:10.035 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:36:09 smithi112 podman[73551]: 2024-09-19 04:36:09.87411569 +0000 UTC m=+3.320626099 container cleanup 8f1c7824f25b3978db8ce5a3ebf553697e4fde293deca684cd7268b9d513d62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T04:36:10.035 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 19 04:36:09 smithi112 bash[73551]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-f 2024-09-19T04:36:10.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T04:36:10.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:10.345 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T04:36:10.345 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T04:36:10.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.i 2024-09-19T04:36:10.784 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:36:10 smithi112 systemd[1]: Stopping Ceph mds.i for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:10.785 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:36:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i[65435]: 2024-09-19T04:36:10.652+0000 7fab8c576640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:10.785 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:36:10 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i[65435]: 2024-09-19T04:36:10.652+0000 7fab8c576640 -1 mds.i *** got signal Terminated *** 2024-09-19T04:36:13.785 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:36:13 smithi112 podman[73671]: 2024-09-19 04:36:13.508870835 +0000 UTC m=+2.947785619 container died 00741c13c1113e5228298c77ce380e31eae0f7c8bddd81b3153c971d7bfbb2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T04:36:14.257 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:36:13 smithi112 podman[73671]: 2024-09-19 04:36:13.902717461 +0000 UTC m=+3.341632238 container cleanup 00741c13c1113e5228298c77ce380e31eae0f7c8bddd81b3153c971d7bfbb2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T04:36:14.257 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:36:13 smithi112 bash[73671]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i 2024-09-19T04:36:14.258 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 19 04:36:14 smithi112 podman[73683]: 2024-09-19 04:36:14.062175907 +0000 UTC m=+0.527893581 container remove 00741c13c1113e5228298c77ce380e31eae0f7c8bddd81b3153c971d7bfbb2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T04:36:14.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T04:36:14.310 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:14.311 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T04:36:14.311 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T04:36:14.311 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.l 2024-09-19T04:36:14.534 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:36:14 smithi112 systemd[1]: Stopping Ceph mds.l for 503c4526-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:15.034 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:36:14 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l[64529]: 2024-09-19T04:36:14.622+0000 7f2740dd4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:15.035 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:36:14 smithi112 ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l[64529]: 2024-09-19T04:36:14.622+0000 7f2740dd4640 -1 mds.l *** got signal Terminated *** 2024-09-19T04:36:16.359 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:36:16 smithi112 podman[73790]: 2024-09-19 04:36:16.055969658 +0000 UTC m=+1.528425395 container stop c9740431c0dfe128be9d4e1d07759377ff433883ed2c366077bb028cdf9aa549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T04:36:16.359 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:36:16 smithi112 podman[73790]: 2024-09-19 04:36:16.081604399 +0000 UTC m=+1.554060144 container died c9740431c0dfe128be9d4e1d07759377ff433883ed2c366077bb028cdf9aa549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T04:36:16.624 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:36:16 smithi112 podman[73790]: 2024-09-19 04:36:16.456510217 +0000 UTC m=+1.928965954 container cleanup c9740431c0dfe128be9d4e1d07759377ff433883ed2c366077bb028cdf9aa549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T04:36:16.625 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 19 04:36:16 smithi112 bash[73790]: ceph-503c4526-763c-11ef-bceb-c7b262605968-mds-l 2024-09-19T04:36:16.826 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-503c4526-763c-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T04:36:16.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:16.858 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T04:36:16.858 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 503c4526-763c-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T04:36:17.060 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:36:19.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 503c4526-763c-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T04:36:19.796 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:36:22.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 503c4526-763c-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T04:36:22.486 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:36:25.005 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:36:25.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:36:25.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:36:25.098 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T04:36:25.100 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624/remote/smithi094/crash 2024-09-19T04:36:25.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/crash -- . 2024-09-19T04:36:25.155 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T04:36:25.156 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-09-19T04:36:25.158 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624/remote/smithi104/crash 2024-09-19T04:36:25.158 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/crash -- . 2024-09-19T04:36:25.201 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T04:36:25.202 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-09-19T04:36:25.203 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624/remote/smithi112/crash 2024-09-19T04:36:25.204 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/crash -- . 2024-09-19T04:36:25.237 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/503c4526-763c-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T04:36:25.237 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-09-19T04:36:25.238 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T04:36:25.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T04:36:25.328 INFO:tasks.cephadm:Compressing logs... 2024-09-19T04:36:25.329 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:36:25.331 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:36:25.334 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:36:25.360 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T04:36:25.360 INFO:teuthology.orchestra.run.smithi104.stderr:: No such file or directory 2024-09-19T04:36:25.360 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T04:36:25.362 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T04:36:25.362 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T04:36:25.362 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T04:36:25.363 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T04:36:25.363 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T04:36:25.363 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.b.log: /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T04:36:25.363 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T04:36:25.364 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T04:36:25.364 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T04:36:25.365 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T04:36:25.366 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T04:36:25.366 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T04:36:25.366 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T04:36:25.366 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T04:36:25.366 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T04:36:25.367 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.4.log: /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T04:36:25.368 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T04:36:25.369 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log 2024-09-19T04:36:25.369 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T04:36:25.369 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T04:36:25.369 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T04:36:25.369 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.l.log: /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T04:36:25.369 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T04:36:25.369 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose --/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.c.log: /var/log/ceph/cephadm.log 2024-09-19T04:36:25.373 INFO:teuthology.orchestra.run.smithi104.stderr: 94.6% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T04:36:25.373 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T04:36:25.373 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log 2024-09-19T04:36:25.373 INFO:teuthology.orchestra.run.smithi104.stderr: 82.0% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T04:36:25.375 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mgr.y.log: 91.0% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T04:36:25.376 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T04:36:25.376 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T04:36:25.377 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T04:36:25.378 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-19T04:36:25.378 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T04:36:25.378 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T04:36:25.378 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T04:36:25.382 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T04:36:25.382 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T04:36:25.382 INFO:teuthology.orchestra.run.smithi104.stderr: 92.3% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T04:36:25.385 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.h.log: 87.5% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T04:36:25.386 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T04:36:25.386 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T04:36:25.400 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T04:36:25.443 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T04:36:25.465 INFO:teuthology.orchestra.run.smithi112.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T04:36:25.471 INFO:teuthology.orchestra.run.smithi112.stderr: 94.7% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T04:36:25.475 INFO:teuthology.orchestra.run.smithi094.stderr: 90.3% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T04:36:25.496 INFO:teuthology.orchestra.run.smithi112.stderr: 82.0% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T04:36:25.532 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T04:36:25.543 INFO:teuthology.orchestra.run.smithi112.stderr: 90.5% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T04:36:25.554 INFO:teuthology.orchestra.run.smithi094.stderr: 90.9% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T04:36:25.563 INFO:teuthology.orchestra.run.smithi112.stderr: 91.0% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T04:36:25.592 INFO:teuthology.orchestra.run.smithi094.stderr: 94.6% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T04:36:25.631 INFO:teuthology.orchestra.run.smithi094.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T04:36:25.632 INFO:teuthology.orchestra.run.smithi094.stderr: 84.5% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T04:36:25.705 INFO:teuthology.orchestra.run.smithi104.stderr: 94.0% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T04:36:25.876 INFO:teuthology.orchestra.run.smithi094.stderr: 94.6% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T04:36:25.880 INFO:teuthology.orchestra.run.smithi094.stderr: 94.6% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T04:36:25.998 INFO:teuthology.orchestra.run.smithi104.stderr: 91.7% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T04:36:26.075 INFO:teuthology.orchestra.run.smithi104.stderr: 95.4% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T04:36:26.995 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T04:36:27.248 INFO:teuthology.orchestra.run.smithi094.stderr: 90.3% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T04:36:28.822 INFO:teuthology.orchestra.run.smithi112.stderr: 94.9% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T04:36:33.256 INFO:teuthology.orchestra.run.smithi094.stderr: 90.3% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T04:36:37.463 INFO:teuthology.orchestra.run.smithi104.stderr: 94.0% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T04:36:39.531 INFO:teuthology.orchestra.run.smithi104.stderr: 94.0% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T04:36:41.167 INFO:teuthology.orchestra.run.smithi104.stderr: 93.8% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T04:36:41.618 INFO:teuthology.orchestra.run.smithi104.stderr: 94.0% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T04:36:49.412 INFO:teuthology.orchestra.run.smithi104.stderr: 94.1% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T04:36:49.414 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-09-19T04:36:49.415 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m24.066s 2024-09-19T04:36:49.415 INFO:teuthology.orchestra.run.smithi104.stderr:user 1m20.965s 2024-09-19T04:36:49.415 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m2.900s 2024-09-19T04:36:58.595 INFO:teuthology.orchestra.run.smithi112.stderr: 93.9% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T04:36:59.043 INFO:teuthology.orchestra.run.smithi112.stderr: 93.8% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T04:37:06.367 INFO:teuthology.orchestra.run.smithi094.stderr: 94.0% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T04:37:06.406 INFO:teuthology.orchestra.run.smithi112.stderr: 93.7% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T04:37:38.442 INFO:teuthology.orchestra.run.smithi112.stderr: 97.2% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T04:37:41.378 INFO:teuthology.orchestra.run.smithi112.stderr: 94.1% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T04:38:21.395 INFO:teuthology.orchestra.run.smithi094.stderr: 93.8% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T04:38:27.526 INFO:teuthology.orchestra.run.smithi094.stderr: 93.9% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T04:38:41.320 INFO:teuthology.orchestra.run.smithi094.stderr: 93.7% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T04:39:02.221 INFO:teuthology.orchestra.run.smithi112.stderr: 94.3% -- replaced with /var/log/ceph/503c4526-763c-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T04:39:02.222 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-19T04:39:02.223 INFO:teuthology.orchestra.run.smithi112.stderr:real 2m36.874s 2024-09-19T04:39:02.223 INFO:teuthology.orchestra.run.smithi112.stderr:user 3m13.104s 2024-09-19T04:39:02.223 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m7.888s 2024-09-19T04:40:38.821 INFO:teuthology.orchestra.run.smithi094.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-19T04:40:38.822 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-19T04:40:38.823 INFO:teuthology.orchestra.run.smithi094.stderr:real 4m13.476s 2024-09-19T04:40:38.823 INFO:teuthology.orchestra.run.smithi094.stderr:user 3m27.294s 2024-09-19T04:40:38.823 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m9.237s 2024-09-19T04:40:38.823 INFO:tasks.cephadm:Archiving logs... 2024-09-19T04:40:38.824 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624/remote/smithi094/log 2024-09-19T04:40:38.825 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:41:10.248 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624/remote/smithi104/log 2024-09-19T04:41:10.249 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:41:21.262 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624/remote/smithi112/log 2024-09-19T04:41:21.264 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:41:48.535 INFO:tasks.cephadm:Removing cluster... 2024-09-19T04:41:48.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 503c4526-763c-11ef-bceb-c7b262605968 --force 2024-09-19T04:41:48.738 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:41:49.089 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 503c4526-763c-11ef-bceb-c7b262605968 --force 2024-09-19T04:41:49.292 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:41:49.636 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 503c4526-763c-11ef-bceb-c7b262605968 --force 2024-09-19T04:41:49.839 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 503c4526-763c-11ef-bceb-c7b262605968 2024-09-19T04:41:50.309 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T04:41:50.309 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T04:41:50.327 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T04:41:50.344 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T04:41:50.360 INFO:tasks.cephadm:Teardown complete 2024-09-19T04:41:50.360 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T04:41:50.371 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T04:41:50.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:41:50.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:41:50.386 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:41:50.506 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T04:41:50.516 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T04:41:50.516 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:41:50.519 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:41:50.521 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:41:50.531 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:41:50.534 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:41:50.535 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:41:50.536 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:41:50.536 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-09-19T04:41:50.536 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 7 377 80 +31us[ +65us] +/- 44ms 2024-09-19T04:41:50.536 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 +83us[ +83us] +/- 75ms 2024-09-19T04:41:50.536 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 -127us[ -127us] +/- 83ms 2024-09-19T04:41:50.536 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:41:50.538 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:41:50.538 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-09-19T04:41:50.538 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 +26us[ +29us] +/- 45ms 2024-09-19T04:41:50.538 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 +93us[ +96us] +/- 75ms 2024-09-19T04:41:50.538 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -128us[ -128us] +/- 83ms 2024-09-19T04:41:50.539 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:41:50.567 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:41:50.567 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-19T04:41:50.567 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +21us[ +19us] +/- 45ms 2024-09-19T04:41:50.567 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +89us[ +88us] +/- 75ms 2024-09-19T04:41:50.567 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -130us[ -132us] +/- 83ms 2024-09-19T04:41:50.568 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:41:50.568 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T04:41:50.579 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T04:41:50.579 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T04:41:50.595 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T04:41:50.741 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T04:41:50.810 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T04:41:50.936 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:41:50.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:41:50.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:41:50.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:41:50.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:41:51.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:41:51.021 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T04:41:51.203 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T04:41:51.227 INFO:teuthology.task.internal:Duration was 2584.329363 seconds 2024-09-19T04:41:51.228 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T04:41:51.253 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T04:41:51.253 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:41:51.255 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:41:51.258 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:41:51.295 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:41:51.332 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:41:51.333 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:41:51.642 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T04:41:51.642 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-09-19T04:41:51.643 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:41:51.663 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-09-19T04:41:51.663 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:41:51.707 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-09-19T04:41:51.708 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:41:51.748 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T04:41:51.748 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:41:51.750 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:41:51.753 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:41:51.815 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T04:41:51.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:41:51.817 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:41:51.820 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:41:53.012 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T04:41:53.023 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T04:41:53.023 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:41:53.080 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:41:53.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:41:53.147 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T04:41:53.158 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:41:53.160 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:41:53.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:41:53.186 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-09-19T04:41:53.191 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-09-19T04:41:53.214 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-09-19T04:41:53.455 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:41:53.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:41:53.470 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:41:53.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:41:53.512 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:41:53.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:41:53.527 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T04:41:53.539 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T04:41:53.540 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624/remote/smithi094 2024-09-19T04:41:53.540 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:41:53.601 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624/remote/smithi104 2024-09-19T04:41:53.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:41:53.640 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911624/remote/smithi112 2024-09-19T04:41:53.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:41:53.678 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T04:41:53.678 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:41:53.681 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:41:53.683 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:41:53.726 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T04:41:53.740 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T04:41:53.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T04:41:53.781 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T04:41:53.782 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:41:53.784 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:41:53.787 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:41:53.796 INFO:teuthology.orchestra.run.smithi094.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:41 /home/ubuntu/cephtest 2024-09-19T04:41:53.801 INFO:teuthology.orchestra.run.smithi104.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:41 /home/ubuntu/cephtest 2024-09-19T04:41:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:41 /home/ubuntu/cephtest 2024-09-19T04:41:53.803 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T04:41:53.814 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T04:41:53.882 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} duration: 2584.3293633461 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T04:41:53.882 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T04:41:53.972 INFO:teuthology.run:pass